fix: Fix Question Service seed-batch bugs and edge cases #35

xGladiate · 2025-10-12T16:14:02Z

Description:

This PR fixes the following issues with the question syncing process:

Only leetcode DB sync is completed when it returns
The cursor was updated to the nearest 200 questions instead of the last sync point.
Failed sync questions are forgotten during syncing. Tracks and remembers skipped questions during syncing.
Seed cursor only validates sync to leetcode-service instead of question-service.

Changes Made

Health Check for Question Service done before fetching leetcode questions into the leetcode-service.questions DB
The next sync now starts from the last successful sync point.
Database operations are strictly limited to inserts—no updates if the question already exists in the database. (it was previously not explicitly guarded)
Tracks and remembers and attempt to resync skipped questions during syncing.

Copilot

Pull Request Overview

This PR fixes synchronization issues in the LeetCode question seeding process by improving cursor management and database operations. It removes the "done" state tracking in favor of more robust pagination handling and optimizes the database schema for better performance.

Replaces "done" state with total-based cursor management to prevent re-fetching completed data
Changes from $set to $setOnInsert to prevent overwriting existing question data
Optimizes database indexes by adding unique constraint on titleSlug and improving compound indexes

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts	Removes done state tracking, improves cursor pagination logic, and changes upsert strategy to preserve existing data
backend-services/leetcode-backend-service/src/db/model/question.ts	Adds unique index on titleSlug and optimizes compound indexes for better query performance

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts

Copilot · 2025-10-12T16:14:42Z

backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts

+        // setOnInsert to avoid overwriting existing entries
+        $setOnInsert: {


Using $setOnInsert instead of $set means that existing questions will never be updated with new data from LeetCode. This could leave stale data in the database if question details change on LeetCode's side.

Suggested change

// setOnInsert to avoid overwriting existing entries

$setOnInsert: {

// set to update existing entries with latest data

$set: {

This is the intended outcome, not to update the database - to prevent issue where update may overwrite update made manually.

backend-services/leetcode-backend-service/src/db/model/question.ts

Copilot

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts

Copilot · 2025-10-12T16:53:38Z

backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts

        },
      },


The updatedAt field was removed from $setOnInsert but upsert: true still allows updates. This could lead to inconsistent timestamp behavior for existing records. Consider either adding updatedAt back or clarifying the intended behavior for record updates.

Suggested change

},

},

updatedAt: new Date(),

},

$set: {

updatedAt: new Date(),

},

In our case, an additional

$set: { updatedAt: new Date(), },

is not required since we only want to insert new records and don't want to update existing ones. Moreover, since the schema already has { timestamps: true }, MongoDB will take care of setting updatedAt when a new document is inserted.

…atch

Copilot

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-15T16:05:06Z

backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts

  const cursor =
    (await SeedCursor.findById(id)) ??
-    new SeedCursor({ _id: id, nextSkip: 0, pageSize: 200, done: false });
+    new SeedCursor({ _id: id, nextSkip: 0, pageSize: 200 });


The pageSize is hardcoded as a magic number. Consider defining this as a named constant at the module level for better maintainability.

Copilot · 2025-10-15T16:05:06Z

backend-services/leetcode-backend-service/src/leetcode/seedBatch.ts

+  if (nextSkip + pageSize > total) {
+    // Prevent future refetching of previously fetched items
+    cursor.nextSkip = total;
+  } else {
+    cursor.nextSkip = nextSkip + pageSize;
+  }


This cursor advancement logic is duplicated from lines 67. Consider extracting this into a helper function to avoid code duplication.

Copilot · 2025-10-15T16:05:06Z

backend-services/leetcode-backend-service/src/health.ts

+    }
+
+    if (attempt < retries) {
+      await delay(250 * 2 ** attempt);


The retry delay calculation uses magic numbers. Consider defining base delay (250ms) as a named constant for better maintainability.

Copilot AI review requested due to automatic review settings October 12, 2025 16:14

Copilot AI reviewed Oct 12, 2025

View reviewed changes

xGladiate changed the title ~~fix: Fix Question Service bugs and edge cases~~ fix: Fix Question Service seed-batch bugs and edge cases Oct 12, 2025

xGladiate self-assigned this Oct 12, 2025

Copilot AI review requested due to automatic review settings October 12, 2025 16:40

This comment was marked as resolved.

Sign in to view

Copilot AI review requested due to automatic review settings October 12, 2025 16:53

Copilot AI reviewed Oct 12, 2025

View reviewed changes

xGladiate force-pushed the fix-question-edge-cases branch from d7a163c to ceaa880 Compare October 14, 2025 15:26

xGladiate added 10 commits October 16, 2025 00:04

Fix sync when retrieving all questions

b8ada77

Replace insertOne with updateOne but with $setOnInsert only

03f1e61

Remove unnecessary unique constrain enforcement

44996a2

Remove initial_count from return

3d12946

Remove fully initial_count from fetchNonPaidQuestionList

e67cedb

Remove done attribute for cursor across files

57cf2f5

Remove entirely initial_count

334c086

Replace magic number total + 1 with total instead

5fd554c

Fix minor naming issues for comments

7615a18

Add question-service health check before question retrieval at seed-b…

c970774

…atch

Copilot AI review requested due to automatic review settings October 15, 2025 16:04

xGladiate force-pushed the fix-question-edge-cases branch from ceaa880 to c970774 Compare October 15, 2025 16:04

Copilot AI reviewed Oct 15, 2025

View reviewed changes

		// setOnInsert to avoid overwriting existing entries
		$setOnInsert: {

-        },
-      },
+          updatedAt: new Date(),
+        },
+        $set: {
+          updatedAt: new Date(),
+        },

fix: Fix Question Service seed-batch bugs and edge cases #35

Are you sure you want to change the base?

fix: Fix Question Service seed-batch bugs and edge cases #35

Conversation

xGladiate commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description:

Changes Made

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

xGladiate Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

xGladiate Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

xGladiate commented Oct 12, 2025 •

edited

Loading

xGladiate Oct 12, 2025 •

edited

Loading