fix(backend): prevent duplicate index jobs from indexedAt race

[brendan-kellam]

Fixes SOU-1150

Problem

Large repositories could be indexed twice within a single reindex interval due to a race in RepoIndexManager.onJobCompleted (packages/backend/src/repoIndexManager.ts).

The handler did three things in order:

1. Mark the index job COMPLETED (one DB write, which also flips repo.latestIndexingJobStatus).
2. Run several git reads — isRepoEmpty, getCommitHashForRefName, getLatestCommitTimestamp, getLocalDefaultBranch.
3. Update repo.indexedAt (a separate DB write).

Between steps 1 and 3 there is a window — the duration of the git reads in step 2 — where the job is already COMPLETED but indexedAt is still stale. The scheduler (scheduleIndexJobs) polls every reindexRepoPollingIntervalMs (default 1s) and creates a new index job when both:

• indexedAt IS NULL OR indexedAt < now - reindexIntervalMs (still true — step 3 hasn't run), and
• there is no INDEX job in PENDING/IN_PROGRESS (now true — step 1 already flipped it to COMPLETED).

So a poll landing in that window schedules a duplicate index job. On large repos the git reads take long enough that the 1s poll hits the window routinely; the Redlock doesn't help because onJobCompleted runs as a BullMQ completed event handler, after the lock from processJob has already been released (and the duplicate job row is created regardless of execution serialization).

Fix

Run the git reads first, then write status = COMPLETED and indexedAt (plus indexedCommitHash, pushedAt, metadata, defaultBranch) together in a single repoIndexingJob.update with a nested repo update.

Now the job remains IN_PROGRESS for the entire git-read window and only flips to COMPLETED at the same instant indexedAt becomes fresh. The scheduler's two guards can never both pass simultaneously: before the write, the active-job guard blocks; after it, the fresh indexedAt guards.

Verification

Reproduced the original bug locally by injecting a 5s sleep between the COMPLETED write and the indexedAt update — a single repo cascaded into 4 index jobs within ~11s. After the fix, with the sleep moved to before the combined write (so the job stays IN_PROGRESS), no duplicate is scheduled.

• yarn workspace @sourcebot/backend build passes.
• yarn workspace @sourcebot/backend test --run repoIndexManager — 15/15 pass. The "updates repo.indexedAt" test now asserts status: COMPLETED and indexedAt are written in the same repoIndexingJob.update call, which serves as the regression guard.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• Bug Fixes
  • Fixed a race condition that could cause large repositories to be indexed twice within a single reindex interval.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: abfc2ef9-a580-474b-bc64-a04d791fa5ef

📥 Commits

Reviewing files that changed from the base of the PR and between e668c3f and 6b2b9ea.

📒 Files selected for processing (3)

• CHANGELOG.md
• packages/backend/src/repoIndexManager.test.ts
• packages/backend/src/repoIndexManager.ts

---

Walkthrough

Refactored RepoIndexManager.onJobCompleted to consolidate Prisma updates: job completion and repo metadata changes now execute in a single nested call instead of sequential operations. First-index detection is precomputed before repo updates modify state. Tests and changelog updated accordingly.

Changes

Nested Prisma Update Pattern

Layer / File(s)	Summary
Implementation of nested update pattern and precomputed first-index flag packages/backend/src/repoIndexManager.ts, CHANGELOG.md	onJobCompleted fetches the job record upfront to capture first-index state, consolidates INDEX job completion and repo field updates (indexedAt, indexedCommitHash, pushedAt, metadata, defaultBranch) into a single nested Prisma repoIndexingJob.update call, and uses the precomputed isFirstIndex flag for backend_repo_first_indexed event emission instead of re-checking post-update state. Changelog documents the race condition fix.
Test updates for nested Prisma semantics packages/backend/src/repoIndexManager.test.ts	Across completion, cleanup, and latestIndexingJobStatus tests, mocked repoIndexingJob.findUniqueOrThrow now returns the full job shape (type, repoId, repo, metadata), and assertions expect repoIndexingJob.update to include nested repo.update operations instead of separate mockPrisma.repo.update calls.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• sourcebot-dev/sourcebot#860: Both PRs modify packages/backend/src/repoIndexManager.ts, specifically the onJobCompleted job-completion flow (one during the groupmq→bullmq/redlock migration, the other refactoring how completion updates repoIndexingJob and nested repo metadata).
• sourcebot-dev/sourcebot#900: Both PRs modify RepoIndexManager.onJobCompleted around the "first successful index" flow—this PR refactors completion to precompute isFirstIndex before updating repo.indexedAt, while the retrieved PR changes PostHog event emission to fire backend_repo_first_indexed only on that first index.
• sourcebot-dev/sourcebot#878: Both PRs modify packages/backend/src/repoIndexManager.ts, with the retrieved PR adding PostHog capture logic during repo index job completion/failure and this PR refactoring the onJobCompleted update behavior.

Suggested reviewers

• msukkari

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(backend): prevent duplicate index jobs from indexedAt race' accurately describes the main change - fixing a race condition that caused duplicate indexing jobs by ensuring repo indexing completion updates happen atomically.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch brendan/fix-SOU-1150

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}