sourcebot/packages/backend
Derek Miller 2c0540f6bf
fix(backend): Limit concurrent git operations to prevent resource exhaustion (#590) (#593)
When syncing generic-git-host connections with thousands of repositories,
unbounded Promise.all caused resource exhaustion (EAGAIN errors) by spawning
too many concurrent git processes. This resulted in valid repositories being
incorrectly skipped during sync.

- Add p-limit to control concurrent git operations (max 100)
- Follow existing pattern from github.ts for consistency
- Prevents file descriptor and process limit exhaustion
- Uses rolling concurrency to avoid head-of-line blocking

Fixes #590
2025-11-01 15:15:09 -07:00
..
src fix(backend): Limit concurrent git operations to prevent resource exhaustion (#590) (#593) 2025-11-01 15:15:09 -07:00
.gitignore v3 effort (#158) 2025-03-31 22:34:42 -07:00
package.json add p-limit to GitHub API calls to avoid overwhelming the node process (or the API rate limits) (#591) 2025-10-31 14:49:43 -07:00
tsconfig.json v3 effort (#158) 2025-03-31 22:34:42 -07:00
vitest.config.ts chore(worker,web): Repo indexing stability improvements + perf improvements to web (#563) 2025-10-18 16:31:22 -07:00