Commit graph

80 commits

Author SHA1 Message Date
bkellam
338993e8cb fix build 2025-03-21 13:15:41 -07:00
Brendan Kellam
4c52059ecc
Declarative connection configuration (#235) 2025-03-21 12:27:33 -07:00
Brendan Kellam
583df1dd77
Massage environment variables from strings to numbers (#234) 2025-03-20 13:34:50 -07:00
Brendan Kellam
ad60c5f1e0
Centralize to using a single .env.development for development workflows (#231) 2025-03-18 13:18:58 -07:00
bkellam
600cefbbd8 Add missing env var 2025-03-17 21:39:56 -07:00
Brendan Kellam
483217bf56
Switch to using t3-env for env-var management (#230) 2025-03-17 21:22:05 -07:00
bkellam
e8acfcca70 Merge branch 'main' into v3 2025-03-16 15:24:04 -07:00
msukkari
bf76cb9358 cherry pick gitignore additions into main 2025-03-07 15:04:14 -08:00
bkellam
98a662c8b5 (fix) Fixed bug with gitlab and gitea not including hostname in the repoName 2025-03-07 10:55:04 -08:00
msukkari
d439933afc add repo index timeout logic 2025-03-03 10:31:56 -08:00
bkellam
d920d0d167 fix private repo clone issue for gitlab 2025-03-01 22:57:07 -08:00
Brendan Kellam
c48c0930a8
backend logging (#224) 2025-03-01 21:40:47 -08:00
msukkari
b27507eb7d catch and rethrow simplegit exceptions 2025-03-01 20:53:33 -08:00
msukkari
d33185c118 switch source map publish to runtime in entrypoint 2025-03-01 20:43:50 -08:00
Michael Sukkarieh
a93ee6527c
add sentry support to backend and webapp (#223)
* add sentry to web app

* set sentry environemnt from env var

* add sentry env replace logic in docker container

* wip add backend sentry

* add sentry to backend

* move dns to env var

* remove test exception
2025-03-01 19:21:17 -08:00
bkellam
51186fe87d Add prometheus metric for pending repo indexing jobs 2025-03-01 15:50:09 -08:00
msukkari
7ce10672e1 remove error msg dump on failed repo index job, and update indexedAt field 2025-02-27 16:04:31 -08:00
msukkari
d0b75ce57d fix repo rm issue, 502 page, condition on test clock 2025-02-27 13:11:30 -08:00
bkellam
fcf0b836ee fix build 2025-02-27 09:43:35 -08:00
Brendan Kellam
46be0440b0
Add back revision support (#215) 2025-02-27 09:39:55 -08:00
Brendan Kellam
386a3b52d7
remove git & local schemas (#213) 2025-02-26 16:55:10 -08:00
Brendan Kellam
50b94b2c46
Connections UX pass + query optimizations (#212) 2025-02-26 15:46:37 -08:00
msukkari
b77f55fa20 wipe existing repo if we've picked up a killed job to ensure good state 2025-02-26 12:46:24 -08:00
msukkari
8531b14a82 rename some prom metrics and cleanup unused 2025-02-25 18:54:01 -08:00
Michael Sukkarieh
3f586dd927
make repo garbage collection async (#211)
* add gc queue logic

* fix missing switch cases for gc status

* style org create form better with new staging domain

* change repo rm logic to be async

* simplify repo for inactive org query

* add grace period for garbage collecting repos

* make prom scrape interval 500ms
2025-02-25 15:49:09 -08:00
Michael Sukkarieh
fc33f26808
add grafana alloy config and setup (#210)
* add grafana alloy config and setup

* add basic repo prom metrics

* nits in dockerfile
2025-02-24 21:11:28 -08:00
Michael Sukkarieh
de44c81cfa
add posthog events on various user actions (#208)
* add page view event support

* add posthog events

* nit: remove unused import

* feedback
2025-02-24 17:06:29 -08:00
msukkari
8a94949d2d remove repo_synced event 2025-02-20 10:10:28 -08:00
msukkari
0ac9509fab log upsert stats 2025-02-20 09:42:54 -08:00
msukkari
c930fe0d8b revert repo compile update logic to upsert loop 2025-02-19 18:54:50 -08:00
Michael Sukkarieh
fdd71cfcfe
add better visualization for connection/repo errors and warnings (#201)
* replace upsert with seperate create many and raw update many calls

* add bulk repo status update and queue addition with priority

* add support for managed redis

* add note for changing raw sql on schema change

* add error package and use BackendException in connection manager

* handle connection failure display on web app

* add warning banner for not found orgs/repos/users

* add failure handling for gerrit

* add gitea notfound warning support

* add warning icon in connections list

* style nits

* add failed repo vis in connections list

* added retry failed repo index buttons

* move nav indicators to client with polling

* fix indicator flash issue and truncate large list results

* display error nav better

* truncate failed repo list in connection list item

* fix merge error

* fix merge bug

* add connection util file [wip]

* refactor notfound fetch logic and add missing error package to dockerfile

* move repeated logic to function and add zod schema for syncStatusMetadata
2025-02-19 18:10:22 -08:00
msukkari
63af99aa59 remove non secret token options 2025-02-19 14:23:26 -08:00
Michael Sukkarieh
5d7a77bd92
db performance improvements and job resilience (#200)
* replace upsert with seperate create many and raw update many calls

* add bulk repo status update and queue addition with priority

* add support for managed redis

* add note for changing raw sql on schema change
2025-02-19 10:26:13 -08:00
msukkari
e09b21f6b9 add config upsert timeout env var 2025-02-15 17:27:24 -08:00
msukkari
e17331a672 fix build errors and add index concurrency env var 2025-02-15 16:58:05 -08:00
Michael Sukkarieh
54d14ea98e
add retries for 429 issues (#196)
* add connection compile retry and hard repo limit

* add more retry checks

* cleanup unused change

* address feedback
2025-02-15 16:37:50 -08:00
Michael Sukkarieh
e0d363420b
connections qol improvements (#195)
* add client side polling to connections list

* properly fetch repo image url

* add client polling to connection management page, and add ability to sync failed connections
2025-02-15 10:00:44 -08:00
Michael Sukkarieh
3be3680ee2
cleanup org's repos and shards if it's inactive (#194)
* add stripe subscription status and webhook

* add inactive org repo cleanup logic

* mark reactivated org connections for sync
2025-02-15 09:58:17 -08:00
Michael Sukkarieh
da33220289
add back gitlab, gitea, and gerrit support (#184)
* add non github config definitions

* refactor github config compilation to seperate file

* add gitlab config compilation

* Connection management (#183)

* wip gitlab repo sync support

* fix gitlab zoekt metadata

* add gitea support

* add gerrit support

* Connection management (#183)

* add gerrit config compilation

* Connection management (#183)

---------

Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
2025-02-14 10:58:53 -08:00
msukkari
875f9b8586 add await for blocking loop in backend 2025-02-13 10:30:14 -08:00
dependabot[bot]
65b61155bf
Bump vitest from 2.1.4 to 2.1.9 (#186)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 2.1.4 to 2.1.9.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Commits](https://github.com/vitest-dev/vitest/commits/v2.1.9/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-12 13:50:24 -08:00
bkellam
8d2c731b88 Domain support 2025-02-12 13:45:12 -08:00
Brendan Kellam
846d73b0e6
Connection management (#183) 2025-02-04 15:04:05 -05:00
Michael Sukkarieh
dffbbae41d
adds garbage collection for repos (#182)
* refactor repo indexing logic into RepoManager

* wip cleanup stale repos

* add rest of gc logic

* set status to indexing properly
2025-01-29 09:05:20 -08:00
msukkari
f83a4db3d6 add maxTrigramCount setting 2025-01-28 14:44:47 -08:00
Michael Sukkarieh
31114a9d95
add concept of secrets (#180)
* add @sourcebot/schemas package

* migrate things to use the schemas package

* Dockerfile support

* add secret table to schema

* Add concept of connection manager

* Rename Config->Connection

* Handle job failures

* Add join table between repo and connection

* nits

* create first version of crypto package

* add crypto package as deps to others

* forgot to add package changes

* add server action for adding and listing secrets, create test page for it

* add secrets page to nav menu

* add secret to config and support fetching it in backend

* reset secret form on successful submission

* add toast feedback for secrets form

* add instructions for adding encryption key to dev instructions

* add encryption key support in docker file

* add delete secret button

* fix nits from pr review

---------

Co-authored-by: bkellam <bshizzle1234@gmail.com>
2025-01-27 14:07:07 -08:00
Brendan Kellam
dd8ff6edb0
Connection management (#178) 2025-01-24 13:16:08 -08:00
Brendan Kellam
bae7ca3438
Add @sourcebot/schemas package (#177) 2025-01-24 10:51:49 -08:00
msukkari
8a3521b080 updated syncedAt date after config sync: 2025-01-23 09:08:30 -08:00
Michael Sukkarieh
7c6adf17aa
Multi tenancy support in config syncer (#171)
* [wip] initial mt support in config syncer

* Move logout button & profile picture into settings dropdown (#172)

* update sync status properly and fix bug with multiple config in db case

* make config path required in single tenant mode

NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR

---------

Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
2025-01-21 11:50:35 -08:00