v3 effort (#158)
* SQL Database (#157)
* point zoekt to v3 branch
* bump zoekt version
* Add tenant ID concept into web app and backend (#160)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add nocheckin comment to tenant id in v2 schema
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* parallelize repo indexing (#163)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add pol of bullmq into backend
* add better error handling and concurrency setting
* spin up redis instance in dockerfile
* cleanup transaction logic when adding repos to index queue
* add NEW index status fetch condition
* move bullmq deps to backend
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* Authentication (#164)
* Add Org table (#167)
* Move logout button & profile picture into settings dropdown (#172)
* Multi tenancy support in config syncer (#171)
* [wip] initial mt support in config syncer
* Move logout button & profile picture into settings dropdown (#172)
* update sync status properly and fix bug with multiple config in db case
* make config path required in single tenant mode
NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* add tenant mode support in docker container:
* Organization switching & active org management (#173)
* updated syncedAt date after config sync:
* Migrate to postgres (#174)
* spin up postgres in docker container
* get initial pol of postgres db working in docker image
* spin up postgres server in dev case
* updated syncedAt date after config sync:
* remove unnecessary port expose in docker file
* Connection creation form (#175)
* fix issue with yarn dev startup
* init (#176)
* Add `@sourcebot/schemas` package (#177)
* Connection management (#178)
* add concept of secrets (#180)
* add @sourcebot/schemas package
* migrate things to use the schemas package
* Dockerfile support
* add secret table to schema
* Add concept of connection manager
* Rename Config->Connection
* Handle job failures
* Add join table between repo and connection
* nits
* create first version of crypto package
* add crypto package as deps to others
* forgot to add package changes
* add server action for adding and listing secrets, create test page for it
* add secrets page to nav menu
* add secret to config and support fetching it in backend
* reset secret form on successful submission
* add toast feedback for secrets form
* add instructions for adding encryption key to dev instructions
* add encryption key support in docker file
* add delete secret button
* fix nits from pr review
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* enforce tenancy on search and repo listing endpoints (#181)
* enforce tenancy on search and repo listing
* remove orgId from request schemas
* adds garbage collection for repos (#182)
* refactor repo indexing logic into RepoManager
* wip cleanup stale repos
* add rest of gc logic
* set status to indexing properly
* add initial logic for staging environment
* try to move encryption key env decleration in docker file to fix build issues
* switch encryption key as build arg to se if that fixes build issues
* add deployment action for staging image
* try using mac github action runners instead
* switch to using arm64 runners on arm64 build
* change workflow names to fix trigger issue
* trigger staging actions to see if it works
* fix working directory typo and pray it doesnt push to prod
* checkout v3 when deploying staging
* try to change into the staging dir manuall
* dummy commit to trigger v3 workflows to test
* update staging deploy script to match new version in main
* reference proper image:tag in staging fly config
* update staging fly config to point to ghcr
* Connection management (#183)
* add invite system and google oauth provider (#185)
* add settings page with members list
* add invite to schema and basic create form
* add invite table
* add basic invite link copy button
* add auth invite accept case
* add non auth logic
* add google oauth provider
* fix reference to header component in connections
* add google logo to google oauth
* fix web build errors
* bump staging resources
* change staging cpu to perf
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* Domain support (#188)
* Update Makefile to include crypto package when doing a make clean
* Add default for AUTH_URL in attempt to fix build
* attempt 2
* fix attempt #3: Do not require a encrpytion key at build time
* Fix generate script race condition
* Attempt #4
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* Generate AUTH_SECRET if not provided (#189)
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
* Add stripe billing logic (#190)
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* fix nits
* remove providers check
* fix more nits
* change stripe init to be behind function
* fix publishible stripe key handling in docker container
* enforce owner perms (#191)
* add make owner logic, and owner perms for removal, invite, and manage subscription
* add change billing email card to billing settings
* enforce owner role in action level
* remove unused hover card component
* cleanup
* add back gitlab, gitea, and gerrit support (#184)
* add non github config definitions
* refactor github config compilation to seperate file
* add gitlab config compilation
* Connection management (#183)
* wip gitlab repo sync support
* fix gitlab zoekt metadata
* add gitea support
* add gerrit support
* Connection management (#183)
* add gerrit config compilation
* Connection management (#183)
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* fix apos usage in redeem page
* change csrf cookie to secure not host
* Credentials provider (#192)
* email password functionality
* feedback
* cleanup org's repos and shards if it's inactive (#194)
* add stripe subscription status and webhook
* add inactive org repo cleanup logic
* mark reactivated org connections for sync
* connections qol improvements (#195)
* add client side polling to connections list
* properly fetch repo image url
* add client polling to connection management page, and add ability to sync failed connections
* Fix build with suspense boundary
* improved fix
* add retries for 429 issues (#196)
* add connection compile retry and hard repo limit
* add more retry checks
* cleanup unused change
* address feedback
* fix build errors and add index concurrency env var
* add config upsert timeout env var
* Membership settings rework (#198)
* Add refined members list
* futher progress on members settings polish
* Remove old components
* feedback
* Magic links (#199)
* wip on magic link support
* Switch to nodemailer / resend for transactional mail
* Further cleanup
* Add stylized email using react-email
* fix
* Fix build
* db performance improvements and job resilience (#200)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* remove non secret token options
* fix token examples in schema
* add better visualization for connection/repo errors and warnings (#201)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* add error package and use BackendException in connection manager
* handle connection failure display on web app
* add warning banner for not found orgs/repos/users
* add failure handling for gerrit
* add gitea notfound warning support
* add warning icon in connections list
* style nits
* add failed repo vis in connections list
* added retry failed repo index buttons
* move nav indicators to client with polling
* fix indicator flash issue and truncate large list results
* display error nav better
* truncate failed repo list in connection list item
* fix merge error
* fix merge bug
* add connection util file [wip]
* refactor notfound fetch logic and add missing error package to dockerfile
* move repeated logic to function and add zod schema for syncStatusMetadata
* add orgid unique constraint to repo
* revert repo compile update logic to upsert loop
* log upsert stats
* [temp] disable polling everywhere (#205)
* add health check endpoint
* Refined onboarding flow (#202)
* Redeem UX pass (#204)
* add log for health check
* fix new connection complete callback route
* add cpu split logic and only wait for postgres if we're going to connec to it
* Inline secret creation (#207)
* use docker scopes to try and improve caching
* Dummy change
* remove cpu split logic
* Add some instrumentation to web
* add posthog events on various user actions (#208)
* add page view event support
* add posthog events
* nit: remove unused import
* feedback
* fix merge error
* use staging posthog papik when building staging image
* fix other merge error and build warnings
* Add invite email (#209)
* wrap posthog provider in suspense to fix build error
* add grafana alloy config and setup (#210)
* add grafana alloy config and setup
* add basic repo prom metrics
* nits in dockerfile
* remove invalid characters when auto filling domain
* add login posthog events
* remove hard coded sourcebot.app references
* make repo garbage collection async (#211)
* add gc queue logic
* fix missing switch cases for gc status
* style org create form better with new staging domain
* change repo rm logic to be async
* simplify repo for inactive org query
* add grace period for garbage collecting repos
* make prom scrape interval 500ms
* fix typo in trial card
* onboarding tweaks
* rename some prom metrics and cleanup unused
* wipe existing repo if we've picked up a killed job to ensure good state
* Connections UX pass + query optimizations (#212)
* remove git & local schemas (#213)
* skip stripe checkout for trial + fix indexing in progress UI + additional schema validation (#214)
* add additional config validation
* wip bypass stripe checkout for trial
* fix stripe trial checkout bypass
* fix indexing in progress ui on home page
* add subscription checks, more schema validation, and fix issue with complete page
* dont display if no indexed repos
* fix skipping onboard complete check
* fix build error
* add back button in onboard connection creation flow
* Add back revision support (#215)
* fix build
* Fix bug with repository snapshot
* fix share links
* fix repo rm issue, 502 page, condition on test clock
* Make login and onboarding mobile friendly
* fix ordering of quick actions
* remove error msg dump on failed repo index job, and update indexedAt field
* Add mobile unsupported splash screne
* cherry pick fix for file links
* [Cherry Pick] Syntax reference guide (#169) (#216)
* Add .env to db gitignore
* fix case where we have repos but they're all failed for repo snapshot
* /settings/secrets page (#217)
* display domain properly in org create form
* Quick action tweaks (#218)
* revamp repo page (#220)
* wip repo table
* new repo page
* add indicator for when feedback is applied in repo page
* add repo button
* fetch connection data in one query
* fix styling
* fix (#219)
* remove / keyboard shortcut hint in search bar
* prevent switching to first page on data update and truncate long repo names in repo list
* General settings + cleanup (#221)
* General settings
* Add alert to org domain change
* First attempt at sending logs to grafana
* logs wip
* add alloy logs
* wip
* [temp] comment out loki for now
* update trial card content and add events for code host selection on onboard
* reduce scraping interval to 15s
* Add prometheus metric for pending repo indexing jobs
* switch magic link to invite code (#222)
* wip magic link codes
* pipe email to email provider properly
* remove magic link data cookie after sign in
* clean up unused imports
* dont remove cookie before we use it
* rm package-lock.json
* revert yarn files to v3 state
* switch email passing from cookie to search param
* add comment for settings dropdown auth update
* remove unused middleware file
* fix build error and warnings
* fix build error with useSearchParam not wrapped in suspense
* add sentry support to backend and webapp (#223)
* add sentry to web app
* set sentry environemnt from env var
* add sentry env replace logic in docker container
* wip add backend sentry
* add sentry to backend
* move dns to env var
* remove test exception
* Fix root domain issue on onboarding
* add setup sentry cli step to github action
* login to sentry
* fix sentry login in action
* Update grafana loki endpoint
* switch source map publish to runtime in entrypoint
* catch and rethrow simplegit exceptions
* alloy nits
* fix alloy
* backend logging (#224)
* revert grafana loki config
* fix login ui nits
* fix quick actions
* fix typo in secret creation
* fix private repo clone issue for gitlab
* add repo index timeout logic
* add posthog identify call after registeration
* various changes to add terms and security info (#225)
* add terms and security to footer
* add security card
* add demo card
* fix build error
* nit fix: center 'get in touch' on security card
* Dark theme improvements (#226)
* (fix) Fixed bug with gitlab and gitea not including hostname in the repoName
* Switch to using t3-env for env-var management (#230)
* Add missing env var
* fix build
* Centralize to using a single .env.development for development workflows (#231)
* Make billing optional (#232)
* Massage environment variables from strings to numbers (#234)
* Single tenancy & auth modes (#233)
* Add docs to this repo
* dummy change
* Declarative connection configuration (#235)
* fix build
* upgrade to next 14.2.25
* Improved database DX
* migrate to yarn v4
* Use origin from header for baseUrl of emails (instead of AUTH_URL). Also removed reference to hide scrollbars
* Remove SOURCEBOT_ENCRYPTION_KEY from build arg
* Fix issue with linking default user to org in single tenant + no-auth mode
* Fix fallback tokens (#242)
* add SECURITY_CARD_ENABLED flag
* Add repository weburl (#243)
* Random fixes and improvements (#244)
* add zoekt max wall time env var
* remove empty warning in docs
* fix reference in sh docs
* add connection manager upsert timeout env var
* Declarative connection cleanup + improvements (#245)
* change contact us footer in app to point to main contact form
* PostHog event pass (#246)
* fix typo
* Add sourcebot cloud environment prop to staging workflow
* Update generated files
* remove AUTH_URL since it unused and (likely) unnecessary
* Revert "remove AUTH_URL since it unused and (likely) unnecessary"
This reverts commit 1f4a5aed22fa94bace899262e8576427fc852f61.
* cleanup GitHub action releases (#252)
* remove alloy, change auth defaul to disabled, add settings page in me dropdown
* enforce connection management perms to owner (#253)
* enforce conneciton management perms to owner
* fix formatting
* more formatting
* naming nits
* fix var name error
* change empty repo set copy if auth is disabled
* add CONTRIBUTING.md file
* hide settings in dropdown with auth isnt enabled
* handle case where gerrit weburl is just gitiles path
* Docs overhall (#251)
* remove nocheckin
* fix build error
* remove v3 trigger from deploy staging
* fix build errors round 2
* another error fix
---------
Co-authored-by: msukkari <michael.sukkarieh@mail.mcgill.ca>
2025-04-01 05:34:42 +00:00
|
|
|
import { checkIfOrgDomainExists } from "@/actions";
|
|
|
|
|
import { RepoIndexingStatus } from "@sourcebot/db";
|
2024-09-10 06:16:41 +00:00
|
|
|
import { z } from "zod";
|
v3 effort (#158)
* SQL Database (#157)
* point zoekt to v3 branch
* bump zoekt version
* Add tenant ID concept into web app and backend (#160)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add nocheckin comment to tenant id in v2 schema
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* parallelize repo indexing (#163)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add pol of bullmq into backend
* add better error handling and concurrency setting
* spin up redis instance in dockerfile
* cleanup transaction logic when adding repos to index queue
* add NEW index status fetch condition
* move bullmq deps to backend
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* Authentication (#164)
* Add Org table (#167)
* Move logout button & profile picture into settings dropdown (#172)
* Multi tenancy support in config syncer (#171)
* [wip] initial mt support in config syncer
* Move logout button & profile picture into settings dropdown (#172)
* update sync status properly and fix bug with multiple config in db case
* make config path required in single tenant mode
NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* add tenant mode support in docker container:
* Organization switching & active org management (#173)
* updated syncedAt date after config sync:
* Migrate to postgres (#174)
* spin up postgres in docker container
* get initial pol of postgres db working in docker image
* spin up postgres server in dev case
* updated syncedAt date after config sync:
* remove unnecessary port expose in docker file
* Connection creation form (#175)
* fix issue with yarn dev startup
* init (#176)
* Add `@sourcebot/schemas` package (#177)
* Connection management (#178)
* add concept of secrets (#180)
* add @sourcebot/schemas package
* migrate things to use the schemas package
* Dockerfile support
* add secret table to schema
* Add concept of connection manager
* Rename Config->Connection
* Handle job failures
* Add join table between repo and connection
* nits
* create first version of crypto package
* add crypto package as deps to others
* forgot to add package changes
* add server action for adding and listing secrets, create test page for it
* add secrets page to nav menu
* add secret to config and support fetching it in backend
* reset secret form on successful submission
* add toast feedback for secrets form
* add instructions for adding encryption key to dev instructions
* add encryption key support in docker file
* add delete secret button
* fix nits from pr review
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* enforce tenancy on search and repo listing endpoints (#181)
* enforce tenancy on search and repo listing
* remove orgId from request schemas
* adds garbage collection for repos (#182)
* refactor repo indexing logic into RepoManager
* wip cleanup stale repos
* add rest of gc logic
* set status to indexing properly
* add initial logic for staging environment
* try to move encryption key env decleration in docker file to fix build issues
* switch encryption key as build arg to se if that fixes build issues
* add deployment action for staging image
* try using mac github action runners instead
* switch to using arm64 runners on arm64 build
* change workflow names to fix trigger issue
* trigger staging actions to see if it works
* fix working directory typo and pray it doesnt push to prod
* checkout v3 when deploying staging
* try to change into the staging dir manuall
* dummy commit to trigger v3 workflows to test
* update staging deploy script to match new version in main
* reference proper image:tag in staging fly config
* update staging fly config to point to ghcr
* Connection management (#183)
* add invite system and google oauth provider (#185)
* add settings page with members list
* add invite to schema and basic create form
* add invite table
* add basic invite link copy button
* add auth invite accept case
* add non auth logic
* add google oauth provider
* fix reference to header component in connections
* add google logo to google oauth
* fix web build errors
* bump staging resources
* change staging cpu to perf
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* Domain support (#188)
* Update Makefile to include crypto package when doing a make clean
* Add default for AUTH_URL in attempt to fix build
* attempt 2
* fix attempt #3: Do not require a encrpytion key at build time
* Fix generate script race condition
* Attempt #4
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* Generate AUTH_SECRET if not provided (#189)
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
* Add stripe billing logic (#190)
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* fix nits
* remove providers check
* fix more nits
* change stripe init to be behind function
* fix publishible stripe key handling in docker container
* enforce owner perms (#191)
* add make owner logic, and owner perms for removal, invite, and manage subscription
* add change billing email card to billing settings
* enforce owner role in action level
* remove unused hover card component
* cleanup
* add back gitlab, gitea, and gerrit support (#184)
* add non github config definitions
* refactor github config compilation to seperate file
* add gitlab config compilation
* Connection management (#183)
* wip gitlab repo sync support
* fix gitlab zoekt metadata
* add gitea support
* add gerrit support
* Connection management (#183)
* add gerrit config compilation
* Connection management (#183)
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* fix apos usage in redeem page
* change csrf cookie to secure not host
* Credentials provider (#192)
* email password functionality
* feedback
* cleanup org's repos and shards if it's inactive (#194)
* add stripe subscription status and webhook
* add inactive org repo cleanup logic
* mark reactivated org connections for sync
* connections qol improvements (#195)
* add client side polling to connections list
* properly fetch repo image url
* add client polling to connection management page, and add ability to sync failed connections
* Fix build with suspense boundary
* improved fix
* add retries for 429 issues (#196)
* add connection compile retry and hard repo limit
* add more retry checks
* cleanup unused change
* address feedback
* fix build errors and add index concurrency env var
* add config upsert timeout env var
* Membership settings rework (#198)
* Add refined members list
* futher progress on members settings polish
* Remove old components
* feedback
* Magic links (#199)
* wip on magic link support
* Switch to nodemailer / resend for transactional mail
* Further cleanup
* Add stylized email using react-email
* fix
* Fix build
* db performance improvements and job resilience (#200)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* remove non secret token options
* fix token examples in schema
* add better visualization for connection/repo errors and warnings (#201)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* add error package and use BackendException in connection manager
* handle connection failure display on web app
* add warning banner for not found orgs/repos/users
* add failure handling for gerrit
* add gitea notfound warning support
* add warning icon in connections list
* style nits
* add failed repo vis in connections list
* added retry failed repo index buttons
* move nav indicators to client with polling
* fix indicator flash issue and truncate large list results
* display error nav better
* truncate failed repo list in connection list item
* fix merge error
* fix merge bug
* add connection util file [wip]
* refactor notfound fetch logic and add missing error package to dockerfile
* move repeated logic to function and add zod schema for syncStatusMetadata
* add orgid unique constraint to repo
* revert repo compile update logic to upsert loop
* log upsert stats
* [temp] disable polling everywhere (#205)
* add health check endpoint
* Refined onboarding flow (#202)
* Redeem UX pass (#204)
* add log for health check
* fix new connection complete callback route
* add cpu split logic and only wait for postgres if we're going to connec to it
* Inline secret creation (#207)
* use docker scopes to try and improve caching
* Dummy change
* remove cpu split logic
* Add some instrumentation to web
* add posthog events on various user actions (#208)
* add page view event support
* add posthog events
* nit: remove unused import
* feedback
* fix merge error
* use staging posthog papik when building staging image
* fix other merge error and build warnings
* Add invite email (#209)
* wrap posthog provider in suspense to fix build error
* add grafana alloy config and setup (#210)
* add grafana alloy config and setup
* add basic repo prom metrics
* nits in dockerfile
* remove invalid characters when auto filling domain
* add login posthog events
* remove hard coded sourcebot.app references
* make repo garbage collection async (#211)
* add gc queue logic
* fix missing switch cases for gc status
* style org create form better with new staging domain
* change repo rm logic to be async
* simplify repo for inactive org query
* add grace period for garbage collecting repos
* make prom scrape interval 500ms
* fix typo in trial card
* onboarding tweaks
* rename some prom metrics and cleanup unused
* wipe existing repo if we've picked up a killed job to ensure good state
* Connections UX pass + query optimizations (#212)
* remove git & local schemas (#213)
* skip stripe checkout for trial + fix indexing in progress UI + additional schema validation (#214)
* add additional config validation
* wip bypass stripe checkout for trial
* fix stripe trial checkout bypass
* fix indexing in progress ui on home page
* add subscription checks, more schema validation, and fix issue with complete page
* dont display if no indexed repos
* fix skipping onboard complete check
* fix build error
* add back button in onboard connection creation flow
* Add back revision support (#215)
* fix build
* Fix bug with repository snapshot
* fix share links
* fix repo rm issue, 502 page, condition on test clock
* Make login and onboarding mobile friendly
* fix ordering of quick actions
* remove error msg dump on failed repo index job, and update indexedAt field
* Add mobile unsupported splash screne
* cherry pick fix for file links
* [Cherry Pick] Syntax reference guide (#169) (#216)
* Add .env to db gitignore
* fix case where we have repos but they're all failed for repo snapshot
* /settings/secrets page (#217)
* display domain properly in org create form
* Quick action tweaks (#218)
* revamp repo page (#220)
* wip repo table
* new repo page
* add indicator for when feedback is applied in repo page
* add repo button
* fetch connection data in one query
* fix styling
* fix (#219)
* remove / keyboard shortcut hint in search bar
* prevent switching to first page on data update and truncate long repo names in repo list
* General settings + cleanup (#221)
* General settings
* Add alert to org domain change
* First attempt at sending logs to grafana
* logs wip
* add alloy logs
* wip
* [temp] comment out loki for now
* update trial card content and add events for code host selection on onboard
* reduce scraping interval to 15s
* Add prometheus metric for pending repo indexing jobs
* switch magic link to invite code (#222)
* wip magic link codes
* pipe email to email provider properly
* remove magic link data cookie after sign in
* clean up unused imports
* dont remove cookie before we use it
* rm package-lock.json
* revert yarn files to v3 state
* switch email passing from cookie to search param
* add comment for settings dropdown auth update
* remove unused middleware file
* fix build error and warnings
* fix build error with useSearchParam not wrapped in suspense
* add sentry support to backend and webapp (#223)
* add sentry to web app
* set sentry environemnt from env var
* add sentry env replace logic in docker container
* wip add backend sentry
* add sentry to backend
* move dns to env var
* remove test exception
* Fix root domain issue on onboarding
* add setup sentry cli step to github action
* login to sentry
* fix sentry login in action
* Update grafana loki endpoint
* switch source map publish to runtime in entrypoint
* catch and rethrow simplegit exceptions
* alloy nits
* fix alloy
* backend logging (#224)
* revert grafana loki config
* fix login ui nits
* fix quick actions
* fix typo in secret creation
* fix private repo clone issue for gitlab
* add repo index timeout logic
* add posthog identify call after registeration
* various changes to add terms and security info (#225)
* add terms and security to footer
* add security card
* add demo card
* fix build error
* nit fix: center 'get in touch' on security card
* Dark theme improvements (#226)
* (fix) Fixed bug with gitlab and gitea not including hostname in the repoName
* Switch to using t3-env for env-var management (#230)
* Add missing env var
* fix build
* Centralize to using a single .env.development for development workflows (#231)
* Make billing optional (#232)
* Massage environment variables from strings to numbers (#234)
* Single tenancy & auth modes (#233)
* Add docs to this repo
* dummy change
* Declarative connection configuration (#235)
* fix build
* upgrade to next 14.2.25
* Improved database DX
* migrate to yarn v4
* Use origin from header for baseUrl of emails (instead of AUTH_URL). Also removed reference to hide scrollbars
* Remove SOURCEBOT_ENCRYPTION_KEY from build arg
* Fix issue with linking default user to org in single tenant + no-auth mode
* Fix fallback tokens (#242)
* add SECURITY_CARD_ENABLED flag
* Add repository weburl (#243)
* Random fixes and improvements (#244)
* add zoekt max wall time env var
* remove empty warning in docs
* fix reference in sh docs
* add connection manager upsert timeout env var
* Declarative connection cleanup + improvements (#245)
* change contact us footer in app to point to main contact form
* PostHog event pass (#246)
* fix typo
* Add sourcebot cloud environment prop to staging workflow
* Update generated files
* remove AUTH_URL since it unused and (likely) unnecessary
* Revert "remove AUTH_URL since it unused and (likely) unnecessary"
This reverts commit 1f4a5aed22fa94bace899262e8576427fc852f61.
* cleanup GitHub action releases (#252)
* remove alloy, change auth defaul to disabled, add settings page in me dropdown
* enforce connection management perms to owner (#253)
* enforce conneciton management perms to owner
* fix formatting
* more formatting
* naming nits
* fix var name error
* change empty repo set copy if auth is disabled
* add CONTRIBUTING.md file
* hide settings in dropdown with auth isnt enabled
* handle case where gerrit weburl is just gitiles path
* Docs overhall (#251)
* remove nocheckin
* fix build error
* remove v3 trigger from deploy staging
* fix build errors round 2
* another error fix
---------
Co-authored-by: msukkari <michael.sukkarieh@mail.mcgill.ca>
2025-04-01 05:34:42 +00:00
|
|
|
import { isServiceError } from "./utils";
|
2024-09-10 06:16:41 +00:00
|
|
|
export const searchRequestSchema = z.object({
|
|
|
|
|
query: z.string(),
|
2024-09-26 06:31:51 +00:00
|
|
|
maxMatchDisplayCount: z.number(),
|
2024-09-10 19:24:12 +00:00
|
|
|
whole: z.boolean().optional(),
|
2024-09-10 06:16:41 +00:00
|
|
|
});
|
|
|
|
|
|
|
|
|
|
|
2024-09-29 21:39:17 +00:00
|
|
|
// @see : https://github.com/sourcebot-dev/zoekt/blob/main/api.go#L212
|
2024-09-26 03:12:20 +00:00
|
|
|
export const locationSchema = z.object({
|
2024-09-11 04:55:00 +00:00
|
|
|
// 0-based byte offset from the beginning of the file
|
2024-09-10 06:16:41 +00:00
|
|
|
ByteOffset: z.number(),
|
2024-09-11 04:55:00 +00:00
|
|
|
// 1-based line number from the beginning of the file
|
2024-09-10 06:16:41 +00:00
|
|
|
LineNumber: z.number(),
|
2024-09-11 04:55:00 +00:00
|
|
|
// 1-based column number (in runes) from the beginning of line
|
2024-09-10 06:16:41 +00:00
|
|
|
Column: z.number(),
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-26 03:12:20 +00:00
|
|
|
export const rangeSchema = z.object({
|
2024-09-10 06:16:41 +00:00
|
|
|
Start: locationSchema,
|
|
|
|
|
End: locationSchema,
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-29 21:39:17 +00:00
|
|
|
// @see : https://github.com/sourcebot-dev/zoekt/blob/3780e68cdb537d5a7ed2c84d9b3784f80c7c5d04/api.go#L350
|
2024-09-17 04:37:34 +00:00
|
|
|
export const searchResponseStats = {
|
|
|
|
|
ContentBytesLoaded: z.number(),
|
|
|
|
|
IndexBytesLoaded: z.number(),
|
|
|
|
|
Crashes: z.number(),
|
|
|
|
|
Duration: z.number(),
|
|
|
|
|
FileCount: z.number(),
|
|
|
|
|
ShardFilesConsidered: z.number(),
|
|
|
|
|
FilesConsidered: z.number(),
|
|
|
|
|
FilesLoaded: z.number(),
|
|
|
|
|
FilesSkipped: z.number(),
|
|
|
|
|
ShardsScanned: z.number(),
|
|
|
|
|
ShardsSkipped: z.number(),
|
|
|
|
|
ShardsSkippedFilter: z.number(),
|
|
|
|
|
MatchCount: z.number(),
|
|
|
|
|
NgramMatches: z.number(),
|
|
|
|
|
NgramLookups: z.number(),
|
|
|
|
|
Wait: z.number(),
|
|
|
|
|
MatchTreeConstruction: z.number(),
|
|
|
|
|
MatchTreeSearch: z.number(),
|
|
|
|
|
RegexpsConsidered: z.number(),
|
|
|
|
|
FlushReason: z.number(),
|
|
|
|
|
}
|
|
|
|
|
|
2024-11-28 21:26:27 +00:00
|
|
|
export const symbolSchema = z.object({
|
|
|
|
|
Sym: z.string(),
|
|
|
|
|
Kind: z.string(),
|
|
|
|
|
Parent: z.string(),
|
|
|
|
|
ParentKind: z.string(),
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-29 21:39:17 +00:00
|
|
|
// @see : https://github.com/sourcebot-dev/zoekt/blob/3780e68cdb537d5a7ed2c84d9b3784f80c7c5d04/api.go#L497
|
2024-11-07 02:28:10 +00:00
|
|
|
export const zoektSearchResponseSchema = z.object({
|
2024-09-10 06:16:41 +00:00
|
|
|
Result: z.object({
|
2024-09-17 04:37:34 +00:00
|
|
|
...searchResponseStats,
|
2024-09-10 06:16:41 +00:00
|
|
|
Files: z.array(z.object({
|
|
|
|
|
FileName: z.string(),
|
|
|
|
|
Repository: z.string(),
|
2024-11-01 17:51:14 +00:00
|
|
|
Version: z.string().optional(),
|
2024-09-10 06:16:41 +00:00
|
|
|
Language: z.string(),
|
2024-11-01 17:51:14 +00:00
|
|
|
Branches: z.array(z.string()).optional(),
|
2024-09-10 06:16:41 +00:00
|
|
|
ChunkMatches: z.array(z.object({
|
|
|
|
|
Content: z.string(),
|
|
|
|
|
Ranges: z.array(rangeSchema),
|
|
|
|
|
FileName: z.boolean(),
|
|
|
|
|
ContentStart: locationSchema,
|
|
|
|
|
Score: z.number(),
|
2024-11-28 21:26:27 +00:00
|
|
|
SymbolInfo: z.array(symbolSchema).nullable(),
|
2024-09-10 06:16:41 +00:00
|
|
|
})),
|
|
|
|
|
Checksum: z.string(),
|
|
|
|
|
Score: z.number(),
|
|
|
|
|
// Set if `whole` is true.
|
2024-09-10 19:24:12 +00:00
|
|
|
Content: z.string().optional(),
|
|
|
|
|
})).nullable(),
|
2024-11-27 05:49:41 +00:00
|
|
|
RepoURLs: z.record(z.string(), z.string()),
|
2024-09-10 06:16:41 +00:00
|
|
|
}),
|
|
|
|
|
});
|
|
|
|
|
|
2024-11-07 02:28:10 +00:00
|
|
|
export const searchResponseSchema = z.object({
|
|
|
|
|
...zoektSearchResponseSchema.shape,
|
|
|
|
|
// Flag when a branch filter was used (e.g., `branch:`, `revision:`, etc.).
|
|
|
|
|
isBranchFilteringEnabled: z.boolean(),
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-10 06:16:41 +00:00
|
|
|
export const fileSourceRequestSchema = z.object({
|
|
|
|
|
fileName: z.string(),
|
2024-11-07 02:28:10 +00:00
|
|
|
repository: z.string(),
|
|
|
|
|
branch: z.string().optional(),
|
2024-09-10 06:16:41 +00:00
|
|
|
});
|
|
|
|
|
|
|
|
|
|
export const fileSourceResponseSchema = z.object({
|
|
|
|
|
source: z.string(),
|
2025-01-07 18:27:42 +00:00
|
|
|
language: z.string(),
|
2024-09-11 04:55:00 +00:00
|
|
|
});
|
|
|
|
|
|
v3 effort (#158)
* SQL Database (#157)
* point zoekt to v3 branch
* bump zoekt version
* Add tenant ID concept into web app and backend (#160)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add nocheckin comment to tenant id in v2 schema
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* parallelize repo indexing (#163)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add pol of bullmq into backend
* add better error handling and concurrency setting
* spin up redis instance in dockerfile
* cleanup transaction logic when adding repos to index queue
* add NEW index status fetch condition
* move bullmq deps to backend
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* Authentication (#164)
* Add Org table (#167)
* Move logout button & profile picture into settings dropdown (#172)
* Multi tenancy support in config syncer (#171)
* [wip] initial mt support in config syncer
* Move logout button & profile picture into settings dropdown (#172)
* update sync status properly and fix bug with multiple config in db case
* make config path required in single tenant mode
NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* add tenant mode support in docker container:
* Organization switching & active org management (#173)
* updated syncedAt date after config sync:
* Migrate to postgres (#174)
* spin up postgres in docker container
* get initial pol of postgres db working in docker image
* spin up postgres server in dev case
* updated syncedAt date after config sync:
* remove unnecessary port expose in docker file
* Connection creation form (#175)
* fix issue with yarn dev startup
* init (#176)
* Add `@sourcebot/schemas` package (#177)
* Connection management (#178)
* add concept of secrets (#180)
* add @sourcebot/schemas package
* migrate things to use the schemas package
* Dockerfile support
* add secret table to schema
* Add concept of connection manager
* Rename Config->Connection
* Handle job failures
* Add join table between repo and connection
* nits
* create first version of crypto package
* add crypto package as deps to others
* forgot to add package changes
* add server action for adding and listing secrets, create test page for it
* add secrets page to nav menu
* add secret to config and support fetching it in backend
* reset secret form on successful submission
* add toast feedback for secrets form
* add instructions for adding encryption key to dev instructions
* add encryption key support in docker file
* add delete secret button
* fix nits from pr review
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* enforce tenancy on search and repo listing endpoints (#181)
* enforce tenancy on search and repo listing
* remove orgId from request schemas
* adds garbage collection for repos (#182)
* refactor repo indexing logic into RepoManager
* wip cleanup stale repos
* add rest of gc logic
* set status to indexing properly
* add initial logic for staging environment
* try to move encryption key env decleration in docker file to fix build issues
* switch encryption key as build arg to se if that fixes build issues
* add deployment action for staging image
* try using mac github action runners instead
* switch to using arm64 runners on arm64 build
* change workflow names to fix trigger issue
* trigger staging actions to see if it works
* fix working directory typo and pray it doesnt push to prod
* checkout v3 when deploying staging
* try to change into the staging dir manuall
* dummy commit to trigger v3 workflows to test
* update staging deploy script to match new version in main
* reference proper image:tag in staging fly config
* update staging fly config to point to ghcr
* Connection management (#183)
* add invite system and google oauth provider (#185)
* add settings page with members list
* add invite to schema and basic create form
* add invite table
* add basic invite link copy button
* add auth invite accept case
* add non auth logic
* add google oauth provider
* fix reference to header component in connections
* add google logo to google oauth
* fix web build errors
* bump staging resources
* change staging cpu to perf
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* Domain support (#188)
* Update Makefile to include crypto package when doing a make clean
* Add default for AUTH_URL in attempt to fix build
* attempt 2
* fix attempt #3: Do not require a encrpytion key at build time
* Fix generate script race condition
* Attempt #4
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* Generate AUTH_SECRET if not provided (#189)
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
* Add stripe billing logic (#190)
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* fix nits
* remove providers check
* fix more nits
* change stripe init to be behind function
* fix publishible stripe key handling in docker container
* enforce owner perms (#191)
* add make owner logic, and owner perms for removal, invite, and manage subscription
* add change billing email card to billing settings
* enforce owner role in action level
* remove unused hover card component
* cleanup
* add back gitlab, gitea, and gerrit support (#184)
* add non github config definitions
* refactor github config compilation to seperate file
* add gitlab config compilation
* Connection management (#183)
* wip gitlab repo sync support
* fix gitlab zoekt metadata
* add gitea support
* add gerrit support
* Connection management (#183)
* add gerrit config compilation
* Connection management (#183)
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* fix apos usage in redeem page
* change csrf cookie to secure not host
* Credentials provider (#192)
* email password functionality
* feedback
* cleanup org's repos and shards if it's inactive (#194)
* add stripe subscription status and webhook
* add inactive org repo cleanup logic
* mark reactivated org connections for sync
* connections qol improvements (#195)
* add client side polling to connections list
* properly fetch repo image url
* add client polling to connection management page, and add ability to sync failed connections
* Fix build with suspense boundary
* improved fix
* add retries for 429 issues (#196)
* add connection compile retry and hard repo limit
* add more retry checks
* cleanup unused change
* address feedback
* fix build errors and add index concurrency env var
* add config upsert timeout env var
* Membership settings rework (#198)
* Add refined members list
* futher progress on members settings polish
* Remove old components
* feedback
* Magic links (#199)
* wip on magic link support
* Switch to nodemailer / resend for transactional mail
* Further cleanup
* Add stylized email using react-email
* fix
* Fix build
* db performance improvements and job resilience (#200)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* remove non secret token options
* fix token examples in schema
* add better visualization for connection/repo errors and warnings (#201)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* add error package and use BackendException in connection manager
* handle connection failure display on web app
* add warning banner for not found orgs/repos/users
* add failure handling for gerrit
* add gitea notfound warning support
* add warning icon in connections list
* style nits
* add failed repo vis in connections list
* added retry failed repo index buttons
* move nav indicators to client with polling
* fix indicator flash issue and truncate large list results
* display error nav better
* truncate failed repo list in connection list item
* fix merge error
* fix merge bug
* add connection util file [wip]
* refactor notfound fetch logic and add missing error package to dockerfile
* move repeated logic to function and add zod schema for syncStatusMetadata
* add orgid unique constraint to repo
* revert repo compile update logic to upsert loop
* log upsert stats
* [temp] disable polling everywhere (#205)
* add health check endpoint
* Refined onboarding flow (#202)
* Redeem UX pass (#204)
* add log for health check
* fix new connection complete callback route
* add cpu split logic and only wait for postgres if we're going to connec to it
* Inline secret creation (#207)
* use docker scopes to try and improve caching
* Dummy change
* remove cpu split logic
* Add some instrumentation to web
* add posthog events on various user actions (#208)
* add page view event support
* add posthog events
* nit: remove unused import
* feedback
* fix merge error
* use staging posthog papik when building staging image
* fix other merge error and build warnings
* Add invite email (#209)
* wrap posthog provider in suspense to fix build error
* add grafana alloy config and setup (#210)
* add grafana alloy config and setup
* add basic repo prom metrics
* nits in dockerfile
* remove invalid characters when auto filling domain
* add login posthog events
* remove hard coded sourcebot.app references
* make repo garbage collection async (#211)
* add gc queue logic
* fix missing switch cases for gc status
* style org create form better with new staging domain
* change repo rm logic to be async
* simplify repo for inactive org query
* add grace period for garbage collecting repos
* make prom scrape interval 500ms
* fix typo in trial card
* onboarding tweaks
* rename some prom metrics and cleanup unused
* wipe existing repo if we've picked up a killed job to ensure good state
* Connections UX pass + query optimizations (#212)
* remove git & local schemas (#213)
* skip stripe checkout for trial + fix indexing in progress UI + additional schema validation (#214)
* add additional config validation
* wip bypass stripe checkout for trial
* fix stripe trial checkout bypass
* fix indexing in progress ui on home page
* add subscription checks, more schema validation, and fix issue with complete page
* dont display if no indexed repos
* fix skipping onboard complete check
* fix build error
* add back button in onboard connection creation flow
* Add back revision support (#215)
* fix build
* Fix bug with repository snapshot
* fix share links
* fix repo rm issue, 502 page, condition on test clock
* Make login and onboarding mobile friendly
* fix ordering of quick actions
* remove error msg dump on failed repo index job, and update indexedAt field
* Add mobile unsupported splash screne
* cherry pick fix for file links
* [Cherry Pick] Syntax reference guide (#169) (#216)
* Add .env to db gitignore
* fix case where we have repos but they're all failed for repo snapshot
* /settings/secrets page (#217)
* display domain properly in org create form
* Quick action tweaks (#218)
* revamp repo page (#220)
* wip repo table
* new repo page
* add indicator for when feedback is applied in repo page
* add repo button
* fetch connection data in one query
* fix styling
* fix (#219)
* remove / keyboard shortcut hint in search bar
* prevent switching to first page on data update and truncate long repo names in repo list
* General settings + cleanup (#221)
* General settings
* Add alert to org domain change
* First attempt at sending logs to grafana
* logs wip
* add alloy logs
* wip
* [temp] comment out loki for now
* update trial card content and add events for code host selection on onboard
* reduce scraping interval to 15s
* Add prometheus metric for pending repo indexing jobs
* switch magic link to invite code (#222)
* wip magic link codes
* pipe email to email provider properly
* remove magic link data cookie after sign in
* clean up unused imports
* dont remove cookie before we use it
* rm package-lock.json
* revert yarn files to v3 state
* switch email passing from cookie to search param
* add comment for settings dropdown auth update
* remove unused middleware file
* fix build error and warnings
* fix build error with useSearchParam not wrapped in suspense
* add sentry support to backend and webapp (#223)
* add sentry to web app
* set sentry environemnt from env var
* add sentry env replace logic in docker container
* wip add backend sentry
* add sentry to backend
* move dns to env var
* remove test exception
* Fix root domain issue on onboarding
* add setup sentry cli step to github action
* login to sentry
* fix sentry login in action
* Update grafana loki endpoint
* switch source map publish to runtime in entrypoint
* catch and rethrow simplegit exceptions
* alloy nits
* fix alloy
* backend logging (#224)
* revert grafana loki config
* fix login ui nits
* fix quick actions
* fix typo in secret creation
* fix private repo clone issue for gitlab
* add repo index timeout logic
* add posthog identify call after registeration
* various changes to add terms and security info (#225)
* add terms and security to footer
* add security card
* add demo card
* fix build error
* nit fix: center 'get in touch' on security card
* Dark theme improvements (#226)
* (fix) Fixed bug with gitlab and gitea not including hostname in the repoName
* Switch to using t3-env for env-var management (#230)
* Add missing env var
* fix build
* Centralize to using a single .env.development for development workflows (#231)
* Make billing optional (#232)
* Massage environment variables from strings to numbers (#234)
* Single tenancy & auth modes (#233)
* Add docs to this repo
* dummy change
* Declarative connection configuration (#235)
* fix build
* upgrade to next 14.2.25
* Improved database DX
* migrate to yarn v4
* Use origin from header for baseUrl of emails (instead of AUTH_URL). Also removed reference to hide scrollbars
* Remove SOURCEBOT_ENCRYPTION_KEY from build arg
* Fix issue with linking default user to org in single tenant + no-auth mode
* Fix fallback tokens (#242)
* add SECURITY_CARD_ENABLED flag
* Add repository weburl (#243)
* Random fixes and improvements (#244)
* add zoekt max wall time env var
* remove empty warning in docs
* fix reference in sh docs
* add connection manager upsert timeout env var
* Declarative connection cleanup + improvements (#245)
* change contact us footer in app to point to main contact form
* PostHog event pass (#246)
* fix typo
* Add sourcebot cloud environment prop to staging workflow
* Update generated files
* remove AUTH_URL since it unused and (likely) unnecessary
* Revert "remove AUTH_URL since it unused and (likely) unnecessary"
This reverts commit 1f4a5aed22fa94bace899262e8576427fc852f61.
* cleanup GitHub action releases (#252)
* remove alloy, change auth defaul to disabled, add settings page in me dropdown
* enforce connection management perms to owner (#253)
* enforce conneciton management perms to owner
* fix formatting
* more formatting
* naming nits
* fix var name error
* change empty repo set copy if auth is disabled
* add CONTRIBUTING.md file
* hide settings in dropdown with auth isnt enabled
* handle case where gerrit weburl is just gitiles path
* Docs overhall (#251)
* remove nocheckin
* fix build error
* remove v3 trigger from deploy staging
* fix build errors round 2
* another error fix
---------
Co-authored-by: msukkari <michael.sukkarieh@mail.mcgill.ca>
2025-04-01 05:34:42 +00:00
|
|
|
export const secretCreateRequestSchema = z.object({
|
|
|
|
|
key: z.string(),
|
|
|
|
|
value: z.string(),
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
export const secreteDeleteRequestSchema = z.object({
|
|
|
|
|
key: z.string(),
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-11 04:55:00 +00:00
|
|
|
|
2024-09-29 21:39:17 +00:00
|
|
|
// @see : https://github.com/sourcebot-dev/zoekt/blob/3780e68cdb537d5a7ed2c84d9b3784f80c7c5d04/api.go#L728
|
2024-09-17 04:37:34 +00:00
|
|
|
const repoStatsSchema = z.object({
|
2024-09-11 04:55:00 +00:00
|
|
|
Repos: z.number(),
|
|
|
|
|
Shards: z.number(),
|
|
|
|
|
Documents: z.number(),
|
|
|
|
|
IndexBytes: z.number(),
|
|
|
|
|
ContentBytes: z.number(),
|
|
|
|
|
NewLinesCount: z.number(),
|
|
|
|
|
DefaultBranchNewLinesCount: z.number(),
|
|
|
|
|
OtherBranchesNewLinesCount: z.number(),
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-29 21:39:17 +00:00
|
|
|
// @see : https://github.com/sourcebot-dev/zoekt/blob/3780e68cdb537d5a7ed2c84d9b3784f80c7c5d04/api.go#L716
|
2024-09-17 04:37:34 +00:00
|
|
|
const indexMetadataSchema = z.object({
|
2024-09-11 04:55:00 +00:00
|
|
|
IndexFormatVersion: z.number(),
|
|
|
|
|
IndexFeatureVersion: z.number(),
|
|
|
|
|
IndexMinReaderVersion: z.number(),
|
|
|
|
|
IndexTime: z.string(),
|
|
|
|
|
PlainASCII: z.boolean(),
|
|
|
|
|
LanguageMap: z.record(z.string(), z.number()),
|
|
|
|
|
ZoektVersion: z.string(),
|
|
|
|
|
ID: z.string(),
|
|
|
|
|
});
|
|
|
|
|
|
2024-09-29 21:39:17 +00:00
|
|
|
// @see : https://github.com/sourcebot-dev/zoekt/blob/3780e68cdb537d5a7ed2c84d9b3784f80c7c5d04/api.go#L555
|
2024-09-26 03:12:20 +00:00
|
|
|
export const repositorySchema = z.object({
|
2024-09-11 04:55:00 +00:00
|
|
|
Name: z.string(),
|
|
|
|
|
URL: z.string(),
|
|
|
|
|
Source: z.string(),
|
|
|
|
|
Branches: z.array(z.object({
|
|
|
|
|
Name: z.string(),
|
|
|
|
|
Version: z.string(),
|
2024-11-01 17:51:14 +00:00
|
|
|
})).nullable(),
|
2024-09-11 04:55:00 +00:00
|
|
|
CommitURLTemplate: z.string(),
|
|
|
|
|
FileURLTemplate: z.string(),
|
|
|
|
|
LineFragmentTemplate: z.string(),
|
2024-11-01 17:51:14 +00:00
|
|
|
RawConfig: z.record(z.string(), z.string()).nullable(),
|
2024-09-11 04:55:00 +00:00
|
|
|
Rank: z.number(),
|
|
|
|
|
IndexOptions: z.string(),
|
|
|
|
|
HasSymbols: z.boolean(),
|
|
|
|
|
Tombstone: z.boolean(),
|
|
|
|
|
LatestCommitDate: z.string(),
|
|
|
|
|
FileTombstones: z.string().optional(),
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
export const listRepositoriesResponseSchema = z.object({
|
|
|
|
|
List: z.object({
|
|
|
|
|
Repos: z.array(z.object({
|
|
|
|
|
Repository: repositorySchema,
|
|
|
|
|
IndexMetadata: indexMetadataSchema,
|
2024-09-17 04:37:34 +00:00
|
|
|
Stats: repoStatsSchema,
|
2024-09-11 04:55:00 +00:00
|
|
|
})),
|
2024-09-17 04:37:34 +00:00
|
|
|
Stats: repoStatsSchema,
|
2024-09-11 04:55:00 +00:00
|
|
|
})
|
|
|
|
|
});
|
v3 effort (#158)
* SQL Database (#157)
* point zoekt to v3 branch
* bump zoekt version
* Add tenant ID concept into web app and backend (#160)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add nocheckin comment to tenant id in v2 schema
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* parallelize repo indexing (#163)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add pol of bullmq into backend
* add better error handling and concurrency setting
* spin up redis instance in dockerfile
* cleanup transaction logic when adding repos to index queue
* add NEW index status fetch condition
* move bullmq deps to backend
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* Authentication (#164)
* Add Org table (#167)
* Move logout button & profile picture into settings dropdown (#172)
* Multi tenancy support in config syncer (#171)
* [wip] initial mt support in config syncer
* Move logout button & profile picture into settings dropdown (#172)
* update sync status properly and fix bug with multiple config in db case
* make config path required in single tenant mode
NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* add tenant mode support in docker container:
* Organization switching & active org management (#173)
* updated syncedAt date after config sync:
* Migrate to postgres (#174)
* spin up postgres in docker container
* get initial pol of postgres db working in docker image
* spin up postgres server in dev case
* updated syncedAt date after config sync:
* remove unnecessary port expose in docker file
* Connection creation form (#175)
* fix issue with yarn dev startup
* init (#176)
* Add `@sourcebot/schemas` package (#177)
* Connection management (#178)
* add concept of secrets (#180)
* add @sourcebot/schemas package
* migrate things to use the schemas package
* Dockerfile support
* add secret table to schema
* Add concept of connection manager
* Rename Config->Connection
* Handle job failures
* Add join table between repo and connection
* nits
* create first version of crypto package
* add crypto package as deps to others
* forgot to add package changes
* add server action for adding and listing secrets, create test page for it
* add secrets page to nav menu
* add secret to config and support fetching it in backend
* reset secret form on successful submission
* add toast feedback for secrets form
* add instructions for adding encryption key to dev instructions
* add encryption key support in docker file
* add delete secret button
* fix nits from pr review
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* enforce tenancy on search and repo listing endpoints (#181)
* enforce tenancy on search and repo listing
* remove orgId from request schemas
* adds garbage collection for repos (#182)
* refactor repo indexing logic into RepoManager
* wip cleanup stale repos
* add rest of gc logic
* set status to indexing properly
* add initial logic for staging environment
* try to move encryption key env decleration in docker file to fix build issues
* switch encryption key as build arg to se if that fixes build issues
* add deployment action for staging image
* try using mac github action runners instead
* switch to using arm64 runners on arm64 build
* change workflow names to fix trigger issue
* trigger staging actions to see if it works
* fix working directory typo and pray it doesnt push to prod
* checkout v3 when deploying staging
* try to change into the staging dir manuall
* dummy commit to trigger v3 workflows to test
* update staging deploy script to match new version in main
* reference proper image:tag in staging fly config
* update staging fly config to point to ghcr
* Connection management (#183)
* add invite system and google oauth provider (#185)
* add settings page with members list
* add invite to schema and basic create form
* add invite table
* add basic invite link copy button
* add auth invite accept case
* add non auth logic
* add google oauth provider
* fix reference to header component in connections
* add google logo to google oauth
* fix web build errors
* bump staging resources
* change staging cpu to perf
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* Domain support (#188)
* Update Makefile to include crypto package when doing a make clean
* Add default for AUTH_URL in attempt to fix build
* attempt 2
* fix attempt #3: Do not require a encrpytion key at build time
* Fix generate script race condition
* Attempt #4
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* Generate AUTH_SECRET if not provided (#189)
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
* Add stripe billing logic (#190)
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* fix nits
* remove providers check
* fix more nits
* change stripe init to be behind function
* fix publishible stripe key handling in docker container
* enforce owner perms (#191)
* add make owner logic, and owner perms for removal, invite, and manage subscription
* add change billing email card to billing settings
* enforce owner role in action level
* remove unused hover card component
* cleanup
* add back gitlab, gitea, and gerrit support (#184)
* add non github config definitions
* refactor github config compilation to seperate file
* add gitlab config compilation
* Connection management (#183)
* wip gitlab repo sync support
* fix gitlab zoekt metadata
* add gitea support
* add gerrit support
* Connection management (#183)
* add gerrit config compilation
* Connection management (#183)
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* fix apos usage in redeem page
* change csrf cookie to secure not host
* Credentials provider (#192)
* email password functionality
* feedback
* cleanup org's repos and shards if it's inactive (#194)
* add stripe subscription status and webhook
* add inactive org repo cleanup logic
* mark reactivated org connections for sync
* connections qol improvements (#195)
* add client side polling to connections list
* properly fetch repo image url
* add client polling to connection management page, and add ability to sync failed connections
* Fix build with suspense boundary
* improved fix
* add retries for 429 issues (#196)
* add connection compile retry and hard repo limit
* add more retry checks
* cleanup unused change
* address feedback
* fix build errors and add index concurrency env var
* add config upsert timeout env var
* Membership settings rework (#198)
* Add refined members list
* futher progress on members settings polish
* Remove old components
* feedback
* Magic links (#199)
* wip on magic link support
* Switch to nodemailer / resend for transactional mail
* Further cleanup
* Add stylized email using react-email
* fix
* Fix build
* db performance improvements and job resilience (#200)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* remove non secret token options
* fix token examples in schema
* add better visualization for connection/repo errors and warnings (#201)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* add error package and use BackendException in connection manager
* handle connection failure display on web app
* add warning banner for not found orgs/repos/users
* add failure handling for gerrit
* add gitea notfound warning support
* add warning icon in connections list
* style nits
* add failed repo vis in connections list
* added retry failed repo index buttons
* move nav indicators to client with polling
* fix indicator flash issue and truncate large list results
* display error nav better
* truncate failed repo list in connection list item
* fix merge error
* fix merge bug
* add connection util file [wip]
* refactor notfound fetch logic and add missing error package to dockerfile
* move repeated logic to function and add zod schema for syncStatusMetadata
* add orgid unique constraint to repo
* revert repo compile update logic to upsert loop
* log upsert stats
* [temp] disable polling everywhere (#205)
* add health check endpoint
* Refined onboarding flow (#202)
* Redeem UX pass (#204)
* add log for health check
* fix new connection complete callback route
* add cpu split logic and only wait for postgres if we're going to connec to it
* Inline secret creation (#207)
* use docker scopes to try and improve caching
* Dummy change
* remove cpu split logic
* Add some instrumentation to web
* add posthog events on various user actions (#208)
* add page view event support
* add posthog events
* nit: remove unused import
* feedback
* fix merge error
* use staging posthog papik when building staging image
* fix other merge error and build warnings
* Add invite email (#209)
* wrap posthog provider in suspense to fix build error
* add grafana alloy config and setup (#210)
* add grafana alloy config and setup
* add basic repo prom metrics
* nits in dockerfile
* remove invalid characters when auto filling domain
* add login posthog events
* remove hard coded sourcebot.app references
* make repo garbage collection async (#211)
* add gc queue logic
* fix missing switch cases for gc status
* style org create form better with new staging domain
* change repo rm logic to be async
* simplify repo for inactive org query
* add grace period for garbage collecting repos
* make prom scrape interval 500ms
* fix typo in trial card
* onboarding tweaks
* rename some prom metrics and cleanup unused
* wipe existing repo if we've picked up a killed job to ensure good state
* Connections UX pass + query optimizations (#212)
* remove git & local schemas (#213)
* skip stripe checkout for trial + fix indexing in progress UI + additional schema validation (#214)
* add additional config validation
* wip bypass stripe checkout for trial
* fix stripe trial checkout bypass
* fix indexing in progress ui on home page
* add subscription checks, more schema validation, and fix issue with complete page
* dont display if no indexed repos
* fix skipping onboard complete check
* fix build error
* add back button in onboard connection creation flow
* Add back revision support (#215)
* fix build
* Fix bug with repository snapshot
* fix share links
* fix repo rm issue, 502 page, condition on test clock
* Make login and onboarding mobile friendly
* fix ordering of quick actions
* remove error msg dump on failed repo index job, and update indexedAt field
* Add mobile unsupported splash screne
* cherry pick fix for file links
* [Cherry Pick] Syntax reference guide (#169) (#216)
* Add .env to db gitignore
* fix case where we have repos but they're all failed for repo snapshot
* /settings/secrets page (#217)
* display domain properly in org create form
* Quick action tweaks (#218)
* revamp repo page (#220)
* wip repo table
* new repo page
* add indicator for when feedback is applied in repo page
* add repo button
* fetch connection data in one query
* fix styling
* fix (#219)
* remove / keyboard shortcut hint in search bar
* prevent switching to first page on data update and truncate long repo names in repo list
* General settings + cleanup (#221)
* General settings
* Add alert to org domain change
* First attempt at sending logs to grafana
* logs wip
* add alloy logs
* wip
* [temp] comment out loki for now
* update trial card content and add events for code host selection on onboard
* reduce scraping interval to 15s
* Add prometheus metric for pending repo indexing jobs
* switch magic link to invite code (#222)
* wip magic link codes
* pipe email to email provider properly
* remove magic link data cookie after sign in
* clean up unused imports
* dont remove cookie before we use it
* rm package-lock.json
* revert yarn files to v3 state
* switch email passing from cookie to search param
* add comment for settings dropdown auth update
* remove unused middleware file
* fix build error and warnings
* fix build error with useSearchParam not wrapped in suspense
* add sentry support to backend and webapp (#223)
* add sentry to web app
* set sentry environemnt from env var
* add sentry env replace logic in docker container
* wip add backend sentry
* add sentry to backend
* move dns to env var
* remove test exception
* Fix root domain issue on onboarding
* add setup sentry cli step to github action
* login to sentry
* fix sentry login in action
* Update grafana loki endpoint
* switch source map publish to runtime in entrypoint
* catch and rethrow simplegit exceptions
* alloy nits
* fix alloy
* backend logging (#224)
* revert grafana loki config
* fix login ui nits
* fix quick actions
* fix typo in secret creation
* fix private repo clone issue for gitlab
* add repo index timeout logic
* add posthog identify call after registeration
* various changes to add terms and security info (#225)
* add terms and security to footer
* add security card
* add demo card
* fix build error
* nit fix: center 'get in touch' on security card
* Dark theme improvements (#226)
* (fix) Fixed bug with gitlab and gitea not including hostname in the repoName
* Switch to using t3-env for env-var management (#230)
* Add missing env var
* fix build
* Centralize to using a single .env.development for development workflows (#231)
* Make billing optional (#232)
* Massage environment variables from strings to numbers (#234)
* Single tenancy & auth modes (#233)
* Add docs to this repo
* dummy change
* Declarative connection configuration (#235)
* fix build
* upgrade to next 14.2.25
* Improved database DX
* migrate to yarn v4
* Use origin from header for baseUrl of emails (instead of AUTH_URL). Also removed reference to hide scrollbars
* Remove SOURCEBOT_ENCRYPTION_KEY from build arg
* Fix issue with linking default user to org in single tenant + no-auth mode
* Fix fallback tokens (#242)
* add SECURITY_CARD_ENABLED flag
* Add repository weburl (#243)
* Random fixes and improvements (#244)
* add zoekt max wall time env var
* remove empty warning in docs
* fix reference in sh docs
* add connection manager upsert timeout env var
* Declarative connection cleanup + improvements (#245)
* change contact us footer in app to point to main contact form
* PostHog event pass (#246)
* fix typo
* Add sourcebot cloud environment prop to staging workflow
* Update generated files
* remove AUTH_URL since it unused and (likely) unnecessary
* Revert "remove AUTH_URL since it unused and (likely) unnecessary"
This reverts commit 1f4a5aed22fa94bace899262e8576427fc852f61.
* cleanup GitHub action releases (#252)
* remove alloy, change auth defaul to disabled, add settings page in me dropdown
* enforce connection management perms to owner (#253)
* enforce conneciton management perms to owner
* fix formatting
* more formatting
* naming nits
* fix var name error
* change empty repo set copy if auth is disabled
* add CONTRIBUTING.md file
* hide settings in dropdown with auth isnt enabled
* handle case where gerrit weburl is just gitiles path
* Docs overhall (#251)
* remove nocheckin
* fix build error
* remove v3 trigger from deploy staging
* fix build errors round 2
* another error fix
---------
Co-authored-by: msukkari <michael.sukkarieh@mail.mcgill.ca>
2025-04-01 05:34:42 +00:00
|
|
|
export const repositoryQuerySchema = z.object({
|
|
|
|
|
codeHostType: z.string(),
|
|
|
|
|
repoId: z.number(),
|
|
|
|
|
repoName: z.string(),
|
2025-04-03 00:50:48 +00:00
|
|
|
repoDisplayName: z.string().optional(),
|
v3 effort (#158)
* SQL Database (#157)
* point zoekt to v3 branch
* bump zoekt version
* Add tenant ID concept into web app and backend (#160)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add nocheckin comment to tenant id in v2 schema
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* parallelize repo indexing (#163)
* hacked together a example of using zoekt grpc api
* provide tenant id to zoekt git indexer
* update zoekt version to point to multitenant branch
* pipe tenant id through header to zoekt
* remove incorrect submodule reference and settings typo
* update zoekt commit
* remove unused yarn script
* remove unused grpc client in web server
* remove unneeded deps and improve tenant id log
* pass tenant id when creating repo in db
* add mt yarn script
* add pol of bullmq into backend
* add better error handling and concurrency setting
* spin up redis instance in dockerfile
* cleanup transaction logic when adding repos to index queue
* add NEW index status fetch condition
* move bullmq deps to backend
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* Authentication (#164)
* Add Org table (#167)
* Move logout button & profile picture into settings dropdown (#172)
* Multi tenancy support in config syncer (#171)
* [wip] initial mt support in config syncer
* Move logout button & profile picture into settings dropdown (#172)
* update sync status properly and fix bug with multiple config in db case
* make config path required in single tenant mode
NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* add tenant mode support in docker container:
* Organization switching & active org management (#173)
* updated syncedAt date after config sync:
* Migrate to postgres (#174)
* spin up postgres in docker container
* get initial pol of postgres db working in docker image
* spin up postgres server in dev case
* updated syncedAt date after config sync:
* remove unnecessary port expose in docker file
* Connection creation form (#175)
* fix issue with yarn dev startup
* init (#176)
* Add `@sourcebot/schemas` package (#177)
* Connection management (#178)
* add concept of secrets (#180)
* add @sourcebot/schemas package
* migrate things to use the schemas package
* Dockerfile support
* add secret table to schema
* Add concept of connection manager
* Rename Config->Connection
* Handle job failures
* Add join table between repo and connection
* nits
* create first version of crypto package
* add crypto package as deps to others
* forgot to add package changes
* add server action for adding and listing secrets, create test page for it
* add secrets page to nav menu
* add secret to config and support fetching it in backend
* reset secret form on successful submission
* add toast feedback for secrets form
* add instructions for adding encryption key to dev instructions
* add encryption key support in docker file
* add delete secret button
* fix nits from pr review
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* bump zoekt version
* enforce tenancy on search and repo listing endpoints (#181)
* enforce tenancy on search and repo listing
* remove orgId from request schemas
* adds garbage collection for repos (#182)
* refactor repo indexing logic into RepoManager
* wip cleanup stale repos
* add rest of gc logic
* set status to indexing properly
* add initial logic for staging environment
* try to move encryption key env decleration in docker file to fix build issues
* switch encryption key as build arg to se if that fixes build issues
* add deployment action for staging image
* try using mac github action runners instead
* switch to using arm64 runners on arm64 build
* change workflow names to fix trigger issue
* trigger staging actions to see if it works
* fix working directory typo and pray it doesnt push to prod
* checkout v3 when deploying staging
* try to change into the staging dir manuall
* dummy commit to trigger v3 workflows to test
* update staging deploy script to match new version in main
* reference proper image:tag in staging fly config
* update staging fly config to point to ghcr
* Connection management (#183)
* add invite system and google oauth provider (#185)
* add settings page with members list
* add invite to schema and basic create form
* add invite table
* add basic invite link copy button
* add auth invite accept case
* add non auth logic
* add google oauth provider
* fix reference to header component in connections
* add google logo to google oauth
* fix web build errors
* bump staging resources
* change staging cpu to perf
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* Domain support (#188)
* Update Makefile to include crypto package when doing a make clean
* Add default for AUTH_URL in attempt to fix build
* attempt 2
* fix attempt #3: Do not require a encrpytion key at build time
* Fix generate script race condition
* Attempt #4
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* Generate AUTH_SECRET if not provided (#189)
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
* Add stripe billing logic (#190)
* add side bar nav in settings page
* improve styling of members page
* wip adding stripe checkout button
* wip onboarding flow
* add stripe subscription id to org
* save stripe session id and add manage subscription button in settings
* properly block access to pages if user isn't in an org
* wip add paywall
* Domain support
* add back paywall and also add support for incrememnting seat count on invite redemption
* prevent self invite
* action button styling in settings and toast on copy
* add ability to remove member from org
* move stripe product id to env var
* add await for blocking loop in backend
* add subscription info to billing page
* handle trial case in billing info page
* add trial duration indicator to nav bar
* check if domain starts or ends with dash
* remove unused no org component
* remove package lock file and fix prisma dep version
* revert dep version updates
* fix yarn.lock
* add auth and membership check to fetchSubscription
* properly handle invite redeem with no valid subscription case
* change back fetch subscription to not require org membership
* add back subscription check in invite redeem page
---------
Co-authored-by: bkellam <bshizzle1234@gmail.com>
* fix nits
* remove providers check
* fix more nits
* change stripe init to be behind function
* fix publishible stripe key handling in docker container
* enforce owner perms (#191)
* add make owner logic, and owner perms for removal, invite, and manage subscription
* add change billing email card to billing settings
* enforce owner role in action level
* remove unused hover card component
* cleanup
* add back gitlab, gitea, and gerrit support (#184)
* add non github config definitions
* refactor github config compilation to seperate file
* add gitlab config compilation
* Connection management (#183)
* wip gitlab repo sync support
* fix gitlab zoekt metadata
* add gitea support
* add gerrit support
* Connection management (#183)
* add gerrit config compilation
* Connection management (#183)
---------
Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>
* fix apos usage in redeem page
* change csrf cookie to secure not host
* Credentials provider (#192)
* email password functionality
* feedback
* cleanup org's repos and shards if it's inactive (#194)
* add stripe subscription status and webhook
* add inactive org repo cleanup logic
* mark reactivated org connections for sync
* connections qol improvements (#195)
* add client side polling to connections list
* properly fetch repo image url
* add client polling to connection management page, and add ability to sync failed connections
* Fix build with suspense boundary
* improved fix
* add retries for 429 issues (#196)
* add connection compile retry and hard repo limit
* add more retry checks
* cleanup unused change
* address feedback
* fix build errors and add index concurrency env var
* add config upsert timeout env var
* Membership settings rework (#198)
* Add refined members list
* futher progress on members settings polish
* Remove old components
* feedback
* Magic links (#199)
* wip on magic link support
* Switch to nodemailer / resend for transactional mail
* Further cleanup
* Add stylized email using react-email
* fix
* Fix build
* db performance improvements and job resilience (#200)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* remove non secret token options
* fix token examples in schema
* add better visualization for connection/repo errors and warnings (#201)
* replace upsert with seperate create many and raw update many calls
* add bulk repo status update and queue addition with priority
* add support for managed redis
* add note for changing raw sql on schema change
* add error package and use BackendException in connection manager
* handle connection failure display on web app
* add warning banner for not found orgs/repos/users
* add failure handling for gerrit
* add gitea notfound warning support
* add warning icon in connections list
* style nits
* add failed repo vis in connections list
* added retry failed repo index buttons
* move nav indicators to client with polling
* fix indicator flash issue and truncate large list results
* display error nav better
* truncate failed repo list in connection list item
* fix merge error
* fix merge bug
* add connection util file [wip]
* refactor notfound fetch logic and add missing error package to dockerfile
* move repeated logic to function and add zod schema for syncStatusMetadata
* add orgid unique constraint to repo
* revert repo compile update logic to upsert loop
* log upsert stats
* [temp] disable polling everywhere (#205)
* add health check endpoint
* Refined onboarding flow (#202)
* Redeem UX pass (#204)
* add log for health check
* fix new connection complete callback route
* add cpu split logic and only wait for postgres if we're going to connec to it
* Inline secret creation (#207)
* use docker scopes to try and improve caching
* Dummy change
* remove cpu split logic
* Add some instrumentation to web
* add posthog events on various user actions (#208)
* add page view event support
* add posthog events
* nit: remove unused import
* feedback
* fix merge error
* use staging posthog papik when building staging image
* fix other merge error and build warnings
* Add invite email (#209)
* wrap posthog provider in suspense to fix build error
* add grafana alloy config and setup (#210)
* add grafana alloy config and setup
* add basic repo prom metrics
* nits in dockerfile
* remove invalid characters when auto filling domain
* add login posthog events
* remove hard coded sourcebot.app references
* make repo garbage collection async (#211)
* add gc queue logic
* fix missing switch cases for gc status
* style org create form better with new staging domain
* change repo rm logic to be async
* simplify repo for inactive org query
* add grace period for garbage collecting repos
* make prom scrape interval 500ms
* fix typo in trial card
* onboarding tweaks
* rename some prom metrics and cleanup unused
* wipe existing repo if we've picked up a killed job to ensure good state
* Connections UX pass + query optimizations (#212)
* remove git & local schemas (#213)
* skip stripe checkout for trial + fix indexing in progress UI + additional schema validation (#214)
* add additional config validation
* wip bypass stripe checkout for trial
* fix stripe trial checkout bypass
* fix indexing in progress ui on home page
* add subscription checks, more schema validation, and fix issue with complete page
* dont display if no indexed repos
* fix skipping onboard complete check
* fix build error
* add back button in onboard connection creation flow
* Add back revision support (#215)
* fix build
* Fix bug with repository snapshot
* fix share links
* fix repo rm issue, 502 page, condition on test clock
* Make login and onboarding mobile friendly
* fix ordering of quick actions
* remove error msg dump on failed repo index job, and update indexedAt field
* Add mobile unsupported splash screne
* cherry pick fix for file links
* [Cherry Pick] Syntax reference guide (#169) (#216)
* Add .env to db gitignore
* fix case where we have repos but they're all failed for repo snapshot
* /settings/secrets page (#217)
* display domain properly in org create form
* Quick action tweaks (#218)
* revamp repo page (#220)
* wip repo table
* new repo page
* add indicator for when feedback is applied in repo page
* add repo button
* fetch connection data in one query
* fix styling
* fix (#219)
* remove / keyboard shortcut hint in search bar
* prevent switching to first page on data update and truncate long repo names in repo list
* General settings + cleanup (#221)
* General settings
* Add alert to org domain change
* First attempt at sending logs to grafana
* logs wip
* add alloy logs
* wip
* [temp] comment out loki for now
* update trial card content and add events for code host selection on onboard
* reduce scraping interval to 15s
* Add prometheus metric for pending repo indexing jobs
* switch magic link to invite code (#222)
* wip magic link codes
* pipe email to email provider properly
* remove magic link data cookie after sign in
* clean up unused imports
* dont remove cookie before we use it
* rm package-lock.json
* revert yarn files to v3 state
* switch email passing from cookie to search param
* add comment for settings dropdown auth update
* remove unused middleware file
* fix build error and warnings
* fix build error with useSearchParam not wrapped in suspense
* add sentry support to backend and webapp (#223)
* add sentry to web app
* set sentry environemnt from env var
* add sentry env replace logic in docker container
* wip add backend sentry
* add sentry to backend
* move dns to env var
* remove test exception
* Fix root domain issue on onboarding
* add setup sentry cli step to github action
* login to sentry
* fix sentry login in action
* Update grafana loki endpoint
* switch source map publish to runtime in entrypoint
* catch and rethrow simplegit exceptions
* alloy nits
* fix alloy
* backend logging (#224)
* revert grafana loki config
* fix login ui nits
* fix quick actions
* fix typo in secret creation
* fix private repo clone issue for gitlab
* add repo index timeout logic
* add posthog identify call after registeration
* various changes to add terms and security info (#225)
* add terms and security to footer
* add security card
* add demo card
* fix build error
* nit fix: center 'get in touch' on security card
* Dark theme improvements (#226)
* (fix) Fixed bug with gitlab and gitea not including hostname in the repoName
* Switch to using t3-env for env-var management (#230)
* Add missing env var
* fix build
* Centralize to using a single .env.development for development workflows (#231)
* Make billing optional (#232)
* Massage environment variables from strings to numbers (#234)
* Single tenancy & auth modes (#233)
* Add docs to this repo
* dummy change
* Declarative connection configuration (#235)
* fix build
* upgrade to next 14.2.25
* Improved database DX
* migrate to yarn v4
* Use origin from header for baseUrl of emails (instead of AUTH_URL). Also removed reference to hide scrollbars
* Remove SOURCEBOT_ENCRYPTION_KEY from build arg
* Fix issue with linking default user to org in single tenant + no-auth mode
* Fix fallback tokens (#242)
* add SECURITY_CARD_ENABLED flag
* Add repository weburl (#243)
* Random fixes and improvements (#244)
* add zoekt max wall time env var
* remove empty warning in docs
* fix reference in sh docs
* add connection manager upsert timeout env var
* Declarative connection cleanup + improvements (#245)
* change contact us footer in app to point to main contact form
* PostHog event pass (#246)
* fix typo
* Add sourcebot cloud environment prop to staging workflow
* Update generated files
* remove AUTH_URL since it unused and (likely) unnecessary
* Revert "remove AUTH_URL since it unused and (likely) unnecessary"
This reverts commit 1f4a5aed22fa94bace899262e8576427fc852f61.
* cleanup GitHub action releases (#252)
* remove alloy, change auth defaul to disabled, add settings page in me dropdown
* enforce connection management perms to owner (#253)
* enforce conneciton management perms to owner
* fix formatting
* more formatting
* naming nits
* fix var name error
* change empty repo set copy if auth is disabled
* add CONTRIBUTING.md file
* hide settings in dropdown with auth isnt enabled
* handle case where gerrit weburl is just gitiles path
* Docs overhall (#251)
* remove nocheckin
* fix build error
* remove v3 trigger from deploy staging
* fix build errors round 2
* another error fix
---------
Co-authored-by: msukkari <michael.sukkarieh@mail.mcgill.ca>
2025-04-01 05:34:42 +00:00
|
|
|
repoCloneUrl: z.string(),
|
|
|
|
|
webUrl: z.string().optional(),
|
|
|
|
|
linkedConnections: z.array(z.object({
|
|
|
|
|
id: z.number(),
|
|
|
|
|
name: z.string(),
|
|
|
|
|
})),
|
|
|
|
|
imageUrl: z.string().optional(),
|
|
|
|
|
indexedAt: z.date().optional(),
|
|
|
|
|
repoIndexingStatus: z.nativeEnum(RepoIndexingStatus),
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
export const verifyCredentialsRequestSchema = z.object({
|
|
|
|
|
email: z.string().email(),
|
|
|
|
|
password: z.string().min(8),
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
export const orgNameSchema = z.string().min(2, { message: "Organization name must be at least 3 characters long." });
|
|
|
|
|
|
|
|
|
|
export const orgDomainSchema = z.string()
|
|
|
|
|
.min(2, { message: "Url must be at least 3 characters long." })
|
|
|
|
|
.max(50, { message: "Url must be at most 50 characters long." })
|
|
|
|
|
.regex(/^[a-z][a-z-]*[a-z]$/, {
|
|
|
|
|
message: "Url must start and end with a letter, and can only contain lowercase letters and dashes.",
|
|
|
|
|
})
|
|
|
|
|
.refine((domain) => {
|
|
|
|
|
const reserved = [
|
|
|
|
|
'api',
|
|
|
|
|
'login',
|
|
|
|
|
'signup',
|
|
|
|
|
'onboard',
|
|
|
|
|
'redeem',
|
|
|
|
|
'account',
|
|
|
|
|
'settings',
|
|
|
|
|
'staging',
|
|
|
|
|
'support',
|
|
|
|
|
'docs',
|
|
|
|
|
'blog',
|
|
|
|
|
'contact',
|
|
|
|
|
'status'
|
|
|
|
|
];
|
|
|
|
|
return !reserved.includes(domain);
|
|
|
|
|
}, "This url is reserved for internal use.")
|
|
|
|
|
.refine(async (domain) => {
|
|
|
|
|
const doesDomainExist = await checkIfOrgDomainExists(domain);
|
|
|
|
|
return isServiceError(doesDomainExist) || !doesDomainExist;
|
|
|
|
|
}, "This url is already taken.");
|
2025-01-28 17:04:27 +00:00
|
|
|
|
|
|
|
|
export const getVersionResponseSchema = z.object({
|
|
|
|
|
version: z.string(),
|
|
|
|
|
});
|