sourcebot/demo-site-config.json
Brendan Kellam 39b92b9e98
v3 effort (#158)
* SQL Database (#157)

* point zoekt to v3 branch

* bump zoekt version

* Add tenant ID concept into web app and backend (#160)

* hacked together a example of using zoekt grpc api

* provide tenant id to zoekt git indexer

* update zoekt version to point to multitenant branch

* pipe tenant id through header to zoekt

* remove incorrect submodule reference and settings typo

* update zoekt commit

* remove unused yarn script

* remove unused grpc client in web server

* remove unneeded deps and improve tenant id log

* pass tenant id when creating repo in db

* add mt yarn script

* add nocheckin comment to tenant id in v2 schema

---------

Co-authored-by: bkellam <bshizzle1234@gmail.com>

* bump zoekt version

* parallelize repo indexing (#163)

* hacked together a example of using zoekt grpc api

* provide tenant id to zoekt git indexer

* update zoekt version to point to multitenant branch

* pipe tenant id through header to zoekt

* remove incorrect submodule reference and settings typo

* update zoekt commit

* remove unused yarn script

* remove unused grpc client in web server

* remove unneeded deps and improve tenant id log

* pass tenant id when creating repo in db

* add mt yarn script

* add pol of bullmq into backend

* add better error handling and concurrency setting

* spin up redis instance in dockerfile

* cleanup transaction logic when adding repos to index queue

* add NEW index status fetch condition

* move bullmq deps to backend

---------

Co-authored-by: bkellam <bshizzle1234@gmail.com>

* Authentication (#164)

* Add Org table (#167)

* Move logout button & profile picture into settings dropdown (#172)

* Multi tenancy support in config syncer (#171)

* [wip] initial mt support in config syncer

* Move logout button & profile picture into settings dropdown (#172)

* update sync status properly and fix bug with multiple config in db case

* make config path required in single tenant mode

NOTE: deleting config/repos is currently not supported in multi tenancy case. Support for this will be added in a future PR

---------

Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>

* add tenant mode support in docker container:

* Organization switching & active org management (#173)

* updated syncedAt date after config sync:

* Migrate to postgres (#174)

* spin up postgres in docker container

* get initial pol of postgres db working in docker image

* spin up postgres server in dev case

* updated syncedAt date after config sync:

* remove unnecessary port expose in docker file

* Connection creation form (#175)

* fix issue with yarn dev startup

* init (#176)

* Add `@sourcebot/schemas` package (#177)

* Connection management (#178)

* add concept of secrets (#180)

* add @sourcebot/schemas package

* migrate things to use the schemas package

* Dockerfile support

* add secret table to schema

* Add concept of connection manager

* Rename Config->Connection

* Handle job failures

* Add join table between repo and connection

* nits

* create first version of crypto package

* add crypto package as deps to others

* forgot to add package changes

* add server action for adding and listing secrets, create test page for it

* add secrets page to nav menu

* add secret to config and support fetching it in backend

* reset secret form on successful submission

* add toast feedback for secrets form

* add instructions for adding encryption key to dev instructions

* add encryption key support in docker file

* add delete secret button

* fix nits from pr review

---------

Co-authored-by: bkellam <bshizzle1234@gmail.com>

* bump zoekt version

* enforce tenancy on search and repo listing endpoints (#181)

* enforce tenancy on search and repo listing

* remove orgId from request schemas

* adds garbage collection for repos (#182)

* refactor repo indexing logic into RepoManager

* wip cleanup stale repos

* add rest of gc logic

* set status to indexing properly

* add initial logic for staging environment

* try to move encryption key env decleration in docker file to fix build issues

* switch encryption key as build arg to se if that fixes build issues

* add deployment action for staging image

* try using mac github action runners instead

* switch to using arm64 runners on arm64 build

* change workflow names to fix trigger issue

* trigger staging actions to see if it works

* fix working directory typo and pray it doesnt push to prod

* checkout v3 when deploying staging

* try to change into the staging dir manuall

* dummy commit to trigger v3 workflows to test

* update staging deploy script to match new version in main

* reference proper image:tag in staging fly config

* update staging fly config to point to ghcr

* Connection management (#183)

* add invite system and google oauth provider (#185)

* add settings page with members list

* add invite to schema and basic create form

* add invite table

* add basic invite link copy button

* add auth invite accept case

* add non auth logic

* add google oauth provider

* fix reference to header component in connections

* add google logo to google oauth

* fix web build errors

* bump staging resources

* change staging cpu to perf

* add side bar nav in settings page

* improve styling of members page

* wip adding stripe checkout button

* wip onboarding flow

* add stripe subscription id to org

* save stripe session id and add manage subscription button in settings

* properly block access to pages if user isn't in an org

* wip add paywall

* Domain support

* Domain support (#188)

* Update Makefile to include crypto package when doing a make clean

* Add default for AUTH_URL in attempt to fix build

* attempt 2

* fix attempt #3: Do not require a encrpytion key at build time

* Fix generate script race condition

* Attempt #4

* add back paywall and also add support for incrememnting seat count on invite redemption

* prevent self invite

* action button styling in settings and toast on copy

* add ability to remove member from org

* move stripe product id to env var

* add await for blocking loop in backend

* add subscription info to billing page

* handle trial case in billing info page

* add trial duration indicator to nav bar

* check if domain starts or ends with dash

* remove unused no org component

* Generate AUTH_SECRET if not provided (#189)

* remove package lock file and fix prisma dep version

* revert dep version updates

* fix yarn.lock

* add auth and membership check to fetchSubscription

* properly handle invite redeem with no valid subscription case

* change back fetch subscription to not require org membership

* add back subscription check in invite redeem page

* Add stripe billing logic (#190)

* add side bar nav in settings page

* improve styling of members page

* wip adding stripe checkout button

* wip onboarding flow

* add stripe subscription id to org

* save stripe session id and add manage subscription button in settings

* properly block access to pages if user isn't in an org

* wip add paywall

* Domain support

* add back paywall and also add support for incrememnting seat count on invite redemption

* prevent self invite

* action button styling in settings and toast on copy

* add ability to remove member from org

* move stripe product id to env var

* add await for blocking loop in backend

* add subscription info to billing page

* handle trial case in billing info page

* add trial duration indicator to nav bar

* check if domain starts or ends with dash

* remove unused no org component

* remove package lock file and fix prisma dep version

* revert dep version updates

* fix yarn.lock

* add auth and membership check to fetchSubscription

* properly handle invite redeem with no valid subscription case

* change back fetch subscription to not require org membership

* add back subscription check in invite redeem page

---------

Co-authored-by: bkellam <bshizzle1234@gmail.com>

* fix nits

* remove providers check

* fix more nits

* change stripe init to be behind function

* fix publishible stripe key handling in docker container

* enforce owner perms (#191)

* add make owner logic, and owner perms for removal, invite, and manage subscription

* add change billing email card to billing settings

* enforce owner role in action level

* remove unused hover card component

* cleanup

* add back gitlab, gitea, and gerrit support (#184)

* add non github config definitions

* refactor github config compilation to seperate file

* add gitlab config compilation

* Connection management (#183)

* wip gitlab repo sync support

* fix gitlab zoekt metadata

* add gitea support

* add gerrit support

* Connection management (#183)

* add gerrit config compilation

* Connection management (#183)

---------

Co-authored-by: Brendan Kellam <bshizzle1234@gmail.com>

* fix apos usage in redeem page

* change csrf cookie to secure not host

* Credentials provider (#192)

* email password functionality

* feedback

* cleanup org's repos and shards if it's inactive (#194)

* add stripe subscription status and webhook

* add inactive org repo cleanup logic

* mark reactivated org connections for sync

* connections qol improvements (#195)

* add client side polling to connections list

* properly fetch repo image url

* add client polling to connection management page, and add ability to sync failed connections

* Fix build with suspense boundary

* improved fix

* add retries for 429 issues (#196)

* add connection compile retry and hard repo limit

* add more retry checks

* cleanup unused change

* address feedback

* fix build errors and add index concurrency env var

* add config upsert timeout env var

* Membership settings rework (#198)

* Add refined members list

* futher progress on members settings polish

* Remove old components

* feedback

* Magic links (#199)

* wip on magic link support

* Switch to nodemailer / resend for transactional mail

* Further cleanup

* Add stylized email using react-email

* fix

* Fix build

* db performance improvements and job resilience  (#200)

* replace upsert with seperate create many and raw update many calls

* add bulk repo status update and queue addition with priority

* add support for managed redis

* add note for changing raw sql on schema change

* remove non secret token options

* fix token examples in schema

* add better visualization for connection/repo errors and warnings (#201)

* replace upsert with seperate create many and raw update many calls

* add bulk repo status update and queue addition with priority

* add support for managed redis

* add note for changing raw sql on schema change

* add error package and use BackendException in connection manager

* handle connection failure display on web app

* add warning banner for not found orgs/repos/users

* add failure handling for gerrit

* add gitea notfound warning support

* add warning icon in connections list

* style nits

* add failed repo vis in connections list

* added retry failed repo index buttons

* move nav indicators to client with polling

* fix indicator flash issue and truncate large list results

* display error nav better

* truncate failed repo list in connection list item

* fix merge error

* fix merge bug

* add connection util file [wip]

* refactor notfound fetch logic and add missing error package to dockerfile

* move repeated logic to function and add zod schema for syncStatusMetadata

* add orgid unique constraint to repo

* revert repo compile update logic to upsert loop

* log upsert stats

* [temp] disable polling everywhere (#205)

* add health check endpoint

* Refined onboarding flow (#202)

* Redeem UX pass (#204)

* add log for health check

* fix new connection complete callback route

* add cpu split logic and only wait for postgres if we're going to connec to it

* Inline secret creation (#207)

* use docker scopes to try and improve caching

* Dummy change

* remove cpu split logic

* Add some instrumentation to web

* add posthog events on various user actions (#208)

* add page view event support

* add posthog events

* nit: remove unused import

* feedback

* fix merge error

* use staging posthog papik when building staging image

* fix other merge error and build warnings

* Add invite email (#209)

* wrap posthog provider in suspense to fix build error

* add grafana alloy config and setup (#210)

* add grafana alloy config and setup

* add basic repo prom metrics

* nits in dockerfile

* remove invalid characters when auto filling domain

* add login posthog events

* remove hard coded sourcebot.app references

* make repo garbage collection async (#211)

* add gc queue logic

* fix missing switch cases for gc status

* style org create form better with new staging domain

* change repo rm logic to be async

* simplify repo for inactive org query

* add grace period for garbage collecting repos

* make prom scrape interval 500ms

* fix typo in trial card

* onboarding tweaks

* rename some prom metrics and cleanup unused

* wipe existing repo if we've picked up a killed job to ensure good state

* Connections UX pass + query optimizations (#212)

* remove git & local schemas (#213)

* skip stripe checkout for trial + fix indexing in progress UI + additional schema validation (#214)

* add additional config validation

* wip bypass stripe checkout for trial

* fix stripe trial checkout bypass

* fix indexing in progress ui on home page

* add subscription checks, more schema validation, and fix issue with complete page

* dont display if no indexed repos

* fix skipping onboard complete check

* fix build error

* add back button in onboard connection creation flow

* Add back revision support (#215)

* fix build

* Fix bug with repository snapshot

* fix share links

* fix repo rm issue, 502 page, condition on test clock

* Make login and onboarding mobile friendly

* fix ordering of quick actions

* remove error msg dump on failed repo index job, and update indexedAt field

* Add mobile unsupported splash screne

* cherry pick fix for file links

* [Cherry Pick] Syntax reference guide (#169) (#216)

* Add .env to db gitignore

* fix case where we have repos but they're all failed for repo snapshot

* /settings/secrets page (#217)

* display domain properly in org create form

* Quick action tweaks (#218)

* revamp repo page (#220)

* wip repo table

* new repo page

* add indicator for when feedback is applied in repo page

* add repo button

* fetch connection data in one query

* fix styling

* fix (#219)

* remove / keyboard shortcut hint in search bar

* prevent switching to first page on data update and truncate long repo names in repo list

* General settings + cleanup (#221)

* General settings

* Add alert to org domain change

* First attempt at sending logs to grafana

* logs wip

* add alloy logs

* wip

* [temp] comment out loki for now

* update trial card content and add events for code host selection on onboard

* reduce scraping interval to 15s

* Add prometheus metric for pending repo indexing jobs

* switch magic link to invite code (#222)

* wip magic link codes

* pipe email to email provider properly

* remove magic link data cookie after sign in

* clean up unused imports

* dont remove cookie before we use it

* rm package-lock.json

* revert yarn files to v3 state

* switch email passing from cookie to search param

* add comment for settings dropdown auth update

* remove unused middleware file

* fix build error and warnings

* fix build error with useSearchParam not wrapped in suspense

* add sentry support to backend and webapp (#223)

* add sentry to web app

* set sentry environemnt from env var

* add sentry env replace logic in docker container

* wip add backend sentry

* add sentry to backend

* move dns to env var

* remove test exception

* Fix root domain issue on onboarding

* add setup sentry cli step to github action

* login to sentry

* fix sentry login in action

* Update grafana loki endpoint

* switch source map publish to runtime in entrypoint

* catch and rethrow simplegit exceptions

* alloy nits

* fix alloy

* backend logging (#224)

* revert grafana loki config

* fix login ui nits

* fix quick actions

* fix typo in secret creation

* fix private repo clone issue for gitlab

* add repo index timeout logic

* add posthog identify call after registeration

* various changes to add terms and security info (#225)

* add terms and security to footer

* add security card

* add demo card

* fix build error

* nit fix: center 'get in touch' on security card

* Dark theme improvements (#226)

* (fix) Fixed bug with gitlab and gitea not including hostname in the repoName

* Switch to using t3-env for env-var management (#230)

* Add missing env var

* fix build

* Centralize to using a single .env.development for development workflows (#231)

* Make billing optional (#232)

* Massage environment variables from strings to numbers (#234)

* Single tenancy & auth modes (#233)

* Add docs to this repo

* dummy change

* Declarative connection configuration (#235)

* fix build

* upgrade to next 14.2.25

* Improved database DX

* migrate to yarn v4

* Use origin from header for baseUrl of emails (instead of AUTH_URL). Also removed reference to hide scrollbars

* Remove SOURCEBOT_ENCRYPTION_KEY from build arg

* Fix issue with linking default user to org in single tenant + no-auth mode

* Fix fallback tokens (#242)

* add SECURITY_CARD_ENABLED flag

* Add repository weburl (#243)

* Random fixes and improvements (#244)

* add zoekt max wall time env var

* remove empty warning in docs

* fix reference in sh docs

* add connection manager upsert timeout env var

* Declarative connection cleanup + improvements (#245)

* change contact us footer in app to point to main contact form

* PostHog event pass (#246)

* fix typo

* Add sourcebot cloud environment prop to staging workflow

* Update generated files

* remove AUTH_URL since it unused and (likely) unnecessary

* Revert "remove AUTH_URL since it unused and (likely) unnecessary"

This reverts commit 1f4a5aed22.

* cleanup GitHub action releases (#252)

* remove alloy, change auth defaul to disabled, add settings page in me dropdown

* enforce connection management perms to owner (#253)

* enforce conneciton management perms to owner

* fix formatting

* more formatting

* naming nits

* fix var name error

* change empty repo set copy if auth is disabled

* add CONTRIBUTING.md file

* hide settings in dropdown with auth isnt enabled

* handle case where gerrit weburl is just gitiles path

* Docs overhall (#251)

* remove nocheckin

* fix build error

* remove v3 trigger from deploy staging

* fix build errors round 2

* another error fix

---------

Co-authored-by: msukkari <michael.sukkarieh@mail.mcgill.ca>
2025-03-31 22:34:42 -07:00

297 lines
No EOL
11 KiB
JSON

{
"$schema": "./schemas/v2/index.json",
"settings": {
"reindexInterval": 86400000, // 24 hours
"resyncInterval": 86400000 // 24 hours
},
"repos": [
{
"type": "github",
"token": {
"env": "GITHUB_TOKEN"
},
"exclude": {
"size": {
"max": 1000000000 // Limit to 1GB
}
},
"repos": [
"torvalds/linux",
"pytorch/pytorch",
"commaai/openpilot",
"ggerganov/whisper.cpp",
"ggerganov/llama.cpp",
"codemirror/dev",
"tailwindlabs/tailwindcss",
"sourcebot-dev/sourcebot",
"freeCodeCamp/freeCodeCamp",
"EbookFoundation/free-programming-books",
"sindresorhus/awesome",
"public-apis/public-apis",
"codecrafters-io/build-your-own-x",
"jwasham/coding-interview-university",
"kamranahmedse/developer-roadmap",
"donnemartin/system-design-primer",
"996icu/996.ICU",
"facebook/react",
"vinta/awesome-python",
"vuejs/vue",
"practical-tutorials/project-based-learning",
"awesome-selfhosted/awesome-selfhosted",
"TheAlgorithms/Python",
"trekhleb/javascript-algorithms",
"tensorflow/tensorflow",
"getify/You-Dont-Know-JS",
"CyC2018/CS-Notes",
"ohmyzsh/ohmyzsh",
"ossu/computer-science",
"twbs/bootstrap",
"Significant-Gravitas/AutoGPT",
"flutter/flutter",
"microsoft/vscode",
"github/gitignore",
"jackfrued/Python-100-Days",
"jlevy/the-art-of-command-line",
"trimstray/the-book-of-secret-knowledge",
"Snailclimb/JavaGuide",
"airbnb/javascript",
"AUTOMATIC1111/stable-diffusion-webui",
"huggingface/transformers",
"avelino/awesome-go",
"ytdl-org/youtube-dl",
"vercel/next.js",
"labuladong/fucking-algorithm",
"golang/go",
"Chalarangelo/30-seconds-of-code",
"yangshun/tech-interview-handbook",
"facebook/react-native",
"electron/electron",
"Genymobile/scrcpy",
"f/awesome-chatgpt-prompts",
"microsoft/PowerToys",
"justjavac/free-programming-books-zh_CN",
"kubernetes/kubernetes",
"d3/d3",
"nodejs/node",
"massgravel/Microsoft-Activation-Scripts",
"axios/axios",
"mrdoob/three.js",
"krahets/hello-algo",
"facebook/create-react-app",
"ollama/ollama",
"microsoft/TypeScript",
"goldbergyoni/nodebestpractices",
"rust-lang/rust",
"denoland/deno",
"angular/angular",
"langchain-ai/langchain",
"microsoft/terminal",
"521xueweihan/HelloGitHub",
"mui/material-ui",
"ant-design/ant-design",
"yt-dlp/yt-dlp",
"ryanmcdermott/clean-code-javascript",
"godotengine/godot",
"ripienaar/free-for-dev",
"iluwatar/java-design-patterns",
"puppeteer/puppeteer",
"papers-we-love/papers-we-love",
"PanJiaChen/vue-element-admin",
"iptv-org/iptv",
"fatedier/frp",
"excalidraw/excalidraw",
"tauri-apps/tauri",
"Hack-with-Github/Awesome-Hacking",
"nvbn/thefuck",
"mtdvio/every-programmer-should-know",
"storybookjs/storybook",
"neovim/neovim",
"microsoft/Web-Dev-For-Beginners",
"django/django",
"florinpop17/app-ideas",
"animate-css/animate.css",
"nvm-sh/nvm",
"gothinkster/realworld",
"bitcoin/bitcoin",
"sveltejs/svelte",
"opencv/opencv",
"gin-gonic/gin",
"laravel/laravel",
"fastapi/fastapi",
"macrozheng/mall",
"jaywcjlove/awesome-mac",
"tonsky/FiraCode",
"ChatGPTNextWeb/ChatGPT-Next-Web",
"rustdesk/rustdesk",
"tensorflow/models",
"doocs/advanced-java",
"shadcn-ui/ui",
"gohugoio/hugo",
"MisterBooo/LeetCodeAnimation",
"spring-projects/spring-boot",
"supabase/supabase",
"oven-sh/bun",
"FortAwesome/Font-Awesome",
"home-assistant/core",
"typicode/json-server",
"mermaid-js/mermaid",
"openai/whisper",
"netdata/netdata",
"vuejs/awesome-vue",
"DopplerHQ/awesome-interview-questions",
"3b1b/manim",
"2dust/v2rayN",
"nomic-ai/gpt4all",
"elastic/elasticsearch",
"anuraghazra/github-readme-stats",
"microsoft/ML-For-Beginners",
"MunGell/awesome-for-beginners",
"fighting41love/funNLP",
"vitejs/vite",
"thedaviddias/Front-End-Checklist",
"coder/code-server",
"moby/moby",
"CompVis/stable-diffusion",
"base-org/node",
"nestjs/nest",
"pallets/flask",
"hakimel/reveal.js",
"Anduin2017/HowToCook",
"microsoft/playwright",
"swiftlang/swift",
"Developer-Y/cs-video-courses",
"redis/redis",
"bregman-arie/devops-exercises",
"josephmisiti/awesome-machine-learning",
"binary-husky/gpt_academic",
"junegunn/fzf",
"syncthing/syncthing",
"hoppscotch/hoppscotch",
"protocolbuffers/protobuf",
"enaqx/awesome-react",
"expressjs/express",
"microsoft/generative-ai-for-beginners",
"grafana/grafana",
"abi/screenshot-to-code",
"ByteByteGoHq/system-design-101",
"chartjs/Chart.js",
"webpack/webpack",
"d2l-ai/d2l-zh",
"sdmg15/Best-websites-a-programmer-should-visit",
"strapi/strapi",
"python/cpython",
"leonardomso/33-js-concepts",
"kdn251/interviews",
"ventoy/Ventoy",
"ansible/ansible",
"apache/superset",
"tesseract-ocr/tesseract",
"lydiahallie/javascript-questions",
"xtekky/gpt4free",
"FuelLabs/sway",
"twitter/the-algorithm",
"keras-team/keras",
"resume/resume.github.com",
"swisskyrepo/PayloadsAllTheThings",
"ocornut/imgui",
"socketio/socket.io",
"awesomedata/awesome-public-datasets",
"louislam/uptime-kuma",
"kelseyhightower/nocode",
"sherlock-project/sherlock",
"reduxjs/redux",
"apache/echarts",
"obsproject/obs-studio",
"openai/openai-cookbook",
"fffaraz/awesome-cpp",
"scikit-learn/scikit-learn",
"TheAlgorithms/Java",
"atom/atom",
"Eugeny/tabby",
"lodash/lodash",
"caddyserver/caddy",
"sindresorhus/awesome-nodejs",
"rust-unofficial/awesome-rust",
"streamich/react-use",
"pocketbase/pocketbase",
"serhii-londar/open-source-mac-os-apps",
"lllyasviel/Fooocus",
"k88hudson/git-flight-rules",
"react-hook-form/react-hook-form",
"koajs/koa",
"SheetJS/sheetjs",
"trpc/trpc",
"LC044/WeChatMsg",
"airbnb/lottie-android",
"huihut/interview",
"jgm/pandoc",
"google/googletest",
"date-fns/date-fns",
"nativefier/nativefier",
"openai/gym",
"files-community/Files",
"sahat/hackathon-starter",
"appsmithorg/appsmith",
"ultralytics/ultralytics",
"slidevjs/slidev",
"xitu/gold-miner",
"sorrycc/awesome-javascript",
"astral-sh/ruff",
"logseq/logseq",
"shadowsocks/shadowsocks",
"ccxt/ccxt",
"netty/netty",
"tw93/Pake",
"fxsjy/jieba",
"atlassian/react-beautiful-dnd",
"ToolJet/ToolJet",
"markedjs/marked",
"typicode/husky",
"laravel/framework",
"TheAlgorithms/JavaScript",
"bilibili/ijkplayer",
"solidjs/solid",
"fastify/fastify",
"huggingface/pytorch-image-models",
"shadowsocks/ShadowsocksX-NG",
"carbon-language/carbon-lang",
"s0md3v/roop",
"ascoders/weekly",
"backstage/backstage",
"servo/servo",
"composer/composer",
"tastejs/todomvc",
"lutzroeder/netron",
"alibaba/canal",
"tinygrad/tinygrad",
"ManimCommunity/manim",
"filebrowser/filebrowser",
"nicolargo/glances",
"iperov/DeepFaceLive",
"StevenBlack/hosts",
"crossoverJie/JCSprout",
"mantinedev/mantine",
"Automattic/mongoose",
"eslint/eslint",
"nextauthjs/next-auth",
"flameshot-org/flameshot",
"envoyproxy/envoy"
]
},
{
"type": "gitlab",
"token": {
"env": "GITLAB_TOKEN"
},
"groups": [
"fdroid"
],
"exclude": {
"projects": [
"fdroid/wiki",
"Matrixcoffee/internal-twif-preview"
]
}
}
]
}