Blazingly fast code search 🏎️

Docker

GitHub

Generate a GitHub Personal Access Token (PAT) [here](https://github.com/settings/tokens/new). If you're only indexing public repositories select the `public_repo` scope; otherwise, select the `repo` scope. You'll need to pass this PAT each time you run Sourcebot, so we recommend adding it as an environment variable. In this guide, we'll add the Github PAT as an environment variable called `GITHUB_TOKEN`: ```sh export GITHUB_TOKEN= ``` If you'd like to persist this environment variable across shell sessions, please add this line to your shell config file (ex. `~/.bashrc`, `~/.bash_profile`, etc)

GitLab

TODO

BitBucket

TODO

GitHub

Run the `sourcebot` docker image, passing in the Github PAT you generated in the previous step as an environment variable called `GITHUB_TOKEN`: ```sh docker run -p 3000:3000 --rm --name sourcebot -v $(pwd):/data -e GITHUB_TOKEN=$GITHUB_TOKEN ghcr.io/taqlaai/sourcebot:main ```

GitLab

```sh docker run -p 3000:3000 --rm --name sourcebot -v $(pwd):/data -e GITLAB_TOKEN= ghcr.io/taqlaai/sourcebot:main ```

BitBucket

TODO

] ... INFO success: node-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) INFO success: zoekt-indexserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) INFO success: zoekt-webserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) ``` zoekt will now index your repositories (at `HEAD`). By default, it will re-index existing repositories every hour, and discover new repositories every 24 hours. 4. Go to `http://localhost:3000` - once an index has been created, you can start searching. ## Building Sourcebot 1. Install go and NodeJS. Note that a NodeJS version of at least `21.1.0` is required. 2. Install [ctags](https://github.com/universal-ctags/ctags) (required by zoekt-indexserver): Mac: `brew install universal-ctags` Ubuntu: `apt-get install universal-ctags` 3. Clone the repository with submodules: ```sh git clone --recurse-submodules https://github.com/TaqlaAI/sourcebot.git ``` 4. Run make to build zoekt and install dependencies: ```sh cd sourcebot make ``` The zoekt binaries and web dependencies are placed into `bin` and `node_modules` respectively. 3. Create a `config.json` file and list the repositories you want to index. The JSON schema defined in [index.json](./schemas/index.json) defines the structure of the config file and the available options. For example, if we want to index Sourcebot on its own code, we could use the following config found in `sample-config.json`: ```json { "$schema": "https://raw.githubusercontent.com/TaqlaAI/sourcebot/main/schemas/index.json", "Configs": [ { "Type": "github", "GitHubOrg": "TaqlaAI", "Name": "^sourcebot$" } ] } ``` 4. Create a Personal Access Token (PAT) to authenticate with a code host:

GitHub

Generate a GitHub Personal Access Token (PAT) [here](https://github.com/settings/tokens/new). If you are indexing public repositories only, you can select the `public_repo` scope, otherwise you will need the `repo` scope. Create a text file named `.github-token` **in your home directory** and paste the token in it. The file should look like: ```sh ghp_... ``` zoekt will [read this file](https://github.com/TaqlaAI/zoekt/blob/6a5753692b46e669f851ab23211e756a3677185d/cmd/zoekt-mirror-github/main.go#L60) to authenticate with GitHub.

GitLab

TODO

BitBucket

TODO

5. Start Sourcebot with the command: ```sh yarn dev ``` A `.sourcebot` directory will be created and zoekt will begin to index the repositories found given `config.json`. 6. Go to `http://localhost:3000` - once an index has been created, you can start searching. ## Telemetry By default, Sourcebot collects anonymized usage data through [PostHog](https://posthog.com/) to help us improve the performance and reliability of our tool. We do not collect or transmit [any information related to your codebase](https://github.com/search?q=repo:TaqlaAI/sourcebot++captureEvent&type=code). All events are [sanitized](https://github.com/TaqlaAI/sourcebot/blob/main/src/app/posthogProvider.tsx) to ensure that no sensitive or identifying details leave your machine. The data we collect includes general usage statistics and metadata such as query performance (e.g., search duration, error rates) to monitor the application's health and functionality. This information helps us better understand how Sourcebot is used and where improvements can be made :) If you'd like to disable all telemetry, you can do so by setting the environment variable `SOURCEBOT_TELEMETRY_DISABLED` to `1` in the docker run command: ```sh docker run -e SOURCEBOT_TELEMETRY_DISABLED=1 /* additional args */ ghcr.io/taqlaai/sourcebot:main ``` Or if you are building locally, add the following to your [.env](./.env) file: ```sh SOURCEBOT_TELEMETRY_DISABLED=1 NEXT_PUBLIC_SOURCEBOT_TELEMETRY_DISABLED=1 ```