ti.sil/sourcebot
1
0
Fork 0
mirror of https://github.com/sourcebot-dev/sourcebot.git synced 2025-12-12 12:25:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

fix(search-contexts): Fix issue where a repository would not appear in a search context if it was created after the search context was created (#354)

Browse source 
## Problem

If a repository is added **after** a search context (e.g., a new repository is synced from the code host), then it will never be added to the context even if it should be included. The workaround is to restart the instance.

## Solution

This PR adds a call to re-sync all search contexts whenever a connection is successfully synced. This PR adds the `@sourcebot/shared` package that contains `syncSearchContexts.ts` (previously in web) and it's dependencies (namely the entitlements system).

## Why another package?

Because the `syncSearchContexts` call is now called from:
1. `initialize.ts` in **web** - handles syncing search contexts on startup and whenever the config is modified in watch mode. This is the same as before.
2. `connectionManager.ts` in **backend** - syncs the search contexts whenever a connection is successfully synced.

## Follow-up devex work
Two things:
1. We have several very thin shared packages (i.e., `crypto`, `error`, and `logger`) that we can probably fold into this "general" shared package. `schemas` and `db` _feels_ like they should remain separate (mostly because they are "code-gen" packages).
2. When running `yarn dev`, any changes made to the shared package will only get picked if you `ctrl+c` and restart the instance. Would be nice if we have watch mode work across package dependencies in the monorepo.
This commit is contained in:
Brendan Kellam 2025-06-17 14:04:25 -07:00 committed by GitHub
parent c0caa5a8d0
commit 22d548e171
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
41 changed files with 321 additions and 172 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
CHANGELOG.md
View file

@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Delete account join request when redeeming an invite. [#352](https://github.com/sourcebot-dev/sourcebot/pull/352)
- Fix issue where a repository would not be included in a search context if the context was created before the repository. [#354](https://github.com/sourcebot-dev/sourcebot/pull/354)
## [4.3.0] - 2025-06-11

5
Dockerfile
View file

@ -43,12 +43,14 @@ COPY ./packages/schemas ./packages/schemas
COPY ./packages/crypto ./packages/crypto
COPY ./packages/error ./packages/error
COPY ./packages/logger ./packages/logger
COPY ./packages/shared ./packages/shared
RUN yarn workspace @sourcebot/db install
RUN yarn workspace @sourcebot/schemas install
RUN yarn workspace @sourcebot/crypto install
RUN yarn workspace @sourcebot/error install
RUN yarn workspace @sourcebot/logger install
RUN yarn workspace @sourcebot/shared install
# ------------------------------------
# ------ Build Web ------
@ -92,6 +94,7 @@ COPY --from=shared-libs-builder /app/packages/schemas ./packages/schemas
COPY --from=shared-libs-builder /app/packages/crypto ./packages/crypto
COPY --from=shared-libs-builder /app/packages/error ./packages/error
COPY --from=shared-libs-builder /app/packages/logger ./packages/logger
COPY --from=shared-libs-builder /app/packages/shared ./packages/shared
# Fixes arm64 timeouts
RUN yarn workspace @sourcebot/web install
@ -132,6 +135,7 @@ COPY --from=shared-libs-builder /app/packages/schemas ./packages/schemas
COPY --from=shared-libs-builder /app/packages/crypto ./packages/crypto
COPY --from=shared-libs-builder /app/packages/error ./packages/error
COPY --from=shared-libs-builder /app/packages/logger ./packages/logger
COPY --from=shared-libs-builder /app/packages/shared ./packages/shared
RUN yarn workspace @sourcebot/backend install
RUN yarn workspace @sourcebot/backend build
@ -215,6 +219,7 @@ COPY --from=shared-libs-builder /app/packages/schemas ./packages/schemas
COPY --from=shared-libs-builder /app/packages/crypto ./packages/crypto
COPY --from=shared-libs-builder /app/packages/error ./packages/error
COPY --from=shared-libs-builder /app/packages/logger ./packages/logger
COPY --from=shared-libs-builder /app/packages/shared ./packages/shared
# Configure dependencies
RUN apk add --no-cache git ca-certificates bind-tools tini jansson wget supervisor uuidgen curl perl jq redis postgresql postgresql-contrib openssl util-linux unzip

2
LICENSE
View file

@ -2,7 +2,7 @@ Copyright (c) 2025 Taqla Inc.
Portions of this software are licensed as follows:
- All content that resides under the "ee/" and "packages/web/src/ee/" directories of this repository, if these directories exist, is licensed under the license defined in "ee/LICENSE".
- All content that resides under the "ee/", "packages/web/src/ee/", and "packages/shared/src/ee/" directories of this repository, if these directories exist, is licensed under the license defined in "ee/LICENSE".
- All third party components incorporated into the Sourcebot Software are licensed under the original license provided by the owner of the applicable component.
- Content outside of the above mentioned directories or restrictions above is available under the "MIT Expat" license as defined below.

2
Makefile
View file

@ -34,6 +34,8 @@ clean:
packages/error/dist \
packages/mcp/node_modules \
packages/mcp/dist \
packages/shared/node_modules \
packages/shared/dist \
.sourcebot
soft-reset:

2
package.json
View file

@ -16,7 +16,7 @@
"dev:prisma:migrate:dev": "yarn with-env yarn workspace @sourcebot/db prisma:migrate:dev",
"dev:prisma:studio": "yarn with-env yarn workspace @sourcebot/db prisma:studio",
"dev:prisma:migrate:reset": "yarn with-env yarn workspace @sourcebot/db prisma:migrate:reset",
"build:deps": "yarn workspaces foreach -R --from '{@sourcebot/schemas,@sourcebot/error,@sourcebot/crypto,@sourcebot/db}' run build"
"build:deps": "yarn workspaces foreach -R --from '{@sourcebot/schemas,@sourcebot/error,@sourcebot/crypto,@sourcebot/db,@sourcebot/shared}' run build"
},
"devDependencies": {
"cross-env": "^7.0.3",

3
packages/backend/package.json
View file

@ -33,9 +33,9 @@
"@sourcebot/error": "workspace:*",
"@sourcebot/logger": "workspace:*",
"@sourcebot/schemas": "workspace:*",
"@sourcebot/shared": "workspace:*",
"@t3-oss/env-core": "^0.12.0",
"@types/express": "^5.0.0",
"ajv": "^8.17.1",
"argparse": "^2.0.1",
"bullmq": "^5.34.10",
"cross-fetch": "^4.0.0",
@ -50,7 +50,6 @@
"posthog-node": "^4.2.1",
"prom-client": "^15.1.3",
"simple-git": "^3.27.0",
"strip-json-comments": "^5.0.1",
"zod": "^3.24.3"
}
}

23
packages/backend/src/connectionManager.ts
View file

@ -9,6 +9,7 @@ import { BackendError, BackendException } from "@sourcebot/error";
import { captureEvent } from "./posthog.js";
import { env } from "./env.js";
import * as Sentry from "@sentry/node";
import { loadConfig, syncSearchContexts } from "@sourcebot/shared";
interface IConnectionManager {
scheduleConnectionSync: (connection: Connection) => Promise<void>;
@ -264,7 +265,7 @@ export class ConnectionManager implements IConnectionManager {
private async onSyncJobCompleted(job: Job<JobPayload>, result: JobResult) {
this.logger.info(`Connection sync job for connection ${job.data.connectionName} (id: ${job.data.connectionId}, jobId: ${job.id}) completed`);
const { connectionId } = job.data;
const { connectionId, orgId } = job.data;
let syncStatusMetadata: Record<string, unknown> = (await this.db.connection.findUnique({
where: { id: connectionId },
@ -289,7 +290,25 @@ export class ConnectionManager implements IConnectionManager {
notFound.repos.length > 0 ? ConnectionSyncStatus.SYNCED_WITH_WARNINGS : ConnectionSyncStatus.SYNCED,
syncedAt: new Date()
}
})
});
// After a connection has synced, we need to re-sync the org's search contexts as
// there may be new repos that match the search context's include/exclude patterns.
if (env.CONFIG_PATH) {
try {
const config = await loadConfig(env.CONFIG_PATH);
await syncSearchContexts({
db: this.db,
orgId,
contexts: config.contexts,
});
} catch (err) {
this.logger.error(`Failed to sync search contexts for connection ${connectionId}: ${err}`);
Sentry.captureException(err);
}
}
captureEvent('backend_connection_sync_job_completed', {
connectionId: connectionId,

4
packages/backend/src/index.ts
View file

@ -10,7 +10,8 @@ import { PrismaClient } from "@sourcebot/db";
import { env } from "./env.js";
import { createLogger } from "@sourcebot/logger";
const logger = createLogger('index');
const logger = createLogger('backend-entrypoint');
// Register handler for normal exit
process.on('exit', (code) => {
@ -72,3 +73,4 @@ main(prisma, context)
.finally(() => {
logger.info("Shutting down...");
});

28
packages/backend/src/main.ts
View file

@ -7,40 +7,16 @@ import { ConnectionManager } from './connectionManager.js';
import { RepoManager } from './repoManager.js';
import { env } from './env.js';
import { PromClient } from './promClient.js';
import { isRemotePath } from './utils.js';
import { readFile } from 'fs/promises';
import stripJsonComments from 'strip-json-comments';
import { SourcebotConfig } from '@sourcebot/schemas/v3/index.type';
import { indexSchema } from '@sourcebot/schemas/v3/index.schema';
import { Ajv } from "ajv";
import { loadConfig } from '@sourcebot/shared';
const logger = createLogger('backend-main');
const ajv = new Ajv({
validateFormats: false,
});
const getSettings = async (configPath?: string) => {
if (!configPath) {
return DEFAULT_SETTINGS;
}
const configContent = await (async () => {
if (isRemotePath(configPath)) {
const response = await fetch(configPath);
if (!response.ok) {
throw new Error(`Failed to fetch config file ${configPath}: ${response.statusText}`);
}
return response.text();
} else {
return readFile(configPath, { encoding: 'utf-8' });
}
})();
const config = JSON.parse(stripJsonComments(configContent)) as SourcebotConfig;
const isValidConfig = ajv.validate(indexSchema, config);
if (!isValidConfig) {
throw new Error(`Config file '${configPath}' is invalid: ${ajv.errorsText(ajv.errors)}`);
}
const config = await loadConfig(configPath);
return {
...DEFAULT_SETTINGS,

3
packages/backend/src/utils.test.ts
View file

@ -1,5 +1,6 @@
import { expect, test } from 'vitest';
import { arraysEqualShallow, isRemotePath } from './utils';
import { arraysEqualShallow } from './utils';
import { isRemotePath } from '@sourcebot/shared';
test('should return true for identical arrays', () => {
expect(arraysEqualShallow([1, 2, 3], [1, 2, 3])).toBe(true);

4
packages/backend/src/utils.ts
View file

@ -20,10 +20,6 @@ export const marshalBool = (value?: boolean) => {
return !!value ? '1' : '0';
}
export const isRemotePath = (path: string) => {
return path.startsWith('https://') || path.startsWith('http://');
}
export const getTokenFromConfig = async (token: any, orgId: number, db: PrismaClient, logger?: Logger) => {
try {
return await getTokenFromConfigBase(token, orgId, db);

2
packages/shared/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
dist/
*.tsbuildinfo

9
packages/shared/README.md Normal file
View file

@ -0,0 +1,9 @@
This package contains shared code between the backend & webapp packages.
### Why two index files?
This package contains two index files: `index.server.ts` and `index.client.ts`. There is some code in this package that will only work in a Node.JS runtime (e.g., because it depends on the `fs` pacakge. Entitlements are a good example of this), and other code that is runtime agnostic (e.g., `constants.ts`). To deal with this, we these two index files export server code and client code, respectively.
For package consumers, the usage would look like the following:
- Server: `import { ... } from @sourcebot/shared`
- Client: `import { ... } from @sourcebot/shared/client`

32
packages/shared/package.json Normal file
View file

@ -0,0 +1,32 @@
{
"name": "@sourcebot/shared",
"version": "0.1.0",
"type": "module",
"private": true,
"scripts": {
"build": "tsc",
"build:watch": "tsc-watch --preserveWatchOutput",
"postinstall": "yarn build"
},
"dependencies": {
"@sourcebot/crypto": "workspace:*",
"@sourcebot/db": "workspace:*",
"@sourcebot/logger": "workspace:*",
"@sourcebot/schemas": "workspace:*",
"@t3-oss/env-core": "^0.12.0",
"ajv": "^8.17.1",
"micromatch": "^4.0.8",
"strip-json-comments": "^5.0.1",
"zod": "^3.24.3"
},
"devDependencies": {
"@types/micromatch": "^4.0.9",
"@types/node": "^22.7.5",
"tsc-watch": "6.2.1",
"typescript": "^5.7.3"
},
"exports": {
".": "./dist/index.server.js",
"./client": "./dist/index.client.js"
}
}

11
packages/shared/src/constants.ts Normal file
View file

@ -0,0 +1,11 @@
export const SOURCEBOT_SUPPORT_EMAIL = 'team@sourcebot.dev';
export const SOURCEBOT_CLOUD_ENVIRONMENT = [
"dev",
"demo",
"staging",
"prod",
] as const;
export const SOURCEBOT_UNLIMITED_SEATS = -1;

45
packages/web/src/ee/features/searchContexts/syncSearchContexts.ts → packages/shared/src/ee/syncSearchContexts.ts
View file

@ -1,31 +1,34 @@
import { env } from "@/env.mjs";
import { getPlan, hasEntitlement } from "@/features/entitlements/server";
import { SINGLE_TENANT_ORG_ID, SOURCEBOT_SUPPORT_EMAIL } from "@/lib/constants";
import { prisma } from "@/prisma";
import { SearchContext } from "@sourcebot/schemas/v3/index.type";
import micromatch from "micromatch";
import { createLogger } from "@sourcebot/logger";
import { PrismaClient } from "@sourcebot/db";
import { getPlan, hasEntitlement } from "../entitlements.js";
import { SOURCEBOT_SUPPORT_EMAIL } from "../constants.js";
import { SearchContext } from "@sourcebot/schemas/v3/index.type";
const logger = createLogger('sync-search-contexts');
export const syncSearchContexts = async (contexts?: { [key: string]: SearchContext }) => {
if (env.SOURCEBOT_TENANCY_MODE !== 'single') {
throw new Error("Search contexts are not supported in this tenancy mode. Set SOURCEBOT_TENANCY_MODE=single in your environment variables.");
interface SyncSearchContextsParams {
contexts?: { [key: string]: SearchContext } | undefined;
orgId: number;
db: PrismaClient;
}
export const syncSearchContexts = async (params: SyncSearchContextsParams) => {
const { contexts, orgId, db } = params;
if (!hasEntitlement("search-contexts")) {
if (contexts) {
const plan = getPlan();
logger.error(`Search contexts are not supported in your current plan: ${plan}. If you have a valid enterprise license key, pass it via SOURCEBOT_EE_LICENSE_KEY. For support, contact ${SOURCEBOT_SUPPORT_EMAIL}.`);
logger.warn(`Skipping search context sync. Reason: "Search contexts are not supported in your current plan: ${plan}. If you have a valid enterprise license key, pass it via SOURCEBOT_EE_LICENSE_KEY. For support, contact ${SOURCEBOT_SUPPORT_EMAIL}."`);
}
return;
return false;
}
if (contexts) {
for (const [key, newContextConfig] of Object.entries(contexts)) {
const allRepos = await prisma.repo.findMany({
const allRepos = await db.repo.findMany({
where: {
orgId: SINGLE_TENANT_ORG_ID,
orgId,
},
select: {
id: true,
@ -44,11 +47,11 @@ export const syncSearchContexts = async (contexts?: { [key: string]: SearchConte
});
}
const currentReposInContext = (await prisma.searchContext.findUnique({
const currentReposInContext = (await db.searchContext.findUnique({
where: {
name_orgId: {
name: key,
orgId: SINGLE_TENANT_ORG_ID,
orgId,
}
},
include: {
@ -56,11 +59,11 @@ export const syncSearchContexts = async (contexts?: { [key: string]: SearchConte
}
}))?.repos ?? [];
await prisma.searchContext.upsert({
await db.searchContext.upsert({
where: {
name_orgId: {
name: key,
orgId: SINGLE_TENANT_ORG_ID,
orgId,
}
},
update: {
@ -81,7 +84,7 @@ export const syncSearchContexts = async (contexts?: { [key: string]: SearchConte
description: newContextConfig.description,
org: {
connect: {
id: SINGLE_TENANT_ORG_ID,
id: orgId,
}
},
repos: {
@ -94,21 +97,23 @@ export const syncSearchContexts = async (contexts?: { [key: string]: SearchConte
}
}
const deletedContexts = await prisma.searchContext.findMany({
const deletedContexts = await db.searchContext.findMany({
where: {
name: {
notIn: Object.keys(contexts ?? {}),
},
orgId: SINGLE_TENANT_ORG_ID,
orgId,
}
});
for (const context of deletedContexts) {
logger.info(`Deleting search context with name '${context.name}'. ID: ${context.id}`);
await prisma.searchContext.delete({
await db.searchContext.delete({
where: {
id: context.id,
}
})
}
return true;
}

44
packages/web/src/features/entitlements/server.ts → packages/shared/src/entitlements.ts
View file

@ -1,15 +1,13 @@
import { env } from "@/env.mjs"
import { Entitlement, entitlementsByPlan, Plan } from "./constants"
import { base64Decode } from "@/lib/utils";
import { base64Decode } from "./utils.js";
import { z } from "zod";
import { SOURCEBOT_SUPPORT_EMAIL } from "@/lib/constants";
import { createLogger } from "@sourcebot/logger";
import { verifySignature } from "@sourcebot/crypto";
import { env } from "./env.js";
import { SOURCEBOT_SUPPORT_EMAIL, SOURCEBOT_UNLIMITED_SEATS } from "./constants.js";
const logger = createLogger('entitlements');
const eeLicenseKeyPrefix = "sourcebot_ee_";
export const SOURCEBOT_UNLIMITED_SEATS = -1;
const eeLicenseKeyPayloadSchema = z.object({
id: z.string(),
@ -21,13 +19,43 @@ const eeLicenseKeyPayloadSchema = z.object({
type LicenseKeyPayload = z.infer<typeof eeLicenseKeyPayloadSchema>;
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const planLabels = {
oss: "OSS",
"cloud:team": "Team",
"cloud:demo": "Demo",
"self-hosted:enterprise": "Enterprise (Self-Hosted)",
"self-hosted:enterprise-unlimited": "Enterprise (Self-Hosted) Unlimited",
} as const;
export type Plan = keyof typeof planLabels;
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const entitlements = [
"search-contexts",
"billing",
"public-access",
"multi-tenancy",
"sso",
"code-nav"
] as const;
export type Entitlement = (typeof entitlements)[number];
const entitlementsByPlan: Record<Plan, Entitlement[]> = {
oss: [],
"cloud:team": ["billing", "multi-tenancy", "sso", "code-nav"],
"self-hosted:enterprise": ["search-contexts", "sso", "code-nav"],
"self-hosted:enterprise-unlimited": ["search-contexts", "public-access", "sso", "code-nav"],
// Special entitlement for https://demo.sourcebot.dev
"cloud:demo": ["public-access", "code-nav", "search-contexts"],
} as const;
const decodeLicenseKeyPayload = (payload: string): LicenseKeyPayload => {
try {
const decodedPayload = base64Decode(payload);
const payloadJson = JSON.parse(decodedPayload);
const licenseData = eeLicenseKeyPayloadSchema.parse(payloadJson);
if (env.SOURCEBOT_PUBLIC_KEY_PATH) {
const dataToVerify = JSON.stringify({
expiryDate: licenseData.expiryDate,
id: licenseData.id,
@ -39,10 +67,6 @@ const decodeLicenseKeyPayload = (payload: string): LicenseKeyPayload => {
logger.error('License key signature verification failed');
process.exit(1);
}
} else {
logger.error('No public key path provided, unable to verify license key signature');
process.exit(1);
}
return licenseData;
} catch (error) {

21
packages/shared/src/env.ts Normal file
View file

@ -0,0 +1,21 @@
import { createEnv } from "@t3-oss/env-core";
import { z } from "zod";
import { SOURCEBOT_CLOUD_ENVIRONMENT } from "./constants.js";
export const env = createEnv({
server: {
SOURCEBOT_EE_LICENSE_KEY: z.string().optional(),
SOURCEBOT_PUBLIC_KEY_PATH: z.string(),
},
client: {
NEXT_PUBLIC_SOURCEBOT_CLOUD_ENVIRONMENT: z.enum(SOURCEBOT_CLOUD_ENVIRONMENT).optional(),
},
clientPrefix: "NEXT_PUBLIC_",
runtimeEnvStrict: {
SOURCEBOT_EE_LICENSE_KEY: process.env.SOURCEBOT_EE_LICENSE_KEY,
SOURCEBOT_PUBLIC_KEY_PATH: process.env.SOURCEBOT_PUBLIC_KEY_PATH,
NEXT_PUBLIC_SOURCEBOT_CLOUD_ENVIRONMENT: process.env.NEXT_PUBLIC_SOURCEBOT_CLOUD_ENVIRONMENT,
},
emptyStringAsUndefined: true,
skipValidation: process.env.SKIP_ENV_VALIDATION === "1",
});

2
packages/shared/src/index.client.ts Normal file
View file

@ -0,0 +1,2 @@
export * from "./constants.js";

20
packages/shared/src/index.server.ts Normal file
View file

@ -0,0 +1,20 @@
export {
hasEntitlement,
getLicenseKey,
getPlan,
getSeats,
getEntitlements,
} from "./entitlements.js";
export type {
Plan,
Entitlement,
} from "./entitlements.js";
export {
base64Decode,
loadConfig,
isRemotePath,
} from "./utils.js";
export {
syncSearchContexts,
} from "./ee/syncSearchContexts.js";
export * from "./constants.js";

42
packages/shared/src/utils.ts Normal file
View file

@ -0,0 +1,42 @@
import { SourcebotConfig } from "@sourcebot/schemas/v3/index.type";
import { indexSchema } from "@sourcebot/schemas/v3/index.schema";
import { readFile } from 'fs/promises';
import stripJsonComments from 'strip-json-comments';
import { Ajv } from "ajv";
const ajv = new Ajv({
validateFormats: false,
});
// From https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem
export const base64Decode = (base64: string): string => {
const binString = atob(base64);
return Buffer.from(Uint8Array.from(binString, (m) => m.codePointAt(0)!).buffer).toString();
}
export const isRemotePath = (path: string) => {
return path.startsWith('https://') || path.startsWith('http://');
}
export const loadConfig = async (configPath: string): Promise<SourcebotConfig> => {
const configContent = await (async () => {
if (isRemotePath(configPath)) {
const response = await fetch(configPath);
if (!response.ok) {
throw new Error(`Failed to fetch config file ${configPath}: ${response.statusText}`);
}
return response.text();
} else {
return readFile(configPath, {
encoding: 'utf-8',
});
}
})();
const config = JSON.parse(stripJsonComments(configContent)) as SourcebotConfig;
const isValidConfig = ajv.validate(indexSchema, config);
if (!isValidConfig) {
throw new Error(`Config file '${configPath}' is invalid: ${ajv.errorsText(ajv.errors)}`);
}
return config;
}

23
packages/shared/tsconfig.json Normal file
View file

@ -0,0 +1,23 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "Node16",
"moduleResolution": "Node16",
"lib": ["ES2023"],
"outDir": "dist",
"rootDir": "src",
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"skipLibCheck": true,
"isolatedModules": true,
"resolveJsonModule": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}

1
packages/web/package.json
View file

@ -76,6 +76,7 @@
"@sourcebot/error": "workspace:*",
"@sourcebot/logger": "workspace:*",
"@sourcebot/schemas": "workspace:*",
"@sourcebot/shared": "workspace:*",
"@ssddanbrown/codemirror-lang-twig": "^1.0.0",
"@stripe/react-stripe-js": "^3.1.1",
"@stripe/stripe-js": "^5.6.0",

3
packages/web/src/actions.ts
View file

@ -31,8 +31,7 @@ import { TenancyMode, ApiKeyPayload } from "./lib/types";
import { decrementOrgSeatCount, getSubscriptionForOrg, incrementOrgSeatCount } from "./ee/features/billing/serverUtils";
import { bitbucketSchema } from "@sourcebot/schemas/v3/bitbucket.schema";
import { genericGitHostSchema } from "@sourcebot/schemas/v3/genericGitHost.schema";
import { getPlan, getSeats, SOURCEBOT_UNLIMITED_SEATS } from "./features/entitlements/server";
import { hasEntitlement } from "./features/entitlements/server";
import { getPlan, getSeats, hasEntitlement, SOURCEBOT_UNLIMITED_SEATS } from "@sourcebot/shared";
import { getPublicAccessStatus } from "./ee/features/publicAccess/publicAccess";
import JoinRequestSubmittedEmail from "./emails/joinRequestSubmittedEmail";
import JoinRequestApprovedEmail from "./emails/joinRequestApprovedEmail";

2
packages/web/src/app/[domain]/layout.tsx
View file

@ -14,7 +14,7 @@ import { IS_BILLING_ENABLED } from "@/ee/features/billing/stripe";
import { notFound, redirect } from "next/navigation";
import { getSubscriptionInfo } from "@/ee/features/billing/actions";
import { PendingApprovalCard } from "./components/pendingApproval";
import { hasEntitlement } from "@/features/entitlements/server";
import { hasEntitlement } from "@sourcebot/shared";
import { getPublicAccessStatus } from "@/ee/features/publicAccess/publicAccess";
import { env } from "@/env.mjs";
import { GcpIapAuth } from "./components/gcpIapAuth";

8
packages/web/src/app/[domain]/settings/license/page.tsx
View file

@ -1,4 +1,4 @@
import { getEntitlements, getLicenseKey, getPlan, SOURCEBOT_UNLIMITED_SEATS } from "@/features/entitlements/server";
import { getLicenseKey, getEntitlements, getPlan, SOURCEBOT_UNLIMITED_SEATS } from "@sourcebot/shared";
import { Button } from "@/components/ui/button";
import { Info, Mail } from "lucide-react";
import { getOrgMembers } from "@/actions";
@ -17,9 +17,9 @@ export default async function LicensePage({ params: { domain } }: LicensePagePro
notFound();
}
const licenseKey = await getLicenseKey();
const entitlements = await getEntitlements();
const plan = await getPlan();
const licenseKey = getLicenseKey();
const entitlements = getEntitlements();
const plan = getPlan();
if (!licenseKey) {
return (

2
packages/web/src/app/[domain]/settings/members/page.tsx
View file

@ -9,7 +9,7 @@ import { InvitesList } from "./components/invitesList";
import { getOrgInvites, getMe, getOrgAccountRequests } from "@/actions";
import { IS_BILLING_ENABLED } from "@/ee/features/billing/stripe";
import { ServiceErrorException } from "@/lib/serviceError";
import { getSeats, SOURCEBOT_UNLIMITED_SEATS } from "@/features/entitlements/server";
import { getSeats, SOURCEBOT_UNLIMITED_SEATS } from "@sourcebot/shared";
import { RequestsList } from "./components/requestsList";
import { OrgRole } from "@prisma/client";

2
packages/web/src/app/layout.tsx
View file

@ -8,7 +8,7 @@ import { TooltipProvider } from "@/components/ui/tooltip";
import { SessionProvider } from "next-auth/react";
import { env } from "@/env.mjs";
import { PlanProvider } from "@/features/entitlements/planProvider";
import { getEntitlements } from "@/features/entitlements/server";
import { getEntitlements } from "@sourcebot/shared";
export const metadata: Metadata = {
title: "Sourcebot",

2
packages/web/src/auth.ts
View file

@ -14,7 +14,7 @@ import { render } from '@react-email/render';
import MagicLinkEmail from './emails/magicLinkEmail';
import bcrypt from 'bcryptjs';
import { getSSOProviders } from '@/ee/sso/sso';
import { hasEntitlement } from '@/features/entitlements/server';
import { hasEntitlement } from '@sourcebot/shared';
import { onCreateUser } from '@/lib/authUtils';
export const runtime = 'nodejs';

2
packages/web/src/ee/features/billing/stripe.ts
View file

@ -1,7 +1,7 @@
import 'server-only';
import { env } from '@/env.mjs'
import Stripe from "stripe";
import { hasEntitlement } from '@/features/entitlements/server';
import { hasEntitlement } from '@sourcebot/shared';
export const IS_BILLING_ENABLED = hasEntitlement('billing') && env.STRIPE_SECRET_KEY !== undefined;

2
packages/web/src/ee/features/publicAccess/publicAccess.tsx
View file

@ -7,7 +7,7 @@ import { ErrorCode } from "@/lib/errorCodes";
import { StatusCodes } from "http-status-codes";
import { prisma } from "@/prisma";
import { sew } from "@/actions";
import { getPlan, hasEntitlement } from "@/features/entitlements/server";
import { getPlan, hasEntitlement } from "@sourcebot/shared";
import { SOURCEBOT_GUEST_USER_EMAIL, SOURCEBOT_GUEST_USER_ID, SOURCEBOT_SUPPORT_EMAIL } from "@/lib/constants";
import { OrgRole } from "@sourcebot/db";

4
packages/web/src/ee/sso/sso.tsx
View file

@ -9,7 +9,7 @@ import MicrosoftEntraID from "next-auth/providers/microsoft-entra-id";
import { prisma } from "@/prisma";
import { notFound, ServiceError } from "@/lib/serviceError";
import { OrgRole } from "@sourcebot/db";
import { getSeats, SOURCEBOT_UNLIMITED_SEATS } from "@/features/entitlements/server";
import { getSeats, SOURCEBOT_UNLIMITED_SEATS } from "@sourcebot/shared";
import { StatusCodes } from "http-status-codes";
import { ErrorCode } from "@/lib/errorCodes";
import { OAuth2Client } from "google-auth-library";
@ -216,7 +216,7 @@ export const handleJITProvisioning = async (userId: string, domain: string): Pro
return true;
}
const seats = await getSeats();
const seats = getSeats();
const memberCount = org.members.length;
if (seats != SOURCEBOT_UNLIMITED_SEATS && memberCount >= seats) {

3
packages/web/src/env.mjs
View file

@ -1,5 +1,6 @@
import { createEnv } from "@t3-oss/env-nextjs";
import { z } from "zod";
import { SOURCEBOT_CLOUD_ENVIRONMENT } from "@sourcebot/shared/client";
// Booleans are specified as 'true' or 'false' strings.
const booleanSchema = z.enum(["true", "false"]);
@ -107,7 +108,7 @@ export const env = createEnv({
NEXT_PUBLIC_SOURCEBOT_VERSION: z.string().default('unknown'),
NEXT_PUBLIC_POLLING_INTERVAL_MS: numberSchema.default(5000),
NEXT_PUBLIC_SOURCEBOT_CLOUD_ENVIRONMENT: z.enum(["dev", "demo", "staging", "prod"]).optional(),
NEXT_PUBLIC_SOURCEBOT_CLOUD_ENVIRONMENT: z.enum(SOURCEBOT_CLOUD_ENVIRONMENT).optional(),
},
// For Next.js >= 13.4.4, you only need to destructure client variables:
experimental__runtimeEnv: {

31
packages/web/src/features/entitlements/constants.ts
View file

@ -1,31 +0,0 @@
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const planLabels = {
oss: "OSS",
"cloud:team": "Team",
"cloud:demo": "Demo",
"self-hosted:enterprise": "Enterprise (Self-Hosted)",
"self-hosted:enterprise-unlimited": "Enterprise (Self-Hosted) Unlimited",
} as const;
export type Plan = keyof typeof planLabels;
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const entitlements = [
"search-contexts",
"billing",
"public-access",
"multi-tenancy",
"sso",
"code-nav"
] as const;
export type Entitlement = (typeof entitlements)[number];
export const entitlementsByPlan: Record<Plan, Entitlement[]> = {
oss: [],
"cloud:team": ["billing", "multi-tenancy", "sso", "code-nav"],
"self-hosted:enterprise": ["search-contexts", "sso", "code-nav"],
"self-hosted:enterprise-unlimited": ["search-contexts", "public-access", "sso", "code-nav"],
// Special entitlement for https://demo.sourcebot.dev
"cloud:demo": ["public-access", "code-nav", "search-contexts"],
} as const;

2
packages/web/src/features/entitlements/planProvider.tsx
View file

@ -1,7 +1,7 @@
'use client';
import { createContext } from "react";
import { Entitlement } from "./constants";
import { Entitlement } from "@sourcebot/shared";
export const PlanContext = createContext<{ entitlements: Entitlement[] }>({ entitlements: [] });

2
packages/web/src/features/entitlements/useHasEntitlement.ts
View file

@ -1,6 +1,6 @@
'use client';
import { Entitlement } from "./constants";
import { Entitlement } from "@sourcebot/shared";
import { useContext } from "react";
import { PlanContext } from "./planProvider";

49
packages/web/src/initialize.ts
View file

@ -1,16 +1,10 @@
import { ConnectionSyncStatus, OrgRole, Prisma, RepoIndexingStatus } from '@sourcebot/db';
import { env } from './env.mjs';
import { prisma } from "@/prisma";
import { SINGLE_TENANT_ORG_ID, SINGLE_TENANT_ORG_DOMAIN, SINGLE_TENANT_ORG_NAME, SOURCEBOT_GUEST_USER_ID } from './lib/constants';
import { readFile } from 'fs/promises';
import { SINGLE_TENANT_ORG_ID, SINGLE_TENANT_ORG_DOMAIN, SOURCEBOT_GUEST_USER_ID, SINGLE_TENANT_ORG_NAME } from './lib/constants';
import { watch } from 'fs';
import stripJsonComments from 'strip-json-comments';
import { SourcebotConfig } from "@sourcebot/schemas/v3/index.type";
import { ConnectionConfig } from '@sourcebot/schemas/v3/connection.type';
import { indexSchema } from '@sourcebot/schemas/v3/index.schema';
import Ajv from 'ajv';
import { syncSearchContexts } from '@/ee/features/searchContexts/syncSearchContexts';
import { hasEntitlement } from '@/features/entitlements/server';
import { hasEntitlement, loadConfig, isRemotePath, syncSearchContexts } from '@sourcebot/shared';
import { createGuestUser, setPublicAccessStatus } from '@/ee/features/publicAccess/publicAccess';
import { isServiceError } from './lib/utils';
import { ServiceErrorException } from './lib/serviceError';
@ -19,14 +13,6 @@ import { createLogger } from "@sourcebot/logger";
const logger = createLogger('web-initialize');
const ajv = new Ajv({
validateFormats: false,
});
const isRemotePath = (path: string) => {
return path.startsWith('https://') || path.startsWith('http://');
}
const syncConnections = async (connections?: { [key: string]: ConnectionConfig }) => {
if (connections) {
for (const [key, newConnectionConfig] of Object.entries(connections)) {
@ -116,31 +102,8 @@ const syncConnections = async (connections?: { [key: string]: ConnectionConfig }
}
}
const readConfig = async (configPath: string): Promise<SourcebotConfig> => {
const configContent = await (async () => {
if (isRemotePath(configPath)) {
const response = await fetch(configPath);
if (!response.ok) {
throw new Error(`Failed to fetch config file ${configPath}: ${response.statusText}`);
}
return response.text();
} else {
return readFile(configPath, {
encoding: 'utf-8',
});
}
})();
const config = JSON.parse(stripJsonComments(configContent)) as SourcebotConfig;
const isValidConfig = ajv.validate(indexSchema, config);
if (!isValidConfig) {
throw new Error(`Config file '${configPath}' is invalid: ${ajv.errorsText(ajv.errors)}`);
}
return config;
}
const syncDeclarativeConfig = async (configPath: string) => {
const config = await readConfig(configPath);
const config = await loadConfig(configPath);
const hasPublicAccessEntitlement = hasEntitlement("public-access");
const enablePublicAccess = config.settings?.enablePublicAccess;
@ -158,7 +121,11 @@ const syncDeclarativeConfig = async (configPath: string) => {
}
await syncConnections(config.connections);
await syncSearchContexts(config.contexts);
await syncSearchContexts({
contexts: config.contexts,
orgId: SINGLE_TENANT_ORG_ID,
db: prisma,
});
}
const pruneOldGuestUser = async () => {

2
packages/web/src/lib/authUtils.ts
View file

@ -3,7 +3,7 @@ import { env } from "@/env.mjs";
import { prisma } from "@/prisma";
import { OrgRole } from "@sourcebot/db";
import { SINGLE_TENANT_ORG_DOMAIN, SINGLE_TENANT_ORG_ID } from "@/lib/constants";
import { hasEntitlement } from "@/features/entitlements/server";
import { hasEntitlement } from "@sourcebot/shared";
import { isServiceError } from "@/lib/utils";
import { ServiceErrorException } from "@/lib/serviceError";
import { createAccountRequest } from "@/actions";

2
packages/web/src/lib/constants.ts
View file

@ -32,4 +32,4 @@ export const SINGLE_TENANT_ORG_ID = 1;
export const SINGLE_TENANT_ORG_DOMAIN = '~';
export const SINGLE_TENANT_ORG_NAME = 'default';
export const SOURCEBOT_SUPPORT_EMAIL = 'team@sourcebot.dev';
export { SOURCEBOT_SUPPORT_EMAIL } from "@sourcebot/shared/client";

24
yarn.lock
View file

@ -5769,12 +5769,12 @@ __metadata:
"@sourcebot/error": "workspace:*"
"@sourcebot/logger": "workspace:*"
"@sourcebot/schemas": "workspace:*"
"@sourcebot/shared": "workspace:*"
"@t3-oss/env-core": "npm:^0.12.0"
"@types/argparse": "npm:^2.0.16"
"@types/express": "npm:^5.0.0"
"@types/micromatch": "npm:^4.0.9"
"@types/node": "npm:^22.7.5"
ajv: "npm:^8.17.1"
argparse: "npm:^2.0.1"
bullmq: "npm:^5.34.10"
cross-env: "npm:^7.0.3"
@ -5791,7 +5791,6 @@ __metadata:
posthog-node: "npm:^4.2.1"
prom-client: "npm:^15.1.3"
simple-git: "npm:^3.27.0"
strip-json-comments: "npm:^5.0.1"
tsc-watch: "npm:^6.2.0"
tsx: "npm:^4.19.1"
typescript: "npm:^5.6.2"
@ -5885,6 +5884,26 @@ __metadata:
languageName: unknown
linkType: soft
"@sourcebot/shared@workspace:*, @sourcebot/shared@workspace:packages/shared":
version: 0.0.0-use.local
resolution: "@sourcebot/shared@workspace:packages/shared"
dependencies:
"@sourcebot/crypto": "workspace:*"
"@sourcebot/db": "workspace:*"
"@sourcebot/logger": "workspace:*"
"@sourcebot/schemas": "workspace:*"
"@t3-oss/env-core": "npm:^0.12.0"
"@types/micromatch": "npm:^4.0.9"
"@types/node": "npm:^22.7.5"
ajv: "npm:^8.17.1"
micromatch: "npm:^4.0.8"
strip-json-comments: "npm:^5.0.1"
tsc-watch: "npm:6.2.1"
typescript: "npm:^5.7.3"
zod: "npm:^3.24.3"
languageName: unknown
linkType: soft
"@sourcebot/web@workspace:packages/web":
version: 0.0.0-use.local
resolution: "@sourcebot/web@workspace:packages/web"
@ -5953,6 +5972,7 @@ __metadata:
"@sourcebot/error": "workspace:*"
"@sourcebot/logger": "workspace:*"
"@sourcebot/schemas": "workspace:*"
"@sourcebot/shared": "workspace:*"
"@ssddanbrown/codemirror-lang-twig": "npm:^1.0.0"
"@stripe/react-stripe-js": "npm:^3.1.1"
"@stripe/stripe-js": "npm:^5.6.0"