fix: return truncated content when token limit exceeded in MCP search_code (#604)

When search results exceed maxTokens limit, now returns partial truncated
content instead of discarding the file completely.

Changes:
- Calculate remaining token budget before breaking
- Truncate file content to fit within remaining tokens (if > 100 tokens left)
- Append truncation marker to indicate content was cut off
- Still add truncation message at end of all results

Benefits:
- Users get partial data instead of nothing
- Better debugging and analysis experience
- More useful for AI-powered code analysis tasks
- Consistent with expected behavior when limits are reached

Example: If file would use 10K tokens but only 2K remain, return
first ~8K chars of content + truncation marker instead of dropping it.

Signed-off-by: Wayne Sun <gsun@redhat.com>

packages/mcp/CHANGELOG.md

@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+- Fixed issue where search results exceeding token limits would be completely discarded instead of returning truncated content. [#604](https://github.com/sourcebot-dev/sourcebot/pull/604)
+
 ## [1.0.7] - 2025-10-28
 - Updated API client to match the latest Sourcebot release. [#555](https://github.com/sourcebot-dev/sourcebot/pull/555)
 

packages/mcp/src/index.ts

@@ -123,6 +123,22 @@ server.tool(
             const tokens = text.length / 4;
 
             if ((totalTokens + tokens) > maxTokens) {
+                // Calculate remaining token budget
+                const remainingTokens = maxTokens - totalTokens;
+
+                if (remainingTokens > 100) {  // Only truncate if meaningful space left
+                    // Truncate text to fit remaining tokens (tokens ≈ chars/4)
+                    const maxLength = Math.floor(remainingTokens * 4);
+                    const truncatedText = text.substring(0, maxLength) + "\n\n...[content truncated due to token limit]";
+
+                    content.push({
+                        type: "text",
+                        text: truncatedText,
+                    });
+
+                    totalTokens += remainingTokens;
+                }
+
                 isResponseTruncated = true;
                 break;
             }