diff --git a/CHANGELOG.md b/CHANGELOG.md index a11c2848ee..91da40b9e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,67 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.6.5] - 2025-04-14 + +### Added + +- 🛂 **Granular Voice Feature Permissions Per User Group**: Admins can now separately manage access to Speech-to-Text (record voice), Text-to-Speech (read aloud), and Tool Calls for each user group—giving teams tighter control over voice features and enhanced governance across roles. +- 🗣️ **Toggle Voice Activity Detection (VAD) for Whisper STT**: New environment variable lets you enable/disable VAD filtering with built-in Whisper speech-to-text, giving you flexibility to optimize for different audio quality and response accuracy levels. +- 📋 **Copy Formatted Response Mode**: You can now enable “Copy Formatted” in Settings > Interface to copy AI responses exactly as styled (with rich formatting, links, and structure preserved), making it faster and cleaner to paste into documents, emails, or reports. +- ⚙️ **Backend Stability and Performance Enhancements**: General backend refactoring improves system resilience, consistency, and overall reliability—offering smoother performance across workflows whether chatting, generating media, or using external tools. +- 🌎 **Translation Refinements Across Multiple Languages**: Updated translations deliver smoother language localization, clearer labels, and improved international usability throughout the UI—ensuring a better experience for non-English speakers. + +### Fixed + +- 🛠️ **LDAP Login Reliability Restored**: Resolved a critical issue where some LDAP setups failed due to attribute parsing—ensuring consistent, secure, and seamless user authentication across enterprise deployments. +- 🖼️ **Image Generation in Temporary Chats Now Works Properly**: Fixed a bug where image outputs weren’t generated during temporary chats—visual content can now be used reliably in all chat modes without interruptions. + +## [0.6.4] - 2025-04-12 + +### Fixed + +- 🛠️ **RAG_TEMPLATE Display Issue Resolved**: Fixed a formatting problem where the custom RAG_TEMPLATE wasn't correctly rendered in the interface—ensuring that custom retrieval prompts now appear exactly as intended for more reliable prompt engineering. + +## [0.6.3] - 2025-04-12 + +### Added + +- 🧪 **Auto-Artifact Detection Toggle**: Automatically detects artifacts in results—but now you can disable this behavior under advanced settings for full control. +- 🖼️ **Widescreen Mode for Shared Chats**: Shared link conversations now support widescreen layouts—perfect for presentations or easier review across wider displays. +- 🔁 **Reindex Knowledge Files on Demand**: Admins can now trigger reindexing of all knowledge files after changing embeddings—ensuring immediate alignment with new models for optimal RAG performance. +- 📄 **OpenAPI YAML Format Support**: External tools can now use YAML-format OpenAPI specs—making integration simpler for developers familiar with YAML-based configurations. +- 💬 **Message Content Copy Behavior**: Copy action now excludes 'details' tags—streamlining clipboard content when sharing or pasting summaries elsewhere. +- 🧭 **Sougou Web Search Integration**: New search engine option added—enhancing global relevance and diversity of search sources for multilingual users. +- 🧰 **Frontend Web Loader Engine Configuration**: Admins can now set preferred web loader engine for RAG workflows directly from the frontend—offering more control across setups. +- 👥 **Multi-Model Chat Permission Control**: Admins can manage access to multi-model chats per user group—allowing tighter governance in team environments. +- 🧱 **Persistent Configuration Can Be Disabled**: New environment variable lets advanced users and hosts turn off persistent configs—ideal for volatile or stateless deployments. +- 🧠 **Elixir Code Highlighting Support**: Elixir syntax is now beautifully rendered in code blocks—perfect for developers using this language in AI or automation projects. +- 🌐 **PWA External Manifest URL Support**: You can now define an external manifest.json—integrate Open WebUI seamlessly in managed or proxy-based PWA environments like Cloudflare Zero Trust. +- 🧪 **Azure AI Speech-to-Text Provider Integration**: Easily transcribe large audio files (up to 200MB) with high accuracy using Microsoft's Azure STT—fully configurable in Audio Settings. +- 🔏 **PKCE (Code Challenge Method) Support for OIDC**: Enhance your OIDC login security with Proof Key for Code Exchange—ideal for zero-trust and native client apps. +- ✨ **General UI/UX Enhancements**: Numerous refinements across layout, styling, and tool interactions—reducing visual noise and improving overall usability across key workflows. +- 🌍 **Translation Updates Across Multiple Languages**: Refined Catalan, Russian, Chinese (Simplified & Traditional), Hungarian, and Spanish translations for clearer navigation and instructions globally. + +### Fixed + +- 💥 **Chat Completion Error with Missing Models Resolved**: Fixed internal server error when referencing a model that doesn’t exist—ensuring graceful fallback and clear error guidance. +- 🔧 **Correct Knowledge Base Citations Restored**: Citations generated by RAG workflows now show accurate references—ensuring verifiability in outputs from sourced content. +- 🎙️ **Broken OGG/WebM Audio Upload Handling for OpenAI Fixed**: Uploading OGG or WebM files now converts properly to WAV before transcription—restoring accurate AI speech recognition workflows. +- 🔐 **Tool Server 'Session' Authentication Restored**: Previously broken session auth on external tool servers is now fully functional—ensuring secure and seamless access to connected tools. +- 🌐 **Folder-Based Chat Rename Now Updates Correctly**: Renaming chats in folders now reflects instantly everywhere—improving chat organization and clarity. +- 📜 **KaTeX Overflow Displays Fixed**: Math expressions now stay neatly within message bounds—preserving layout consistency even with long formulas. +- 🚫 **Stopping Ongoing Chat Fixed**: You can now return to an active (ongoing) chat and stop generation at any time—ensuring full control over sessions. +- 🔧 **TOOL_SERVERS / TOOL_SERVER_CONNECTIONS Indexing Issue Fixed**: Fixed a mismatch between tool lists and their access paths—restoring full function and preventing confusion in tool management. +- 🔐 **LDAP Login Handles Multiple Emails**: When LDAP returns multiple email attributes, the first valid one is now used—ensuring login success and account consistency. +- 🧩 **Model Visibility Toggle Fix**: Toggling model visibility now works even for untouched models—letting admins smoothly manage user access across base models. +- ⚙️ **Cross-Origin manifest.json Now Loads Properly**: Compatibility issues with Cloudflare Zero Trust (and others) resolved, allowing manifest.json to load behind authenticated proxies. + +### Changed + +- 🔒 **Default Access Scopes Set to Private for All Resources**: Models, tools, and knowledge are now private by default when created—ensuring better baseline security and visibility controls. +- 🧱 **General Backend Refactoring for Stability**: Numerous invisible improvements enhance backend scalability, security, and maintainability—powering upcoming features with a stronger foundation. +- 🧩 **Stable Dependency Upgrades**: Updated key platform libraries—Chromadb (0.6.3), pgvector (0.4.0), Azure Identity (1.21.0), and Youtube Transcript API (1.0.3)—for improved compatibility, functionality, and security. + ## [0.6.2] - 2025-04-06 ### Added diff --git a/backend/open_webui/__init__.py b/backend/open_webui/__init__.py index 0c70cb63a9..ff386957c4 100644 --- a/backend/open_webui/__init__.py +++ b/backend/open_webui/__init__.py @@ -76,11 +76,11 @@ def serve( from open_webui.env import UVICORN_WORKERS # Import the workers setting uvicorn.run( - open_webui.main.app, - host=host, - port=port, + open_webui.main.app, + host=host, + port=port, forwarded_allow_ips="*", - workers=UVICORN_WORKERS + workers=UVICORN_WORKERS, ) diff --git a/backend/open_webui/config.py b/backend/open_webui/config.py index 9f5395154c..3b40977f27 100644 --- a/backend/open_webui/config.py +++ b/backend/open_webui/config.py @@ -201,7 +201,10 @@ def save_config(config): T = TypeVar("T") -ENABLE_PERSISTENT_CONFIG = os.environ.get("ENABLE_PERSISTENT_CONFIG", "True").lower() == "true" +ENABLE_PERSISTENT_CONFIG = ( + os.environ.get("ENABLE_PERSISTENT_CONFIG", "True").lower() == "true" +) + class PersistentConfig(Generic[T]): def __init__(self, env_name: str, config_path: str, env_value: T): @@ -612,10 +615,16 @@ def load_oauth_providers(): "scope": OAUTH_SCOPES.value, } - if OAUTH_CODE_CHALLENGE_METHOD.value and OAUTH_CODE_CHALLENGE_METHOD.value == "S256": + if ( + OAUTH_CODE_CHALLENGE_METHOD.value + and OAUTH_CODE_CHALLENGE_METHOD.value == "S256" + ): client_kwargs["code_challenge_method"] = "S256" elif OAUTH_CODE_CHALLENGE_METHOD.value: - raise Exception('Code challenge methods other than "%s" not supported. Given: "%s"' % ("S256", OAUTH_CODE_CHALLENGE_METHOD.value)) + raise Exception( + 'Code challenge methods other than "%s" not supported. Given: "%s"' + % ("S256", OAUTH_CODE_CHALLENGE_METHOD.value) + ) client.register( name="oidc", @@ -1053,6 +1062,22 @@ USER_PERMISSIONS_CHAT_EDIT = ( os.environ.get("USER_PERMISSIONS_CHAT_EDIT", "True").lower() == "true" ) +USER_PERMISSIONS_CHAT_STT = ( + os.environ.get("USER_PERMISSIONS_CHAT_STT", "True").lower() == "true" +) + +USER_PERMISSIONS_CHAT_TTS = ( + os.environ.get("USER_PERMISSIONS_CHAT_TTS", "True").lower() == "true" +) + +USER_PERMISSIONS_CHAT_CALL = ( + os.environ.get("USER_PERMISSIONS_CHAT_CALL", "True").lower() == "true" +) + +USER_PERMISSIONS_CHAT_MULTIPLE_MODELS = ( + os.environ.get("USER_PERMISSIONS_CHAT_MULTIPLE_MODELS", "True").lower() == "true" +) + USER_PERMISSIONS_CHAT_TEMPORARY = ( os.environ.get("USER_PERMISSIONS_CHAT_TEMPORARY", "True").lower() == "true" ) @@ -1062,6 +1087,7 @@ USER_PERMISSIONS_CHAT_TEMPORARY_ENFORCED = ( == "true" ) + USER_PERMISSIONS_FEATURES_DIRECT_TOOL_SERVERS = ( os.environ.get("USER_PERMISSIONS_FEATURES_DIRECT_TOOL_SERVERS", "False").lower() == "true" @@ -1100,6 +1126,10 @@ DEFAULT_USER_PERMISSIONS = { "file_upload": USER_PERMISSIONS_CHAT_FILE_UPLOAD, "delete": USER_PERMISSIONS_CHAT_DELETE, "edit": USER_PERMISSIONS_CHAT_EDIT, + "stt": USER_PERMISSIONS_CHAT_STT, + "tts": USER_PERMISSIONS_CHAT_TTS, + "call": USER_PERMISSIONS_CHAT_CALL, + "multiple_models": USER_PERMISSIONS_CHAT_MULTIPLE_MODELS, "temporary": USER_PERMISSIONS_CHAT_TEMPORARY, "temporary_enforced": USER_PERMISSIONS_CHAT_TEMPORARY_ENFORCED, }, @@ -1820,12 +1850,6 @@ RAG_FILE_MAX_SIZE = PersistentConfig( ), ) -ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION = PersistentConfig( - "ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION", - "rag.enable_web_loader_ssl_verification", - os.environ.get("ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION", "True").lower() == "true", -) - RAG_EMBEDDING_ENGINE = PersistentConfig( "RAG_EMBEDDING_ENGINE", "rag.embedding_engine", @@ -1990,16 +2014,20 @@ YOUTUBE_LOADER_PROXY_URL = PersistentConfig( ) -ENABLE_RAG_WEB_SEARCH = PersistentConfig( - "ENABLE_RAG_WEB_SEARCH", +#################################### +# Web Search (RAG) +#################################### + +ENABLE_WEB_SEARCH = PersistentConfig( + "ENABLE_WEB_SEARCH", "rag.web.search.enable", - os.getenv("ENABLE_RAG_WEB_SEARCH", "False").lower() == "true", + os.getenv("ENABLE_WEB_SEARCH", "False").lower() == "true", ) -RAG_WEB_SEARCH_ENGINE = PersistentConfig( - "RAG_WEB_SEARCH_ENGINE", +WEB_SEARCH_ENGINE = PersistentConfig( + "WEB_SEARCH_ENGINE", "rag.web.search.engine", - os.getenv("RAG_WEB_SEARCH_ENGINE", ""), + os.getenv("WEB_SEARCH_ENGINE", ""), ) BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL = PersistentConfig( @@ -2008,10 +2036,18 @@ BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL = PersistentConfig( os.getenv("BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL", "False").lower() == "true", ) + +WEB_SEARCH_RESULT_COUNT = PersistentConfig( + "WEB_SEARCH_RESULT_COUNT", + "rag.web.search.result_count", + int(os.getenv("WEB_SEARCH_RESULT_COUNT", "3")), +) + + # You can provide a list of your own websites to filter after performing a web search. # This ensures the highest level of safety and reliability of the information sources. -RAG_WEB_SEARCH_DOMAIN_FILTER_LIST = PersistentConfig( - "RAG_WEB_SEARCH_DOMAIN_FILTER_LIST", +WEB_SEARCH_DOMAIN_FILTER_LIST = PersistentConfig( + "WEB_SEARCH_DOMAIN_FILTER_LIST", "rag.web.search.domain.filter_list", [ # "wikipedia.com", @@ -2020,6 +2056,30 @@ RAG_WEB_SEARCH_DOMAIN_FILTER_LIST = PersistentConfig( ], ) +WEB_SEARCH_CONCURRENT_REQUESTS = PersistentConfig( + "WEB_SEARCH_CONCURRENT_REQUESTS", + "rag.web.search.concurrent_requests", + int(os.getenv("WEB_SEARCH_CONCURRENT_REQUESTS", "10")), +) + +WEB_LOADER_ENGINE = PersistentConfig( + "WEB_LOADER_ENGINE", + "rag.web.loader.engine", + os.environ.get("WEB_LOADER_ENGINE", ""), +) + +ENABLE_WEB_LOADER_SSL_VERIFICATION = PersistentConfig( + "ENABLE_WEB_LOADER_SSL_VERIFICATION", + "rag.web.loader.ssl_verification", + os.environ.get("ENABLE_WEB_LOADER_SSL_VERIFICATION", "True").lower() == "true", +) + +WEB_SEARCH_TRUST_ENV = PersistentConfig( + "WEB_SEARCH_TRUST_ENV", + "rag.web.search.trust_env", + os.getenv("WEB_SEARCH_TRUST_ENV", "False").lower() == "true", +) + SEARXNG_QUERY_URL = PersistentConfig( "SEARXNG_QUERY_URL", @@ -2087,18 +2147,6 @@ SERPLY_API_KEY = PersistentConfig( os.getenv("SERPLY_API_KEY", ""), ) -TAVILY_API_KEY = PersistentConfig( - "TAVILY_API_KEY", - "rag.web.search.tavily_api_key", - os.getenv("TAVILY_API_KEY", ""), -) - -TAVILY_EXTRACT_DEPTH = PersistentConfig( - "TAVILY_EXTRACT_DEPTH", - "rag.web.search.tavily_extract_depth", - os.getenv("TAVILY_EXTRACT_DEPTH", "basic"), -) - JINA_API_KEY = PersistentConfig( "JINA_API_KEY", "rag.web.search.jina_api_key", @@ -2167,54 +2215,43 @@ SOUGOU_API_SK = PersistentConfig( os.getenv("SOUGOU_API_SK", ""), ) -RAG_WEB_SEARCH_RESULT_COUNT = PersistentConfig( - "RAG_WEB_SEARCH_RESULT_COUNT", - "rag.web.search.result_count", - int(os.getenv("RAG_WEB_SEARCH_RESULT_COUNT", "3")), +TAVILY_API_KEY = PersistentConfig( + "TAVILY_API_KEY", + "rag.web.search.tavily_api_key", + os.getenv("TAVILY_API_KEY", ""), ) -RAG_WEB_SEARCH_CONCURRENT_REQUESTS = PersistentConfig( - "RAG_WEB_SEARCH_CONCURRENT_REQUESTS", - "rag.web.search.concurrent_requests", - int(os.getenv("RAG_WEB_SEARCH_CONCURRENT_REQUESTS", "10")), +TAVILY_EXTRACT_DEPTH = PersistentConfig( + "TAVILY_EXTRACT_DEPTH", + "rag.web.search.tavily_extract_depth", + os.getenv("TAVILY_EXTRACT_DEPTH", "basic"), ) -RAG_WEB_LOADER_ENGINE = PersistentConfig( - "RAG_WEB_LOADER_ENGINE", - "rag.web.loader.engine", - os.environ.get("RAG_WEB_LOADER_ENGINE", "safe_web"), -) - -RAG_WEB_SEARCH_TRUST_ENV = PersistentConfig( - "RAG_WEB_SEARCH_TRUST_ENV", - "rag.web.search.trust_env", - os.getenv("RAG_WEB_SEARCH_TRUST_ENV", "False").lower() == "true", -) - -PLAYWRIGHT_WS_URI = PersistentConfig( - "PLAYWRIGHT_WS_URI", - "rag.web.loader.engine.playwright.ws.uri", - os.environ.get("PLAYWRIGHT_WS_URI", None), +PLAYWRIGHT_WS_URL = PersistentConfig( + "PLAYWRIGHT_WS_URL", + "rag.web.loader.playwright_ws_url", + os.environ.get("PLAYWRIGHT_WS_URL", ""), ) PLAYWRIGHT_TIMEOUT = PersistentConfig( "PLAYWRIGHT_TIMEOUT", - "rag.web.loader.engine.playwright.timeout", - int(os.environ.get("PLAYWRIGHT_TIMEOUT", "10")), + "rag.web.loader.playwright_timeout", + int(os.environ.get("PLAYWRIGHT_TIMEOUT", "10000")), ) FIRECRAWL_API_KEY = PersistentConfig( "FIRECRAWL_API_KEY", - "firecrawl.api_key", + "rag.web.loader.firecrawl_api_key", os.environ.get("FIRECRAWL_API_KEY", ""), ) FIRECRAWL_API_BASE_URL = PersistentConfig( "FIRECRAWL_API_BASE_URL", - "firecrawl.api_url", + "rag.web.loader.firecrawl_api_url", os.environ.get("FIRECRAWL_API_BASE_URL", "https://api.firecrawl.dev"), ) + #################################### # Images #################################### @@ -2467,6 +2504,13 @@ WHISPER_MODEL_AUTO_UPDATE = ( and os.environ.get("WHISPER_MODEL_AUTO_UPDATE", "").lower() == "true" ) +WHISPER_VAD_FILTER = PersistentConfig( + "WHISPER_VAD_FILTER", + "audio.stt.whisper_vad_filter", + os.getenv("WHISPER_VAD_FILTER", "False").lower() == "true", +) + + # Add Deepgram configuration DEEPGRAM_API_KEY = PersistentConfig( "DEEPGRAM_API_KEY", @@ -2474,6 +2518,7 @@ DEEPGRAM_API_KEY = PersistentConfig( os.getenv("DEEPGRAM_API_KEY", ""), ) + AUDIO_STT_OPENAI_API_BASE_URL = PersistentConfig( "AUDIO_STT_OPENAI_API_BASE_URL", "audio.stt.openai.api_base_url", diff --git a/backend/open_webui/env.py b/backend/open_webui/env.py index dda239dce4..fff8bd9a97 100644 --- a/backend/open_webui/env.py +++ b/backend/open_webui/env.py @@ -498,4 +498,4 @@ PIP_PACKAGE_INDEX_OPTIONS = os.getenv("PIP_PACKAGE_INDEX_OPTIONS", "").split() # PROGRESSIVE WEB APP OPTIONS #################################### -EXTERNAL_PWA_MANIFEST_URL = os.environ.get("EXTERNAL_PWA_MANIFEST_URL") \ No newline at end of file +EXTERNAL_PWA_MANIFEST_URL = os.environ.get("EXTERNAL_PWA_MANIFEST_URL") diff --git a/backend/open_webui/main.py b/backend/open_webui/main.py index 01af453583..afac86c6e8 100644 --- a/backend/open_webui/main.py +++ b/backend/open_webui/main.py @@ -160,12 +160,13 @@ from open_webui.config import ( AUDIO_TTS_VOICE, AUDIO_TTS_AZURE_SPEECH_REGION, AUDIO_TTS_AZURE_SPEECH_OUTPUT_FORMAT, - PLAYWRIGHT_WS_URI, + PLAYWRIGHT_WS_URL, PLAYWRIGHT_TIMEOUT, FIRECRAWL_API_BASE_URL, FIRECRAWL_API_KEY, - RAG_WEB_LOADER_ENGINE, + WEB_LOADER_ENGINE, WHISPER_MODEL, + WHISPER_VAD_FILTER, DEEPGRAM_API_KEY, WHISPER_MODEL_AUTO_UPDATE, WHISPER_MODEL_DIR, @@ -205,12 +206,13 @@ from open_webui.config import ( YOUTUBE_LOADER_LANGUAGE, YOUTUBE_LOADER_PROXY_URL, # Retrieval (Web Search) - RAG_WEB_SEARCH_ENGINE, + ENABLE_WEB_SEARCH, + WEB_SEARCH_ENGINE, BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL, - RAG_WEB_SEARCH_RESULT_COUNT, - RAG_WEB_SEARCH_CONCURRENT_REQUESTS, - RAG_WEB_SEARCH_TRUST_ENV, - RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + WEB_SEARCH_RESULT_COUNT, + WEB_SEARCH_CONCURRENT_REQUESTS, + WEB_SEARCH_TRUST_ENV, + WEB_SEARCH_DOMAIN_FILTER_LIST, JINA_API_KEY, SEARCHAPI_API_KEY, SEARCHAPI_ENGINE, @@ -240,8 +242,7 @@ from open_webui.config import ( ONEDRIVE_CLIENT_ID, ENABLE_RAG_HYBRID_SEARCH, ENABLE_RAG_LOCAL_WEB_FETCH, - ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION, - ENABLE_RAG_WEB_SEARCH, + ENABLE_WEB_LOADER_SSL_VERIFICATION, ENABLE_GOOGLE_DRIVE_INTEGRATION, ENABLE_ONEDRIVE_INTEGRATION, UPLOAD_DIR, @@ -373,7 +374,11 @@ from open_webui.utils.auth import ( from open_webui.utils.oauth import OAuthManager from open_webui.utils.security_headers import SecurityHeadersMiddleware -from open_webui.tasks import stop_task, list_tasks # Import from tasks.py +from open_webui.tasks import ( + list_task_ids_by_chat_id, + stop_task, + list_tasks, +) # Import from tasks.py from open_webui.utils.redis import get_sentinels_from_env @@ -596,9 +601,7 @@ app.state.config.FILE_MAX_COUNT = RAG_FILE_MAX_COUNT app.state.config.RAG_FULL_CONTEXT = RAG_FULL_CONTEXT app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL = BYPASS_EMBEDDING_AND_RETRIEVAL app.state.config.ENABLE_RAG_HYBRID_SEARCH = ENABLE_RAG_HYBRID_SEARCH -app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION = ( - ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION -) +app.state.config.ENABLE_WEB_LOADER_SSL_VERIFICATION = ENABLE_WEB_LOADER_SSL_VERIFICATION app.state.config.CONTENT_EXTRACTION_ENGINE = CONTENT_EXTRACTION_ENGINE app.state.config.TIKA_SERVER_URL = TIKA_SERVER_URL @@ -631,12 +634,16 @@ app.state.config.YOUTUBE_LOADER_LANGUAGE = YOUTUBE_LOADER_LANGUAGE app.state.config.YOUTUBE_LOADER_PROXY_URL = YOUTUBE_LOADER_PROXY_URL -app.state.config.ENABLE_RAG_WEB_SEARCH = ENABLE_RAG_WEB_SEARCH -app.state.config.RAG_WEB_SEARCH_ENGINE = RAG_WEB_SEARCH_ENGINE +app.state.config.ENABLE_WEB_SEARCH = ENABLE_WEB_SEARCH +app.state.config.WEB_SEARCH_ENGINE = WEB_SEARCH_ENGINE +app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST = WEB_SEARCH_DOMAIN_FILTER_LIST +app.state.config.WEB_SEARCH_RESULT_COUNT = WEB_SEARCH_RESULT_COUNT +app.state.config.WEB_SEARCH_CONCURRENT_REQUESTS = WEB_SEARCH_CONCURRENT_REQUESTS +app.state.config.WEB_LOADER_ENGINE = WEB_LOADER_ENGINE +app.state.config.WEB_SEARCH_TRUST_ENV = WEB_SEARCH_TRUST_ENV app.state.config.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL = ( BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL ) -app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST = RAG_WEB_SEARCH_DOMAIN_FILTER_LIST app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION = ENABLE_GOOGLE_DRIVE_INTEGRATION app.state.config.ENABLE_ONEDRIVE_INTEGRATION = ENABLE_ONEDRIVE_INTEGRATION @@ -664,11 +671,8 @@ app.state.config.PERPLEXITY_API_KEY = PERPLEXITY_API_KEY app.state.config.SOUGOU_API_SID = SOUGOU_API_SID app.state.config.SOUGOU_API_SK = SOUGOU_API_SK -app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = RAG_WEB_SEARCH_RESULT_COUNT -app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = RAG_WEB_SEARCH_CONCURRENT_REQUESTS -app.state.config.RAG_WEB_LOADER_ENGINE = RAG_WEB_LOADER_ENGINE -app.state.config.RAG_WEB_SEARCH_TRUST_ENV = RAG_WEB_SEARCH_TRUST_ENV -app.state.config.PLAYWRIGHT_WS_URI = PLAYWRIGHT_WS_URI + +app.state.config.PLAYWRIGHT_WS_URL = PLAYWRIGHT_WS_URL app.state.config.PLAYWRIGHT_TIMEOUT = PLAYWRIGHT_TIMEOUT app.state.config.FIRECRAWL_API_BASE_URL = FIRECRAWL_API_BASE_URL app.state.config.FIRECRAWL_API_KEY = FIRECRAWL_API_KEY @@ -788,6 +792,7 @@ app.state.config.STT_ENGINE = AUDIO_STT_ENGINE app.state.config.STT_MODEL = AUDIO_STT_MODEL app.state.config.WHISPER_MODEL = WHISPER_MODEL +app.state.config.WHISPER_VAD_FILTER = WHISPER_VAD_FILTER app.state.config.DEEPGRAM_API_KEY = DEEPGRAM_API_KEY app.state.config.AUDIO_STT_AZURE_API_KEY = AUDIO_STT_AZURE_API_KEY @@ -1022,14 +1027,19 @@ async def get_models(request: Request, user=Depends(get_verified_user)): if "pipeline" in model and model["pipeline"].get("type", None) == "filter": continue - model_tags = [ - tag.get("name") - for tag in model.get("info", {}).get("meta", {}).get("tags", []) - ] - tags = [tag.get("name") for tag in model.get("tags", [])] + try: + model_tags = [ + tag.get("name") + for tag in model.get("info", {}).get("meta", {}).get("tags", []) + ] + tags = [tag.get("name") for tag in model.get("tags", [])] - tags = list(set(model_tags + tags)) - model["tags"] = [{"name": tag} for tag in tags] + tags = list(set(model_tags + tags)) + model["tags"] = [{"name": tag} for tag in tags] + except Exception as e: + log.debug(f"Error processing model tags: {e}") + model["tags"] = [] + pass models.append(model) @@ -1199,7 +1209,7 @@ async def chat_action( @app.post("/api/tasks/stop/{task_id}") async def stop_task_endpoint(task_id: str, user=Depends(get_verified_user)): try: - result = await stop_task(task_id) # Use the function from tasks.py + result = await stop_task(task_id) return result except ValueError as e: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(e)) @@ -1207,7 +1217,19 @@ async def stop_task_endpoint(task_id: str, user=Depends(get_verified_user)): @app.get("/api/tasks") async def list_tasks_endpoint(user=Depends(get_verified_user)): - return {"tasks": list_tasks()} # Use the function from tasks.py + return {"tasks": list_tasks()} + + +@app.get("/api/tasks/chat/{chat_id}") +async def list_tasks_by_chat_id_endpoint(chat_id: str, user=Depends(get_verified_user)): + chat = Chats.get_chat_by_id(chat_id) + if chat is None or chat.user_id != user.id: + return {"task_ids": []} + + task_ids = list_task_ids_by_chat_id(chat_id) + + print(f"Task IDs for chat {chat_id}: {task_ids}") + return {"task_ids": task_ids} ################################## @@ -1263,7 +1285,7 @@ async def get_app_config(request: Request): { "enable_direct_connections": app.state.config.ENABLE_DIRECT_CONNECTIONS, "enable_channels": app.state.config.ENABLE_CHANNELS, - "enable_web_search": app.state.config.ENABLE_RAG_WEB_SEARCH, + "enable_web_search": app.state.config.ENABLE_WEB_SEARCH, "enable_code_execution": app.state.config.ENABLE_CODE_EXECUTION, "enable_code_interpreter": app.state.config.ENABLE_CODE_INTERPRETER, "enable_image_generation": app.state.config.ENABLE_IMAGE_GENERATION, diff --git a/backend/open_webui/models/memories.py b/backend/open_webui/models/memories.py index c8dae97267..8b10a77cf9 100644 --- a/backend/open_webui/models/memories.py +++ b/backend/open_webui/models/memories.py @@ -63,14 +63,15 @@ class MemoriesTable: else: return None - def update_memory_by_id( + def update_memory_by_id_and_user_id( self, id: str, + user_id: str, content: str, ) -> Optional[MemoryModel]: with get_db() as db: try: - db.query(Memory).filter_by(id=id).update( + db.query(Memory).filter_by(id=id, user_id=user_id).update( {"content": content, "updated_at": int(time.time())} ) db.commit() diff --git a/backend/open_webui/retrieval/utils.py b/backend/open_webui/retrieval/utils.py index a00e6982cf..2b23cad17f 100644 --- a/backend/open_webui/retrieval/utils.py +++ b/backend/open_webui/retrieval/utils.py @@ -297,7 +297,9 @@ def query_collection_with_hybrid_search( collection_results = {} for collection_name in collection_names: try: - log.debug(f"query_collection_with_hybrid_search:VECTOR_DB_CLIENT.get:collection {collection_name}") + log.debug( + f"query_collection_with_hybrid_search:VECTOR_DB_CLIENT.get:collection {collection_name}" + ) collection_results[collection_name] = VECTOR_DB_CLIENT.get( collection_name=collection_name ) @@ -619,7 +621,9 @@ def generate_openai_batch_embeddings( user: UserModel = None, ) -> Optional[list[list[float]]]: try: - log.debug(f"generate_openai_batch_embeddings:model {model} batch size: {len(texts)}") + log.debug( + f"generate_openai_batch_embeddings:model {model} batch size: {len(texts)}" + ) json_data = {"input": texts, "model": model} if isinstance(RAG_EMBEDDING_PREFIX_FIELD_NAME, str) and isinstance(prefix, str): json_data[RAG_EMBEDDING_PREFIX_FIELD_NAME] = prefix @@ -662,7 +666,9 @@ def generate_ollama_batch_embeddings( user: UserModel = None, ) -> Optional[list[list[float]]]: try: - log.debug(f"generate_ollama_batch_embeddings:model {model} batch size: {len(texts)}") + log.debug( + f"generate_ollama_batch_embeddings:model {model} batch size: {len(texts)}" + ) json_data = {"input": texts, "model": model} if isinstance(RAG_EMBEDDING_PREFIX_FIELD_NAME, str) and isinstance(prefix, str): json_data[RAG_EMBEDDING_PREFIX_FIELD_NAME] = prefix diff --git a/backend/open_webui/retrieval/web/utils.py b/backend/open_webui/retrieval/web/utils.py index 942cb8483f..718cfe52fa 100644 --- a/backend/open_webui/retrieval/web/utils.py +++ b/backend/open_webui/retrieval/web/utils.py @@ -28,9 +28,9 @@ from open_webui.retrieval.loaders.tavily import TavilyLoader from open_webui.constants import ERROR_MESSAGES from open_webui.config import ( ENABLE_RAG_LOCAL_WEB_FETCH, - PLAYWRIGHT_WS_URI, + PLAYWRIGHT_WS_URL, PLAYWRIGHT_TIMEOUT, - RAG_WEB_LOADER_ENGINE, + WEB_LOADER_ENGINE, FIRECRAWL_API_BASE_URL, FIRECRAWL_API_KEY, TAVILY_API_KEY, @@ -584,13 +584,6 @@ class SafeWebBaseLoader(WebBaseLoader): return [document async for document in self.alazy_load()] -RAG_WEB_LOADER_ENGINES = defaultdict(lambda: SafeWebBaseLoader) -RAG_WEB_LOADER_ENGINES["playwright"] = SafePlaywrightURLLoader -RAG_WEB_LOADER_ENGINES["safe_web"] = SafeWebBaseLoader -RAG_WEB_LOADER_ENGINES["firecrawl"] = SafeFireCrawlLoader -RAG_WEB_LOADER_ENGINES["tavily"] = SafeTavilyLoader - - def get_web_loader( urls: Union[str, Sequence[str]], verify_ssl: bool = True, @@ -608,27 +601,36 @@ def get_web_loader( "trust_env": trust_env, } - if RAG_WEB_LOADER_ENGINE.value == "playwright": + if WEB_LOADER_ENGINE.value == "" or WEB_LOADER_ENGINE.value == "safe_web": + WebLoaderClass = SafeWebBaseLoader + if WEB_LOADER_ENGINE.value == "playwright": + WebLoaderClass = SafePlaywrightURLLoader web_loader_args["playwright_timeout"] = PLAYWRIGHT_TIMEOUT.value * 1000 - if PLAYWRIGHT_WS_URI.value: - web_loader_args["playwright_ws_url"] = PLAYWRIGHT_WS_URI.value + if PLAYWRIGHT_WS_URL.value: + web_loader_args["playwright_ws_url"] = PLAYWRIGHT_WS_URL.value - if RAG_WEB_LOADER_ENGINE.value == "firecrawl": + if WEB_LOADER_ENGINE.value == "firecrawl": + WebLoaderClass = SafeFireCrawlLoader web_loader_args["api_key"] = FIRECRAWL_API_KEY.value web_loader_args["api_url"] = FIRECRAWL_API_BASE_URL.value - if RAG_WEB_LOADER_ENGINE.value == "tavily": + if WEB_LOADER_ENGINE.value == "tavily": + WebLoaderClass = SafeTavilyLoader web_loader_args["api_key"] = TAVILY_API_KEY.value web_loader_args["extract_depth"] = TAVILY_EXTRACT_DEPTH.value - # Create the appropriate WebLoader based on the configuration - WebLoaderClass = RAG_WEB_LOADER_ENGINES[RAG_WEB_LOADER_ENGINE.value] - web_loader = WebLoaderClass(**web_loader_args) + if WebLoaderClass: + web_loader = WebLoaderClass(**web_loader_args) - log.debug( - "Using RAG_WEB_LOADER_ENGINE %s for %s URLs", - web_loader.__class__.__name__, - len(safe_urls), - ) + log.debug( + "Using WEB_LOADER_ENGINE %s for %s URLs", + web_loader.__class__.__name__, + len(safe_urls), + ) - return web_loader + return web_loader + else: + raise ValueError( + f"Invalid WEB_LOADER_ENGINE: {WEB_LOADER_ENGINE.value}. " + "Please set it to 'safe_web', 'playwright', 'firecrawl', or 'tavily'." + ) diff --git a/backend/open_webui/routers/audio.py b/backend/open_webui/routers/audio.py index 3fa066acf4..da51d1ecfc 100644 --- a/backend/open_webui/routers/audio.py +++ b/backend/open_webui/routers/audio.py @@ -330,7 +330,7 @@ async def speech(request: Request, user=Depends(get_verified_user)): detail = f"External: {e}" raise HTTPException( - status_code=getattr(r, "status", 500), + status_code=getattr(r, "status", 500) if r else 500, detail=detail if detail else "Open WebUI: Server Connection Error", ) @@ -384,7 +384,7 @@ async def speech(request: Request, user=Depends(get_verified_user)): detail = f"External: {e}" raise HTTPException( - status_code=getattr(r, "status", 500), + status_code=getattr(r, "status", 500) if r else 500, detail=detail if detail else "Open WebUI: Server Connection Error", ) @@ -440,7 +440,7 @@ async def speech(request: Request, user=Depends(get_verified_user)): detail = f"External: {e}" raise HTTPException( - status_code=getattr(r, "status", 500), + status_code=getattr(r, "status", 500) if r else 500, detail=detail if detail else "Open WebUI: Server Connection Error", ) @@ -497,7 +497,11 @@ def transcribe(request: Request, file_path): ) model = request.app.state.faster_whisper_model - segments, info = model.transcribe(file_path, beam_size=5) + segments, info = model.transcribe( + file_path, + beam_size=5, + vad_filter=request.app.state.config.WHISPER_VAD_FILTER, + ) log.info( "Detected language '%s' with probability %f" % (info.language, info.language_probability) @@ -624,10 +628,7 @@ def transcribe(request: Request, file_path): elif request.app.state.config.STT_ENGINE == "azure": # Check file exists and size if not os.path.exists(file_path): - raise HTTPException( - status_code=400, - detail="Audio file not found" - ) + raise HTTPException(status_code=400, detail="Audio file not found") # Check file size (Azure has a larger limit of 200MB) file_size = os.path.getsize(file_path) @@ -643,11 +644,22 @@ def transcribe(request: Request, file_path): # IF NO LOCALES, USE DEFAULTS if len(locales) < 2: - locales = ['en-US', 'es-ES', 'es-MX', 'fr-FR', 'hi-IN', - 'it-IT','de-DE', 'en-GB', 'en-IN', 'ja-JP', - 'ko-KR', 'pt-BR', 'zh-CN'] - locales = ','.join(locales) - + locales = [ + "en-US", + "es-ES", + "es-MX", + "fr-FR", + "hi-IN", + "it-IT", + "de-DE", + "en-GB", + "en-IN", + "ja-JP", + "ko-KR", + "pt-BR", + "zh-CN", + ] + locales = ",".join(locales) if not api_key or not region: raise HTTPException( @@ -658,22 +670,26 @@ def transcribe(request: Request, file_path): r = None try: # Prepare the request - data = {'definition': json.dumps({ - "locales": locales.split(','), - "diarization": {"maxSpeakers": 3,"enabled": True} - } if locales else {} - ) + data = { + "definition": json.dumps( + { + "locales": locales.split(","), + "diarization": {"maxSpeakers": 3, "enabled": True}, + } + if locales + else {} + ) } url = f"https://{region}.api.cognitive.microsoft.com/speechtotext/transcriptions:transcribe?api-version=2024-11-15" - + # Use context manager to ensure file is properly closed - with open(file_path, 'rb') as audio_file: + with open(file_path, "rb") as audio_file: r = requests.post( url=url, - files={'audio': audio_file}, + files={"audio": audio_file}, data=data, headers={ - 'Ocp-Apim-Subscription-Key': api_key, + "Ocp-Apim-Subscription-Key": api_key, }, ) @@ -681,11 +697,11 @@ def transcribe(request: Request, file_path): response = r.json() # Extract transcript from response - if not response.get('combinedPhrases'): + if not response.get("combinedPhrases"): raise ValueError("No transcription found in response") # Get the full transcript from combinedPhrases - transcript = response['combinedPhrases'][0].get('text', '').strip() + transcript = response["combinedPhrases"][0].get("text", "").strip() if not transcript: raise ValueError("Empty transcript in response") @@ -718,7 +734,7 @@ def transcribe(request: Request, file_path): detail = f"External: {e}" raise HTTPException( - status_code=getattr(r, 'status_code', 500) if r else 500, + status_code=getattr(r, "status_code", 500) if r else 500, detail=detail if detail else "Open WebUI: Server Connection Error", ) diff --git a/backend/open_webui/routers/auths.py b/backend/open_webui/routers/auths.py index 5c91fce29d..7f42488354 100644 --- a/backend/open_webui/routers/auths.py +++ b/backend/open_webui/routers/auths.py @@ -231,13 +231,15 @@ async def ldap_auth(request: Request, response: Response, form_data: LdapForm): entry = connection_app.entries[0] username = str(entry[f"{LDAP_ATTRIBUTE_FOR_USERNAME}"]).lower() - email = entry[f"{LDAP_ATTRIBUTE_FOR_MAIL}"] + email = entry[f"{LDAP_ATTRIBUTE_FOR_MAIL}"].value # retrive the Attribute value if not email: raise HTTPException(400, "User does not have a valid email address.") elif isinstance(email, str): email = email.lower() elif isinstance(email, list): email = email[0].lower() + else: + email = str(email).lower() cn = str(entry["cn"]) user_dn = entry.entry_dn diff --git a/backend/open_webui/routers/chats.py b/backend/open_webui/routers/chats.py index 74bb96c947..5fd44ab9f0 100644 --- a/backend/open_webui/routers/chats.py +++ b/backend/open_webui/routers/chats.py @@ -579,7 +579,12 @@ async def clone_chat_by_id( @router.post("/{id}/clone/shared", response_model=Optional[ChatResponse]) async def clone_shared_chat_by_id(id: str, user=Depends(get_verified_user)): - chat = Chats.get_chat_by_share_id(id) + + if user.role == "admin": + chat = Chats.get_chat_by_id(id) + else: + chat = Chats.get_chat_by_share_id(id) + if chat: updated_chat = { **chat.chat, diff --git a/backend/open_webui/routers/knowledge.py b/backend/open_webui/routers/knowledge.py index ab745cf840..15547afa7c 100644 --- a/backend/open_webui/routers/knowledge.py +++ b/backend/open_webui/routers/knowledge.py @@ -159,7 +159,6 @@ async def create_new_knowledge( status_code=status.HTTP_400_BAD_REQUEST, detail=ERROR_MESSAGES.FILE_EXISTS, ) - ############################ @@ -168,20 +167,17 @@ async def create_new_knowledge( @router.post("/reindex", response_model=bool) -async def reindex_knowledge_files( - request: Request, - user=Depends(get_verified_user) -): +async def reindex_knowledge_files(request: Request, user=Depends(get_verified_user)): if user.role != "admin": raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.UNAUTHORIZED, ) - + knowledge_bases = Knowledges.get_knowledge_bases() - + log.info(f"Starting reindexing for {len(knowledge_bases)} knowledge bases") - + for knowledge_base in knowledge_bases: try: files = Files.get_files_by_ids(knowledge_base.data.get("file_ids", [])) @@ -195,34 +191,40 @@ async def reindex_knowledge_files( log.error(f"Error deleting collection {knowledge_base.id}: {str(e)}") raise HTTPException( status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, - detail=f"Error deleting vector DB collection" + detail=f"Error deleting vector DB collection", ) - + failed_files = [] for file in files: try: process_file( request, - ProcessFileForm(file_id=file.id, collection_name=knowledge_base.id), + ProcessFileForm( + file_id=file.id, collection_name=knowledge_base.id + ), user=user, ) except Exception as e: - log.error(f"Error processing file {file.filename} (ID: {file.id}): {str(e)}") + log.error( + f"Error processing file {file.filename} (ID: {file.id}): {str(e)}" + ) failed_files.append({"file_id": file.id, "error": str(e)}) continue - + except Exception as e: log.error(f"Error processing knowledge base {knowledge_base.id}: {str(e)}") raise HTTPException( status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, - detail=f"Error processing knowledge base" + detail=f"Error processing knowledge base", ) - + if failed_files: - log.warning(f"Failed to process {len(failed_files)} files in knowledge base {knowledge_base.id}") + log.warning( + f"Failed to process {len(failed_files)} files in knowledge base {knowledge_base.id}" + ) for failed in failed_files: log.warning(f"File ID: {failed['file_id']}, Error: {failed['error']}") - + log.info("Reindexing completed successfully") return True @@ -742,6 +744,3 @@ def add_files_to_knowledge_batch( return KnowledgeFilesResponse( **knowledge.model_dump(), files=Files.get_files_by_ids(existing_file_ids) ) - - - diff --git a/backend/open_webui/routers/memories.py b/backend/open_webui/routers/memories.py index e660ef852b..6d54c9c170 100644 --- a/backend/open_webui/routers/memories.py +++ b/backend/open_webui/routers/memories.py @@ -153,7 +153,9 @@ async def update_memory_by_id( form_data: MemoryUpdateModel, user=Depends(get_verified_user), ): - memory = Memories.update_memory_by_id(memory_id, form_data.content) + memory = Memories.update_memory_by_id_and_user_id( + memory_id, user.id, form_data.content + ) if memory is None: raise HTTPException(status_code=404, detail="Memory not found") diff --git a/backend/open_webui/routers/retrieval.py b/backend/open_webui/routers/retrieval.py index 8e1708c65b..13f012483d 100644 --- a/backend/open_webui/routers/retrieval.py +++ b/backend/open_webui/routers/retrieval.py @@ -352,439 +352,436 @@ async def update_reranking_config( async def get_rag_config(request: Request, user=Depends(get_admin_user)): return { "status": True, - "pdf_extract_images": request.app.state.config.PDF_EXTRACT_IMAGES, - "RAG_FULL_CONTEXT": request.app.state.config.RAG_FULL_CONTEXT, + # RAG settings + "RAG_TEMPLATE": request.app.state.config.RAG_TEMPLATE, + "TOP_K": request.app.state.config.TOP_K, "BYPASS_EMBEDDING_AND_RETRIEVAL": request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL, - "enable_google_drive_integration": request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION, - "enable_onedrive_integration": request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION, - "content_extraction": { - "engine": request.app.state.config.CONTENT_EXTRACTION_ENGINE, - "tika_server_url": request.app.state.config.TIKA_SERVER_URL, - "docling_server_url": request.app.state.config.DOCLING_SERVER_URL, - "document_intelligence_config": { - "endpoint": request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT, - "key": request.app.state.config.DOCUMENT_INTELLIGENCE_KEY, - }, - "mistral_ocr_config": { - "api_key": request.app.state.config.MISTRAL_OCR_API_KEY, - }, - }, - "chunk": { - "text_splitter": request.app.state.config.TEXT_SPLITTER, - "chunk_size": request.app.state.config.CHUNK_SIZE, - "chunk_overlap": request.app.state.config.CHUNK_OVERLAP, - }, - "file": { - "max_size": request.app.state.config.FILE_MAX_SIZE, - "max_count": request.app.state.config.FILE_MAX_COUNT, - }, - "youtube": { - "language": request.app.state.config.YOUTUBE_LOADER_LANGUAGE, - "translation": request.app.state.YOUTUBE_LOADER_TRANSLATION, - "proxy_url": request.app.state.config.YOUTUBE_LOADER_PROXY_URL, - }, + "RAG_FULL_CONTEXT": request.app.state.config.RAG_FULL_CONTEXT, + # Hybrid search settings + "ENABLE_RAG_HYBRID_SEARCH": request.app.state.config.ENABLE_RAG_HYBRID_SEARCH, + "TOP_K_RERANKER": request.app.state.config.TOP_K_RERANKER, + "RELEVANCE_THRESHOLD": request.app.state.config.RELEVANCE_THRESHOLD, + # Content extraction settings + "CONTENT_EXTRACTION_ENGINE": request.app.state.config.CONTENT_EXTRACTION_ENGINE, + "PDF_EXTRACT_IMAGES": request.app.state.config.PDF_EXTRACT_IMAGES, + "TIKA_SERVER_URL": request.app.state.config.TIKA_SERVER_URL, + "DOCLING_SERVER_URL": request.app.state.config.DOCLING_SERVER_URL, + "DOCUMENT_INTELLIGENCE_ENDPOINT": request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT, + "DOCUMENT_INTELLIGENCE_KEY": request.app.state.config.DOCUMENT_INTELLIGENCE_KEY, + "MISTRAL_OCR_API_KEY": request.app.state.config.MISTRAL_OCR_API_KEY, + # Chunking settings + "TEXT_SPLITTER": request.app.state.config.TEXT_SPLITTER, + "CHUNK_SIZE": request.app.state.config.CHUNK_SIZE, + "CHUNK_OVERLAP": request.app.state.config.CHUNK_OVERLAP, + # File upload settings + "FILE_MAX_SIZE": request.app.state.config.FILE_MAX_SIZE, + "FILE_MAX_COUNT": request.app.state.config.FILE_MAX_COUNT, + # Integration settings + "ENABLE_GOOGLE_DRIVE_INTEGRATION": request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION, + "ENABLE_ONEDRIVE_INTEGRATION": request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION, + # Web search settings "web": { - "ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION": request.app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION, + "ENABLE_WEB_SEARCH": request.app.state.config.ENABLE_WEB_SEARCH, + "WEB_SEARCH_ENGINE": request.app.state.config.WEB_SEARCH_ENGINE, + "WEB_SEARCH_TRUST_ENV": request.app.state.config.WEB_SEARCH_TRUST_ENV, + "WEB_SEARCH_RESULT_COUNT": request.app.state.config.WEB_SEARCH_RESULT_COUNT, + "WEB_SEARCH_CONCURRENT_REQUESTS": request.app.state.config.WEB_SEARCH_CONCURRENT_REQUESTS, + "WEB_SEARCH_DOMAIN_FILTER_LIST": request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, "BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL": request.app.state.config.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL, - "search": { - "enabled": request.app.state.config.ENABLE_RAG_WEB_SEARCH, - "drive": request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION, - "onedrive": request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION, - "engine": request.app.state.config.RAG_WEB_SEARCH_ENGINE, - "searxng_query_url": request.app.state.config.SEARXNG_QUERY_URL, - "google_pse_api_key": request.app.state.config.GOOGLE_PSE_API_KEY, - "google_pse_engine_id": request.app.state.config.GOOGLE_PSE_ENGINE_ID, - "brave_search_api_key": request.app.state.config.BRAVE_SEARCH_API_KEY, - "kagi_search_api_key": request.app.state.config.KAGI_SEARCH_API_KEY, - "mojeek_search_api_key": request.app.state.config.MOJEEK_SEARCH_API_KEY, - "bocha_search_api_key": request.app.state.config.BOCHA_SEARCH_API_KEY, - "serpstack_api_key": request.app.state.config.SERPSTACK_API_KEY, - "serpstack_https": request.app.state.config.SERPSTACK_HTTPS, - "serper_api_key": request.app.state.config.SERPER_API_KEY, - "serply_api_key": request.app.state.config.SERPLY_API_KEY, - "tavily_api_key": request.app.state.config.TAVILY_API_KEY, - "searchapi_api_key": request.app.state.config.SEARCHAPI_API_KEY, - "searchapi_engine": request.app.state.config.SEARCHAPI_ENGINE, - "serpapi_api_key": request.app.state.config.SERPAPI_API_KEY, - "serpapi_engine": request.app.state.config.SERPAPI_ENGINE, - "jina_api_key": request.app.state.config.JINA_API_KEY, - "bing_search_v7_endpoint": request.app.state.config.BING_SEARCH_V7_ENDPOINT, - "bing_search_v7_subscription_key": request.app.state.config.BING_SEARCH_V7_SUBSCRIPTION_KEY, - "exa_api_key": request.app.state.config.EXA_API_KEY, - "perplexity_api_key": request.app.state.config.PERPLEXITY_API_KEY, - "sougou_api_sid": request.app.state.config.SOUGOU_API_SID, - "sougou_api_sk": request.app.state.config.SOUGOU_API_SK, - "result_count": request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - "trust_env": request.app.state.config.RAG_WEB_SEARCH_TRUST_ENV, - "concurrent_requests": request.app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS, - "domain_filter_list": request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, - }, + "SEARXNG_QUERY_URL": request.app.state.config.SEARXNG_QUERY_URL, + "GOOGLE_PSE_API_KEY": request.app.state.config.GOOGLE_PSE_API_KEY, + "GOOGLE_PSE_ENGINE_ID": request.app.state.config.GOOGLE_PSE_ENGINE_ID, + "BRAVE_SEARCH_API_KEY": request.app.state.config.BRAVE_SEARCH_API_KEY, + "KAGI_SEARCH_API_KEY": request.app.state.config.KAGI_SEARCH_API_KEY, + "MOJEEK_SEARCH_API_KEY": request.app.state.config.MOJEEK_SEARCH_API_KEY, + "BOCHA_SEARCH_API_KEY": request.app.state.config.BOCHA_SEARCH_API_KEY, + "SERPSTACK_API_KEY": request.app.state.config.SERPSTACK_API_KEY, + "SERPSTACK_HTTPS": request.app.state.config.SERPSTACK_HTTPS, + "SERPER_API_KEY": request.app.state.config.SERPER_API_KEY, + "SERPLY_API_KEY": request.app.state.config.SERPLY_API_KEY, + "TAVILY_API_KEY": request.app.state.config.TAVILY_API_KEY, + "SEARCHAPI_API_KEY": request.app.state.config.SEARCHAPI_API_KEY, + "SEARCHAPI_ENGINE": request.app.state.config.SEARCHAPI_ENGINE, + "SERPAPI_API_KEY": request.app.state.config.SERPAPI_API_KEY, + "SERPAPI_ENGINE": request.app.state.config.SERPAPI_ENGINE, + "JINA_API_KEY": request.app.state.config.JINA_API_KEY, + "BING_SEARCH_V7_ENDPOINT": request.app.state.config.BING_SEARCH_V7_ENDPOINT, + "BING_SEARCH_V7_SUBSCRIPTION_KEY": request.app.state.config.BING_SEARCH_V7_SUBSCRIPTION_KEY, + "EXA_API_KEY": request.app.state.config.EXA_API_KEY, + "PERPLEXITY_API_KEY": request.app.state.config.PERPLEXITY_API_KEY, + "SOUGOU_API_SID": request.app.state.config.SOUGOU_API_SID, + "SOUGOU_API_SK": request.app.state.config.SOUGOU_API_SK, + "WEB_LOADER_ENGINE": request.app.state.config.WEB_LOADER_ENGINE, + "ENABLE_WEB_LOADER_SSL_VERIFICATION": request.app.state.config.ENABLE_WEB_LOADER_SSL_VERIFICATION, + "PLAYWRIGHT_WS_URL": request.app.state.config.PLAYWRIGHT_WS_URL, + "PLAYWRIGHT_TIMEOUT": request.app.state.config.PLAYWRIGHT_TIMEOUT, + "FIRECRAWL_API_KEY": request.app.state.config.FIRECRAWL_API_KEY, + "FIRECRAWL_API_BASE_URL": request.app.state.config.FIRECRAWL_API_BASE_URL, + "TAVILY_EXTRACT_DEPTH": request.app.state.config.TAVILY_EXTRACT_DEPTH, + "YOUTUBE_LOADER_LANGUAGE": request.app.state.config.YOUTUBE_LOADER_LANGUAGE, + "YOUTUBE_LOADER_PROXY_URL": request.app.state.config.YOUTUBE_LOADER_PROXY_URL, + "YOUTUBE_LOADER_TRANSLATION": request.app.state.YOUTUBE_LOADER_TRANSLATION, }, } -class FileConfig(BaseModel): - max_size: Optional[int] = None - max_count: Optional[int] = None - - -class DocumentIntelligenceConfigForm(BaseModel): - endpoint: str - key: str - - -class MistralOCRConfigForm(BaseModel): - api_key: str - - -class ContentExtractionConfig(BaseModel): - engine: str = "" - tika_server_url: Optional[str] = None - docling_server_url: Optional[str] = None - document_intelligence_config: Optional[DocumentIntelligenceConfigForm] = None - mistral_ocr_config: Optional[MistralOCRConfigForm] = None - - -class ChunkParamUpdateForm(BaseModel): - text_splitter: Optional[str] = None - chunk_size: int - chunk_overlap: int - - -class YoutubeLoaderConfig(BaseModel): - language: list[str] - translation: Optional[str] = None - proxy_url: str = "" - - -class WebSearchConfig(BaseModel): - enabled: bool - engine: Optional[str] = None - searxng_query_url: Optional[str] = None - google_pse_api_key: Optional[str] = None - google_pse_engine_id: Optional[str] = None - brave_search_api_key: Optional[str] = None - kagi_search_api_key: Optional[str] = None - mojeek_search_api_key: Optional[str] = None - bocha_search_api_key: Optional[str] = None - serpstack_api_key: Optional[str] = None - serpstack_https: Optional[bool] = None - serper_api_key: Optional[str] = None - serply_api_key: Optional[str] = None - tavily_api_key: Optional[str] = None - searchapi_api_key: Optional[str] = None - searchapi_engine: Optional[str] = None - serpapi_api_key: Optional[str] = None - serpapi_engine: Optional[str] = None - jina_api_key: Optional[str] = None - bing_search_v7_endpoint: Optional[str] = None - bing_search_v7_subscription_key: Optional[str] = None - exa_api_key: Optional[str] = None - perplexity_api_key: Optional[str] = None - sougou_api_sid: Optional[str] = None - sougou_api_sk: Optional[str] = None - result_count: Optional[int] = None - concurrent_requests: Optional[int] = None - trust_env: Optional[bool] = None - domain_filter_list: Optional[List[str]] = [] - - class WebConfig(BaseModel): - search: WebSearchConfig - ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION: Optional[bool] = None + ENABLE_WEB_SEARCH: Optional[bool] = None + WEB_SEARCH_ENGINE: Optional[str] = None + WEB_SEARCH_TRUST_ENV: Optional[bool] = None + WEB_SEARCH_RESULT_COUNT: Optional[int] = None + WEB_SEARCH_CONCURRENT_REQUESTS: Optional[int] = None + WEB_SEARCH_DOMAIN_FILTER_LIST: Optional[List[str]] = [] BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL: Optional[bool] = None + SEARXNG_QUERY_URL: Optional[str] = None + GOOGLE_PSE_API_KEY: Optional[str] = None + GOOGLE_PSE_ENGINE_ID: Optional[str] = None + BRAVE_SEARCH_API_KEY: Optional[str] = None + KAGI_SEARCH_API_KEY: Optional[str] = None + MOJEEK_SEARCH_API_KEY: Optional[str] = None + BOCHA_SEARCH_API_KEY: Optional[str] = None + SERPSTACK_API_KEY: Optional[str] = None + SERPSTACK_HTTPS: Optional[bool] = None + SERPER_API_KEY: Optional[str] = None + SERPLY_API_KEY: Optional[str] = None + TAVILY_API_KEY: Optional[str] = None + SEARCHAPI_API_KEY: Optional[str] = None + SEARCHAPI_ENGINE: Optional[str] = None + SERPAPI_API_KEY: Optional[str] = None + SERPAPI_ENGINE: Optional[str] = None + JINA_API_KEY: Optional[str] = None + BING_SEARCH_V7_ENDPOINT: Optional[str] = None + BING_SEARCH_V7_SUBSCRIPTION_KEY: Optional[str] = None + EXA_API_KEY: Optional[str] = None + PERPLEXITY_API_KEY: Optional[str] = None + SOUGOU_API_SID: Optional[str] = None + SOUGOU_API_SK: Optional[str] = None + WEB_LOADER_ENGINE: Optional[str] = None + ENABLE_WEB_LOADER_SSL_VERIFICATION: Optional[bool] = None + PLAYWRIGHT_WS_URL: Optional[str] = None + PLAYWRIGHT_TIMEOUT: Optional[int] = None + FIRECRAWL_API_KEY: Optional[str] = None + FIRECRAWL_API_BASE_URL: Optional[str] = None + TAVILY_EXTRACT_DEPTH: Optional[str] = None + YOUTUBE_LOADER_LANGUAGE: Optional[List[str]] = None + YOUTUBE_LOADER_PROXY_URL: Optional[str] = None + YOUTUBE_LOADER_TRANSLATION: Optional[str] = None -class ConfigUpdateForm(BaseModel): - RAG_FULL_CONTEXT: Optional[bool] = None +class ConfigForm(BaseModel): + # RAG settings + RAG_TEMPLATE: Optional[str] = None + TOP_K: Optional[int] = None BYPASS_EMBEDDING_AND_RETRIEVAL: Optional[bool] = None - pdf_extract_images: Optional[bool] = None - enable_google_drive_integration: Optional[bool] = None - enable_onedrive_integration: Optional[bool] = None - file: Optional[FileConfig] = None - content_extraction: Optional[ContentExtractionConfig] = None - chunk: Optional[ChunkParamUpdateForm] = None - youtube: Optional[YoutubeLoaderConfig] = None + RAG_FULL_CONTEXT: Optional[bool] = None + + # Hybrid search settings + ENABLE_RAG_HYBRID_SEARCH: Optional[bool] = None + TOP_K_RERANKER: Optional[int] = None + RELEVANCE_THRESHOLD: Optional[float] = None + + # Content extraction settings + CONTENT_EXTRACTION_ENGINE: Optional[str] = None + PDF_EXTRACT_IMAGES: Optional[bool] = None + TIKA_SERVER_URL: Optional[str] = None + DOCLING_SERVER_URL: Optional[str] = None + DOCUMENT_INTELLIGENCE_ENDPOINT: Optional[str] = None + DOCUMENT_INTELLIGENCE_KEY: Optional[str] = None + MISTRAL_OCR_API_KEY: Optional[str] = None + + # Chunking settings + TEXT_SPLITTER: Optional[str] = None + CHUNK_SIZE: Optional[int] = None + CHUNK_OVERLAP: Optional[int] = None + + # File upload settings + FILE_MAX_SIZE: Optional[int] = None + FILE_MAX_COUNT: Optional[int] = None + + # Integration settings + ENABLE_GOOGLE_DRIVE_INTEGRATION: Optional[bool] = None + ENABLE_ONEDRIVE_INTEGRATION: Optional[bool] = None + + # Web search settings web: Optional[WebConfig] = None @router.post("/config/update") async def update_rag_config( - request: Request, form_data: ConfigUpdateForm, user=Depends(get_admin_user) + request: Request, form_data: ConfigForm, user=Depends(get_admin_user) ): - request.app.state.config.PDF_EXTRACT_IMAGES = ( - form_data.pdf_extract_images - if form_data.pdf_extract_images is not None - else request.app.state.config.PDF_EXTRACT_IMAGES + # RAG settings + request.app.state.config.RAG_TEMPLATE = ( + form_data.RAG_TEMPLATE + if form_data.RAG_TEMPLATE is not None + else request.app.state.config.RAG_TEMPLATE + ) + request.app.state.config.TOP_K = ( + form_data.TOP_K + if form_data.TOP_K is not None + else request.app.state.config.TOP_K + ) + request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL = ( + form_data.BYPASS_EMBEDDING_AND_RETRIEVAL + if form_data.BYPASS_EMBEDDING_AND_RETRIEVAL is not None + else request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL ) - request.app.state.config.RAG_FULL_CONTEXT = ( form_data.RAG_FULL_CONTEXT if form_data.RAG_FULL_CONTEXT is not None else request.app.state.config.RAG_FULL_CONTEXT ) - request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL = ( - form_data.BYPASS_EMBEDDING_AND_RETRIEVAL - if form_data.BYPASS_EMBEDDING_AND_RETRIEVAL is not None - else request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL - ) - - request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION = ( - form_data.enable_google_drive_integration - if form_data.enable_google_drive_integration is not None - else request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION - ) - - request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION = ( - form_data.enable_onedrive_integration - if form_data.enable_onedrive_integration is not None - else request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION - ) - - if form_data.file is not None: - request.app.state.config.FILE_MAX_SIZE = form_data.file.max_size - request.app.state.config.FILE_MAX_COUNT = form_data.file.max_count - - if form_data.content_extraction is not None: - log.info( - f"Updating content extraction: {request.app.state.config.CONTENT_EXTRACTION_ENGINE} to {form_data.content_extraction.engine}" - ) - request.app.state.config.CONTENT_EXTRACTION_ENGINE = ( - form_data.content_extraction.engine - ) - request.app.state.config.TIKA_SERVER_URL = ( - form_data.content_extraction.tika_server_url - ) - request.app.state.config.DOCLING_SERVER_URL = ( - form_data.content_extraction.docling_server_url - ) - if form_data.content_extraction.document_intelligence_config is not None: - request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT = ( - form_data.content_extraction.document_intelligence_config.endpoint - ) - request.app.state.config.DOCUMENT_INTELLIGENCE_KEY = ( - form_data.content_extraction.document_intelligence_config.key - ) - if form_data.content_extraction.mistral_ocr_config is not None: - request.app.state.config.MISTRAL_OCR_API_KEY = ( - form_data.content_extraction.mistral_ocr_config.api_key - ) - - if form_data.chunk is not None: - request.app.state.config.TEXT_SPLITTER = form_data.chunk.text_splitter - request.app.state.config.CHUNK_SIZE = form_data.chunk.chunk_size - request.app.state.config.CHUNK_OVERLAP = form_data.chunk.chunk_overlap - - if form_data.youtube is not None: - request.app.state.config.YOUTUBE_LOADER_LANGUAGE = form_data.youtube.language - request.app.state.config.YOUTUBE_LOADER_PROXY_URL = form_data.youtube.proxy_url - request.app.state.YOUTUBE_LOADER_TRANSLATION = form_data.youtube.translation - - if form_data.web is not None: - request.app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION = ( - # Note: When UI "Bypass SSL verification for Websites"=True then ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION=False - form_data.web.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION - ) - - request.app.state.config.ENABLE_RAG_WEB_SEARCH = form_data.web.search.enabled - request.app.state.config.RAG_WEB_SEARCH_ENGINE = form_data.web.search.engine - - request.app.state.config.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL = ( - form_data.web.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL - ) - - request.app.state.config.SEARXNG_QUERY_URL = ( - form_data.web.search.searxng_query_url - ) - request.app.state.config.GOOGLE_PSE_API_KEY = ( - form_data.web.search.google_pse_api_key - ) - request.app.state.config.GOOGLE_PSE_ENGINE_ID = ( - form_data.web.search.google_pse_engine_id - ) - request.app.state.config.BRAVE_SEARCH_API_KEY = ( - form_data.web.search.brave_search_api_key - ) - request.app.state.config.KAGI_SEARCH_API_KEY = ( - form_data.web.search.kagi_search_api_key - ) - request.app.state.config.MOJEEK_SEARCH_API_KEY = ( - form_data.web.search.mojeek_search_api_key - ) - request.app.state.config.BOCHA_SEARCH_API_KEY = ( - form_data.web.search.bocha_search_api_key - ) - request.app.state.config.SERPSTACK_API_KEY = ( - form_data.web.search.serpstack_api_key - ) - request.app.state.config.SERPSTACK_HTTPS = form_data.web.search.serpstack_https - request.app.state.config.SERPER_API_KEY = form_data.web.search.serper_api_key - request.app.state.config.SERPLY_API_KEY = form_data.web.search.serply_api_key - request.app.state.config.TAVILY_API_KEY = form_data.web.search.tavily_api_key - request.app.state.config.SEARCHAPI_API_KEY = ( - form_data.web.search.searchapi_api_key - ) - request.app.state.config.SEARCHAPI_ENGINE = ( - form_data.web.search.searchapi_engine - ) - - request.app.state.config.SERPAPI_API_KEY = form_data.web.search.serpapi_api_key - request.app.state.config.SERPAPI_ENGINE = form_data.web.search.serpapi_engine - - request.app.state.config.JINA_API_KEY = form_data.web.search.jina_api_key - request.app.state.config.BING_SEARCH_V7_ENDPOINT = ( - form_data.web.search.bing_search_v7_endpoint - ) - request.app.state.config.BING_SEARCH_V7_SUBSCRIPTION_KEY = ( - form_data.web.search.bing_search_v7_subscription_key - ) - - request.app.state.config.EXA_API_KEY = form_data.web.search.exa_api_key - - request.app.state.config.PERPLEXITY_API_KEY = ( - form_data.web.search.perplexity_api_key - ) - request.app.state.config.SOUGOU_API_SID = ( - form_data.web.search.sougou_api_sid - ) - request.app.state.config.SOUGOU_API_SK = ( - form_data.web.search.sougou_api_sk - ) - - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = ( - form_data.web.search.result_count - ) - request.app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = ( - form_data.web.search.concurrent_requests - ) - request.app.state.config.RAG_WEB_SEARCH_TRUST_ENV = ( - form_data.web.search.trust_env - ) - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST = ( - form_data.web.search.domain_filter_list - ) - - return { - "status": True, - "pdf_extract_images": request.app.state.config.PDF_EXTRACT_IMAGES, - "RAG_FULL_CONTEXT": request.app.state.config.RAG_FULL_CONTEXT, - "BYPASS_EMBEDDING_AND_RETRIEVAL": request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL, - "file": { - "max_size": request.app.state.config.FILE_MAX_SIZE, - "max_count": request.app.state.config.FILE_MAX_COUNT, - }, - "content_extraction": { - "engine": request.app.state.config.CONTENT_EXTRACTION_ENGINE, - "tika_server_url": request.app.state.config.TIKA_SERVER_URL, - "docling_server_url": request.app.state.config.DOCLING_SERVER_URL, - "document_intelligence_config": { - "endpoint": request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT, - "key": request.app.state.config.DOCUMENT_INTELLIGENCE_KEY, - }, - "mistral_ocr_config": { - "api_key": request.app.state.config.MISTRAL_OCR_API_KEY, - }, - }, - "chunk": { - "text_splitter": request.app.state.config.TEXT_SPLITTER, - "chunk_size": request.app.state.config.CHUNK_SIZE, - "chunk_overlap": request.app.state.config.CHUNK_OVERLAP, - }, - "youtube": { - "language": request.app.state.config.YOUTUBE_LOADER_LANGUAGE, - "proxy_url": request.app.state.config.YOUTUBE_LOADER_PROXY_URL, - "translation": request.app.state.YOUTUBE_LOADER_TRANSLATION, - }, - "web": { - "ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION": request.app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION, - "BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL": request.app.state.config.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL, - "search": { - "enabled": request.app.state.config.ENABLE_RAG_WEB_SEARCH, - "engine": request.app.state.config.RAG_WEB_SEARCH_ENGINE, - "searxng_query_url": request.app.state.config.SEARXNG_QUERY_URL, - "google_pse_api_key": request.app.state.config.GOOGLE_PSE_API_KEY, - "google_pse_engine_id": request.app.state.config.GOOGLE_PSE_ENGINE_ID, - "brave_search_api_key": request.app.state.config.BRAVE_SEARCH_API_KEY, - "kagi_search_api_key": request.app.state.config.KAGI_SEARCH_API_KEY, - "mojeek_search_api_key": request.app.state.config.MOJEEK_SEARCH_API_KEY, - "bocha_search_api_key": request.app.state.config.BOCHA_SEARCH_API_KEY, - "serpstack_api_key": request.app.state.config.SERPSTACK_API_KEY, - "serpstack_https": request.app.state.config.SERPSTACK_HTTPS, - "serper_api_key": request.app.state.config.SERPER_API_KEY, - "serply_api_key": request.app.state.config.SERPLY_API_KEY, - "serachapi_api_key": request.app.state.config.SEARCHAPI_API_KEY, - "searchapi_engine": request.app.state.config.SEARCHAPI_ENGINE, - "serpapi_api_key": request.app.state.config.SERPAPI_API_KEY, - "serpapi_engine": request.app.state.config.SERPAPI_ENGINE, - "tavily_api_key": request.app.state.config.TAVILY_API_KEY, - "jina_api_key": request.app.state.config.JINA_API_KEY, - "bing_search_v7_endpoint": request.app.state.config.BING_SEARCH_V7_ENDPOINT, - "bing_search_v7_subscription_key": request.app.state.config.BING_SEARCH_V7_SUBSCRIPTION_KEY, - "exa_api_key": request.app.state.config.EXA_API_KEY, - "perplexity_api_key": request.app.state.config.PERPLEXITY_API_KEY, - "sougou_api_sid": request.app.state.config.SOUGOU_API_SID, - "sougou_api_sk": request.app.state.config.SOUGOU_API_SK, - "result_count": request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - "concurrent_requests": request.app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS, - "trust_env": request.app.state.config.RAG_WEB_SEARCH_TRUST_ENV, - "domain_filter_list": request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, - }, - }, - } - - -@router.get("/template") -async def get_rag_template(request: Request, user=Depends(get_verified_user)): - return { - "status": True, - "template": request.app.state.config.RAG_TEMPLATE, - } - - -@router.get("/query/settings") -async def get_query_settings(request: Request, user=Depends(get_admin_user)): - return { - "status": True, - "template": request.app.state.config.RAG_TEMPLATE, - "k": request.app.state.config.TOP_K, - "k_reranker": request.app.state.config.TOP_K_RERANKER, - "r": request.app.state.config.RELEVANCE_THRESHOLD, - "hybrid": request.app.state.config.ENABLE_RAG_HYBRID_SEARCH, - } - - -class QuerySettingsForm(BaseModel): - k: Optional[int] = None - k_reranker: Optional[int] = None - r: Optional[float] = None - template: Optional[str] = None - hybrid: Optional[bool] = None - - -@router.post("/query/settings/update") -async def update_query_settings( - request: Request, form_data: QuerySettingsForm, user=Depends(get_admin_user) -): - request.app.state.config.RAG_TEMPLATE = form_data.template - request.app.state.config.TOP_K = form_data.k if form_data.k else 4 - request.app.state.config.TOP_K_RERANKER = form_data.k_reranker or 4 - request.app.state.config.RELEVANCE_THRESHOLD = form_data.r if form_data.r else 0.0 - + # Hybrid search settings request.app.state.config.ENABLE_RAG_HYBRID_SEARCH = ( - form_data.hybrid if form_data.hybrid else False + form_data.ENABLE_RAG_HYBRID_SEARCH + if form_data.ENABLE_RAG_HYBRID_SEARCH is not None + else request.app.state.config.ENABLE_RAG_HYBRID_SEARCH ) - + # Free up memory if hybrid search is disabled if not request.app.state.config.ENABLE_RAG_HYBRID_SEARCH: request.app.state.rf = None + request.app.state.config.TOP_K_RERANKER = ( + form_data.TOP_K_RERANKER + if form_data.TOP_K_RERANKER is not None + else request.app.state.config.TOP_K_RERANKER + ) + request.app.state.config.RELEVANCE_THRESHOLD = ( + form_data.RELEVANCE_THRESHOLD + if form_data.RELEVANCE_THRESHOLD is not None + else request.app.state.config.RELEVANCE_THRESHOLD + ) + + # Content extraction settings + request.app.state.config.CONTENT_EXTRACTION_ENGINE = ( + form_data.CONTENT_EXTRACTION_ENGINE + if form_data.CONTENT_EXTRACTION_ENGINE is not None + else request.app.state.config.CONTENT_EXTRACTION_ENGINE + ) + request.app.state.config.PDF_EXTRACT_IMAGES = ( + form_data.PDF_EXTRACT_IMAGES + if form_data.PDF_EXTRACT_IMAGES is not None + else request.app.state.config.PDF_EXTRACT_IMAGES + ) + request.app.state.config.TIKA_SERVER_URL = ( + form_data.TIKA_SERVER_URL + if form_data.TIKA_SERVER_URL is not None + else request.app.state.config.TIKA_SERVER_URL + ) + request.app.state.config.DOCLING_SERVER_URL = ( + form_data.DOCLING_SERVER_URL + if form_data.DOCLING_SERVER_URL is not None + else request.app.state.config.DOCLING_SERVER_URL + ) + request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT = ( + form_data.DOCUMENT_INTELLIGENCE_ENDPOINT + if form_data.DOCUMENT_INTELLIGENCE_ENDPOINT is not None + else request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT + ) + request.app.state.config.DOCUMENT_INTELLIGENCE_KEY = ( + form_data.DOCUMENT_INTELLIGENCE_KEY + if form_data.DOCUMENT_INTELLIGENCE_KEY is not None + else request.app.state.config.DOCUMENT_INTELLIGENCE_KEY + ) + request.app.state.config.MISTRAL_OCR_API_KEY = ( + form_data.MISTRAL_OCR_API_KEY + if form_data.MISTRAL_OCR_API_KEY is not None + else request.app.state.config.MISTRAL_OCR_API_KEY + ) + + # Chunking settings + request.app.state.config.TEXT_SPLITTER = ( + form_data.TEXT_SPLITTER + if form_data.TEXT_SPLITTER is not None + else request.app.state.config.TEXT_SPLITTER + ) + request.app.state.config.CHUNK_SIZE = ( + form_data.CHUNK_SIZE + if form_data.CHUNK_SIZE is not None + else request.app.state.config.CHUNK_SIZE + ) + request.app.state.config.CHUNK_OVERLAP = ( + form_data.CHUNK_OVERLAP + if form_data.CHUNK_OVERLAP is not None + else request.app.state.config.CHUNK_OVERLAP + ) + + # File upload settings + request.app.state.config.FILE_MAX_SIZE = ( + form_data.FILE_MAX_SIZE + if form_data.FILE_MAX_SIZE is not None + else request.app.state.config.FILE_MAX_SIZE + ) + request.app.state.config.FILE_MAX_COUNT = ( + form_data.FILE_MAX_COUNT + if form_data.FILE_MAX_COUNT is not None + else request.app.state.config.FILE_MAX_COUNT + ) + + # Integration settings + request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION = ( + form_data.ENABLE_GOOGLE_DRIVE_INTEGRATION + if form_data.ENABLE_GOOGLE_DRIVE_INTEGRATION is not None + else request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION + ) + request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION = ( + form_data.ENABLE_ONEDRIVE_INTEGRATION + if form_data.ENABLE_ONEDRIVE_INTEGRATION is not None + else request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION + ) + + if form_data.web is not None: + # Web search settings + request.app.state.config.ENABLE_WEB_SEARCH = form_data.web.ENABLE_WEB_SEARCH + request.app.state.config.WEB_SEARCH_ENGINE = form_data.web.WEB_SEARCH_ENGINE + request.app.state.config.WEB_SEARCH_TRUST_ENV = ( + form_data.web.WEB_SEARCH_TRUST_ENV + ) + request.app.state.config.WEB_SEARCH_RESULT_COUNT = ( + form_data.web.WEB_SEARCH_RESULT_COUNT + ) + request.app.state.config.WEB_SEARCH_CONCURRENT_REQUESTS = ( + form_data.web.WEB_SEARCH_CONCURRENT_REQUESTS + ) + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST = ( + form_data.web.WEB_SEARCH_DOMAIN_FILTER_LIST + ) + request.app.state.config.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL = ( + form_data.web.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL + ) + request.app.state.config.SEARXNG_QUERY_URL = form_data.web.SEARXNG_QUERY_URL + request.app.state.config.GOOGLE_PSE_API_KEY = form_data.web.GOOGLE_PSE_API_KEY + request.app.state.config.GOOGLE_PSE_ENGINE_ID = ( + form_data.web.GOOGLE_PSE_ENGINE_ID + ) + request.app.state.config.BRAVE_SEARCH_API_KEY = ( + form_data.web.BRAVE_SEARCH_API_KEY + ) + request.app.state.config.KAGI_SEARCH_API_KEY = form_data.web.KAGI_SEARCH_API_KEY + request.app.state.config.MOJEEK_SEARCH_API_KEY = ( + form_data.web.MOJEEK_SEARCH_API_KEY + ) + request.app.state.config.BOCHA_SEARCH_API_KEY = ( + form_data.web.BOCHA_SEARCH_API_KEY + ) + request.app.state.config.SERPSTACK_API_KEY = form_data.web.SERPSTACK_API_KEY + request.app.state.config.SERPSTACK_HTTPS = form_data.web.SERPSTACK_HTTPS + request.app.state.config.SERPER_API_KEY = form_data.web.SERPER_API_KEY + request.app.state.config.SERPLY_API_KEY = form_data.web.SERPLY_API_KEY + request.app.state.config.TAVILY_API_KEY = form_data.web.TAVILY_API_KEY + request.app.state.config.SEARCHAPI_API_KEY = form_data.web.SEARCHAPI_API_KEY + request.app.state.config.SEARCHAPI_ENGINE = form_data.web.SEARCHAPI_ENGINE + request.app.state.config.SERPAPI_API_KEY = form_data.web.SERPAPI_API_KEY + request.app.state.config.SERPAPI_ENGINE = form_data.web.SERPAPI_ENGINE + request.app.state.config.JINA_API_KEY = form_data.web.JINA_API_KEY + request.app.state.config.BING_SEARCH_V7_ENDPOINT = ( + form_data.web.BING_SEARCH_V7_ENDPOINT + ) + request.app.state.config.BING_SEARCH_V7_SUBSCRIPTION_KEY = ( + form_data.web.BING_SEARCH_V7_SUBSCRIPTION_KEY + ) + request.app.state.config.EXA_API_KEY = form_data.web.EXA_API_KEY + request.app.state.config.PERPLEXITY_API_KEY = form_data.web.PERPLEXITY_API_KEY + request.app.state.config.SOUGOU_API_SID = form_data.web.SOUGOU_API_SID + request.app.state.config.SOUGOU_API_SK = form_data.web.SOUGOU_API_SK + + # Web loader settings + request.app.state.config.WEB_LOADER_ENGINE = form_data.web.WEB_LOADER_ENGINE + request.app.state.config.ENABLE_WEB_LOADER_SSL_VERIFICATION = ( + form_data.web.ENABLE_WEB_LOADER_SSL_VERIFICATION + ) + request.app.state.config.PLAYWRIGHT_WS_URL = form_data.web.PLAYWRIGHT_WS_URL + request.app.state.config.PLAYWRIGHT_TIMEOUT = form_data.web.PLAYWRIGHT_TIMEOUT + request.app.state.config.FIRECRAWL_API_KEY = form_data.web.FIRECRAWL_API_KEY + request.app.state.config.FIRECRAWL_API_BASE_URL = ( + form_data.web.FIRECRAWL_API_BASE_URL + ) + request.app.state.config.TAVILY_EXTRACT_DEPTH = ( + form_data.web.TAVILY_EXTRACT_DEPTH + ) + request.app.state.config.YOUTUBE_LOADER_LANGUAGE = ( + form_data.web.YOUTUBE_LOADER_LANGUAGE + ) + request.app.state.config.YOUTUBE_LOADER_PROXY_URL = ( + form_data.web.YOUTUBE_LOADER_PROXY_URL + ) + request.app.state.YOUTUBE_LOADER_TRANSLATION = ( + form_data.web.YOUTUBE_LOADER_TRANSLATION + ) + return { "status": True, - "template": request.app.state.config.RAG_TEMPLATE, - "k": request.app.state.config.TOP_K, - "k_reranker": request.app.state.config.TOP_K_RERANKER, - "r": request.app.state.config.RELEVANCE_THRESHOLD, - "hybrid": request.app.state.config.ENABLE_RAG_HYBRID_SEARCH, + # RAG settings + "RAG_TEMPLATE": request.app.state.config.RAG_TEMPLATE, + "TOP_K": request.app.state.config.TOP_K, + "BYPASS_EMBEDDING_AND_RETRIEVAL": request.app.state.config.BYPASS_EMBEDDING_AND_RETRIEVAL, + "RAG_FULL_CONTEXT": request.app.state.config.RAG_FULL_CONTEXT, + # Hybrid search settings + "ENABLE_RAG_HYBRID_SEARCH": request.app.state.config.ENABLE_RAG_HYBRID_SEARCH, + "TOP_K_RERANKER": request.app.state.config.TOP_K_RERANKER, + "RELEVANCE_THRESHOLD": request.app.state.config.RELEVANCE_THRESHOLD, + # Content extraction settings + "CONTENT_EXTRACTION_ENGINE": request.app.state.config.CONTENT_EXTRACTION_ENGINE, + "PDF_EXTRACT_IMAGES": request.app.state.config.PDF_EXTRACT_IMAGES, + "TIKA_SERVER_URL": request.app.state.config.TIKA_SERVER_URL, + "DOCLING_SERVER_URL": request.app.state.config.DOCLING_SERVER_URL, + "DOCUMENT_INTELLIGENCE_ENDPOINT": request.app.state.config.DOCUMENT_INTELLIGENCE_ENDPOINT, + "DOCUMENT_INTELLIGENCE_KEY": request.app.state.config.DOCUMENT_INTELLIGENCE_KEY, + "MISTRAL_OCR_API_KEY": request.app.state.config.MISTRAL_OCR_API_KEY, + # Chunking settings + "TEXT_SPLITTER": request.app.state.config.TEXT_SPLITTER, + "CHUNK_SIZE": request.app.state.config.CHUNK_SIZE, + "CHUNK_OVERLAP": request.app.state.config.CHUNK_OVERLAP, + # File upload settings + "FILE_MAX_SIZE": request.app.state.config.FILE_MAX_SIZE, + "FILE_MAX_COUNT": request.app.state.config.FILE_MAX_COUNT, + # Integration settings + "ENABLE_GOOGLE_DRIVE_INTEGRATION": request.app.state.config.ENABLE_GOOGLE_DRIVE_INTEGRATION, + "ENABLE_ONEDRIVE_INTEGRATION": request.app.state.config.ENABLE_ONEDRIVE_INTEGRATION, + # Web search settings + "web": { + "ENABLE_WEB_SEARCH": request.app.state.config.ENABLE_WEB_SEARCH, + "WEB_SEARCH_ENGINE": request.app.state.config.WEB_SEARCH_ENGINE, + "WEB_SEARCH_TRUST_ENV": request.app.state.config.WEB_SEARCH_TRUST_ENV, + "WEB_SEARCH_RESULT_COUNT": request.app.state.config.WEB_SEARCH_RESULT_COUNT, + "WEB_SEARCH_CONCURRENT_REQUESTS": request.app.state.config.WEB_SEARCH_CONCURRENT_REQUESTS, + "WEB_SEARCH_DOMAIN_FILTER_LIST": request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, + "BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL": request.app.state.config.BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL, + "SEARXNG_QUERY_URL": request.app.state.config.SEARXNG_QUERY_URL, + "GOOGLE_PSE_API_KEY": request.app.state.config.GOOGLE_PSE_API_KEY, + "GOOGLE_PSE_ENGINE_ID": request.app.state.config.GOOGLE_PSE_ENGINE_ID, + "BRAVE_SEARCH_API_KEY": request.app.state.config.BRAVE_SEARCH_API_KEY, + "KAGI_SEARCH_API_KEY": request.app.state.config.KAGI_SEARCH_API_KEY, + "MOJEEK_SEARCH_API_KEY": request.app.state.config.MOJEEK_SEARCH_API_KEY, + "BOCHA_SEARCH_API_KEY": request.app.state.config.BOCHA_SEARCH_API_KEY, + "SERPSTACK_API_KEY": request.app.state.config.SERPSTACK_API_KEY, + "SERPSTACK_HTTPS": request.app.state.config.SERPSTACK_HTTPS, + "SERPER_API_KEY": request.app.state.config.SERPER_API_KEY, + "SERPLY_API_KEY": request.app.state.config.SERPLY_API_KEY, + "TAVILY_API_KEY": request.app.state.config.TAVILY_API_KEY, + "SEARCHAPI_API_KEY": request.app.state.config.SEARCHAPI_API_KEY, + "SEARCHAPI_ENGINE": request.app.state.config.SEARCHAPI_ENGINE, + "SERPAPI_API_KEY": request.app.state.config.SERPAPI_API_KEY, + "SERPAPI_ENGINE": request.app.state.config.SERPAPI_ENGINE, + "JINA_API_KEY": request.app.state.config.JINA_API_KEY, + "BING_SEARCH_V7_ENDPOINT": request.app.state.config.BING_SEARCH_V7_ENDPOINT, + "BING_SEARCH_V7_SUBSCRIPTION_KEY": request.app.state.config.BING_SEARCH_V7_SUBSCRIPTION_KEY, + "EXA_API_KEY": request.app.state.config.EXA_API_KEY, + "PERPLEXITY_API_KEY": request.app.state.config.PERPLEXITY_API_KEY, + "SOUGOU_API_SID": request.app.state.config.SOUGOU_API_SID, + "SOUGOU_API_SK": request.app.state.config.SOUGOU_API_SK, + "WEB_LOADER_ENGINE": request.app.state.config.WEB_LOADER_ENGINE, + "ENABLE_WEB_LOADER_SSL_VERIFICATION": request.app.state.config.ENABLE_WEB_LOADER_SSL_VERIFICATION, + "PLAYWRIGHT_WS_URL": request.app.state.config.PLAYWRIGHT_WS_URL, + "PLAYWRIGHT_TIMEOUT": request.app.state.config.PLAYWRIGHT_TIMEOUT, + "FIRECRAWL_API_KEY": request.app.state.config.FIRECRAWL_API_KEY, + "FIRECRAWL_API_BASE_URL": request.app.state.config.FIRECRAWL_API_BASE_URL, + "TAVILY_EXTRACT_DEPTH": request.app.state.config.TAVILY_EXTRACT_DEPTH, + "YOUTUBE_LOADER_LANGUAGE": request.app.state.config.YOUTUBE_LOADER_LANGUAGE, + "YOUTUBE_LOADER_PROXY_URL": request.app.state.config.YOUTUBE_LOADER_PROXY_URL, + "YOUTUBE_LOADER_TRANSLATION": request.app.state.YOUTUBE_LOADER_TRANSLATION, + }, } @@ -1228,8 +1225,8 @@ def process_web( loader = get_web_loader( form_data.url, - verify_ssl=request.app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION, - requests_per_second=request.app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS, + verify_ssl=request.app.state.config.ENABLE_WEB_LOADER_SSL_VERIFICATION, + requests_per_second=request.app.state.config.WEB_SEARCH_CONCURRENT_REQUESTS, ) docs = loader.load() content = " ".join([doc.page_content for doc in docs]) @@ -1293,8 +1290,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_searxng( request.app.state.config.SEARXNG_QUERY_URL, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No SEARXNG_QUERY_URL found in environment variables") @@ -1307,8 +1304,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: request.app.state.config.GOOGLE_PSE_API_KEY, request.app.state.config.GOOGLE_PSE_ENGINE_ID, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception( @@ -1319,8 +1316,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_brave( request.app.state.config.BRAVE_SEARCH_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No BRAVE_SEARCH_API_KEY found in environment variables") @@ -1329,8 +1326,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_kagi( request.app.state.config.KAGI_SEARCH_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No KAGI_SEARCH_API_KEY found in environment variables") @@ -1339,8 +1336,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_mojeek( request.app.state.config.MOJEEK_SEARCH_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No MOJEEK_SEARCH_API_KEY found in environment variables") @@ -1349,8 +1346,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_bocha( request.app.state.config.BOCHA_SEARCH_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No BOCHA_SEARCH_API_KEY found in environment variables") @@ -1359,8 +1356,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_serpstack( request.app.state.config.SERPSTACK_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, https_enabled=request.app.state.config.SERPSTACK_HTTPS, ) else: @@ -1370,8 +1367,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_serper( request.app.state.config.SERPER_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No SERPER_API_KEY found in environment variables") @@ -1380,24 +1377,24 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_serply( request.app.state.config.SERPLY_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No SERPLY_API_KEY found in environment variables") elif engine == "duckduckgo": return search_duckduckgo( query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) elif engine == "tavily": if request.app.state.config.TAVILY_API_KEY: return search_tavily( request.app.state.config.TAVILY_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No TAVILY_API_KEY found in environment variables") @@ -1407,8 +1404,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: request.app.state.config.SEARCHAPI_API_KEY, request.app.state.config.SEARCHAPI_ENGINE, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No SEARCHAPI_API_KEY found in environment variables") @@ -1418,8 +1415,8 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: request.app.state.config.SERPAPI_API_KEY, request.app.state.config.SERPAPI_ENGINE, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: raise Exception("No SERPAPI_API_KEY found in environment variables") @@ -1427,7 +1424,7 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: return search_jina( request.app.state.config.JINA_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, ) elif engine == "bing": return search_bing( @@ -1435,34 +1432,39 @@ def search_web(request: Request, engine: str, query: str) -> list[SearchResult]: request.app.state.config.BING_SEARCH_V7_ENDPOINT, str(DEFAULT_LOCALE), query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) elif engine == "exa": return search_exa( request.app.state.config.EXA_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) elif engine == "perplexity": return search_perplexity( request.app.state.config.PERPLEXITY_API_KEY, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) - elif engine == 'sougou': - if request.app.state.config.SOUGOU_API_SID and request.app.state.config.SOUGOU_API_SK: + elif engine == "sougou": + if ( + request.app.state.config.SOUGOU_API_SID + and request.app.state.config.SOUGOU_API_SK + ): return search_sougou( request.app.state.config.SOUGOU_API_SID, request.app.state.config.SOUGOU_API_SK, query, - request.app.state.config.RAG_WEB_SEARCH_RESULT_COUNT, - request.app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST, + request.app.state.config.WEB_SEARCH_RESULT_COUNT, + request.app.state.config.WEB_SEARCH_DOMAIN_FILTER_LIST, ) else: - raise Exception("No SOUGOU_API_SID or SOUGOU_API_SK found in environment variables") + raise Exception( + "No SOUGOU_API_SID or SOUGOU_API_SK found in environment variables" + ) else: raise Exception("No search engine API key found in environment variables") @@ -1473,10 +1475,10 @@ async def process_web_search( ): try: logging.info( - f"trying to web search with {request.app.state.config.RAG_WEB_SEARCH_ENGINE, form_data.query}" + f"trying to web search with {request.app.state.config.WEB_SEARCH_ENGINE, form_data.query}" ) web_results = search_web( - request, request.app.state.config.RAG_WEB_SEARCH_ENGINE, form_data.query + request, request.app.state.config.WEB_SEARCH_ENGINE, form_data.query ) except Exception as e: log.exception(e) @@ -1492,9 +1494,9 @@ async def process_web_search( urls = [result.link for result in web_results] loader = get_web_loader( urls, - verify_ssl=request.app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION, - requests_per_second=request.app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS, - trust_env=request.app.state.config.RAG_WEB_SEARCH_TRUST_ENV, + verify_ssl=request.app.state.config.ENABLE_WEB_LOADER_SSL_VERIFICATION, + requests_per_second=request.app.state.config.WEB_SEARCH_CONCURRENT_REQUESTS, + trust_env=request.app.state.config.WEB_SEARCH_TRUST_ENV, ) docs = await loader.aload() urls = [ @@ -1518,20 +1520,20 @@ async def process_web_search( else: collection_names = [] for doc_idx, doc in enumerate(docs): - collection_name = f"web-search-{calculate_sha256_string(form_data.query + '-' + urls[doc_idx])}"[ - :63 - ] + if doc and doc.page_content: + collection_name = f"web-search-{calculate_sha256_string(form_data.query + '-' + urls[doc_idx])}"[ + :63 + ] - collection_names.append(collection_name) - - await run_in_threadpool( - save_docs_to_vector_db, - request, - [doc], - collection_name, - overwrite=True, - user=user, - ) + collection_names.append(collection_name) + await run_in_threadpool( + save_docs_to_vector_db, + request, + [doc], + collection_name, + overwrite=True, + user=user, + ) return { "status": True, diff --git a/backend/open_webui/routers/users.py b/backend/open_webui/routers/users.py index d1046bcedb..a9ac34e2fb 100644 --- a/backend/open_webui/routers/users.py +++ b/backend/open_webui/routers/users.py @@ -88,6 +88,10 @@ class ChatPermissions(BaseModel): file_upload: bool = True delete: bool = True edit: bool = True + stt: bool = True + tts: bool = True + call: bool = True + multiple_models: bool = True temporary: bool = True temporary_enforced: bool = False diff --git a/backend/open_webui/socket/main.py b/backend/open_webui/socket/main.py index 83dd74fff1..282f4db956 100644 --- a/backend/open_webui/socket/main.py +++ b/backend/open_webui/socket/main.py @@ -9,9 +9,8 @@ from open_webui.models.users import Users, UserNameResponse from open_webui.models.channels import Channels from open_webui.models.chats import Chats from open_webui.utils.redis import ( - parse_redis_sentinel_url, get_sentinels_from_env, - AsyncRedisSentinelManager, + get_sentinel_url_from_env, ) from open_webui.env import ( @@ -38,15 +37,10 @@ log.setLevel(SRC_LOG_LEVELS["SOCKET"]) if WEBSOCKET_MANAGER == "redis": if WEBSOCKET_SENTINEL_HOSTS: - redis_config = parse_redis_sentinel_url(WEBSOCKET_REDIS_URL) - mgr = AsyncRedisSentinelManager( - WEBSOCKET_SENTINEL_HOSTS.split(","), - sentinel_port=int(WEBSOCKET_SENTINEL_PORT), - redis_port=redis_config["port"], - service=redis_config["service"], - db=redis_config["db"], - username=redis_config["username"], - password=redis_config["password"], + mgr = socketio.AsyncRedisManager( + get_sentinel_url_from_env( + WEBSOCKET_REDIS_URL, WEBSOCKET_SENTINEL_HOSTS, WEBSOCKET_SENTINEL_PORT + ) ) else: mgr = socketio.AsyncRedisManager(WEBSOCKET_REDIS_URL) @@ -345,16 +339,17 @@ def get_event_emitter(request_info, update_db=True): request_info["message_id"], ) - content = message.get("content", "") - content += event_data.get("data", {}).get("content", "") + if message: + content = message.get("content", "") + content += event_data.get("data", {}).get("content", "") - Chats.upsert_message_to_chat_by_id_and_message_id( - request_info["chat_id"], - request_info["message_id"], - { - "content": content, - }, - ) + Chats.upsert_message_to_chat_by_id_and_message_id( + request_info["chat_id"], + request_info["message_id"], + { + "content": content, + }, + ) if "type" in event_data and event_data["type"] == "replace": content = event_data.get("data", {}).get("content", "") diff --git a/backend/open_webui/tasks.py b/backend/open_webui/tasks.py index 2740ecb5aa..e575e6885c 100644 --- a/backend/open_webui/tasks.py +++ b/backend/open_webui/tasks.py @@ -5,16 +5,23 @@ from uuid import uuid4 # A dictionary to keep track of active tasks tasks: Dict[str, asyncio.Task] = {} +chat_tasks = {} -def cleanup_task(task_id: str): +def cleanup_task(task_id: str, id=None): """ Remove a completed or canceled task from the global `tasks` dictionary. """ tasks.pop(task_id, None) # Remove the task if it exists + # If an ID is provided, remove the task from the chat_tasks dictionary + if id and task_id in chat_tasks.get(id, []): + chat_tasks[id].remove(task_id) + if not chat_tasks[id]: # If no tasks left for this ID, remove the entry + chat_tasks.pop(id, None) -def create_task(coroutine): + +def create_task(coroutine, id=None): """ Create a new asyncio task and add it to the global task dictionary. """ @@ -22,9 +29,15 @@ def create_task(coroutine): task = asyncio.create_task(coroutine) # Create the task # Add a done callback for cleanup - task.add_done_callback(lambda t: cleanup_task(task_id)) - + task.add_done_callback(lambda t: cleanup_task(task_id, id)) tasks[task_id] = task + + # If an ID is provided, associate the task with that ID + if chat_tasks.get(id): + chat_tasks[id].append(task_id) + else: + chat_tasks[id] = [task_id] + return task_id, task @@ -42,6 +55,13 @@ def list_tasks(): return list(tasks.keys()) +def list_task_ids_by_chat_id(id): + """ + List all tasks associated with a specific ID. + """ + return chat_tasks.get(id, []) + + async def stop_task(task_id: str): """ Cancel a running task and remove it from the global task list. diff --git a/backend/open_webui/utils/middleware.py b/backend/open_webui/utils/middleware.py index f246bc84b3..4070bc697f 100644 --- a/backend/open_webui/utils/middleware.py +++ b/backend/open_webui/utils/middleware.py @@ -235,46 +235,30 @@ async def chat_completion_tools_handler( if isinstance(tool_result, str): tool = tools[tool_function_name] tool_id = tool.get("tool_id", "") + + tool_name = ( + f"{tool_id}/{tool_function_name}" + if tool_id + else f"{tool_function_name}" + ) if tool.get("metadata", {}).get("citation", False) or tool.get( "direct", False ): - + # Citation is enabled for this tool sources.append( { "source": { - "name": ( - f"TOOL:" + f"{tool_id}/{tool_function_name}" - if tool_id - else f"{tool_function_name}" - ), + "name": (f"TOOL:{tool_name}"), }, - "document": [tool_result, *tool_result_files], - "metadata": [ - { - "source": ( - f"TOOL:" + f"{tool_id}/{tool_function_name}" - if tool_id - else f"{tool_function_name}" - ) - } - ], + "document": [tool_result], + "metadata": [{"source": (f"TOOL:{tool_name}")}], } ) else: - sources.append( - { - "source": {}, - "document": [tool_result, *tool_result_files], - "metadata": [ - { - "source": ( - f"TOOL:" + f"{tool_id}/{tool_function_name}" - if tool_id - else f"{tool_function_name}" - ) - } - ], - } + # Citation is not enabled for this tool + body["messages"] = add_or_update_user_message( + f"\nTool `{tool_name}` Output: {tool_result}", + body["messages"], ) if ( @@ -550,13 +534,20 @@ async def chat_image_generation_handler( } ) - for image in images: - await __event_emitter__( - { - "type": "message", - "data": {"content": f"![Generated Image]({image['url']})\n"}, - } - ) + await __event_emitter__( + { + "type": "files", + "data": { + "files": [ + { + "type": "image", + "url": image["url"], + } + for image in images + ] + }, + } + ) system_message_content = "User is shown the generated image, tell the user that the image has been generated" except Exception as e: @@ -2261,7 +2252,9 @@ async def process_chat_response( await response.background() # background_tasks.add_task(post_response_handler, response, events) - task_id, _ = create_task(post_response_handler(response, events)) + task_id, _ = create_task( + post_response_handler(response, events), id=metadata["chat_id"] + ) return {"status": True, "task_id": task_id} else: diff --git a/backend/open_webui/utils/models.py b/backend/open_webui/utils/models.py index b631c2ae33..95d360bed8 100644 --- a/backend/open_webui/utils/models.py +++ b/backend/open_webui/utils/models.py @@ -114,9 +114,12 @@ async def get_all_models(request, user: UserModel = None): for custom_model in custom_models: if custom_model.base_model_id is None: for model in models: - if ( - custom_model.id == model["id"] - or custom_model.id == model["id"].split(":")[0] + if custom_model.id == model["id"] or ( + model.get("owned_by") == "ollama" + and custom_model.id + == model["id"].split(":")[ + 0 + ] # Ollama may return model ids in different formats (e.g., 'llama3' vs. 'llama3:7b') ): if custom_model.is_active: model["name"] = custom_model.name diff --git a/backend/open_webui/utils/redis.py b/backend/open_webui/utils/redis.py index baccb16ad6..e0a53e73d1 100644 --- a/backend/open_webui/utils/redis.py +++ b/backend/open_webui/utils/redis.py @@ -4,7 +4,7 @@ from redis import asyncio as aioredis from urllib.parse import urlparse -def parse_redis_sentinel_url(redis_url): +def parse_redis_service_url(redis_url): parsed_url = urlparse(redis_url) if parsed_url.scheme != "redis": raise ValueError("Invalid Redis URL scheme. Must be 'redis'.") @@ -20,7 +20,7 @@ def parse_redis_sentinel_url(redis_url): def get_redis_connection(redis_url, redis_sentinels, decode_responses=True): if redis_sentinels: - redis_config = parse_redis_sentinel_url(redis_url) + redis_config = parse_redis_service_url(redis_url) sentinel = redis.sentinel.Sentinel( redis_sentinels, port=redis_config["port"], @@ -45,65 +45,14 @@ def get_sentinels_from_env(sentinel_hosts_env, sentinel_port_env): return [] -class AsyncRedisSentinelManager(socketio.AsyncRedisManager): - def __init__( - self, - sentinel_hosts, - sentinel_port=26379, - redis_port=6379, - service="mymaster", - db=0, - username=None, - password=None, - channel="socketio", - write_only=False, - logger=None, - redis_options=None, - ): - """ - Initialize the Redis Sentinel Manager. - This implementation mostly replicates the __init__ of AsyncRedisManager and - overrides _redis_connect() with a version that uses Redis Sentinel - - :param sentinel_hosts: List of Sentinel hosts - :param sentinel_port: Sentinel Port - :param redis_port: Redis Port (currently unsupported by aioredis!) - :param service: Master service name in Sentinel - :param db: Redis database to use - :param username: Redis username (if any) (currently unsupported by aioredis!) - :param password: Redis password (if any) - :param channel: The channel name on which the server sends and receives - notifications. Must be the same in all the servers. - :param write_only: If set to ``True``, only initialize to emit events. The - default of ``False`` initializes the class for emitting - and receiving. - :param redis_options: additional keyword arguments to be passed to - ``aioredis.from_url()``. - """ - self._sentinels = [(host, sentinel_port) for host in sentinel_hosts] - self._redis_port = redis_port - self._service = service - self._db = db - self._username = username - self._password = password - self._channel = channel - self.redis_options = redis_options or {} - - # connect and call grandparent constructor - self._redis_connect() - super(socketio.AsyncRedisManager, self).__init__( - channel=channel, write_only=write_only, logger=logger - ) - - def _redis_connect(self): - """Establish connections to Redis through Sentinel.""" - sentinel = aioredis.sentinel.Sentinel( - self._sentinels, - port=self._redis_port, - db=self._db, - password=self._password, - **self.redis_options, - ) - - self.redis = sentinel.master_for(self._service) - self.pubsub = self.redis.pubsub(ignore_subscribe_messages=True) +def get_sentinel_url_from_env(redis_url, sentinel_hosts_env, sentinel_port_env): + redis_config = parse_redis_service_url(redis_url) + username = redis_config["username"] or "" + password = redis_config["password"] or "" + auth_part = "" + if username or password: + auth_part = f"{username}:{password}@" + hosts_part = ",".join( + f"{host}:{sentinel_port_env}" for host in sentinel_hosts_env.split(",") + ) + return f"redis+sentinel://{auth_part}{hosts_part}/{redis_config['db']}/{redis_config['service']}" diff --git a/backend/open_webui/utils/task.py b/backend/open_webui/utils/task.py index 3a8b4b0a42..66bdb4b3e2 100644 --- a/backend/open_webui/utils/task.py +++ b/backend/open_webui/utils/task.py @@ -152,6 +152,8 @@ def rag_template(template: str, context: str, query: str): if template.strip() == "": template = DEFAULT_RAG_TEMPLATE + template = prompt_template(template) + if "[context]" not in template and "{{CONTEXT}}" not in template: log.debug( "WARNING: The RAG template does not contain the '[context]' or '{{CONTEXT}}' placeholder." diff --git a/backend/requirements.txt b/backend/requirements.txt index def5cfe233..f0cf262ee6 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -3,7 +3,7 @@ uvicorn[standard]==0.34.0 pydantic==2.10.6 python-multipart==0.0.20 -python-socketio==5.11.3 +python-socketio==5.13.0 python-jose==3.4.0 passlib[bcrypt]==1.7.4 diff --git a/backend/start.sh b/backend/start.sh index b9a30fd3da..4588e4c348 100755 --- a/backend/start.sh +++ b/backend/start.sh @@ -4,8 +4,8 @@ SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) cd "$SCRIPT_DIR" || exit # Add conditional Playwright browser installation -if [[ "${RAG_WEB_LOADER_ENGINE,,}" == "playwright" ]]; then - if [[ -z "${PLAYWRIGHT_WS_URI}" ]]; then +if [[ "${WEB_LOADER_ENGINE,,}" == "playwright" ]]; then + if [[ -z "${PLAYWRIGHT_WS_URL}" ]]; then echo "Installing Playwright browsers..." playwright install chromium playwright install-deps chromium diff --git a/backend/start_windows.bat b/backend/start_windows.bat index 661ecc494e..8d9aae3ac6 100644 --- a/backend/start_windows.bat +++ b/backend/start_windows.bat @@ -7,8 +7,8 @@ SET "SCRIPT_DIR=%~dp0" cd /d "%SCRIPT_DIR%" || exit /b :: Add conditional Playwright browser installation -IF /I "%RAG_WEB_LOADER_ENGINE%" == "playwright" ( - IF "%PLAYWRIGHT_WS_URI%" == "" ( +IF /I "%WEB_LOADER_ENGINE%" == "playwright" ( + IF "%PLAYWRIGHT_WS_URL%" == "" ( echo Installing Playwright browsers... playwright install chromium playwright install-deps chromium diff --git a/docker-compose.playwright.yaml b/docker-compose.playwright.yaml index fe570bed01..fa2b49ff9a 100644 --- a/docker-compose.playwright.yaml +++ b/docker-compose.playwright.yaml @@ -6,5 +6,5 @@ services: open-webui: environment: - - 'RAG_WEB_LOADER_ENGINE=playwright' - - 'PLAYWRIGHT_WS_URI=ws://playwright:3000' \ No newline at end of file + - 'WEB_LOADER_ENGINE=playwright' + - 'PLAYWRIGHT_WS_URL=ws://playwright:3000' diff --git a/package-lock.json b/package-lock.json index 86b36e96b1..c4f6948bd4 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "open-webui", - "version": "0.6.2", + "version": "0.6.5", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "open-webui", - "version": "0.6.2", + "version": "0.6.5", "dependencies": { "@azure/msal-browser": "^4.5.0", "@codemirror/lang-javascript": "^6.2.2", @@ -33,7 +33,7 @@ "codemirror-lang-hcl": "^0.0.0-beta.2", "crc-32": "^1.2.2", "dayjs": "^1.11.10", - "dompurify": "^3.1.6", + "dompurify": "^3.2.5", "eventsource-parser": "^1.1.2", "file-saver": "^2.0.5", "fuse.js": "^7.0.0", @@ -49,7 +49,7 @@ "katex": "^0.16.21", "kokoro-js": "^1.1.1", "marked": "^9.1.0", - "mermaid": "^10.9.3", + "mermaid": "^11.6.0", "paneforge": "^0.0.6", "panzoom": "^9.4.3", "prosemirror-commands": "^1.6.0", @@ -140,6 +140,28 @@ "node": ">=6.0.0" } }, + "node_modules/@antfu/install-pkg": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/@antfu/install-pkg/-/install-pkg-1.0.0.tgz", + "integrity": "sha512-xvX6P/lo1B3ej0OsaErAjqgFYzYVcJpamjLAFLYh9vRJngBrMoUG7aVnrGTeqM7yxbyTD5p3F2+0/QUEh8Vzhw==", + "license": "MIT", + "dependencies": { + "package-manager-detector": "^0.2.8", + "tinyexec": "^0.3.2" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/@antfu/utils": { + "version": "8.1.1", + "resolved": "https://registry.npmjs.org/@antfu/utils/-/utils-8.1.1.tgz", + "integrity": "sha512-Mex9nXf9vR6AhcXmMrlz/HVgYYZpVGJ6YlPgwl7UnaFpnshXs6EK/oa5Gpf3CzENMjkvEx2tQtntGnb7UtSTOQ==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, "node_modules/@azure/msal-browser": { "version": "4.5.0", "resolved": "https://registry.npmjs.org/@azure/msal-browser/-/msal-browser-4.5.0.tgz", @@ -174,9 +196,10 @@ } }, "node_modules/@braintree/sanitize-url": { - "version": "6.0.4", - "resolved": "https://registry.npmjs.org/@braintree/sanitize-url/-/sanitize-url-6.0.4.tgz", - "integrity": "sha512-s3jaWicZd0pkP0jf5ysyHUI/RE7MHos6qlToFcGWXVp+ykHOy77OUMrfbgJ9it2C5bow7OIQwYYaHjk9XlBQ2A==" + "version": "7.1.1", + "resolved": "https://registry.npmjs.org/@braintree/sanitize-url/-/sanitize-url-7.1.1.tgz", + "integrity": "sha512-i1L7noDNxtFyL5DmZafWy1wRVhGehQmzZaz1HiN5e7iylJMSZR7ekOV7NsIqa5qBldlLrsKv4HbgFUVlQrz8Mw==", + "license": "MIT" }, "node_modules/@bufbuild/protobuf": { "version": "2.2.2", @@ -185,6 +208,45 @@ "devOptional": true, "license": "(Apache-2.0 AND BSD-3-Clause)" }, + "node_modules/@chevrotain/cst-dts-gen": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/@chevrotain/cst-dts-gen/-/cst-dts-gen-11.0.3.tgz", + "integrity": "sha512-BvIKpRLeS/8UbfxXxgC33xOumsacaeCKAjAeLyOn7Pcp95HiRbrpl14S+9vaZLolnbssPIUuiUd8IvgkRyt6NQ==", + "license": "Apache-2.0", + "dependencies": { + "@chevrotain/gast": "11.0.3", + "@chevrotain/types": "11.0.3", + "lodash-es": "4.17.21" + } + }, + "node_modules/@chevrotain/gast": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/@chevrotain/gast/-/gast-11.0.3.tgz", + "integrity": "sha512-+qNfcoNk70PyS/uxmj3li5NiECO+2YKZZQMbmjTqRI3Qchu8Hig/Q9vgkHpI3alNjr7M+a2St5pw5w5F6NL5/Q==", + "license": "Apache-2.0", + "dependencies": { + "@chevrotain/types": "11.0.3", + "lodash-es": "4.17.21" + } + }, + "node_modules/@chevrotain/regexp-to-ast": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/@chevrotain/regexp-to-ast/-/regexp-to-ast-11.0.3.tgz", + "integrity": "sha512-1fMHaBZxLFvWI067AVbGJav1eRY7N8DDvYCTwGBiE/ytKBgP8azTdgyrKyWZ9Mfh09eHWb5PgTSO8wi7U824RA==", + "license": "Apache-2.0" + }, + "node_modules/@chevrotain/types": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/@chevrotain/types/-/types-11.0.3.tgz", + "integrity": "sha512-gsiM3G8b58kZC2HaWR50gu6Y1440cHiJ+i3JUvcp/35JchYejb2+5MVeJK0iKThYpAa/P2PYFV4hoi44HD+aHQ==", + "license": "Apache-2.0" + }, + "node_modules/@chevrotain/utils": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/@chevrotain/utils/-/utils-11.0.3.tgz", + "integrity": "sha512-YslZMgtJUyuMbZ+aKvfF3x1f5liK4mWNxghFRv7jqRR9C3R3fAOGTTKvxXDa2Y1s9zSbcpuO0cAxDYsc9SrXoQ==", + "license": "Apache-2.0" + }, "node_modules/@codemirror/autocomplete": { "version": "6.16.2", "resolved": "https://registry.npmjs.org/@codemirror/autocomplete/-/autocomplete-6.16.2.tgz", @@ -1241,6 +1303,103 @@ "integrity": "sha512-6EwiSjwWYP7pTckG6I5eyFANjPhmPjUX9JRLUSfNPC7FX7zK9gyZAfUEaECL6ALTpGX5AjnBq3C9XmVWPitNpw==", "dev": true }, + "node_modules/@iconify/types": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@iconify/types/-/types-2.0.0.tgz", + "integrity": "sha512-+wluvCrRhXrhyOmRDJ3q8mux9JkKy5SJ/v8ol2tu4FVjyYvtEzkc/3pK15ET6RKg4b4w4BmTk1+gsCUhf21Ykg==", + "license": "MIT" + }, + "node_modules/@iconify/utils": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/@iconify/utils/-/utils-2.3.0.tgz", + "integrity": "sha512-GmQ78prtwYW6EtzXRU1rY+KwOKfz32PD7iJh6Iyqw68GiKuoZ2A6pRtzWONz5VQJbp50mEjXh/7NkumtrAgRKA==", + "license": "MIT", + "dependencies": { + "@antfu/install-pkg": "^1.0.0", + "@antfu/utils": "^8.1.0", + "@iconify/types": "^2.0.0", + "debug": "^4.4.0", + "globals": "^15.14.0", + "kolorist": "^1.8.0", + "local-pkg": "^1.0.0", + "mlly": "^1.7.4" + } + }, + "node_modules/@iconify/utils/node_modules/confbox": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/confbox/-/confbox-0.2.2.tgz", + "integrity": "sha512-1NB+BKqhtNipMsov4xI/NnhCKp9XG9NamYp5PVm9klAT0fsrNPjaFICsCFhNhwZJKNh7zB/3q8qXz0E9oaMNtQ==", + "license": "MIT" + }, + "node_modules/@iconify/utils/node_modules/debug": { + "version": "4.4.0", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.0.tgz", + "integrity": "sha512-6WTZ/IxCY/T6BALoZHaE4ctp9xm+Z5kY/pzYaCHRFeyVhojxlrm+46y68HA6hr0TcwEssoxNiDEUJQjfPZ/RYA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/@iconify/utils/node_modules/globals": { + "version": "15.15.0", + "resolved": "https://registry.npmjs.org/globals/-/globals-15.15.0.tgz", + "integrity": "sha512-7ACyT3wmyp3I61S4fG682L0VA2RGD9otkqGJIwNUMF1SWUombIIk+af1unuDYgMm082aHYwD+mzJvv9Iu8dsgg==", + "license": "MIT", + "engines": { + "node": ">=18" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/@iconify/utils/node_modules/local-pkg": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/local-pkg/-/local-pkg-1.1.1.tgz", + "integrity": "sha512-WunYko2W1NcdfAFpuLUoucsgULmgDBRkdxHxWQ7mK0cQqwPiy8E1enjuRBrhLtZkB5iScJ1XIPdhVEFK8aOLSg==", + "license": "MIT", + "dependencies": { + "mlly": "^1.7.4", + "pkg-types": "^2.0.1", + "quansync": "^0.2.8" + }, + "engines": { + "node": ">=14" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/@iconify/utils/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, + "node_modules/@iconify/utils/node_modules/pathe": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz", + "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==", + "license": "MIT" + }, + "node_modules/@iconify/utils/node_modules/pkg-types": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/pkg-types/-/pkg-types-2.1.0.tgz", + "integrity": "sha512-wmJwA+8ihJixSoHKxZJRBQG1oY8Yr9pGLzRmSsNms0iNWyHHAlZCa7mmKiFR10YPZuz/2k169JiS/inOjBCZ2A==", + "license": "MIT", + "dependencies": { + "confbox": "^0.2.1", + "exsolve": "^1.0.1", + "pathe": "^2.0.3" + } + }, "node_modules/@img/sharp-darwin-arm64": { "version": "0.33.5", "resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.33.5.tgz", @@ -1881,6 +2040,15 @@ "svelte": ">=3 <5" } }, + "node_modules/@mermaid-js/parser": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/@mermaid-js/parser/-/parser-0.4.0.tgz", + "integrity": "sha512-wla8XOWvQAwuqy+gxiZqY+c7FokraOTHRWMsbB4AgRx9Sy7zKslNyejy7E+a77qHfey5GXw/ik3IXv/NHMJgaA==", + "license": "MIT", + "dependencies": { + "langium": "3.3.1" + } + }, "node_modules/@mixmark-io/domino": { "version": "2.2.0", "resolved": "https://registry.npmjs.org/@mixmark-io/domino/-/domino-2.2.0.tgz", @@ -3111,11 +3279,101 @@ "resolved": "https://registry.npmjs.org/@types/cookie/-/cookie-0.6.0.tgz", "integrity": "sha512-4Kh9a6B2bQciAhf7FSuMRRkUWecJgJu9nPnx3yzpsfXX/c50REIqpHY4C82bXP90qrLtXtkDxTZosYO3UpOwlA==" }, + "node_modules/@types/d3": { + "version": "7.4.3", + "resolved": "https://registry.npmjs.org/@types/d3/-/d3-7.4.3.tgz", + "integrity": "sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww==", + "license": "MIT", + "dependencies": { + "@types/d3-array": "*", + "@types/d3-axis": "*", + "@types/d3-brush": "*", + "@types/d3-chord": "*", + "@types/d3-color": "*", + "@types/d3-contour": "*", + "@types/d3-delaunay": "*", + "@types/d3-dispatch": "*", + "@types/d3-drag": "*", + "@types/d3-dsv": "*", + "@types/d3-ease": "*", + "@types/d3-fetch": "*", + "@types/d3-force": "*", + "@types/d3-format": "*", + "@types/d3-geo": "*", + "@types/d3-hierarchy": "*", + "@types/d3-interpolate": "*", + "@types/d3-path": "*", + "@types/d3-polygon": "*", + "@types/d3-quadtree": "*", + "@types/d3-random": "*", + "@types/d3-scale": "*", + "@types/d3-scale-chromatic": "*", + "@types/d3-selection": "*", + "@types/d3-shape": "*", + "@types/d3-time": "*", + "@types/d3-time-format": "*", + "@types/d3-timer": "*", + "@types/d3-transition": "*", + "@types/d3-zoom": "*" + } + }, + "node_modules/@types/d3-array": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/@types/d3-array/-/d3-array-3.2.1.tgz", + "integrity": "sha512-Y2Jn2idRrLzUfAKV2LyRImR+y4oa2AntrgID95SHJxuMUrkNXmanDSed71sRNZysveJVt1hLLemQZIady0FpEg==", + "license": "MIT" + }, + "node_modules/@types/d3-axis": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-axis/-/d3-axis-3.0.6.tgz", + "integrity": "sha512-pYeijfZuBd87T0hGn0FO1vQ/cgLk6E1ALJjfkC0oJ8cbwkZl3TpgS8bVBLZN+2jjGgg38epgxb2zmoGtSfvgMw==", + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-brush": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-brush/-/d3-brush-3.0.6.tgz", + "integrity": "sha512-nH60IZNNxEcrh6L1ZSMNA28rj27ut/2ZmI3r96Zd+1jrZD++zD3LsMIjWlvg4AYrHn/Pqz4CF3veCxGjtbqt7A==", + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-chord": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-chord/-/d3-chord-3.0.6.tgz", + "integrity": "sha512-LFYWWd8nwfwEmTZG9PfQxd17HbNPksHBiJHaKuY1XeqscXacsS2tyoo6OdRsjf+NQYeB6XrNL3a25E3gH69lcg==", + "license": "MIT" + }, "node_modules/@types/d3-color": { "version": "3.1.3", "resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-3.1.3.tgz", "integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==" }, + "node_modules/@types/d3-contour": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-contour/-/d3-contour-3.0.6.tgz", + "integrity": "sha512-BjzLgXGnCWjUSYGfH1cpdo41/hgdWETu4YxpezoztawmqsvCeep+8QGfiY6YbDvfgHz/DkjeIkkZVJavB4a3rg==", + "license": "MIT", + "dependencies": { + "@types/d3-array": "*", + "@types/geojson": "*" + } + }, + "node_modules/@types/d3-delaunay": { + "version": "6.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-delaunay/-/d3-delaunay-6.0.4.tgz", + "integrity": "sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==", + "license": "MIT" + }, + "node_modules/@types/d3-dispatch": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-dispatch/-/d3-dispatch-3.0.6.tgz", + "integrity": "sha512-4fvZhzMeeuBJYZXRXrRIQnvUYfyXwYmLsdiN7XXmVNQKKw1cM8a5WdID0g1hVFZDqT9ZqZEY5pD44p24VS7iZQ==", + "license": "MIT" + }, "node_modules/@types/d3-drag": { "version": "3.0.7", "resolved": "https://registry.npmjs.org/@types/d3-drag/-/d3-drag-3.0.7.tgz", @@ -3124,6 +3382,54 @@ "@types/d3-selection": "*" } }, + "node_modules/@types/d3-dsv": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/@types/d3-dsv/-/d3-dsv-3.0.7.tgz", + "integrity": "sha512-n6QBF9/+XASqcKK6waudgL0pf/S5XHPPI8APyMLLUHd8NqouBGLsU8MgtO7NINGtPBtk9Kko/W4ea0oAspwh9g==", + "license": "MIT" + }, + "node_modules/@types/d3-ease": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz", + "integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==", + "license": "MIT" + }, + "node_modules/@types/d3-fetch": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/@types/d3-fetch/-/d3-fetch-3.0.7.tgz", + "integrity": "sha512-fTAfNmxSb9SOWNB9IoG5c8Hg6R+AzUHDRlsXsDZsNp6sxAEOP0tkP3gKkNSO/qmHPoBFTxNrjDprVHDQDvo5aA==", + "license": "MIT", + "dependencies": { + "@types/d3-dsv": "*" + } + }, + "node_modules/@types/d3-force": { + "version": "3.0.10", + "resolved": "https://registry.npmjs.org/@types/d3-force/-/d3-force-3.0.10.tgz", + "integrity": "sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==", + "license": "MIT" + }, + "node_modules/@types/d3-format": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-format/-/d3-format-3.0.4.tgz", + "integrity": "sha512-fALi2aI6shfg7vM5KiR1wNJnZ7r6UuggVqtDA+xiEdPZQwy/trcQaHnwShLuLdta2rTymCNpxYTiMZX/e09F4g==", + "license": "MIT" + }, + "node_modules/@types/d3-geo": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@types/d3-geo/-/d3-geo-3.1.0.tgz", + "integrity": "sha512-856sckF0oP/diXtS4jNsiQw/UuK5fQG8l/a9VVLeSouf1/PPbBE1i1W852zVwKwYCBkFJJB7nCFTbk6UMEXBOQ==", + "license": "MIT", + "dependencies": { + "@types/geojson": "*" + } + }, + "node_modules/@types/d3-hierarchy": { + "version": "3.1.7", + "resolved": "https://registry.npmjs.org/@types/d3-hierarchy/-/d3-hierarchy-3.1.7.tgz", + "integrity": "sha512-tJFtNoYBtRtkNysX1Xq4sxtjK8YgoWUNpIiUee0/jHGRwqvzYxkq0hGVbbOGSz+JgFxxRu4K8nb3YpG3CMARtg==", + "license": "MIT" + }, "node_modules/@types/d3-interpolate": { "version": "3.0.4", "resolved": "https://registry.npmjs.org/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz", @@ -3132,28 +3438,76 @@ "@types/d3-color": "*" } }, + "node_modules/@types/d3-path": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/@types/d3-path/-/d3-path-3.1.1.tgz", + "integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==", + "license": "MIT" + }, + "node_modules/@types/d3-polygon": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-polygon/-/d3-polygon-3.0.2.tgz", + "integrity": "sha512-ZuWOtMaHCkN9xoeEMr1ubW2nGWsp4nIql+OPQRstu4ypeZ+zk3YKqQT0CXVe/PYqrKpZAi+J9mTs05TKwjXSRA==", + "license": "MIT" + }, + "node_modules/@types/d3-quadtree": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/@types/d3-quadtree/-/d3-quadtree-3.0.6.tgz", + "integrity": "sha512-oUzyO1/Zm6rsxKRHA1vH0NEDG58HrT5icx/azi9MF1TWdtttWl0UIUsjEQBBh+SIkrpd21ZjEv7ptxWys1ncsg==", + "license": "MIT" + }, + "node_modules/@types/d3-random": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/@types/d3-random/-/d3-random-3.0.3.tgz", + "integrity": "sha512-Imagg1vJ3y76Y2ea0871wpabqp613+8/r0mCLEBfdtqC7xMSfj9idOnmBYyMoULfHePJyxMAw3nWhJxzc+LFwQ==", + "license": "MIT" + }, "node_modules/@types/d3-scale": { - "version": "4.0.8", - "resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.8.tgz", - "integrity": "sha512-gkK1VVTr5iNiYJ7vWDI+yUFFlszhNMtVeneJ6lUTKPjprsvLLI9/tgEGiXJOnlINJA8FyA88gfnQsHbybVZrYQ==", + "version": "4.0.9", + "resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.9.tgz", + "integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==", + "license": "MIT", "dependencies": { "@types/d3-time": "*" } }, "node_modules/@types/d3-scale-chromatic": { - "version": "3.0.3", - "resolved": "https://registry.npmjs.org/@types/d3-scale-chromatic/-/d3-scale-chromatic-3.0.3.tgz", - "integrity": "sha512-laXM4+1o5ImZv3RpFAsTRn3TEkzqkytiOY0Dz0sq5cnd1dtNlk6sHLon4OvqaiJb28T0S/TdsBI3Sjsy+keJrw==" + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@types/d3-scale-chromatic/-/d3-scale-chromatic-3.1.0.tgz", + "integrity": "sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==", + "license": "MIT" }, "node_modules/@types/d3-selection": { "version": "3.0.10", "resolved": "https://registry.npmjs.org/@types/d3-selection/-/d3-selection-3.0.10.tgz", "integrity": "sha512-cuHoUgS/V3hLdjJOLTT691+G2QoqAjCVLmr4kJXR4ha56w1Zdu8UUQ5TxLRqudgNjwXeQxKMq4j+lyf9sWuslg==" }, + "node_modules/@types/d3-shape": { + "version": "3.1.7", + "resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.7.tgz", + "integrity": "sha512-VLvUQ33C+3J+8p+Daf+nYSOsjB4GXp19/S/aGo60m9h1v6XaxjiT82lKVWJCfzhtuZ3yD7i/TPeC/fuKLLOSmg==", + "license": "MIT", + "dependencies": { + "@types/d3-path": "*" + } + }, "node_modules/@types/d3-time": { - "version": "3.0.3", - "resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.3.tgz", - "integrity": "sha512-2p6olUZ4w3s+07q3Tm2dbiMZy5pCDfYwtLXXHUnVzXgQlZ/OyPtUz6OL382BkOuGlLXqfT+wqv8Fw2v8/0geBw==" + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.4.tgz", + "integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==", + "license": "MIT" + }, + "node_modules/@types/d3-time-format": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/@types/d3-time-format/-/d3-time-format-4.0.3.tgz", + "integrity": "sha512-5xg9rC+wWL8kdDj153qZcsJ0FWiFt0J5RB6LYUNZjwSnesfblqrI/bJ1wBdJ8OQfncgbJG5+2F+qfqnqyzYxyg==", + "license": "MIT" + }, + "node_modules/@types/d3-timer": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-timer/-/d3-timer-3.0.2.tgz", + "integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==", + "license": "MIT" }, "node_modules/@types/d3-transition": { "version": "3.0.8", @@ -3172,19 +3526,17 @@ "@types/d3-selection": "*" } }, - "node_modules/@types/debug": { - "version": "4.1.12", - "resolved": "https://registry.npmjs.org/@types/debug/-/debug-4.1.12.tgz", - "integrity": "sha512-vIChWdVG3LG1SMxEvI/AK+FWJthlrqlTu7fbrlywTkkaONwk/UAGaULXRlf8vkzFBLVm0zkMdCquhL5aOjhXPQ==", - "dependencies": { - "@types/ms": "*" - } - }, "node_modules/@types/estree": { "version": "1.0.5", "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.5.tgz", "integrity": "sha512-/kYRxGDLWzHOB7q+wtSUQlFrtcdUccpfy+X+9iMBpHK8QLLhx2wIPYuS5DYtR9Wa/YlZAbIovy7qVdB1Aq6Lyw==" }, + "node_modules/@types/geojson": { + "version": "7946.0.16", + "resolved": "https://registry.npmjs.org/@types/geojson/-/geojson-7946.0.16.tgz", + "integrity": "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==", + "license": "MIT" + }, "node_modules/@types/hast": { "version": "3.0.4", "resolved": "https://registry.npmjs.org/@types/hast/-/hast-3.0.4.tgz", @@ -3215,14 +3567,6 @@ "@types/mdurl": "^2" } }, - "node_modules/@types/mdast": { - "version": "3.0.15", - "resolved": "https://registry.npmjs.org/@types/mdast/-/mdast-3.0.15.tgz", - "integrity": "sha512-LnwD+mUEfxWMa1QpDraczIn6k0Ee3SMicuYSSzS6ZYl2gKS09EClnJYGd8Du6rfc5r/GZEk5o1mRb8TaTj03sQ==", - "dependencies": { - "@types/unist": "^2" - } - }, "node_modules/@types/mdurl": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/@types/mdurl/-/mdurl-2.0.0.tgz", @@ -3234,11 +3578,6 @@ "integrity": "sha512-Klz949h02Gz2uZCMGwDUSDS1YBlTdDDgbWHi+81l29tQALUtvz4rAYi5uoVhE5Lagoq6DeqAUlbrHvW/mXDgdQ==", "dev": true }, - "node_modules/@types/ms": { - "version": "0.7.34", - "resolved": "https://registry.npmjs.org/@types/ms/-/ms-0.7.34.tgz", - "integrity": "sha512-nG96G3Wp6acyAgJqGasjODb+acrI7KltPiRxzHPXnP3NgI28bpQDRv53olbqGXbfcgF5aiiHmO3xpwEpS5Ld9g==" - }, "node_modules/@types/node": { "version": "20.11.30", "resolved": "https://registry.npmjs.org/@types/node/-/node-20.11.30.tgz", @@ -3299,7 +3638,8 @@ "node_modules/@types/unist": { "version": "2.0.10", "resolved": "https://registry.npmjs.org/@types/unist/-/unist-2.0.10.tgz", - "integrity": "sha512-IfYcSBWE3hLpBg8+X2SEa8LVkJdJEkT2Ese2aaLs3ptGdVtABxndrMaxuFlQ1qdFf9Q5rDvDpxI3WwgvKFAsQA==" + "integrity": "sha512-IfYcSBWE3hLpBg8+X2SEa8LVkJdJEkT2Ese2aaLs3ptGdVtABxndrMaxuFlQ1qdFf9Q5rDvDpxI3WwgvKFAsQA==", + "peer": true }, "node_modules/@types/yauzl": { "version": "2.10.3", @@ -3657,9 +3997,10 @@ } }, "node_modules/acorn": { - "version": "8.11.3", - "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.11.3.tgz", - "integrity": "sha512-Y9rRfJG5jcKOE0CLisYbojUjIrIEE7AGMzA/Sm4BslANhbS+cDMpgBdcPT91oJ7OuJ9hYJBx59RjbhxVnrF8Xg==", + "version": "8.14.1", + "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.14.1.tgz", + "integrity": "sha512-OvQ/2pUDKmgfCg++xsTX1wGxfTaszcHVcTctW4UJB4hibJx2HXxxO5UmVgyjMa+ZDsiaf5wWLXYpRWMmBI0QHg==", + "license": "MIT", "bin": { "acorn": "bin/acorn" }, @@ -4330,15 +4671,6 @@ "url": "https://github.com/chalk/chalk?sponsor=1" } }, - "node_modules/character-entities": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/character-entities/-/character-entities-2.0.2.tgz", - "integrity": "sha512-shx7oQ0Awen/BRIdkjkvz54PnEEI/EjwXDSIZp86/KKdbafHh1Df/RYGBhn4hbe2+uKC9FnT5UCEdyPz3ai9hQ==", - "funding": { - "type": "github", - "url": "https://github.com/sponsors/wooorm" - } - }, "node_modules/check-error": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/check-error/-/check-error-1.0.3.tgz", @@ -4415,6 +4747,32 @@ "node": ">=18.17" } }, + "node_modules/chevrotain": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/chevrotain/-/chevrotain-11.0.3.tgz", + "integrity": "sha512-ci2iJH6LeIkvP9eJW6gpueU8cnZhv85ELY8w8WiFtNjMHA5ad6pQLaJo9mEly/9qUyCpvqX8/POVUTf18/HFdw==", + "license": "Apache-2.0", + "dependencies": { + "@chevrotain/cst-dts-gen": "11.0.3", + "@chevrotain/gast": "11.0.3", + "@chevrotain/regexp-to-ast": "11.0.3", + "@chevrotain/types": "11.0.3", + "@chevrotain/utils": "11.0.3", + "lodash-es": "4.17.21" + } + }, + "node_modules/chevrotain-allstar": { + "version": "0.3.1", + "resolved": "https://registry.npmjs.org/chevrotain-allstar/-/chevrotain-allstar-0.3.1.tgz", + "integrity": "sha512-b7g+y9A0v4mxCW1qUhf3BSVPg+/NvGErk/dOkrDaHA0nQIQGAtrOjlX//9OQtRlSCy+x9rfB5N8yC71lH1nvMw==", + "license": "MIT", + "dependencies": { + "lodash-es": "^4.17.21" + }, + "peerDependencies": { + "chevrotain": "^11.0.0" + } + }, "node_modules/chokidar": { "version": "3.6.0", "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz", @@ -4762,10 +5120,10 @@ "dev": true }, "node_modules/confbox": { - "version": "0.1.7", - "resolved": "https://registry.npmjs.org/confbox/-/confbox-0.1.7.tgz", - "integrity": "sha512-uJcB/FKZtBMCJpK8MQji6bJHgu1tixKPxRLeGkNzBoOZzpnZUJm0jm2/sBDWcuBx1dYgxV4JU+g5hmNxCyAmdA==", - "dev": true + "version": "0.1.8", + "resolved": "https://registry.npmjs.org/confbox/-/confbox-0.1.8.tgz", + "integrity": "sha512-RMtmw0iFkeR4YV+fUOSucriAQNb9g8zFR52MWCtl+cCZOFRNL6zeB395vPzFhEjjn4fMxXudmELnl/KF/WrK6w==", + "license": "MIT" }, "node_modules/convert-source-map": { "version": "2.0.0", @@ -5022,9 +5380,10 @@ } }, "node_modules/cytoscape": { - "version": "3.29.2", - "resolved": "https://registry.npmjs.org/cytoscape/-/cytoscape-3.29.2.tgz", - "integrity": "sha512-2G1ycU28Nh7OHT9rkXRLpCDP30MKH1dXJORZuBhtEhEW7pKwgPi77ImqlCWinouyE1PNepIOGZBOrE84DG7LyQ==", + "version": "3.31.2", + "resolved": "https://registry.npmjs.org/cytoscape/-/cytoscape-3.31.2.tgz", + "integrity": "sha512-/eOXg2uGdMdpGlEes5Sf6zE+jUG+05f3htFNQIxLxduOH/SsaUZiPBfAwP1btVIVzsnhiNOdi+hvDRLYfMZjGw==", + "license": "MIT", "engines": { "node": ">=0.10" } @@ -5040,10 +5399,38 @@ "cytoscape": "^3.2.0" } }, + "node_modules/cytoscape-fcose": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/cytoscape-fcose/-/cytoscape-fcose-2.2.0.tgz", + "integrity": "sha512-ki1/VuRIHFCzxWNrsshHYPs6L7TvLu3DL+TyIGEsRcvVERmxokbf5Gdk7mFxZnTdiGtnA4cfSmjZJMviqSuZrQ==", + "license": "MIT", + "dependencies": { + "cose-base": "^2.2.0" + }, + "peerDependencies": { + "cytoscape": "^3.2.0" + } + }, + "node_modules/cytoscape-fcose/node_modules/cose-base": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/cose-base/-/cose-base-2.2.0.tgz", + "integrity": "sha512-AzlgcsCbUMymkADOJtQm3wO9S3ltPfYOFD5033keQn9NJzIbtnZj+UdBJe7DYml/8TdbtHJW3j58SOnKhWY/5g==", + "license": "MIT", + "dependencies": { + "layout-base": "^2.0.0" + } + }, + "node_modules/cytoscape-fcose/node_modules/layout-base": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/layout-base/-/layout-base-2.0.1.tgz", + "integrity": "sha512-dp3s92+uNI1hWIpPGH3jK2kxE2lMjdXdr+DH8ynZHpd6PUlH6x6cbuXnoMmiNumznqaNO31xu9e79F0uuZ0JFg==", + "license": "MIT" + }, "node_modules/d3": { "version": "7.9.0", "resolved": "https://registry.npmjs.org/d3/-/d3-7.9.0.tgz", "integrity": "sha512-e1U46jVP+w7Iut8Jt8ri1YsPOvFpg46k+K8TpCb0P+zjCkjkPnV7WzfDJzMHy1LnA+wj5pLT1wjO901gLXeEhA==", + "license": "ISC", "dependencies": { "d3-array": "3", "d3-axis": "3", @@ -5084,6 +5471,7 @@ "version": "3.2.4", "resolved": "https://registry.npmjs.org/d3-array/-/d3-array-3.2.4.tgz", "integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==", + "license": "ISC", "dependencies": { "internmap": "1 - 2" }, @@ -5095,6 +5483,7 @@ "version": "3.0.0", "resolved": "https://registry.npmjs.org/d3-axis/-/d3-axis-3.0.0.tgz", "integrity": "sha512-IH5tgjV4jE/GhHkRV0HiVYPDtvfjHQlQfJHs0usq7M30XcSBvOotpmH1IgkcXsO/5gEQZD43B//fc7SRT5S+xw==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5103,6 +5492,7 @@ "version": "3.0.0", "resolved": "https://registry.npmjs.org/d3-brush/-/d3-brush-3.0.0.tgz", "integrity": "sha512-ALnjWlVYkXsVIGlOsuWH1+3udkYFI48Ljihfnh8FZPF2QS9o+PzGLBslO0PjzVoHLZ2KCVgAM8NVkXPJB2aNnQ==", + "license": "ISC", "dependencies": { "d3-dispatch": "1 - 3", "d3-drag": "2 - 3", @@ -5118,6 +5508,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/d3-chord/-/d3-chord-3.0.1.tgz", "integrity": "sha512-VE5S6TNa+j8msksl7HwjxMHDM2yNK3XCkusIlpX5kwauBfXuyLAtNg9jCp/iHH61tgI4sb6R/EIMWCqEIdjT/g==", + "license": "ISC", "dependencies": { "d3-path": "1 - 3" }, @@ -5137,6 +5528,7 @@ "version": "4.0.2", "resolved": "https://registry.npmjs.org/d3-contour/-/d3-contour-4.0.2.tgz", "integrity": "sha512-4EzFTRIikzs47RGmdxbeUvLWtGedDUNkTcmzoeyg4sP/dvCexO47AaQL7VKy/gul85TOxw+IBgA8US2xwbToNA==", + "license": "ISC", "dependencies": { "d3-array": "^3.2.0" }, @@ -5148,6 +5540,7 @@ "version": "6.0.4", "resolved": "https://registry.npmjs.org/d3-delaunay/-/d3-delaunay-6.0.4.tgz", "integrity": "sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==", + "license": "ISC", "dependencies": { "delaunator": "5" }, @@ -5179,6 +5572,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/d3-dsv/-/d3-dsv-3.0.1.tgz", "integrity": "sha512-UG6OvdI5afDIFP9w4G0mNq50dSOsXHJaRE8arAS5o9ApWnIElp8GZw1Dun8vP8OyHOZ/QJUKUJwxiiCCnUwm+Q==", + "license": "ISC", "dependencies": { "commander": "7", "iconv-lite": "0.6", @@ -5203,6 +5597,7 @@ "version": "7.2.0", "resolved": "https://registry.npmjs.org/commander/-/commander-7.2.0.tgz", "integrity": "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw==", + "license": "MIT", "engines": { "node": ">= 10" } @@ -5219,6 +5614,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/d3-fetch/-/d3-fetch-3.0.1.tgz", "integrity": "sha512-kpkQIM20n3oLVBKGg6oHrUchHM3xODkTzjMoj7aWQFq5QEM+R6E4WkzT5+tojDY7yjez8KgCBRoj4aEr99Fdqw==", + "license": "ISC", "dependencies": { "d3-dsv": "1 - 3" }, @@ -5230,6 +5626,7 @@ "version": "3.0.0", "resolved": "https://registry.npmjs.org/d3-force/-/d3-force-3.0.0.tgz", "integrity": "sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==", + "license": "ISC", "dependencies": { "d3-dispatch": "1 - 3", "d3-quadtree": "1 - 3", @@ -5243,6 +5640,7 @@ "version": "3.1.0", "resolved": "https://registry.npmjs.org/d3-format/-/d3-format-3.1.0.tgz", "integrity": "sha512-YyUI6AEuY/Wpt8KWLgZHsIU86atmikuoOmCfommt0LYHiQSPjvX2AcFc38PX0CBpr2RCyZhjex+NS/LPOv6YqA==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5251,6 +5649,7 @@ "version": "3.1.1", "resolved": "https://registry.npmjs.org/d3-geo/-/d3-geo-3.1.1.tgz", "integrity": "sha512-637ln3gXKXOwhalDzinUgY83KzNWZRKbYubaG+fGVuc/dxO64RRljtCTnf5ecMyE1RIdtqpkVcq0IbtU2S8j2Q==", + "license": "ISC", "dependencies": { "d3-array": "2.5.0 - 3" }, @@ -5262,6 +5661,7 @@ "version": "3.1.2", "resolved": "https://registry.npmjs.org/d3-hierarchy/-/d3-hierarchy-3.1.2.tgz", "integrity": "sha512-FX/9frcub54beBdugHjDCdikxThEqjnR93Qt7PvQTOHxyiNCAlvMrHhclk3cD5VeAaq9fxmfRp+CnWw9rEMBuA==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5281,6 +5681,7 @@ "version": "3.1.0", "resolved": "https://registry.npmjs.org/d3-path/-/d3-path-3.1.0.tgz", "integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5289,6 +5690,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/d3-polygon/-/d3-polygon-3.0.1.tgz", "integrity": "sha512-3vbA7vXYwfe1SYhED++fPUQlWSYTTGmFmQiany/gdbiWgU/iEyQzyymwL9SkJjFFuCS4902BSzewVGsHHmHtXg==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5297,6 +5699,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/d3-quadtree/-/d3-quadtree-3.0.1.tgz", "integrity": "sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5305,6 +5708,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/d3-random/-/d3-random-3.0.1.tgz", "integrity": "sha512-FXMe9GfxTxqd5D6jFsQ+DJ8BJS4E/fT5mqqdjovykEB2oFbTMDVdg1MGFxfQW+FBOGoB++k8swBrgwSHT1cUXQ==", + "license": "ISC", "engines": { "node": ">=12" } @@ -5348,6 +5752,7 @@ "version": "4.0.2", "resolved": "https://registry.npmjs.org/d3-scale/-/d3-scale-4.0.2.tgz", "integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==", + "license": "ISC", "dependencies": { "d3-array": "2.10.0 - 3", "d3-format": "1 - 3", @@ -5363,6 +5768,7 @@ "version": "3.1.0", "resolved": "https://registry.npmjs.org/d3-scale-chromatic/-/d3-scale-chromatic-3.1.0.tgz", "integrity": "sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==", + "license": "ISC", "dependencies": { "d3-color": "1 - 3", "d3-interpolate": "1 - 3" @@ -5383,6 +5789,7 @@ "version": "3.2.0", "resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz", "integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==", + "license": "ISC", "dependencies": { "d3-path": "^3.1.0" }, @@ -5394,6 +5801,7 @@ "version": "3.1.0", "resolved": "https://registry.npmjs.org/d3-time/-/d3-time-3.1.0.tgz", "integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==", + "license": "ISC", "dependencies": { "d3-array": "2 - 3" }, @@ -5405,6 +5813,7 @@ "version": "4.1.0", "resolved": "https://registry.npmjs.org/d3-time-format/-/d3-time-format-4.1.0.tgz", "integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==", + "license": "ISC", "dependencies": { "d3-time": "1 - 3" }, @@ -5454,11 +5863,12 @@ } }, "node_modules/dagre-d3-es": { - "version": "7.0.10", - "resolved": "https://registry.npmjs.org/dagre-d3-es/-/dagre-d3-es-7.0.10.tgz", - "integrity": "sha512-qTCQmEhcynucuaZgY5/+ti3X/rnszKZhEQH/ZdWdtP1tA/y3VoHJzcVrO9pjjJCNpigfscAtoUB5ONcd2wNn0A==", + "version": "7.0.11", + "resolved": "https://registry.npmjs.org/dagre-d3-es/-/dagre-d3-es-7.0.11.tgz", + "integrity": "sha512-tvlJLyQf834SylNKax8Wkzco/1ias1OPw8DcUMDE7oUIoSEW25riQVuiu/0OWEFqT0cxHT3Pa9/D82Jr47IONw==", + "license": "MIT", "dependencies": { - "d3": "^7.8.2", + "d3": "^7.9.0", "lodash-es": "^4.17.21" } }, @@ -5475,9 +5885,10 @@ } }, "node_modules/dayjs": { - "version": "1.11.10", - "resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.11.10.tgz", - "integrity": "sha512-vjAczensTgRcqDERK0SR2XMwsF/tSvnvlv6VcF2GIhg6Sx4yOIt/irsr1RDJsKiIyBzJDpCoXiWWq28MqH2cnQ==" + "version": "1.11.13", + "resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.11.13.tgz", + "integrity": "sha512-oaMBel6gjolK862uaPQOVTA7q3TZhuSvuMQAAglQDOWYO9A91IrAOUJEyKVlqJlHE0vq5p5UXxzdPfMH/x6xNg==", + "license": "MIT" }, "node_modules/debug": { "version": "4.3.4", @@ -5495,18 +5906,6 @@ } } }, - "node_modules/decode-named-character-reference": { - "version": "1.0.2", - "resolved": "https://registry.npmjs.org/decode-named-character-reference/-/decode-named-character-reference-1.0.2.tgz", - "integrity": "sha512-O8x12RzrUF8xyVcY0KJowWsmaJxQbmy0/EtnNtHRpsOcT7dFk5W598coHqBVpmWo1oQQfsCqfCmkZN5DJrZVdg==", - "dependencies": { - "character-entities": "^2.0.0" - }, - "funding": { - "type": "github", - "url": "https://github.com/sponsors/wooorm" - } - }, "node_modules/deep-eql": { "version": "4.1.4", "resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-4.1.4.tgz", @@ -5555,6 +5954,7 @@ "version": "5.0.1", "resolved": "https://registry.npmjs.org/delaunator/-/delaunator-5.0.1.tgz", "integrity": "sha512-8nvh+XBe96aCESrGOqMp/84b13H9cdKbG5P2ejQCh4d4sK9RL4371qou9drQjMhvnPmhWl5hnmqbEE0fXr9Xnw==", + "license": "ISC", "dependencies": { "robust-predicates": "^3.0.2" } @@ -5612,14 +6012,6 @@ "url": "https://github.com/sponsors/wooorm" } }, - "node_modules/diff": { - "version": "5.2.0", - "resolved": "https://registry.npmjs.org/diff/-/diff-5.2.0.tgz", - "integrity": "sha512-uIFDxqpRZGZ6ThOk84hEfqWoHx2devRFvpTZcTHur85vImfaxUbTW9Ryh4CpCuDnToOP1CEtXKIgytHBPVff5A==", - "engines": { - "node": ">=0.3.1" - } - }, "node_modules/diff-sequences": { "version": "29.6.3", "resolved": "https://registry.npmjs.org/diff-sequences/-/diff-sequences-29.6.3.tgz", @@ -5699,9 +6091,13 @@ } }, "node_modules/dompurify": { - "version": "3.1.6", - "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.1.6.tgz", - "integrity": "sha512-cTOAhc36AalkjtBpfG6O8JimdTMWNXjiePT2xQH/ppBGi/4uIpmj8eKyIkMJErXWARyINV/sB38yf8JCLF5pbQ==" + "version": "3.2.5", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.2.5.tgz", + "integrity": "sha512-mLPd29uoRe9HpvwP2TxClGQBzGXeEC/we/q+bFlmPPmj2p2Ugl3r6ATu/UU1v77DXNcehiBg9zsr1dREyA/dJQ==", + "license": "(MPL-2.0 OR Apache-2.0)", + "optionalDependencies": { + "@types/trusted-types": "^2.0.7" + } }, "node_modules/domutils": { "version": "3.2.2", @@ -5733,11 +6129,6 @@ "safer-buffer": "^2.1.0" } }, - "node_modules/elkjs": { - "version": "0.9.3", - "resolved": "https://registry.npmjs.org/elkjs/-/elkjs-0.9.3.tgz", - "integrity": "sha512-f/ZeWvW/BCXbhGEf1Ujp29EASo/lk1FDnETgNKwJrsVvGZhUWCZyg3xLJjAsxfOmt8KjswHmI5EwCQcPMpOYhQ==" - }, "node_modules/emoji-regex": { "version": "9.2.2", "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-9.2.2.tgz", @@ -6231,6 +6622,12 @@ "node": ">=4" } }, + "node_modules/exsolve": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/exsolve/-/exsolve-1.0.4.tgz", + "integrity": "sha512-xsZH6PXaER4XoV+NiT7JHp1bJodJVT+cxeSH1G0f0tlT0lJqYuHUP3bUx2HtfTDvOagMINYp8rsqusxud3RXhw==", + "license": "MIT" + }, "node_modules/extend": { "version": "3.0.2", "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz", @@ -6815,6 +7212,12 @@ "through2": "^2.0.1" } }, + "node_modules/hachure-fill": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/hachure-fill/-/hachure-fill-0.5.2.tgz", + "integrity": "sha512-3GKBOn+m2LX9iq+JC1064cSFprJY4jL1jCXTcpnfER5HYE2l/4EfWSGzkPa/ZDBmYI0ZOEj5VHV/eKnPGkHuOg==", + "license": "MIT" + }, "node_modules/has-flag": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz", @@ -7204,6 +7607,7 @@ "version": "2.0.3", "resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz", "integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==", + "license": "ISC", "engines": { "node": ">=12" } @@ -7499,16 +7903,6 @@ "html2canvas": "^1.0.0-rc.5" } }, - "node_modules/jspdf/node_modules/dompurify": { - "version": "3.2.4", - "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.2.4.tgz", - "integrity": "sha512-ysFSFEDVduQpyhzAob/kkuJjf5zWkZD8/A9ywSp1byueyuCfHamrCBa14/Oc2iiB0e51B+NpxSl5gmzn+Ms/mg==", - "license": "(MPL-2.0 OR Apache-2.0)", - "optional": true, - "optionalDependencies": { - "@types/trusted-types": "^2.0.7" - } - }, "node_modules/jsprim": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/jsprim/-/jsprim-2.0.2.tgz", @@ -7586,6 +7980,28 @@ "phonemizer": "^1.2.1" } }, + "node_modules/kolorist": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/kolorist/-/kolorist-1.8.0.tgz", + "integrity": "sha512-Y+60/zizpJ3HRH8DCss+q95yr6145JXZo46OTpFvDZWLfRCE4qChOyk1b26nMaNpfHHgxagk9dXT5OP0Tfe+dQ==", + "license": "MIT" + }, + "node_modules/langium": { + "version": "3.3.1", + "resolved": "https://registry.npmjs.org/langium/-/langium-3.3.1.tgz", + "integrity": "sha512-QJv/h939gDpvT+9SiLVlY7tZC3xB2qK57v0J04Sh9wpMb6MP1q8gB21L3WIo8T5P1MSMg3Ep14L7KkDCFG3y4w==", + "license": "MIT", + "dependencies": { + "chevrotain": "~11.0.3", + "chevrotain-allstar": "~0.3.0", + "vscode-languageserver": "~9.0.1", + "vscode-languageserver-textdocument": "~1.0.11", + "vscode-uri": "~3.0.8" + }, + "engines": { + "node": ">=16.0.0" + } + }, "node_modules/layout-base": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/layout-base/-/layout-base-1.0.2.tgz", @@ -8003,7 +8419,8 @@ "node_modules/lodash-es": { "version": "4.17.21", "resolved": "https://registry.npmjs.org/lodash-es/-/lodash-es-4.17.21.tgz", - "integrity": "sha512-mKnC+QJ9pWVzv+C4/U3rRsHapFfHvQFoFB92e52xeyGMcX6/OlIl78je1u8vePzYZSkkogMPJ2yjxxsb89cxyw==" + "integrity": "sha512-mKnC+QJ9pWVzv+C4/U3rRsHapFfHvQFoFB92e52xeyGMcX6/OlIl78je1u8vePzYZSkkogMPJ2yjxxsb89cxyw==", + "license": "MIT" }, "node_modules/lodash.castarray": { "version": "4.4.0", @@ -8216,41 +8633,6 @@ "node": "*" } }, - "node_modules/mdast-util-from-markdown": { - "version": "1.3.1", - "resolved": "https://registry.npmjs.org/mdast-util-from-markdown/-/mdast-util-from-markdown-1.3.1.tgz", - "integrity": "sha512-4xTO/M8c82qBcnQc1tgpNtubGUW/Y1tBQ1B0i5CtSoelOLKFYlElIr3bvgREYYO5iRqbMY1YuqZng0GVOI8Qww==", - "dependencies": { - "@types/mdast": "^3.0.0", - "@types/unist": "^2.0.0", - "decode-named-character-reference": "^1.0.0", - "mdast-util-to-string": "^3.1.0", - "micromark": "^3.0.0", - "micromark-util-decode-numeric-character-reference": "^1.0.0", - "micromark-util-decode-string": "^1.0.0", - "micromark-util-normalize-identifier": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0", - "unist-util-stringify-position": "^3.0.0", - "uvu": "^0.5.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/unified" - } - }, - "node_modules/mdast-util-to-string": { - "version": "3.2.0", - "resolved": "https://registry.npmjs.org/mdast-util-to-string/-/mdast-util-to-string-3.2.0.tgz", - "integrity": "sha512-V4Zn/ncyN1QNSqSBxTrMOLpjr+IKdHl2v3KVLoWmDPscP4r9GcCi71gjgvUV1SFSKh92AjAG4peFuBl2/YgCJg==", - "dependencies": { - "@types/mdast": "^3.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/unified" - } - }, "node_modules/mdn-data": { "version": "2.0.30", "resolved": "https://registry.npmjs.org/mdn-data/-/mdn-data-2.0.30.tgz", @@ -8276,453 +8658,58 @@ } }, "node_modules/mermaid": { - "version": "10.9.3", - "resolved": "https://registry.npmjs.org/mermaid/-/mermaid-10.9.3.tgz", - "integrity": "sha512-V80X1isSEvAewIL3xhmz/rVmc27CVljcsbWxkxlWJWY/1kQa4XOABqpDl2qQLGKzpKm6WbTfUEKImBlUfFYArw==", + "version": "11.6.0", + "resolved": "https://registry.npmjs.org/mermaid/-/mermaid-11.6.0.tgz", + "integrity": "sha512-PE8hGUy1LDlWIHWBP05SFdqUHGmRcCcK4IzpOKPE35eOw+G9zZgcnMpyunJVUEOgb//KBORPjysKndw8bFLuRg==", + "license": "MIT", "dependencies": { - "@braintree/sanitize-url": "^6.0.1", - "@types/d3-scale": "^4.0.3", - "@types/d3-scale-chromatic": "^3.0.0", - "cytoscape": "^3.28.1", + "@braintree/sanitize-url": "^7.0.4", + "@iconify/utils": "^2.1.33", + "@mermaid-js/parser": "^0.4.0", + "@types/d3": "^7.4.3", + "cytoscape": "^3.29.3", "cytoscape-cose-bilkent": "^4.1.0", - "d3": "^7.4.0", + "cytoscape-fcose": "^2.2.0", + "d3": "^7.9.0", "d3-sankey": "^0.12.3", - "dagre-d3-es": "7.0.10", - "dayjs": "^1.11.7", - "dompurify": "^3.0.5 <3.1.7", - "elkjs": "^0.9.0", + "dagre-d3-es": "7.0.11", + "dayjs": "^1.11.13", + "dompurify": "^3.2.4", "katex": "^0.16.9", - "khroma": "^2.0.0", + "khroma": "^2.1.0", "lodash-es": "^4.17.21", - "mdast-util-from-markdown": "^1.3.0", - "non-layered-tidy-tree-layout": "^2.0.2", - "stylis": "^4.1.3", + "marked": "^15.0.7", + "roughjs": "^4.6.6", + "stylis": "^4.3.6", "ts-dedent": "^2.2.0", - "uuid": "^9.0.0", - "web-worker": "^1.2.0" + "uuid": "^11.1.0" } }, - "node_modules/micromark": { - "version": "3.2.0", - "resolved": "https://registry.npmjs.org/micromark/-/micromark-3.2.0.tgz", - "integrity": "sha512-uD66tJj54JLYq0De10AhWycZWGQNUvDI55xPgk2sQM5kn1JYlhbCMTtEeT27+vAhW2FBQxLlOmS3pmA7/2z4aA==", + "node_modules/mermaid/node_modules/marked": { + "version": "15.0.8", + "resolved": "https://registry.npmjs.org/marked/-/marked-15.0.8.tgz", + "integrity": "sha512-rli4l2LyZqpQuRve5C0rkn6pj3hT8EWPC+zkAxFTAJLxRbENfTAhEQq9itrmf1Y81QtAX5D/MYlGlIomNgj9lA==", + "license": "MIT", + "bin": { + "marked": "bin/marked.js" + }, + "engines": { + "node": ">= 18" + } + }, + "node_modules/mermaid/node_modules/uuid": { + "version": "11.1.0", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-11.1.0.tgz", + "integrity": "sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A==", "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } + "https://github.com/sponsors/broofa", + "https://github.com/sponsors/ctavan" ], - "dependencies": { - "@types/debug": "^4.0.0", - "debug": "^4.0.0", - "decode-named-character-reference": "^1.0.0", - "micromark-core-commonmark": "^1.0.1", - "micromark-factory-space": "^1.0.0", - "micromark-util-character": "^1.0.0", - "micromark-util-chunked": "^1.0.0", - "micromark-util-combine-extensions": "^1.0.0", - "micromark-util-decode-numeric-character-reference": "^1.0.0", - "micromark-util-encode": "^1.0.0", - "micromark-util-normalize-identifier": "^1.0.0", - "micromark-util-resolve-all": "^1.0.0", - "micromark-util-sanitize-uri": "^1.0.0", - "micromark-util-subtokenize": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.1", - "uvu": "^0.5.0" + "license": "MIT", + "bin": { + "uuid": "dist/esm/bin/uuid" } }, - "node_modules/micromark-core-commonmark": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-core-commonmark/-/micromark-core-commonmark-1.1.0.tgz", - "integrity": "sha512-BgHO1aRbolh2hcrzL2d1La37V0Aoz73ymF8rAcKnohLy93titmv62E0gP8Hrx9PKcKrqCZ1BbLGbP3bEhoXYlw==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "decode-named-character-reference": "^1.0.0", - "micromark-factory-destination": "^1.0.0", - "micromark-factory-label": "^1.0.0", - "micromark-factory-space": "^1.0.0", - "micromark-factory-title": "^1.0.0", - "micromark-factory-whitespace": "^1.0.0", - "micromark-util-character": "^1.0.0", - "micromark-util-chunked": "^1.0.0", - "micromark-util-classify-character": "^1.0.0", - "micromark-util-html-tag-name": "^1.0.0", - "micromark-util-normalize-identifier": "^1.0.0", - "micromark-util-resolve-all": "^1.0.0", - "micromark-util-subtokenize": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.1", - "uvu": "^0.5.0" - } - }, - "node_modules/micromark-factory-destination": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-factory-destination/-/micromark-factory-destination-1.1.0.tgz", - "integrity": "sha512-XaNDROBgx9SgSChd69pjiGKbV+nfHGDPVYFs5dOoDd7ZnMAE+Cuu91BCpsY8RT2NP9vo/B8pds2VQNCLiu0zhg==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-character": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-factory-label": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-factory-label/-/micromark-factory-label-1.1.0.tgz", - "integrity": "sha512-OLtyez4vZo/1NjxGhcpDSbHQ+m0IIGnT8BoPamh+7jVlzLJBH98zzuCoUeMxvM6WsNeh8wx8cKvqLiPHEACn0w==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-character": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0", - "uvu": "^0.5.0" - } - }, - "node_modules/micromark-factory-space": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-factory-space/-/micromark-factory-space-1.1.0.tgz", - "integrity": "sha512-cRzEj7c0OL4Mw2v6nwzttyOZe8XY/Z8G0rzmWQZTBi/jjwyw/U4uqKtUORXQrR5bAZZnbTI/feRV/R7hc4jQYQ==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-character": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-factory-title": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-factory-title/-/micromark-factory-title-1.1.0.tgz", - "integrity": "sha512-J7n9R3vMmgjDOCY8NPw55jiyaQnH5kBdV2/UXCtZIpnHH3P6nHUKaH7XXEYuWwx/xUJcawa8plLBEjMPU24HzQ==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-factory-space": "^1.0.0", - "micromark-util-character": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-factory-whitespace": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-factory-whitespace/-/micromark-factory-whitespace-1.1.0.tgz", - "integrity": "sha512-v2WlmiymVSp5oMg+1Q0N1Lxmt6pMhIHD457whWM7/GUlEks1hI9xj5w3zbc4uuMKXGisksZk8DzP2UyGbGqNsQ==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-factory-space": "^1.0.0", - "micromark-util-character": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-util-character": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/micromark-util-character/-/micromark-util-character-1.2.0.tgz", - "integrity": "sha512-lXraTwcX3yH/vMDaFWCQJP1uIszLVebzUa3ZHdrgxr7KEU/9mL4mVgCpGbyhvNLNlauROiNUq7WN5u7ndbY6xg==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-util-chunked": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-chunked/-/micromark-util-chunked-1.1.0.tgz", - "integrity": "sha512-Ye01HXpkZPNcV6FiyoW2fGZDUw4Yc7vT0E9Sad83+bEDiCJ1uXu0S3mr8WLpsz3HaG3x2q0HM6CTuPdcZcluFQ==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-symbol": "^1.0.0" - } - }, - "node_modules/micromark-util-classify-character": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-classify-character/-/micromark-util-classify-character-1.1.0.tgz", - "integrity": "sha512-SL0wLxtKSnklKSUplok1WQFoGhUdWYKggKUiqhX+Swala+BtptGCu5iPRc+xvzJ4PXE/hwM3FNXsfEVgoZsWbw==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-character": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-util-combine-extensions": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-combine-extensions/-/micromark-util-combine-extensions-1.1.0.tgz", - "integrity": "sha512-Q20sp4mfNf9yEqDL50WwuWZHUrCO4fEyeDCnMGmG5Pr0Cz15Uo7KBs6jq+dq0EgX4DPwwrh9m0X+zPV1ypFvUA==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-chunked": "^1.0.0", - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-util-decode-numeric-character-reference": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-decode-numeric-character-reference/-/micromark-util-decode-numeric-character-reference-1.1.0.tgz", - "integrity": "sha512-m9V0ExGv0jB1OT21mrWcuf4QhP46pH1KkfWy9ZEezqHKAxkj4mPCy3nIH1rkbdMlChLHX531eOrymlwyZIf2iw==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-symbol": "^1.0.0" - } - }, - "node_modules/micromark-util-decode-string": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-decode-string/-/micromark-util-decode-string-1.1.0.tgz", - "integrity": "sha512-YphLGCK8gM1tG1bd54azwyrQRjCFcmgj2S2GoJDNnh4vYtnL38JS8M4gpxzOPNyHdNEpheyWXCTnnTDY3N+NVQ==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "decode-named-character-reference": "^1.0.0", - "micromark-util-character": "^1.0.0", - "micromark-util-decode-numeric-character-reference": "^1.0.0", - "micromark-util-symbol": "^1.0.0" - } - }, - "node_modules/micromark-util-encode": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-encode/-/micromark-util-encode-1.1.0.tgz", - "integrity": "sha512-EuEzTWSTAj9PA5GOAs992GzNh2dGQO52UvAbtSOMvXTxv3Criqb6IOzJUBCmEqrrXSblJIJBbFFv6zPxpreiJw==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ] - }, - "node_modules/micromark-util-html-tag-name": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/micromark-util-html-tag-name/-/micromark-util-html-tag-name-1.2.0.tgz", - "integrity": "sha512-VTQzcuQgFUD7yYztuQFKXT49KghjtETQ+Wv/zUjGSGBioZnkA4P1XXZPT1FHeJA6RwRXSF47yvJ1tsJdoxwO+Q==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ] - }, - "node_modules/micromark-util-normalize-identifier": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-normalize-identifier/-/micromark-util-normalize-identifier-1.1.0.tgz", - "integrity": "sha512-N+w5vhqrBihhjdpM8+5Xsxy71QWqGn7HYNUvch71iV2PM7+E3uWGox1Qp90loa1ephtCxG2ftRV/Conitc6P2Q==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-symbol": "^1.0.0" - } - }, - "node_modules/micromark-util-resolve-all": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-resolve-all/-/micromark-util-resolve-all-1.1.0.tgz", - "integrity": "sha512-b/G6BTMSg+bX+xVCshPTPyAu2tmA0E4X98NSR7eIbeC6ycCqCeE7wjfDIgzEbkzdEVJXRtOG4FbEm/uGbCRouA==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-types": "^1.0.0" - } - }, - "node_modules/micromark-util-sanitize-uri": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/micromark-util-sanitize-uri/-/micromark-util-sanitize-uri-1.2.0.tgz", - "integrity": "sha512-QO4GXv0XZfWey4pYFndLUKEAktKkG5kZTdUNaTAkzbuJxn2tNBOr+QtxR2XpWaMhbImT2dPzyLrPXLlPhph34A==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-character": "^1.0.0", - "micromark-util-encode": "^1.0.0", - "micromark-util-symbol": "^1.0.0" - } - }, - "node_modules/micromark-util-subtokenize": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-subtokenize/-/micromark-util-subtokenize-1.1.0.tgz", - "integrity": "sha512-kUQHyzRoxvZO2PuLzMt2P/dwVsTiivCK8icYTeR+3WgbuPqfHgPPy7nFKbeqRivBvn/3N3GBiNC+JRTMSxEC7A==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ], - "dependencies": { - "micromark-util-chunked": "^1.0.0", - "micromark-util-symbol": "^1.0.0", - "micromark-util-types": "^1.0.0", - "uvu": "^0.5.0" - } - }, - "node_modules/micromark-util-symbol": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-symbol/-/micromark-util-symbol-1.1.0.tgz", - "integrity": "sha512-uEjpEYY6KMs1g7QfJ2eX1SQEV+ZT4rUD3UcF6l57acZvLNK7PBZL+ty82Z1qhK1/yXIY4bdx04FKMgR0g4IAag==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ] - }, - "node_modules/micromark-util-types": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/micromark-util-types/-/micromark-util-types-1.1.0.tgz", - "integrity": "sha512-ukRBgie8TIAcacscVHSiddHjO4k/q3pnedmzMQ4iwDcK0FtFCohKOlFbaOL/mPgfnPsL3C1ZyxJa4sbWrBl3jg==", - "funding": [ - { - "type": "GitHub Sponsors", - "url": "https://github.com/sponsors/unifiedjs" - }, - { - "type": "OpenCollective", - "url": "https://opencollective.com/unified" - } - ] - }, "node_modules/micromatch": { "version": "4.0.8", "resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.8.tgz", @@ -8906,15 +8893,32 @@ } }, "node_modules/mlly": { - "version": "1.7.0", - "resolved": "https://registry.npmjs.org/mlly/-/mlly-1.7.0.tgz", - "integrity": "sha512-U9SDaXGEREBYQgfejV97coK0UL1r+qnF2SyO9A3qcI8MzKnsIFKHNVEkrDyNncQTKQQumsasmeq84eNMdBfsNQ==", - "dev": true, + "version": "1.7.4", + "resolved": "https://registry.npmjs.org/mlly/-/mlly-1.7.4.tgz", + "integrity": "sha512-qmdSIPC4bDJXgZTCR7XosJiNKySV7O215tsPtDN9iEO/7q/76b/ijtgRu/+epFXSJhijtTCCGp3DWS549P3xKw==", + "license": "MIT", "dependencies": { - "acorn": "^8.11.3", - "pathe": "^1.1.2", - "pkg-types": "^1.1.0", - "ufo": "^1.5.3" + "acorn": "^8.14.0", + "pathe": "^2.0.1", + "pkg-types": "^1.3.0", + "ufo": "^1.5.4" + } + }, + "node_modules/mlly/node_modules/pathe": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz", + "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==", + "license": "MIT" + }, + "node_modules/mlly/node_modules/pkg-types": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/pkg-types/-/pkg-types-1.3.1.tgz", + "integrity": "sha512-/Jm5M4RvtBFVkKWRu2BLUTNP8/M2a+UwuAX+ae4770q1qVGtfjG+WTCupoZixokjmHiry8uI+dlY8KXYV5HVVQ==", + "license": "MIT", + "dependencies": { + "confbox": "^0.1.8", + "mlly": "^1.7.4", + "pathe": "^2.0.1" } }, "node_modules/mri": { @@ -8968,11 +8972,6 @@ "resolved": "https://registry.npmjs.org/ngraph.events/-/ngraph.events-1.2.2.tgz", "integrity": "sha512-JsUbEOzANskax+WSYiAPETemLWYXmixuPAlmZmhIbIj6FH/WDgEGCGnRwUQBK0GjOnVm8Ui+e5IJ+5VZ4e32eQ==" }, - "node_modules/non-layered-tidy-tree-layout": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/non-layered-tidy-tree-layout/-/non-layered-tidy-tree-layout-2.0.2.tgz", - "integrity": "sha512-gkXMxRzUH+PB0ax9dUN0yYF0S25BqeAYqhgMaLUFmpXLEk7Fcu8f4emJuOAY0V8kjDICxROIKsTAKsV/v355xw==" - }, "node_modules/normalize-path": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz", @@ -9183,6 +9182,15 @@ "integrity": "sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw==", "license": "BlueOak-1.0.0" }, + "node_modules/package-manager-detector": { + "version": "0.2.11", + "resolved": "https://registry.npmjs.org/package-manager-detector/-/package-manager-detector-0.2.11.tgz", + "integrity": "sha512-BEnLolu+yuz22S56CU1SUKq3XC3PkwD5wv4ikR4MfGvnRVcmzXR9DwSlW2fEamyTPyXHomBJRzgapeuBvRNzJQ==", + "license": "MIT", + "dependencies": { + "quansync": "^0.2.7" + } + }, "node_modules/paneforge": { "version": "0.0.6", "resolved": "https://registry.npmjs.org/paneforge/-/paneforge-0.0.6.tgz", @@ -9256,6 +9264,12 @@ "url": "https://github.com/inikulin/parse5?sponsor=1" } }, + "node_modules/path-data-parser": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/path-data-parser/-/path-data-parser-0.1.0.tgz", + "integrity": "sha512-NOnmBpt5Y2RWbuv0LMzsayp3lVylAHLPUTut412ZA3l+C4uw4ZVkQbjShYCQ8TCpUMdPapr4YjUqLYD6v68j+w==", + "license": "MIT" + }, "node_modules/path-exists": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz", @@ -9432,6 +9446,22 @@ "integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==", "license": "MIT" }, + "node_modules/points-on-curve": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/points-on-curve/-/points-on-curve-0.2.0.tgz", + "integrity": "sha512-0mYKnYYe9ZcqMCWhUjItv/oHjvgEsfKvnUTg8sAtnHr3GVy7rGkXCb6d5cSyqrWqL4k81b9CPg3urd+T7aop3A==", + "license": "MIT" + }, + "node_modules/points-on-path": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/points-on-path/-/points-on-path-0.2.1.tgz", + "integrity": "sha512-25ClnWWuw7JbWZcgqY/gJ4FQWadKxGWk+3kR/7kD0tCaDtPPMj7oHu2ToLaVhfpnHrZzYby2w6tUA0eOIuUg8g==", + "license": "MIT", + "dependencies": { + "path-data-parser": "0.1.0", + "points-on-curve": "0.2.0" + } + }, "node_modules/polyscript": { "version": "0.12.8", "resolved": "https://registry.npmjs.org/polyscript/-/polyscript-0.12.8.tgz", @@ -9992,6 +10022,22 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/quansync": { + "version": "0.2.10", + "resolved": "https://registry.npmjs.org/quansync/-/quansync-0.2.10.tgz", + "integrity": "sha512-t41VRkMYbkHyCYmOvx/6URnN80H7k4X0lLdBMGsz+maAwrJQYB1djpV6vHrQIBE0WBSGqhtEHrK9U3DWWH8v7A==", + "funding": [ + { + "type": "individual", + "url": "https://github.com/sponsors/antfu" + }, + { + "type": "individual", + "url": "https://github.com/sponsors/sxzz" + } + ], + "license": "MIT" + }, "node_modules/querystringify": { "version": "2.2.0", "resolved": "https://registry.npmjs.org/querystringify/-/querystringify-2.2.0.tgz", @@ -10307,7 +10353,8 @@ "node_modules/robust-predicates": { "version": "3.0.2", "resolved": "https://registry.npmjs.org/robust-predicates/-/robust-predicates-3.0.2.tgz", - "integrity": "sha512-IXgzBWvWQwE6PrDI05OvmXUIruQTcoMDzRsOd5CDvHCVLcLHMTSYvOK5Cm46kWqlV3yAbuSpBZdJ5oP5OUoStg==" + "integrity": "sha512-IXgzBWvWQwE6PrDI05OvmXUIruQTcoMDzRsOd5CDvHCVLcLHMTSYvOK5Cm46kWqlV3yAbuSpBZdJ5oP5OUoStg==", + "license": "Unlicense" }, "node_modules/rollup": { "version": "4.22.4", @@ -10348,6 +10395,18 @@ "resolved": "https://registry.npmjs.org/rope-sequence/-/rope-sequence-1.3.4.tgz", "integrity": "sha512-UT5EDe2cu2E/6O4igUr5PSFs23nvvukicWHx6GnOPlHAiiYbzNuCRQCuiUdHJQcqKalLKlrYJnjY0ySGsXNQXQ==" }, + "node_modules/roughjs": { + "version": "4.6.6", + "resolved": "https://registry.npmjs.org/roughjs/-/roughjs-4.6.6.tgz", + "integrity": "sha512-ZUz/69+SYpFN/g/lUlo2FXcIjRkSu3nDarreVdGGndHEBJ6cXPdKguS8JGxwj5HA5xIbVKSmLgr5b3AWxtRfvQ==", + "license": "MIT", + "dependencies": { + "hachure-fill": "^0.5.2", + "path-data-parser": "^0.1.0", + "points-on-curve": "^0.2.0", + "points-on-path": "^0.2.1" + } + }, "node_modules/rsvp": { "version": "4.8.5", "resolved": "https://registry.npmjs.org/rsvp/-/rsvp-4.8.5.tgz", @@ -10382,7 +10441,8 @@ "node_modules/rw": { "version": "1.3.3", "resolved": "https://registry.npmjs.org/rw/-/rw-1.3.3.tgz", - "integrity": "sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==" + "integrity": "sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==", + "license": "BSD-3-Clause" }, "node_modules/rxjs": { "version": "7.8.1", @@ -11324,9 +11384,10 @@ "integrity": "sha512-wnD1HyVqpJUI2+eKZ+eo1UwghftP6yuFheBqqe+bWCotBjC2K1YnteJILRMs3SM4V/0dLEW1SC27MWP5y+mwmw==" }, "node_modules/stylis": { - "version": "4.3.2", - "resolved": "https://registry.npmjs.org/stylis/-/stylis-4.3.2.tgz", - "integrity": "sha512-bhtUjWd/z6ltJiQwg0dUfxEJ+W+jdqQd8TbWLWyeIJHlnsqmGLRFFd8e5mA0AZi/zx90smXRlN66YMTcaSFifg==" + "version": "4.3.6", + "resolved": "https://registry.npmjs.org/stylis/-/stylis-4.3.6.tgz", + "integrity": "sha512-yQ3rwFWRfwNUY7H5vpU0wfdkNSnvnJinhF9830Swlaxl03zsOjCfmX0ugac+3LtK0lYSgwL/KXc8oYL3mG4YFQ==", + "license": "MIT" }, "node_modules/supports-color": { "version": "7.2.0", @@ -11677,6 +11738,12 @@ "integrity": "sha512-1/eK7zUnIklz4JUUlL+658n58XO2hHLQfSk1Zf2LKieUjxidN16eKFEoDEfjHc3ohofSSqK3X5yO6VGb6iW8Lw==", "dev": true }, + "node_modules/tinyexec": { + "version": "0.3.2", + "resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-0.3.2.tgz", + "integrity": "sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==", + "license": "MIT" + }, "node_modules/tinypool": { "version": "0.8.4", "resolved": "https://registry.npmjs.org/tinypool/-/tinypool-0.8.4.tgz", @@ -11884,10 +11951,10 @@ "integrity": "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A==" }, "node_modules/ufo": { - "version": "1.5.3", - "resolved": "https://registry.npmjs.org/ufo/-/ufo-1.5.3.tgz", - "integrity": "sha512-Y7HYmWaFwPUmkoQCUIAYpKqkOf+SbVj/2fJJZ4RJMCfZp0rTGwRbzQD+HghfnhKOjL9E01okqz+ncJskGYfBNw==", - "dev": true + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/ufo/-/ufo-1.6.1.tgz", + "integrity": "sha512-9a4/uxlTWJ4+a5i0ooc1rU7C7YOw3wT+UGqdeNNHWnOF9qcMBgLRS+4IYUqbczewFx4mLEig6gawh7X6mFlEkA==", + "license": "MIT" }, "node_modules/underscore.string": { "version": "3.3.6", @@ -11916,18 +11983,6 @@ "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz", "integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA==" }, - "node_modules/unist-util-stringify-position": { - "version": "3.0.3", - "resolved": "https://registry.npmjs.org/unist-util-stringify-position/-/unist-util-stringify-position-3.0.3.tgz", - "integrity": "sha512-k5GzIBZ/QatR8N5X2y+drfpWG8IDBzdnVj6OInRNWm1oXrzydiaAT2OQiA8DPRRZyAKb9b6I2a6PxYklZD0gKg==", - "dependencies": { - "@types/unist": "^2.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/unified" - } - }, "node_modules/universalify": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.1.tgz", @@ -11991,23 +12046,6 @@ "uuid": "dist/bin/uuid" } }, - "node_modules/uvu": { - "version": "0.5.6", - "resolved": "https://registry.npmjs.org/uvu/-/uvu-0.5.6.tgz", - "integrity": "sha512-+g8ENReyr8YsOc6fv/NVJs2vFdHBnBNdfE49rshrTzDWOlUx4Gq7KOS2GD8eqhy2j+Ejq29+SbKH8yjkAqXqoA==", - "dependencies": { - "dequal": "^2.0.0", - "diff": "^5.0.0", - "kleur": "^4.0.3", - "sade": "^1.7.3" - }, - "bin": { - "uvu": "bin.js" - }, - "engines": { - "node": ">=8" - } - }, "node_modules/value-or-function": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/value-or-function/-/value-or-function-4.0.0.tgz", @@ -12798,6 +12836,55 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/vscode-jsonrpc": { + "version": "8.2.0", + "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-8.2.0.tgz", + "integrity": "sha512-C+r0eKJUIfiDIfwJhria30+TYWPtuHJXHtI7J0YlOmKAo7ogxP20T0zxB7HZQIFhIyvoBPwWskjxrvAtfjyZfA==", + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/vscode-languageserver": { + "version": "9.0.1", + "resolved": "https://registry.npmjs.org/vscode-languageserver/-/vscode-languageserver-9.0.1.tgz", + "integrity": "sha512-woByF3PDpkHFUreUa7Hos7+pUWdeWMXRd26+ZX2A8cFx6v/JPTtd4/uN0/jB6XQHYaOlHbio03NTHCqrgG5n7g==", + "license": "MIT", + "dependencies": { + "vscode-languageserver-protocol": "3.17.5" + }, + "bin": { + "installServerIntoExtension": "bin/installServerIntoExtension" + } + }, + "node_modules/vscode-languageserver-protocol": { + "version": "3.17.5", + "resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.17.5.tgz", + "integrity": "sha512-mb1bvRJN8SVznADSGWM9u/b07H7Ecg0I3OgXDuLdn307rl/J3A9YD6/eYOssqhecL27hK1IPZAsaqh00i/Jljg==", + "license": "MIT", + "dependencies": { + "vscode-jsonrpc": "8.2.0", + "vscode-languageserver-types": "3.17.5" + } + }, + "node_modules/vscode-languageserver-textdocument": { + "version": "1.0.12", + "resolved": "https://registry.npmjs.org/vscode-languageserver-textdocument/-/vscode-languageserver-textdocument-1.0.12.tgz", + "integrity": "sha512-cxWNPesCnQCcMPeenjKKsOCKQZ/L6Tv19DTRIGuLWe32lyzWhihGVJ/rcckZXJxfdKCFvRLS3fpBIsV/ZGX4zA==", + "license": "MIT" + }, + "node_modules/vscode-languageserver-types": { + "version": "3.17.5", + "resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.17.5.tgz", + "integrity": "sha512-Ld1VelNuX9pdF39h2Hgaeb5hEZM2Z3jUrrMgWQAu82jMtZp7p3vJT3BzToKtZI7NgQssZje5o0zryOrhQvzQAg==", + "license": "MIT" + }, + "node_modules/vscode-uri": { + "version": "3.0.8", + "resolved": "https://registry.npmjs.org/vscode-uri/-/vscode-uri-3.0.8.tgz", + "integrity": "sha512-AyFQ0EVmsOZOlAnxoFOGOq1SQDWAB7C6aqMGS23svWAllfOaxbuFvcT8D1i8z3Gyn8fraVeZNNmN6e9bxxXkKw==", + "license": "MIT" + }, "node_modules/w3c-keyname": { "version": "2.2.8", "resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz", @@ -12840,11 +12927,6 @@ "node": "*" } }, - "node_modules/web-worker": { - "version": "1.3.0", - "resolved": "https://registry.npmjs.org/web-worker/-/web-worker-1.3.0.tgz", - "integrity": "sha512-BSR9wyRsy/KOValMgd5kMyr3JzpdeoR9KVId8u5GVlTTAtNChlsE4yTxeY7zMdNSyOmoKBv8NH2qeRY9Tg+IaA==" - }, "node_modules/whatwg-encoding": { "version": "3.1.1", "resolved": "https://registry.npmjs.org/whatwg-encoding/-/whatwg-encoding-3.1.1.tgz", diff --git a/package.json b/package.json index 346a7ddc81..0a982196e4 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "open-webui", - "version": "0.6.2", + "version": "0.6.5", "private": true, "scripts": { "dev": "npm run pyodide:fetch && vite dev --host", @@ -72,11 +72,11 @@ "async": "^3.2.5", "bits-ui": "^0.19.7", "codemirror": "^6.0.1", - "codemirror-lang-hcl": "^0.0.0-beta.2", "codemirror-lang-elixir": "^4.0.0", + "codemirror-lang-hcl": "^0.0.0-beta.2", "crc-32": "^1.2.2", "dayjs": "^1.11.10", - "dompurify": "^3.1.6", + "dompurify": "^3.2.5", "eventsource-parser": "^1.1.2", "file-saver": "^2.0.5", "fuse.js": "^7.0.0", @@ -92,7 +92,7 @@ "katex": "^0.16.21", "kokoro-js": "^1.1.1", "marked": "^9.1.0", - "mermaid": "^10.9.3", + "mermaid": "^11.6.0", "paneforge": "^0.0.6", "panzoom": "^9.4.3", "prosemirror-commands": "^1.6.0", diff --git a/pyproject.toml b/pyproject.toml index b8bf2ff67a..8a48c90fac 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,9 +9,9 @@ dependencies = [ "fastapi==0.115.7", "uvicorn[standard]==0.34.0", "pydantic==2.10.6", - "python-multipart==0.0.18", + "python-multipart==0.0.20", - "python-socketio==5.11.3", + "python-socketio==5.13.0", "python-jose==3.4.0", "passlib[bcrypt]==1.7.4", @@ -26,7 +26,7 @@ dependencies = [ "peewee==3.17.9", "peewee-migrate==1.12.2", "psycopg2-binary==2.9.9", - "pgvector==0.3.5", + "pgvector==0.4.0", "PyMySQL==1.1.1", "bcrypt==4.3.0", @@ -52,7 +52,7 @@ dependencies = [ "langchain-community==0.3.18", "fake-useragent==2.1.0", - "chromadb==0.6.2", + "chromadb==0.6.3", "pymilvus==2.5.0", "qdrant-client~=1.12.0", "opensearch-py==2.8.0", @@ -99,7 +99,7 @@ dependencies = [ "black==25.1.0", "langfuse==2.44.0", - "youtube-transcript-api==0.6.3", + "youtube-transcript-api==1.0.3", "pytube==15.0.0", "extract_msg", @@ -113,7 +113,6 @@ dependencies = [ "docker~=7.1.0", "pytest~=8.3.2", "pytest-docker~=3.1.1", - "moto[s3]>=5.0.26", "googleapis-common-protos==1.63.2", "google-cloud-storage==2.19.0", @@ -128,6 +127,9 @@ dependencies = [ "tencentcloud-sdk-python==3.0.1336", "gcp-storage-emulator>=2024.8.3", + + "moto[s3]>=5.0.26", + ] readme = "README.md" requires-python = ">= 3.11, < 3.13.0a1" diff --git a/src/app.css b/src/app.css index 250a29283d..d0bd50aced 100644 --- a/src/app.css +++ b/src/app.css @@ -218,6 +218,28 @@ input[type='number'] { -moz-appearance: textfield; /* Firefox */ } +.katex-display { + @apply overflow-y-hidden overflow-x-auto max-w-full; +} + +.katex-display::-webkit-scrollbar { + height: 0.4rem; + width: 0.4rem; +} + +.katex-display:active::-webkit-scrollbar-thumb, +.katex-display:focus::-webkit-scrollbar-thumb, +.katex-display:hover::-webkit-scrollbar-thumb { + visibility: visible; +} +.katex-display::-webkit-scrollbar-thumb { + visibility: hidden; +} + +.katex-display::-webkit-scrollbar-corner { + display: none; +} + .cm-editor { height: 100%; width: 100%; diff --git a/src/lib/apis/index.ts b/src/lib/apis/index.ts index 30364fabf2..0929e0a698 100644 --- a/src/lib/apis/index.ts +++ b/src/lib/apis/index.ts @@ -260,6 +260,38 @@ export const stopTask = async (token: string, id: string) => { return res; }; +export const getTaskIdsByChatId = async (token: string, chat_id: string) => { + let error = null; + + const res = await fetch(`${WEBUI_BASE_URL}/api/tasks/chat/${chat_id}`, { + method: 'GET', + headers: { + Accept: 'application/json', + 'Content-Type': 'application/json', + ...(token && { authorization: `Bearer ${token}` }) + } + }) + .then(async (res) => { + if (!res.ok) throw await res.json(); + return res.json(); + }) + .catch((err) => { + console.log(err); + if ('detail' in err) { + error = err.detail; + } else { + error = err; + } + return null; + }); + + if (error) { + throw error; + } + + return res; +}; + export const getToolServerData = async (token: string, url: string) => { let error = null; diff --git a/src/lib/apis/knowledge/index.ts b/src/lib/apis/knowledge/index.ts index a1b80dbe45..92fda2a953 100644 --- a/src/lib/apis/knowledge/index.ts +++ b/src/lib/apis/knowledge/index.ts @@ -346,7 +346,6 @@ export const deleteKnowledgeById = async (token: string, id: string) => { return res; }; - export const reindexKnowledgeFiles = async (token: string) => { let error = null; @@ -373,4 +372,4 @@ export const reindexKnowledgeFiles = async (token: string) => { } return res; -}; \ No newline at end of file +}; diff --git a/src/lib/apis/retrieval/index.ts b/src/lib/apis/retrieval/index.ts index 31317fe0b9..f4b937b68f 100644 --- a/src/lib/apis/retrieval/index.ts +++ b/src/lib/apis/retrieval/index.ts @@ -50,9 +50,9 @@ type YoutubeConfigForm = { }; type RAGConfigForm = { - pdf_extract_images?: boolean; - enable_google_drive_integration?: boolean; - enable_onedrive_integration?: boolean; + PDF_EXTRACT_IMAGES?: boolean; + ENABLE_GOOGLE_DRIVE_INTEGRATION?: boolean; + ENABLE_ONEDRIVE_INTEGRATION?: boolean; chunk?: ChunkConfigForm; content_extraction?: ContentExtractConfigForm; web_loader_ssl_verification?: boolean; @@ -89,33 +89,6 @@ export const updateRAGConfig = async (token: string, payload: RAGConfigForm) => return res; }; -export const getRAGTemplate = async (token: string) => { - let error = null; - - const res = await fetch(`${RETRIEVAL_API_BASE_URL}/template`, { - method: 'GET', - headers: { - 'Content-Type': 'application/json', - Authorization: `Bearer ${token}` - } - }) - .then(async (res) => { - if (!res.ok) throw await res.json(); - return res.json(); - }) - .catch((err) => { - console.log(err); - error = err.detail; - return null; - }); - - if (error) { - throw error; - } - - return res?.template ?? ''; -}; - export const getQuerySettings = async (token: string) => { let error = null; diff --git a/src/lib/components/AddServerModal.svelte b/src/lib/components/AddServerModal.svelte index 1ce7369e44..a5f0ca5c71 100644 --- a/src/lib/components/AddServerModal.svelte +++ b/src/lib/components/AddServerModal.svelte @@ -35,7 +35,7 @@ let auth_type = 'bearer'; let key = ''; - let accessControl = null; + let accessControl = {}; let enable = true; diff --git a/src/lib/components/admin/Settings/Audio.svelte b/src/lib/components/admin/Settings/Audio.svelte index cda6cf03cd..52b8749352 100644 --- a/src/lib/components/admin/Settings/Audio.svelte +++ b/src/lib/components/admin/Settings/Audio.svelte @@ -106,15 +106,15 @@ AZURE_SPEECH_OUTPUT_FORMAT: TTS_AZURE_SPEECH_OUTPUT_FORMAT }, stt: { - OPENAI_API_BASE_URL: STT_OPENAI_API_BASE_URL, - OPENAI_API_KEY: STT_OPENAI_API_KEY, - ENGINE: STT_ENGINE, - MODEL: STT_MODEL, - WHISPER_MODEL: STT_WHISPER_MODEL, - DEEPGRAM_API_KEY: STT_DEEPGRAM_API_KEY, - AZURE_API_KEY: STT_AZURE_API_KEY, - AZURE_REGION: STT_AZURE_REGION, - AZURE_LOCALES: STT_AZURE_LOCALES + OPENAI_API_BASE_URL: STT_OPENAI_API_BASE_URL, + OPENAI_API_KEY: STT_OPENAI_API_KEY, + ENGINE: STT_ENGINE, + MODEL: STT_MODEL, + WHISPER_MODEL: STT_WHISPER_MODEL, + DEEPGRAM_API_KEY: STT_DEEPGRAM_API_KEY, + AZURE_API_KEY: STT_AZURE_API_KEY, + AZURE_REGION: STT_AZURE_REGION, + AZURE_LOCALES: STT_AZURE_LOCALES } }); @@ -150,7 +150,7 @@ STT_OPENAI_API_BASE_URL = res.stt.OPENAI_API_BASE_URL; STT_OPENAI_API_KEY = res.stt.OPENAI_API_KEY; - + STT_ENGINE = res.stt.ENGINE; STT_MODEL = res.stt.MODEL; STT_WHISPER_MODEL = res.stt.WHISPER_MODEL; @@ -190,7 +190,7 @@ - + @@ -259,34 +259,38 @@ {:else if STT_ENGINE === 'azure'} -
-
- - -
- -
- -
-
{$i18n.t('Language Locales')}
-
-
- -
-
-
-
+
+
+ + +
+ +
+ +
+
{$i18n.t('Language Locales')}
+
+
+ +
+
+
+
{:else if STT_ENGINE === ''} -
+
{$i18n.t('STT Model')}
@@ -464,8 +468,7 @@ {voice.name} {/each} diff --git a/src/lib/components/admin/Settings/Documents.svelte b/src/lib/components/admin/Settings/Documents.svelte index 97c62c6a41..2047a07e76 100644 --- a/src/lib/components/admin/Settings/Documents.svelte +++ b/src/lib/components/admin/Settings/Documents.svelte @@ -17,8 +17,8 @@ updateRAGConfig } from '$lib/apis/retrieval'; - import { reindexKnowledgeFiles} from '$lib/apis/knowledge'; - import { deleteAllFiles } from '$lib/apis/files'; + import { reindexKnowledgeFiles } from '$lib/apis/knowledge'; + import { deleteAllFiles } from '$lib/apis/files'; import ResetUploadDirConfirmDialog from '$lib/components/common/ConfirmDialog.svelte'; import ResetVectorDBConfirmDialog from '$lib/components/common/ConfirmDialog.svelte'; @@ -27,6 +27,7 @@ import Tooltip from '$lib/components/common/Tooltip.svelte'; import Switch from '$lib/components/common/Switch.svelte'; import Textarea from '$lib/components/common/Textarea.svelte'; + import Spinner from '$lib/components/common/Spinner.svelte'; const i18n = getContext('i18n'); @@ -42,31 +43,6 @@ let embeddingBatchSize = 1; let rerankingModel = ''; - let fileMaxSize = null; - let fileMaxCount = null; - - let contentExtractionEngine = 'default'; - let tikaServerUrl = ''; - let showTikaServerUrl = false; - let doclingServerUrl = ''; - let showDoclingServerUrl = false; - let documentIntelligenceEndpoint = ''; - let documentIntelligenceKey = ''; - let showDocumentIntelligenceConfig = false; - let mistralApiKey = ''; - let showMistralOcrConfig = false; - - let textSplitter = ''; - let chunkSize = 0; - let chunkOverlap = 0; - let pdfExtractImages = true; - - let RAG_FULL_CONTEXT = false; - let BYPASS_EMBEDDING_AND_RETRIEVAL = false; - - let enableGoogleDriveIntegration = false; - let enableOneDriveIntegration = false; - let OpenAIUrl = ''; let OpenAIKey = ''; @@ -81,6 +57,8 @@ hybrid: false }; + let RAGConfig = null; + const embeddingModelUpdateHandler = async () => { if (embeddingEngine === '' && embeddingModel.split('/').length - 1 > 1) { toast.error( @@ -175,65 +153,40 @@ }; const submitHandler = async () => { - if (contentExtractionEngine === 'tika' && tikaServerUrl === '') { + if (RAGConfig.CONTENT_EXTRACTION_ENGINE === 'tika' && RAGConfig.TIKA_SERVER_URL === '') { toast.error($i18n.t('Tika Server URL required.')); return; } - if (contentExtractionEngine === 'docling' && doclingServerUrl === '') { + if (RAGConfig.CONTENT_EXTRACTION_ENGINE === 'docling' && RAGConfig.DOCLING_SERVER_URL === '') { toast.error($i18n.t('Docling Server URL required.')); return; } + if ( - contentExtractionEngine === 'document_intelligence' && - (documentIntelligenceEndpoint === '' || documentIntelligenceKey === '') + RAGConfig.CONTENT_EXTRACTION_ENGINE === 'document_intelligence' && + (RAGConfig.DOCUMENT_INTELLIGENCE_ENDPOINT === '' || + RAGConfig.DOCUMENT_INTELLIGENCE_KEY === '') ) { toast.error($i18n.t('Document Intelligence endpoint and key required.')); return; } - if (contentExtractionEngine === 'mistral_ocr' && mistralApiKey === '') { + if ( + RAGConfig.CONTENT_EXTRACTION_ENGINE === 'mistral_ocr' && + RAGConfig.MISTRAL_OCR_API_KEY === '' + ) { toast.error($i18n.t('Mistral OCR API Key required.')); return; } - if (!BYPASS_EMBEDDING_AND_RETRIEVAL) { + if (!RAGConfig.BYPASS_EMBEDDING_AND_RETRIEVAL) { await embeddingModelUpdateHandler(); - if (querySettings.hybrid) { + if (RAGConfig.ENABLE_RAG_HYBRID_SEARCH) { await rerankingModelUpdateHandler(); } } - const res = await updateRAGConfig(localStorage.token, { - pdf_extract_images: pdfExtractImages, - enable_google_drive_integration: enableGoogleDriveIntegration, - enable_onedrive_integration: enableOneDriveIntegration, - file: { - max_size: fileMaxSize === '' ? null : fileMaxSize, - max_count: fileMaxCount === '' ? null : fileMaxCount - }, - RAG_FULL_CONTEXT: RAG_FULL_CONTEXT, - BYPASS_EMBEDDING_AND_RETRIEVAL: BYPASS_EMBEDDING_AND_RETRIEVAL, - chunk: { - text_splitter: textSplitter, - chunk_overlap: chunkOverlap, - chunk_size: chunkSize - }, - content_extraction: { - engine: contentExtractionEngine, - tika_server_url: tikaServerUrl, - docling_server_url: doclingServerUrl, - document_intelligence_config: { - key: documentIntelligenceKey, - endpoint: documentIntelligenceEndpoint - }, - mistral_ocr_config: { - api_key: mistralApiKey - } - } - }); - - await updateQuerySettings(localStorage.token, querySettings); - + const res = await updateRAGConfig(localStorage.token, RAGConfig); dispatch('save'); }; @@ -261,46 +214,11 @@ } }; - const toggleHybridSearch = async () => { - querySettings = await updateQuerySettings(localStorage.token, querySettings); - }; - onMount(async () => { await setEmbeddingConfig(); await setRerankingConfig(); - querySettings = await getQuerySettings(localStorage.token); - - const res = await getRAGConfig(localStorage.token); - - if (res) { - pdfExtractImages = res.pdf_extract_images; - - textSplitter = res.chunk.text_splitter; - chunkSize = res.chunk.chunk_size; - chunkOverlap = res.chunk.chunk_overlap; - - RAG_FULL_CONTEXT = res.RAG_FULL_CONTEXT; - BYPASS_EMBEDDING_AND_RETRIEVAL = res.BYPASS_EMBEDDING_AND_RETRIEVAL; - - contentExtractionEngine = res.content_extraction.engine; - tikaServerUrl = res.content_extraction.tika_server_url; - doclingServerUrl = res.content_extraction.docling_server_url; - - showTikaServerUrl = contentExtractionEngine === 'tika'; - showDoclingServerUrl = contentExtractionEngine === 'docling'; - documentIntelligenceEndpoint = res.content_extraction.document_intelligence_config.endpoint; - documentIntelligenceKey = res.content_extraction.document_intelligence_config.key; - showDocumentIntelligenceConfig = contentExtractionEngine === 'document_intelligence'; - mistralApiKey = res.content_extraction.mistral_ocr_config.api_key; - showMistralOcrConfig = contentExtractionEngine === 'mistral_ocr'; - - fileMaxSize = res?.file.max_size ?? ''; - fileMaxCount = res?.file.max_count ?? ''; - - enableGoogleDriveIntegration = res.enable_google_drive_integration; - enableOneDriveIntegration = res.enable_onedrive_integration; - } + RAGConfig = await getRAGConfig(localStorage.token); }); @@ -332,7 +250,6 @@ }} /> - { @@ -353,339 +270,93 @@ submitHandler(); }} > -
-
-
-
{$i18n.t('General')}
- -
- -
-
-
- {$i18n.t('Content Extraction Engine')} -
-
- -
-
- {#if contentExtractionEngine === 'tika'} -
-
- -
-
- {:else if contentExtractionEngine === 'docling'} -
- -
- {:else if contentExtractionEngine === 'document_intelligence'} -
- - -
- {:else if contentExtractionEngine === 'mistral_ocr'} -
- -
- {/if} -
- - {#if contentExtractionEngine === ''} -
-
- {$i18n.t('PDF Extract Images (OCR)')} -
-
- -
-
- {/if} - -
-
- - {$i18n.t('Bypass Embedding and Retrieval')} - -
-
- - - -
-
- - {#if !BYPASS_EMBEDDING_AND_RETRIEVAL} -
-
{$i18n.t('Text Splitter')}
-
- -
-
- -
-
-
-
- {$i18n.t('Chunk Size')} -
-
- -
-
- -
-
- {$i18n.t('Chunk Overlap')} -
- -
- -
-
-
-
- {/if} -
- - {#if !BYPASS_EMBEDDING_AND_RETRIEVAL} + {#if RAGConfig} +
+
-
{$i18n.t('Embedding')}
+
{$i18n.t('General')}

-
+
-
- {$i18n.t('Embedding Model Engine')} +
+ {$i18n.t('Content Extraction Engine')}
-
+
- {#if embeddingEngine === 'openai'} -
- - - + {#if RAGConfig.CONTENT_EXTRACTION_ENGINE === ''} +
+
+
+ {$i18n.t('PDF Extract Images (OCR)')} +
+
+ +
+
- {:else if embeddingEngine === 'ollama'} + {:else if RAGConfig.CONTENT_EXTRACTION_ENGINE === 'tika'} +
+
+ +
+
+ {:else if RAGConfig.CONTENT_EXTRACTION_ENGINE === 'docling'} +
+ +
+ {:else if RAGConfig.CONTENT_EXTRACTION_ENGINE === 'document_intelligence'}
- +
+ {:else if RAGConfig.CONTENT_EXTRACTION_ENGINE === 'mistral_ocr'} +
+
{/if}
-
-
{$i18n.t('Embedding Model')}
- -
- {#if embeddingEngine === 'ollama'} -
-
- -
-
- {:else} -
-
- -
- - {#if embeddingEngine === ''} - - {/if} -
- {/if} -
- -
- {$i18n.t( - 'Warning: If you update or change your embedding model, you will need to re-import all documents.' - )} -
-
- - {#if embeddingEngine === 'ollama' || embeddingEngine === 'openai'} -
-
{$i18n.t('Embedding Batch Size')}
- -
- -
-
- {/if} -
- -
-
{$i18n.t('Retrieval')}
- -
-
-
{$i18n.t('Full Context Mode')}
+
+ + {$i18n.t('Bypass Embedding and Retrieval')} + +
- +
- {#if !RAG_FULL_CONTEXT} + {#if !RAGConfig.BYPASS_EMBEDDING_AND_RETRIEVAL}
-
{$i18n.t('Hybrid Search')}
+
{$i18n.t('Text Splitter')}
- { - toggleHybridSearch(); - }} - /> +
- {#if querySettings.hybrid === true} -
-
{$i18n.t('Reranking Model')}
+
+
+
+
+ {$i18n.t('Chunk Size')} +
+
+ +
+
-
+
+
+ {$i18n.t('Chunk Overlap')} +
+ +
+ +
+
+
+
+ {/if} +
+ + {#if !RAGConfig.BYPASS_EMBEDDING_AND_RETRIEVAL} +
+
{$i18n.t('Embedding')}
+ +
+ +
+
+
+ {$i18n.t('Embedding Model Engine')} +
+
+ +
+
+ + {#if embeddingEngine === 'openai'} +
+ + + +
+ {:else if embeddingEngine === 'ollama'} +
+ + + +
+ {/if} +
+ +
+
{$i18n.t('Embedding Model')}
+ +
+ {#if embeddingEngine === 'ollama'}
+
+
+ {:else} +
+
+
- + {/if} + + {/if}
-
+ {/if}
- {/if} -
-
{$i18n.t('Top K')}
-
- +
+ {$i18n.t( + 'Warning: If you update or change your embedding model, you will need to re-import all documents.' + )}
- {#if querySettings.hybrid === true} -
-
{$i18n.t('Top K Reranker')}
+ {#if embeddingEngine === 'ollama' || embeddingEngine === 'openai'} +
+
+ {$i18n.t('Embedding Batch Size')} +
+ +
+ +
+
+ {/if} +
+ +
+
{$i18n.t('Retrieval')}
+ +
+ +
+
{$i18n.t('Full Context Mode')}
+
+ + + +
+
+ + {#if !RAGConfig.RAG_FULL_CONTEXT} +
+
{$i18n.t('Hybrid Search')}
+
+ { + submitHandler(); + }} + /> +
+
+ + {#if RAGConfig.ENABLE_RAG_HYBRID_SEARCH === true} +
+
{$i18n.t('Reranking Model')}
+ +
+
+
+ +
+ +
+
+
+ {/if} + +
+
{$i18n.t('Top K')}
- {/if} - {#if querySettings.hybrid === true} -
-
-
{$i18n.t('Minimum Score')}
+ {#if RAGConfig.ENABLE_RAG_HYBRID_SEARCH === true} +
+
{$i18n.t('Top K Reranker')}
-
- {$i18n.t( - 'Note: If you set a minimum score, the search will only return documents with a score greater than or equal to the minimum score.' - )} -
-
- {/if} - {/if} + {/if} -
-
{$i18n.t('RAG Template')}
-
- -