ti.sil/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2025-12-13 21:05:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Merge pull request #5051 from open-webui/dev

Browse source 
0.3.17
This commit is contained in:
Timothy Jaeryang Baek 2024-09-04 18:40:16 +02:00 committed by GitHub
commit a9801147b8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
178 changed files with 2047 additions and 1371 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.github/workflows/release-pypi.yml vendored
View file

@ -4,6 +4,7 @@ on:
push:
branches:
- main # or whatever branch you want to use
- pypi-release
jobs:
release:

25
CHANGELOG.md
View file

@ -5,6 +5,31 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.3.17] - 2024-09-04
### Added
- **🔄 Import/Export Configuration**: Users can now import and export webui configurations from admin settings > Database, simplifying setup replication across systems.
- **🌍 Web Search via URL Parameter**: Added support for activating web search directly through URL by setting 'web-search=true'.
- **🌐 SearchApi Integration**: Added support for SearchApi as an alternative web search provider, enhancing search capabilities within the platform.
- **🔍 Literal Type Support in Tools**: Tools now support the Literal type.
- **🌍 Updated Translations**: Improved translations for Chinese, Ukrainian, and Catalan.
### Fixed
- **🔧 Pip Install Issue**: Resolved the issue where pip install failed due to missing 'alembic.ini', ensuring smoother installation processes.
- **🌃 Automatic Theme Update**: Fixed an issue where the color theme did not update dynamically with system changes.
- **🛠️ User Agent in ComfyUI**: Added default headers in ComfyUI to fix access issues, improving reliability in network communications.
- **🔄 Missing Chat Completion Response Headers**: Ensured proper return of proxied response headers during chat completion, improving API reliability.
- **🔗 Websocket Connection Prioritization**: Modified socket.io configuration to prefer websockets and more reliably fallback to polling, enhancing connection stability.
- **🎭 Accessibility Enhancements**: Added missing ARIA labels for buttons, improving accessibility for visually impaired users.
- **⚖️ Advanced Parameter**: Fixed an issue ensuring that advanced parameters are correctly applied in all scenarios, ensuring consistent behavior of user-defined settings.
### Changed
- **🔁 Namespace Reorganization**: Reorganized all Python files under the 'open_webui' namespace to streamline the project structure and improve maintainability. Tools and functions importing from 'utils' should now use 'open_webui.utils'.
- **🚧 Dependency Updates**: Updated several backend dependencies like 'aiohttp', 'authlib', 'duckduckgo-search', 'flask-cors', and 'langchain' to their latest versions, enhancing performance and security.
## [0.3.16] - 2024-08-27
### Added

2
README.md
View file

@ -37,7 +37,7 @@ Open WebUI is an [extensible](https://github.com/open-webui/pipelines), feature-
- 📚 **Local RAG Integration**: Dive into the future of chat interactions with groundbreaking Retrieval Augmented Generation (RAG) support. This feature seamlessly integrates document interactions into your chat experience. You can load documents directly into the chat or add files to your document library, effortlessly accessing them using the `#` command before a query.
- 🔍 **Web Search for RAG**: Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, `serper`, `Serply`, `DuckDuckGo` and `TavilySearch` and inject the results directly into your chat experience.
- 🔍 **Web Search for RAG**: Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, `serper`, `Serply`, `DuckDuckGo`, `TavilySearch` and `SearchApi` and inject the results directly into your chat experience.
- 🌐 **Web Browsing Capability**: Seamlessly integrate websites into your chat experience using the `#` command followed by a URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions.

4
backend/data/litellm/config.yaml
View file

@ -1,4 +0,0 @@
general_settings: {}
litellm_settings: {}
model_list: []
router_settings: {}

2
backend/data/readme.txt
View file

@ -1 +1 @@
dir for backend files (db, documents, etc.)
docker dir for backend files (db, documents, etc.)

2
backend/dev.sh
View file

@ -1,2 +1,2 @@
PORT="${PORT:-8080}"
uvicorn main:app --port $PORT --host 0.0.0.0 --forwarded-allow-ips '*' --reload
uvicorn open_webui.main:app --port $PORT --host 0.0.0.0 --forwarded-allow-ips '*' --reload

12
backend/open_webui/__init__.py
View file

@ -9,8 +9,6 @@ import uvicorn
app = typer.Typer()
KEY_FILE = Path.cwd() / ".webui_secret_key"
if (frontend_build_dir := Path(__file__).parent / "frontend").exists():
os.environ["FRONTEND_BUILD_DIR"] = str(frontend_build_dir)
@app.command()
@ -40,9 +38,9 @@ def serve(
"/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib",
]
)
import main # we need set environment variables before importing main
import open_webui.main # we need set environment variables before importing main
uvicorn.run(main.app, host=host, port=port, forwarded_allow_ips="*")
uvicorn.run(open_webui.main.app, host=host, port=port, forwarded_allow_ips="*")
@app.command()
@ -52,7 +50,11 @@ def dev(
reload: bool = True,
):
uvicorn.run(
"main:app", host=host, port=port, reload=reload, forwarded_allow_ips="*"
"open_webui.main:app",
host=host,
port=port,
reload=reload,
forwarded_allow_ips="*",
)

0
backend/alembic.ini → backend/open_webui/alembic.ini
View file

63
backend/apps/audio/main.py → backend/open_webui/apps/audio/main.py
View file

@ -7,46 +7,33 @@ from functools import lru_cache
from pathlib import Path
import requests
from fastapi import (
FastAPI,
Request,
Depends,
HTTPException,
status,
UploadFile,
File,
from open_webui.config import (
AUDIO_STT_ENGINE,
AUDIO_STT_MODEL,
AUDIO_STT_OPENAI_API_BASE_URL,
AUDIO_STT_OPENAI_API_KEY,
AUDIO_TTS_API_KEY,
AUDIO_TTS_ENGINE,
AUDIO_TTS_MODEL,
AUDIO_TTS_OPENAI_API_BASE_URL,
AUDIO_TTS_OPENAI_API_KEY,
AUDIO_TTS_SPLIT_ON,
AUDIO_TTS_VOICE,
CACHE_DIR,
CORS_ALLOW_ORIGIN,
DEVICE_TYPE,
WHISPER_MODEL,
WHISPER_MODEL_AUTO_UPDATE,
WHISPER_MODEL_DIR,
AppConfig,
)
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import Depends, FastAPI, File, HTTPException, Request, UploadFile, status
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse
from pydantic import BaseModel
from config import (
SRC_LOG_LEVELS,
CACHE_DIR,
WHISPER_MODEL,
WHISPER_MODEL_DIR,
WHISPER_MODEL_AUTO_UPDATE,
DEVICE_TYPE,
AUDIO_STT_OPENAI_API_BASE_URL,
AUDIO_STT_OPENAI_API_KEY,
AUDIO_TTS_OPENAI_API_BASE_URL,
AUDIO_TTS_OPENAI_API_KEY,
AUDIO_TTS_API_KEY,
AUDIO_STT_ENGINE,
AUDIO_STT_MODEL,
AUDIO_TTS_ENGINE,
AUDIO_TTS_MODEL,
AUDIO_TTS_VOICE,
AUDIO_TTS_SPLIT_ON,
AppConfig,
CORS_ALLOW_ORIGIN,
)
from constants import ERROR_MESSAGES
from utils.utils import (
get_current_user,
get_verified_user,
get_admin_user,
)
from open_webui.utils.utils import get_admin_user, get_current_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["AUDIO"])
@ -211,7 +198,7 @@ async def speech(request: Request, user=Depends(get_verified_user)):
body = json.loads(body)
body["model"] = app.state.config.TTS_MODEL
body = json.dumps(body).encode("utf-8")
except Exception as e:
except Exception:
pass
r = None
@ -488,7 +475,7 @@ def get_available_voices() -> dict:
elif app.state.config.TTS_ENGINE == "elevenlabs":
try:
ret = get_elevenlabs_voices()
except Exception as e:
except Exception:
# Avoided @lru_cache with exception
pass

68
backend/apps/images/main.py → backend/open_webui/apps/images/main.py
View file

@ -1,52 +1,42 @@
from fastapi import (
FastAPI,
Request,
Depends,
HTTPException,
)
from fastapi.middleware.cors import CORSMiddleware
from typing import Optional
from pydantic import BaseModel
from pathlib import Path
import mimetypes
import uuid
import asyncio
import base64
import json
import logging
import mimetypes
import re
import uuid
from pathlib import Path
from typing import Optional
import requests
import asyncio
from utils.utils import (
get_verified_user,
get_admin_user,
)
from apps.images.utils.comfyui import (
ComfyUIWorkflow,
from open_webui.apps.images.utils.comfyui import (
ComfyUIGenerateImageForm,
ComfyUIWorkflow,
comfyui_generate_image,
)
from constants import ERROR_MESSAGES
from config import (
SRC_LOG_LEVELS,
CACHE_DIR,
IMAGE_GENERATION_ENGINE,
ENABLE_IMAGE_GENERATION,
AUTOMATIC1111_BASE_URL,
from open_webui.config import (
AUTOMATIC1111_API_AUTH,
AUTOMATIC1111_BASE_URL,
CACHE_DIR,
COMFYUI_BASE_URL,
COMFYUI_WORKFLOW,
COMFYUI_WORKFLOW_NODES,
IMAGES_OPENAI_API_BASE_URL,
IMAGES_OPENAI_API_KEY,
CORS_ALLOW_ORIGIN,
ENABLE_IMAGE_GENERATION,
IMAGE_GENERATION_ENGINE,
IMAGE_GENERATION_MODEL,
IMAGE_SIZE,
IMAGE_STEPS,
CORS_ALLOW_ORIGIN,
IMAGES_OPENAI_API_BASE_URL,
IMAGES_OPENAI_API_KEY,
AppConfig,
)
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import Depends, FastAPI, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from open_webui.utils.utils import get_admin_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["IMAGES"])
@ -143,7 +133,7 @@ async def update_config(form_data: ConfigForm, user=Depends(get_admin_user)):
form_data.automatic1111.AUTOMATIC1111_API_AUTH
)
app.state.config.COMFYUI_BASE_URL = form_data.comfyui.COMFYUI_BASE_URL
app.state.config.COMFYUI_BASE_URL = form_data.comfyui.COMFYUI_BASE_URL.strip("/")
app.state.config.COMFYUI_WORKFLOW = form_data.comfyui.COMFYUI_WORKFLOW
app.state.config.COMFYUI_WORKFLOW_NODES = form_data.comfyui.COMFYUI_WORKFLOW_NODES
@ -186,7 +176,7 @@ async def verify_url(user=Depends(get_admin_user)):
)
r.raise_for_status()
return True
except Exception as e:
except Exception:
app.state.config.ENABLED = False
raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL)
elif app.state.config.ENGINE == "comfyui":
@ -194,7 +184,7 @@ async def verify_url(user=Depends(get_admin_user)):
r = requests.get(url=f"{app.state.config.COMFYUI_BASE_URL}/object_info")
r.raise_for_status()
return True
except Exception as e:
except Exception:
app.state.config.ENABLED = False
raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL)
else:
@ -202,6 +192,7 @@ async def verify_url(user=Depends(get_admin_user)):
def set_image_model(model: str):
log.info(f"Setting image model to {model}")
app.state.config.MODEL = model
if app.state.config.ENGINE in ["", "automatic1111"]:
api_auth = get_automatic1111_api_auth()
@ -255,7 +246,8 @@ async def get_image_config(user=Depends(get_admin_user)):
@app.post("/image/config/update")
async def update_image_config(form_data: ImageConfigForm, user=Depends(get_admin_user)):
app.state.config.MODEL = form_data.MODEL
set_image_model(form_data.MODEL)
pattern = r"^\d+x\d+$"
if re.match(pattern, form_data.IMAGE_SIZE):
@ -397,7 +389,6 @@ def save_url_image(url):
r = requests.get(url)
r.raise_for_status()
if r.headers["content-type"].split("/")[0] == "image":
mime_type = r.headers["content-type"]
image_format = mimetypes.guess_extension(mime_type)
@ -412,7 +403,7 @@ def save_url_image(url):
image_file.write(chunk)
return image_filename
else:
log.error(f"Url does not point to an image.")
log.error("Url does not point to an image.")
return None
except Exception as e:
@ -430,7 +421,6 @@ async def image_generations(
r = None
try:
if app.state.config.ENGINE == "openai":
headers = {}
headers["Authorization"] = f"Bearer {app.state.config.OPENAI_API_KEY}"
headers["Content-Type"] = "application/json"

39
backend/apps/images/utils/comfyui.py → backend/open_webui/apps/images/utils/comfyui.py
View file

@ -1,34 +1,45 @@
import asyncio
import websocket # NOTE: websocket-client (https://github.com/websocket-client/websocket-client)
import json
import urllib.request
import urllib.parse
import random
import logging
import random
import urllib.parse
import urllib.request
from typing import Optional
from config import SRC_LOG_LEVELS
import websocket # NOTE: websocket-client (https://github.com/websocket-client/websocket-client)
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["COMFYUI"])
from pydantic import BaseModel
from typing import Optional
default_headers = {"User-Agent": "Mozilla/5.0"}
def queue_prompt(prompt, client_id, base_url):
log.info("queue_prompt")
p = {"prompt": prompt, "client_id": client_id}
data = json.dumps(p).encode("utf-8")
req = urllib.request.Request(f"{base_url}/prompt", data=data)
return json.loads(urllib.request.urlopen(req).read())
log.debug(f"queue_prompt data: {data}")
try:
req = urllib.request.Request(
f"{base_url}/prompt", data=data, headers=default_headers
)
response = urllib.request.urlopen(req).read()
return json.loads(response)
except Exception as e:
log.exception(f"Error while queuing prompt: {e}")
raise e
def get_image(filename, subfolder, folder_type, base_url):
log.info("get_image")
data = {"filename": filename, "subfolder": subfolder, "type": folder_type}
url_values = urllib.parse.urlencode(data)
with urllib.request.urlopen(f"{base_url}/view?{url_values}") as response:
req = urllib.request.Request(
f"{base_url}/view?{url_values}", headers=default_headers
)
with urllib.request.urlopen(req) as response:
return response.read()
@ -41,7 +52,11 @@ def get_image_url(filename, subfolder, folder_type, base_url):
def get_history(prompt_id, base_url):
log.info("get_history")
with urllib.request.urlopen(f"{base_url}/history/{prompt_id}") as response:
req = urllib.request.Request(
f"{base_url}/history/{prompt_id}", headers=default_headers
)
with urllib.request.urlopen(req) as response:
return json.loads(response.read())

58
backend/apps/ollama/main.py → backend/open_webui/apps/ollama/main.py
View file

@ -1,54 +1,40 @@
from fastapi import (
FastAPI,
Request,
HTTPException,
Depends,
UploadFile,
File,
)
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, ConfigDict
import os
import re
import random
import requests
import json
import aiohttp
import asyncio
import json
import logging
import os
import random
import re
import time
from urllib.parse import urlparse
from typing import Optional, Union
from urllib.parse import urlparse
from starlette.background import BackgroundTask
from apps.webui.models.models import Models
from constants import ERROR_MESSAGES
from utils.utils import (
get_verified_user,
get_admin_user,
)
from config import (
SRC_LOG_LEVELS,
OLLAMA_BASE_URLS,
ENABLE_OLLAMA_API,
import aiohttp
import requests
from open_webui.apps.webui.models.models import Models
from open_webui.config import (
AIOHTTP_CLIENT_TIMEOUT,
CORS_ALLOW_ORIGIN,
ENABLE_MODEL_FILTER,
ENABLE_OLLAMA_API,
MODEL_FILTER_LIST,
OLLAMA_BASE_URLS,
UPLOAD_DIR,
AppConfig,
CORS_ALLOW_ORIGIN,
)
from utils.misc import (
calculate_sha256,
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import Depends, FastAPI, File, HTTPException, Request, UploadFile
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, ConfigDict
from starlette.background import BackgroundTask
from open_webui.utils.misc import (
apply_model_params_to_body_ollama,
apply_model_params_to_body_openai,
apply_model_system_prompt_to_body,
calculate_sha256,
)
from open_webui.utils.utils import get_admin_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["OLLAMA"])

56
backend/apps/openai/main.py → backend/open_webui/apps/openai/main.py
View file

@ -1,44 +1,36 @@
from fastapi import FastAPI, Request, HTTPException, Depends
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse, FileResponse
import requests
import aiohttp
import asyncio
import hashlib
import json
import logging
from pathlib import Path
from typing import Literal, Optional, overload
import aiohttp
import requests
from open_webui.apps.webui.models.models import Models
from open_webui.config import (
AIOHTTP_CLIENT_TIMEOUT,
CACHE_DIR,
CORS_ALLOW_ORIGIN,
ENABLE_MODEL_FILTER,
ENABLE_OPENAI_API,
MODEL_FILTER_LIST,
OPENAI_API_BASE_URLS,
OPENAI_API_KEYS,
AppConfig,
)
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import Depends, FastAPI, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, StreamingResponse
from pydantic import BaseModel
from starlette.background import BackgroundTask
from apps.webui.models.models import Models
from constants import ERROR_MESSAGES
from utils.utils import (
get_verified_user,
get_admin_user,
)
from utils.misc import (
from open_webui.utils.misc import (
apply_model_params_to_body_openai,
apply_model_system_prompt_to_body,
)
from config import (
SRC_LOG_LEVELS,
ENABLE_OPENAI_API,
AIOHTTP_CLIENT_TIMEOUT,
OPENAI_API_BASE_URLS,
OPENAI_API_KEYS,
CACHE_DIR,
ENABLE_MODEL_FILTER,
MODEL_FILTER_LIST,
AppConfig,
CORS_ALLOW_ORIGIN,
)
from typing import Optional, Literal, overload
import hashlib
from pathlib import Path
from open_webui.utils.utils import get_admin_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["OPENAI"])

236
backend/apps/rag/main.py → backend/open_webui/apps/rag/main.py
View file

@ -1,140 +1,118 @@
from fastapi import (
FastAPI,
Depends,
HTTPException,
status,
UploadFile,
File,
Form,
)
from fastapi.middleware.cors import CORSMiddleware
import requests
import os, shutil, logging, re
from datetime import datetime
from pathlib import Path
from typing import Union, Sequence, Iterator, Any
from chromadb.utils.batch_utils import create_batches
from langchain_core.documents import Document
from langchain_community.document_loaders import (
WebBaseLoader,
TextLoader,
PyPDFLoader,
CSVLoader,
BSHTMLLoader,
Docx2txtLoader,
UnstructuredEPubLoader,
UnstructuredWordDocumentLoader,
UnstructuredMarkdownLoader,
UnstructuredXMLLoader,
UnstructuredRSTLoader,
UnstructuredExcelLoader,
UnstructuredPowerPointLoader,
YoutubeLoader,
OutlookMessageLoader,
)
from langchain.text_splitter import RecursiveCharacterTextSplitter
import validators
import urllib.parse
import socket
from pydantic import BaseModel
from typing import Optional
import mimetypes
import uuid
import json
import logging
import mimetypes
import os
import shutil
import socket
import urllib.parse
import uuid
from datetime import datetime
from pathlib import Path
from typing import Iterator, Optional, Sequence, Union
from apps.webui.models.documents import (
Documents,
DocumentForm,
DocumentResponse,
)
from apps.webui.models.files import (
Files,
)
from apps.rag.utils import (
get_model_path,
import requests
import validators
from open_webui.apps.rag.search.brave import search_brave
from open_webui.apps.rag.search.duckduckgo import search_duckduckgo
from open_webui.apps.rag.search.google_pse import search_google_pse
from open_webui.apps.rag.search.jina_search import search_jina
from open_webui.apps.rag.search.main import SearchResult
from open_webui.apps.rag.search.searchapi import search_searchapi
from open_webui.apps.rag.search.searxng import search_searxng
from open_webui.apps.rag.search.serper import search_serper
from open_webui.apps.rag.search.serply import search_serply
from open_webui.apps.rag.search.serpstack import search_serpstack
from open_webui.apps.rag.search.tavily import search_tavily
from open_webui.apps.rag.utils import (
get_embedding_function,
query_doc,
query_doc_with_hybrid_search,
get_model_path,
query_collection,
query_collection_with_hybrid_search,
query_doc,
query_doc_with_hybrid_search,
)
from apps.rag.search.brave import search_brave
from apps.rag.search.google_pse import search_google_pse
from apps.rag.search.main import SearchResult
from apps.rag.search.searxng import search_searxng
from apps.rag.search.serper import search_serper
from apps.rag.search.serpstack import search_serpstack
from apps.rag.search.serply import search_serply
from apps.rag.search.duckduckgo import search_duckduckgo
from apps.rag.search.tavily import search_tavily
from apps.rag.search.jina_search import search_jina
from utils.misc import (
calculate_sha256,
calculate_sha256_string,
sanitize_filename,
extract_folders_after_data_docs,
)
from utils.utils import get_verified_user, get_admin_user
from config import (
AppConfig,
ENV,
SRC_LOG_LEVELS,
UPLOAD_DIR,
DOCS_DIR,
from open_webui.apps.webui.models.documents import DocumentForm, Documents
from open_webui.apps.webui.models.files import Files
from chromadb.utils.batch_utils import create_batches
from open_webui.config import (
BRAVE_SEARCH_API_KEY,
CHROMA_CLIENT,
CHUNK_OVERLAP,
CHUNK_SIZE,
CONTENT_EXTRACTION_ENGINE,
TIKA_SERVER_URL,
RAG_TOP_K,
RAG_RELEVANCE_THRESHOLD,
RAG_FILE_MAX_SIZE,
RAG_FILE_MAX_COUNT,
CORS_ALLOW_ORIGIN,
DEVICE_TYPE,
DOCS_DIR,
ENABLE_RAG_HYBRID_SEARCH,
ENABLE_RAG_LOCAL_WEB_FETCH,
ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
ENABLE_RAG_WEB_SEARCH,
ENV,
GOOGLE_PSE_API_KEY,
GOOGLE_PSE_ENGINE_ID,
PDF_EXTRACT_IMAGES,
RAG_EMBEDDING_ENGINE,
RAG_EMBEDDING_MODEL,
RAG_EMBEDDING_MODEL_AUTO_UPDATE,
RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE,
ENABLE_RAG_HYBRID_SEARCH,
ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
RAG_RERANKING_MODEL,
PDF_EXTRACT_IMAGES,
RAG_RERANKING_MODEL_AUTO_UPDATE,
RAG_RERANKING_MODEL_TRUST_REMOTE_CODE,
RAG_EMBEDDING_OPENAI_BATCH_SIZE,
RAG_FILE_MAX_COUNT,
RAG_FILE_MAX_SIZE,
RAG_OPENAI_API_BASE_URL,
RAG_OPENAI_API_KEY,
DEVICE_TYPE,
CHROMA_CLIENT,
CHUNK_SIZE,
CHUNK_OVERLAP,
RAG_RELEVANCE_THRESHOLD,
RAG_RERANKING_MODEL,
RAG_RERANKING_MODEL_AUTO_UPDATE,
RAG_RERANKING_MODEL_TRUST_REMOTE_CODE,
RAG_TEMPLATE,
ENABLE_RAG_LOCAL_WEB_FETCH,
YOUTUBE_LOADER_LANGUAGE,
ENABLE_RAG_WEB_SEARCH,
RAG_WEB_SEARCH_ENGINE,
RAG_TOP_K,
RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
RAG_WEB_SEARCH_DOMAIN_FILTER_LIST,
RAG_WEB_SEARCH_ENGINE,
RAG_WEB_SEARCH_RESULT_COUNT,
SEARCHAPI_API_KEY,
SEARCHAPI_ENGINE,
SEARXNG_QUERY_URL,
GOOGLE_PSE_API_KEY,
GOOGLE_PSE_ENGINE_ID,
BRAVE_SEARCH_API_KEY,
SERPSTACK_API_KEY,
SERPSTACK_HTTPS,
SERPER_API_KEY,
SERPLY_API_KEY,
SERPSTACK_API_KEY,
SERPSTACK_HTTPS,
TAVILY_API_KEY,
RAG_WEB_SEARCH_RESULT_COUNT,
RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
RAG_EMBEDDING_OPENAI_BATCH_SIZE,
CORS_ALLOW_ORIGIN,
TIKA_SERVER_URL,
UPLOAD_DIR,
YOUTUBE_LOADER_LANGUAGE,
AppConfig,
)
from constants import ERROR_MESSAGES
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import Depends, FastAPI, File, Form, HTTPException, UploadFile, status
from fastapi.middleware.cors import CORSMiddleware
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import (
BSHTMLLoader,
CSVLoader,
Docx2txtLoader,
OutlookMessageLoader,
PyPDFLoader,
TextLoader,
UnstructuredEPubLoader,
UnstructuredExcelLoader,
UnstructuredMarkdownLoader,
UnstructuredPowerPointLoader,
UnstructuredRSTLoader,
UnstructuredXMLLoader,
WebBaseLoader,
YoutubeLoader,
)
from langchain_core.documents import Document
from pydantic import BaseModel
from open_webui.utils.misc import (
calculate_sha256,
calculate_sha256_string,
extract_folders_after_data_docs,
sanitize_filename,
)
from open_webui.utils.utils import get_admin_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
@ -189,6 +167,8 @@ app.state.config.SERPSTACK_HTTPS = SERPSTACK_HTTPS
app.state.config.SERPER_API_KEY = SERPER_API_KEY
app.state.config.SERPLY_API_KEY = SERPLY_API_KEY
app.state.config.TAVILY_API_KEY = TAVILY_API_KEY
app.state.config.SEARCHAPI_API_KEY = SEARCHAPI_API_KEY
app.state.config.SEARCHAPI_ENGINE = SEARCHAPI_ENGINE
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = RAG_WEB_SEARCH_RESULT_COUNT
app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = RAG_WEB_SEARCH_CONCURRENT_REQUESTS
@ -427,6 +407,8 @@ async def get_rag_config(user=Depends(get_admin_user)):
"serper_api_key": app.state.config.SERPER_API_KEY,
"serply_api_key": app.state.config.SERPLY_API_KEY,
"tavily_api_key": app.state.config.TAVILY_API_KEY,
"searchapi_api_key": app.state.config.SEARCHAPI_API_KEY,
"seaarchapi_engine": app.state.config.SEARCHAPI_ENGINE,
"result_count": app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
"concurrent_requests": app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
},
@ -466,6 +448,8 @@ class WebSearchConfig(BaseModel):
serper_api_key: Optional[str] = None
serply_api_key: Optional[str] = None
tavily_api_key: Optional[str] = None
searchapi_api_key: Optional[str] = None
searchapi_engine: Optional[str] = None
result_count: Optional[int] = None
concurrent_requests: Optional[int] = None
@ -529,6 +513,8 @@ async def update_rag_config(form_data: ConfigUpdateForm, user=Depends(get_admin_
app.state.config.SERPER_API_KEY = form_data.web.search.serper_api_key
app.state.config.SERPLY_API_KEY = form_data.web.search.serply_api_key
app.state.config.TAVILY_API_KEY = form_data.web.search.tavily_api_key
app.state.config.SEARCHAPI_API_KEY = form_data.web.search.searchapi_api_key
app.state.config.SEARCHAPI_ENGINE = form_data.web.search.searchapi_engine
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = form_data.web.search.result_count
app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = (
form_data.web.search.concurrent_requests
@ -566,6 +552,8 @@ async def update_rag_config(form_data: ConfigUpdateForm, user=Depends(get_admin_
"serpstack_https": app.state.config.SERPSTACK_HTTPS,
"serper_api_key": app.state.config.SERPER_API_KEY,
"serply_api_key": app.state.config.SERPLY_API_KEY,
"serachapi_api_key": app.state.config.SEARCHAPI_API_KEY,
"searchapi_engine": app.state.config.SEARCHAPI_ENGINE,
"tavily_api_key": app.state.config.TAVILY_API_KEY,
"result_count": app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
"concurrent_requests": app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
@ -817,6 +805,7 @@ def search_web(engine: str, query: str) -> list[SearchResult]:
- SERPER_API_KEY
- SERPLY_API_KEY
- TAVILY_API_KEY
- SEARCHAPI_API_KEY + SEARCHAPI_ENGINE (by default `google`)
Args:
query (str): The query to search for
"""
@ -904,6 +893,17 @@ def search_web(engine: str, query: str) -> list[SearchResult]:
)
else:
raise Exception("No TAVILY_API_KEY found in environment variables")
elif engine == "searchapi":
if app.state.config.SEARCHAPI_API_KEY:
return search_searchapi(
app.state.config.SEARCHAPI_API_KEY,
app.state.config.SEARCHAPI_ENGINE,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
app.state.config.RAG_WEB_SEARCH_DOMAIN_FILTER_LIST,
)
else:
raise Exception("No SEARCHAPI_API_KEY found in environment variables")
elif engine == "jina":
return search_jina(query, app.state.config.RAG_WEB_SEARCH_RESULT_COUNT)
else:
@ -954,7 +954,6 @@ def store_web_search(form_data: SearchForm, user=Depends(get_verified_user)):
def store_data_in_vector_db(
data, collection_name, metadata: Optional[dict] = None, overwrite: bool = False
) -> bool:
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=app.state.config.CHUNK_SIZE,
chunk_overlap=app.state.config.CHUNK_OVERLAP,
@ -1315,7 +1314,6 @@ def store_text(
form_data: TextRAGForm,
user=Depends(get_verified_user),
):
collection_name = form_data.collection_name
if collection_name is None:
collection_name = calculate_sha256_string(form_data.content)

6
backend/apps/rag/search/brave.py → backend/open_webui/apps/rag/search/brave.py
View file

@ -1,9 +1,9 @@
import logging
from typing import Optional
import requests
from apps.rag.search.main import SearchResult, get_filtered_results
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

5
backend/apps/rag/search/duckduckgo.py → backend/open_webui/apps/rag/search/duckduckgo.py
View file

@ -1,8 +1,9 @@
import logging
from typing import Optional
from apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from duckduckgo_search import DDGS
from config import SRC_LOG_LEVELS
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

7
backend/apps/rag/search/google_pse.py → backend/open_webui/apps/rag/search/google_pse.py
View file

@ -1,10 +1,9 @@
import json
import logging
from typing import Optional
import requests
from apps.rag.search.main import SearchResult, get_filtered_results
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

8
backend/apps/rag/search/jina_search.py → backend/open_webui/apps/rag/search/jina_search.py
View file

@ -1,9 +1,9 @@
import logging
import requests
from yarl import URL
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult
from open_webui.env import SRC_LOG_LEVELS
from yarl import URL
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

4
backend/apps/rag/search/main.py → backend/open_webui/apps/rag/search/main.py
View file

@ -1,5 +1,6 @@
from typing import Optional
from urllib.parse import urlparse
from pydantic import BaseModel
@ -8,7 +9,8 @@ def get_filtered_results(results, filter_list):
return results
filtered_results = []
for result in results:
domain = urlparse(result["url"]).netloc
url = result.get("url") or result.get("link", "")
domain = urlparse(url).netloc
if any(domain.endswith(filtered_domain) for filtered_domain in filter_list):
filtered_results.append(result)
return filtered_results

48
backend/open_webui/apps/rag/search/searchapi.py Normal file
View file

@ -0,0 +1,48 @@
import logging
from typing import Optional
from urllib.parse import urlencode
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_searchapi(
api_key: str,
engine: str,
query: str,
count: int,
filter_list: Optional[list[str]] = None,
) -> list[SearchResult]:
"""Search using searchapi.io's API and return the results as a list of SearchResult objects.
Args:
api_key (str): A searchapi.io API key
query (str): The query to search for
"""
url = "https://www.searchapi.io/api/v1/search"
engine = engine or "google"
payload = {"engine": engine, "q": query, "api_key": api_key}
url = f"{url}?{urlencode(payload)}"
response = requests.request("GET", url)
json_response = response.json()
log.info(f"results from searchapi search: {json_response}")
results = sorted(
json_response.get("organic_results", []), key=lambda x: x.get("position", 0)
)
if filter_list:
results = get_filtered_results(results, filter_list)
return [
SearchResult(
link=result["link"], title=result["title"], snippet=result["snippet"]
)
for result in results[:count]
]

7
backend/apps/rag/search/searxng.py → backend/open_webui/apps/rag/search/searxng.py
View file

@ -1,10 +1,9 @@
import logging
import requests
from typing import Optional
from apps.rag.search.main import SearchResult, get_filtered_results
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

6
backend/apps/rag/search/serper.py → backend/open_webui/apps/rag/search/serper.py
View file

@ -1,10 +1,10 @@
import json
import logging
from typing import Optional
import requests
from apps.rag.search.main import SearchResult, get_filtered_results
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

7
backend/apps/rag/search/serply.py → backend/open_webui/apps/rag/search/serply.py
View file

@ -1,11 +1,10 @@
import json
import logging
from typing import Optional
import requests
from urllib.parse import urlencode
from apps.rag.search.main import SearchResult, get_filtered_results
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

7
backend/apps/rag/search/serpstack.py → backend/open_webui/apps/rag/search/serpstack.py
View file

@ -1,10 +1,9 @@
import json
import logging
from typing import Optional
import requests
from apps.rag.search.main import SearchResult, get_filtered_results
from config import SRC_LOG_LEVELS
import requests
from open_webui.apps.rag.search.main import SearchResult, get_filtered_results
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

5
backend/apps/rag/search/tavily.py → backend/open_webui/apps/rag/search/tavily.py
View file

@ -1,9 +1,8 @@
import logging
import requests
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS
from open_webui.apps.rag.search.main import SearchResult
from open_webui.env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])

0
backend/apps/rag/search/testdata/brave.json → backend/open_webui/apps/rag/search/testdata/brave.json vendored
View file

0
backend/apps/rag/search/testdata/google_pse.json → backend/open_webui/apps/rag/search/testdata/google_pse.json vendored
View file

357
backend/open_webui/apps/rag/search/testdata/searchapi.json vendored Normal file
View file

File diff suppressed because one or more lines are too long

0
backend/apps/rag/search/testdata/searxng.json → backend/open_webui/apps/rag/search/testdata/searxng.json vendored
View file

0
backend/apps/rag/search/testdata/serper.json → backend/open_webui/apps/rag/search/testdata/serper.json vendored
View file

0
backend/apps/rag/search/testdata/serply.json → backend/open_webui/apps/rag/search/testdata/serply.json vendored
View file

0
backend/apps/rag/search/testdata/serpstack.json → backend/open_webui/apps/rag/search/testdata/serpstack.json vendored
View file

33
backend/apps/rag/utils.py → backend/open_webui/apps/rag/utils.py
View file

@ -1,27 +1,19 @@
import os
import logging
import os
from typing import Optional, Union
import requests
from typing import Union
from apps.ollama.main import (
generate_ollama_embeddings,
from open_webui.apps.ollama.main import (
GenerateEmbeddingsForm,
generate_ollama_embeddings,
)
from open_webui.config import CHROMA_CLIENT
from open_webui.env import SRC_LOG_LEVELS
from huggingface_hub import snapshot_download
from langchain_core.documents import Document
from langchain.retrievers import ContextualCompressionRetriever, EnsembleRetriever
from langchain_community.retrievers import BM25Retriever
from langchain.retrievers import (
ContextualCompressionRetriever,
EnsembleRetriever,
)
from typing import Optional
from utils.misc import get_last_user_message, add_or_update_system_message
from config import SRC_LOG_LEVELS, CHROMA_CLIENT
from langchain_core.documents import Document
from open_webui.utils.misc import get_last_user_message
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
@ -401,8 +393,8 @@ def generate_openai_batch_embeddings(
from typing import Any
from langchain_core.retrievers import BaseRetriever
from langchain_core.callbacks import CallbackManagerForRetrieverRun
from langchain_core.retrievers import BaseRetriever
class ChromaRetriever(BaseRetriever):
@ -439,11 +431,10 @@ class ChromaRetriever(BaseRetriever):
import operator
from typing import Optional, Sequence
from langchain_core.documents import BaseDocumentCompressor, Document
from langchain_core.callbacks import Callbacks
from langchain_core.documents import BaseDocumentCompressor, Document
from langchain_core.pydantic_v1 import Extra

7
backend/apps/socket/main.py → backend/open_webui/apps/socket/main.py
View file

@ -1,9 +1,8 @@
import socketio
import asyncio
from apps.webui.models.users import Users
from utils.utils import decode_token
import socketio
from open_webui.apps.webui.models.users import Users
from open_webui.utils.utils import decode_token
sio = socketio.AsyncServer(cors_allowed_origins=[], async_mode="asgi")
app = socketio.ASGIApp(sio, socketio_path="/ws/socket.io")

25
backend/apps/webui/internal/db.py → backend/open_webui/apps/webui/internal/db.py
View file

@ -1,21 +1,16 @@
import os
import logging
import json
import logging
from contextlib import contextmanager
from typing import Any, Optional
from typing import Optional, Any
from typing_extensions import Self
from sqlalchemy import create_engine, types, Dialect
from sqlalchemy.sql.type_api import _T
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, scoped_session
from open_webui.apps.webui.internal.wrappers import register_connection
from open_webui.env import OPEN_WEBUI_DIR, DATABASE_URL, SRC_LOG_LEVELS
from peewee_migrate import Router
from apps.webui.internal.wrappers import register_connection
from env import SRC_LOG_LEVELS, BACKEND_DIR, DATABASE_URL
from sqlalchemy import Dialect, create_engine, types
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.sql.type_api import _T
from typing_extensions import Self
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["DB"])
@ -50,7 +45,7 @@ def handle_peewee_migration(DATABASE_URL):
try:
# Replace the postgresql:// with postgres:// to handle the peewee migration
db = register_connection(DATABASE_URL.replace("postgresql://", "postgres://"))
migrate_dir = BACKEND_DIR / "apps" / "webui" / "internal" / "migrations"
migrate_dir = OPEN_WEBUI_DIR / "apps" / "webui" / "internal" / "migrations"
router = Router(db, logger=log, migrate_dir=migrate_dir)
router.run()
db.close()

0
backend/apps/webui/internal/migrations/001_initial_schema.py → backend/open_webui/apps/webui/internal/migrations/001_initial_schema.py
View file

0
backend/apps/webui/internal/migrations/002_add_local_sharing.py → backend/open_webui/apps/webui/internal/migrations/002_add_local_sharing.py
View file

0
backend/apps/webui/internal/migrations/003_add_auth_api_key.py → backend/open_webui/apps/webui/internal/migrations/003_add_auth_api_key.py
View file

0
backend/apps/webui/internal/migrations/004_add_archived.py → backend/open_webui/apps/webui/internal/migrations/004_add_archived.py
View file

0
backend/apps/webui/internal/migrations/005_add_updated_at.py → backend/open_webui/apps/webui/internal/migrations/005_add_updated_at.py
View file

0
backend/apps/webui/internal/migrations/006_migrate_timestamps_and_charfields.py → backend/open_webui/apps/webui/internal/migrations/006_migrate_timestamps_and_charfields.py
View file

0
backend/apps/webui/internal/migrations/007_add_user_last_active_at.py → backend/open_webui/apps/webui/internal/migrations/007_add_user_last_active_at.py
View file

0
backend/apps/webui/internal/migrations/008_add_memory.py → backend/open_webui/apps/webui/internal/migrations/008_add_memory.py
View file

0
backend/apps/webui/internal/migrations/009_add_models.py → backend/open_webui/apps/webui/internal/migrations/009_add_models.py
View file

2
backend/apps/webui/internal/migrations/010_migrate_modelfiles_to_models.py → backend/open_webui/apps/webui/internal/migrations/010_migrate_modelfiles_to_models.py
View file

@ -30,7 +30,7 @@ import peewee as pw
from peewee_migrate import Migrator
import json
from utils.misc import parse_ollama_modelfile
from open_webui.utils.misc import parse_ollama_modelfile
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext

0
backend/apps/webui/internal/migrations/011_add_user_settings.py → backend/open_webui/apps/webui/internal/migrations/011_add_user_settings.py
View file

0
backend/apps/webui/internal/migrations/012_add_tools.py → backend/open_webui/apps/webui/internal/migrations/012_add_tools.py
View file

0
backend/apps/webui/internal/migrations/013_add_user_info.py → backend/open_webui/apps/webui/internal/migrations/013_add_user_info.py
View file

0
backend/apps/webui/internal/migrations/014_add_files.py → backend/open_webui/apps/webui/internal/migrations/014_add_files.py
View file

0
backend/apps/webui/internal/migrations/015_add_functions.py → backend/open_webui/apps/webui/internal/migrations/015_add_functions.py
View file

0
backend/apps/webui/internal/migrations/016_add_valves_and_is_active.py → backend/open_webui/apps/webui/internal/migrations/016_add_valves_and_is_active.py
View file

0
backend/apps/webui/internal/migrations/017_add_user_oauth_sub.py → backend/open_webui/apps/webui/internal/migrations/017_add_user_oauth_sub.py
View file

0
backend/apps/webui/internal/migrations/018_add_function_is_global.py → backend/open_webui/apps/webui/internal/migrations/018_add_function_is_global.py
View file

12
backend/apps/webui/internal/wrappers.py → backend/open_webui/apps/webui/internal/wrappers.py
View file

@ -1,13 +1,13 @@
from contextvars import ContextVar
from peewee import *
from peewee import PostgresqlDatabase, InterfaceError as PeeWeeInterfaceError
import logging
from contextvars import ContextVar
from open_webui.env import SRC_LOG_LEVELS
from peewee import *
from peewee import InterfaceError as PeeWeeInterfaceError
from peewee import PostgresqlDatabase
from playhouse.db_url import connect, parse
from playhouse.shortcuts import ReconnectMixin
from env import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["DB"])

111
backend/apps/webui/main.py → backend/open_webui/apps/webui/main.py
View file

@ -1,65 +1,62 @@
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from apps.webui.routers import (
auths,
users,
chats,
documents,
tools,
models,
prompts,
configs,
memories,
utils,
files,
functions,
)
from apps.webui.models.functions import Functions
from apps.webui.models.models import Models
from apps.webui.utils import load_function_module_by_id
from utils.misc import (
openai_chat_chunk_message_template,
openai_chat_completion_message_template,
apply_model_params_to_body_openai,
apply_model_system_prompt_to_body,
)
from utils.tools import get_tools
from config import (
SHOW_ADMIN_DETAILS,
ADMIN_EMAIL,
WEBUI_AUTH,
DEFAULT_MODELS,
DEFAULT_PROMPT_SUGGESTIONS,
DEFAULT_USER_ROLE,
ENABLE_SIGNUP,
ENABLE_LOGIN_FORM,
USER_PERMISSIONS,
WEBHOOK_URL,
WEBUI_AUTH_TRUSTED_EMAIL_HEADER,
WEBUI_AUTH_TRUSTED_NAME_HEADER,
JWT_EXPIRES_IN,
WEBUI_BANNERS,
ENABLE_COMMUNITY_SHARING,
ENABLE_MESSAGE_RATING,
AppConfig,
OAUTH_USERNAME_CLAIM,
OAUTH_PICTURE_CLAIM,
OAUTH_EMAIL_CLAIM,
CORS_ALLOW_ORIGIN,
)
from apps.socket.main import get_event_call, get_event_emitter
import inspect
import json
import logging
from typing import AsyncGenerator, Generator, Iterator
from typing import Iterator, Generator, AsyncGenerator
from open_webui.apps.socket.main import get_event_call, get_event_emitter
from open_webui.apps.webui.models.functions import Functions
from open_webui.apps.webui.models.models import Models
from open_webui.apps.webui.routers import (
auths,
chats,
configs,
documents,
files,
functions,
memories,
models,
prompts,
tools,
users,
utils,
)
from open_webui.apps.webui.utils import load_function_module_by_id
from open_webui.config import (
ADMIN_EMAIL,
CORS_ALLOW_ORIGIN,
DEFAULT_MODELS,
DEFAULT_PROMPT_SUGGESTIONS,
DEFAULT_USER_ROLE,
ENABLE_COMMUNITY_SHARING,
ENABLE_LOGIN_FORM,
ENABLE_MESSAGE_RATING,
ENABLE_SIGNUP,
JWT_EXPIRES_IN,
OAUTH_EMAIL_CLAIM,
OAUTH_PICTURE_CLAIM,
OAUTH_USERNAME_CLAIM,
SHOW_ADMIN_DETAILS,
USER_PERMISSIONS,
WEBHOOK_URL,
WEBUI_AUTH,
WEBUI_BANNERS,
AppConfig,
)
from open_webui.env import (
WEBUI_AUTH_TRUSTED_EMAIL_HEADER,
WEBUI_AUTH_TRUSTED_NAME_HEADER,
)
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from open_webui.utils.misc import (
apply_model_params_to_body_openai,
apply_model_system_prompt_to_body,
openai_chat_chunk_message_template,
openai_chat_completion_message_template,
)
from open_webui.utils.tools import get_tools
app = FastAPI()

22
backend/apps/webui/models/auths.py → backend/open_webui/apps/webui/models/auths.py
View file

@ -1,15 +1,13 @@
from pydantic import BaseModel
from typing import Optional
import uuid
import logging
from sqlalchemy import String, Column, Boolean, Text
import uuid
from typing import Optional
from utils.utils import verify_password
from apps.webui.models.users import UserModel, Users
from apps.webui.internal.db import Base, get_db
from env import SRC_LOG_LEVELS
from open_webui.apps.webui.internal.db import Base, get_db
from open_webui.apps.webui.models.users import UserModel, Users
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel
from sqlalchemy import Boolean, Column, String, Text
from open_webui.utils.utils import verify_password
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -92,7 +90,6 @@ class AddUserForm(SignupForm):
class AuthsTable:
def insert_new_auth(
self,
email: str,
@ -103,7 +100,6 @@ class AuthsTable:
oauth_sub: Optional[str] = None,
) -> Optional[UserModel]:
with get_db() as db:
log.info("insert_new_auth")
id = str(uuid.uuid4())
@ -130,7 +126,6 @@ class AuthsTable:
log.info(f"authenticate_user: {email}")
try:
with get_db() as db:
auth = db.query(Auth).filter_by(email=email, active=True).first()
if auth:
if verify_password(password, auth.password):
@ -189,7 +184,6 @@ class AuthsTable:
def delete_auth_by_id(self, id: str) -> bool:
try:
with get_db() as db:
# Delete User
result = Users.delete_user_by_id(id)

38
backend/apps/webui/models/chats.py → backend/open_webui/apps/webui/models/chats.py
View file

@ -1,14 +1,11 @@
from pydantic import BaseModel, ConfigDict
from typing import Union, Optional
import json
import uuid
import time
import uuid
from typing import Optional
from sqlalchemy import Column, String, BigInteger, Boolean, Text
from apps.webui.internal.db import Base, get_db
from open_webui.apps.webui.internal.db import Base, get_db
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Boolean, Column, String, Text
####################
# Chat DB Schema
@ -77,10 +74,8 @@ class ChatTitleIdResponse(BaseModel):
class ChatTable:
def insert_new_chat(self, user_id: str, form_data: ChatForm) -> Optional[ChatModel]:
with get_db() as db:
id = str(uuid.uuid4())
chat = ChatModel(
**{
@ -106,7 +101,6 @@ class ChatTable:
def update_chat_by_id(self, id: str, chat: dict) -> Optional[ChatModel]:
try:
with get_db() as db:
chat_obj = db.get(Chat, id)
chat_obj.chat = json.dumps(chat)
chat_obj.title = chat["title"] if "title" in chat else "New Chat"
@ -115,12 +109,11 @@ class ChatTable:
db.refresh(chat_obj)
return ChatModel.model_validate(chat_obj)
except Exception as e:
except Exception:
return None
def insert_shared_chat_by_chat_id(self, chat_id: str) -> Optional[ChatModel]:
with get_db() as db:
# Get the existing chat to share
chat = db.get(Chat, chat_id)
# Check if the chat is already shared
@ -154,7 +147,6 @@ class ChatTable:
def update_shared_chat_by_chat_id(self, chat_id: str) -> Optional[ChatModel]:
try:
with get_db() as db:
print("update_shared_chat_by_id")
chat = db.get(Chat, chat_id)
print(chat)
@ -170,7 +162,6 @@ class ChatTable:
def delete_shared_chat_by_chat_id(self, chat_id: str) -> bool:
try:
with get_db() as db:
db.query(Chat).filter_by(user_id=f"shared-{chat_id}").delete()
db.commit()
@ -183,7 +174,6 @@ class ChatTable:
) -> Optional[ChatModel]:
try:
with get_db() as db:
chat = db.get(Chat, id)
chat.share_id = share_id
db.commit()
@ -195,7 +185,6 @@ class ChatTable:
def toggle_chat_archive_by_id(self, id: str) -> Optional[ChatModel]:
try:
with get_db() as db:
chat = db.get(Chat, id)
chat.archived = not chat.archived
db.commit()
@ -217,7 +206,6 @@ class ChatTable:
self, user_id: str, skip: int = 0, limit: int = 50
) -> list[ChatModel]:
with get_db() as db:
all_chats = (
db.query(Chat)
.filter_by(user_id=user_id, archived=True)
@ -297,7 +285,6 @@ class ChatTable:
def get_chat_by_id(self, id: str) -> Optional[ChatModel]:
try:
with get_db() as db:
chat = db.get(Chat, id)
return ChatModel.model_validate(chat)
except Exception:
@ -306,20 +293,18 @@ class ChatTable:
def get_chat_by_share_id(self, id: str) -> Optional[ChatModel]:
try:
with get_db() as db:
chat = db.query(Chat).filter_by(share_id=id).first()
if chat:
return self.get_chat_by_id(id)
else:
return None
except Exception as e:
except Exception:
return None
def get_chat_by_id_and_user_id(self, id: str, user_id: str) -> Optional[ChatModel]:
try:
with get_db() as db:
chat = db.query(Chat).filter_by(id=id, user_id=user_id).first()
return ChatModel.model_validate(chat)
except Exception:
@ -327,7 +312,6 @@ class ChatTable:
def get_chats(self, skip: int = 0, limit: int = 50) -> list[ChatModel]:
with get_db() as db:
all_chats = (
db.query(Chat)
# .limit(limit).offset(skip)
@ -337,7 +321,6 @@ class ChatTable:
def get_chats_by_user_id(self, user_id: str) -> list[ChatModel]:
with get_db() as db:
all_chats = (
db.query(Chat)
.filter_by(user_id=user_id)
@ -347,7 +330,6 @@ class ChatTable:
def get_archived_chats_by_user_id(self, user_id: str) -> list[ChatModel]:
with get_db() as db:
all_chats = (
db.query(Chat)
.filter_by(user_id=user_id, archived=True)
@ -358,7 +340,6 @@ class ChatTable:
def delete_chat_by_id(self, id: str) -> bool:
try:
with get_db() as db:
db.query(Chat).filter_by(id=id).delete()
db.commit()
@ -369,7 +350,6 @@ class ChatTable:
def delete_chat_by_id_and_user_id(self, id: str, user_id: str) -> bool:
try:
with get_db() as db:
db.query(Chat).filter_by(id=id, user_id=user_id).delete()
db.commit()
@ -379,9 +359,7 @@ class ChatTable:
def delete_chats_by_user_id(self, user_id: str) -> bool:
try:
with get_db() as db:
self.delete_shared_chats_by_user_id(user_id)
db.query(Chat).filter_by(user_id=user_id).delete()
@ -393,9 +371,7 @@ class ChatTable:
def delete_shared_chats_by_user_id(self, user_id: str) -> bool:
try:
with get_db() as db:
chats_by_user = db.query(Chat).filter_by(user_id=user_id).all()
shared_chat_ids = [f"shared-{chat.id}" for chat in chats_by_user]

24
backend/apps/webui/models/documents.py → backend/open_webui/apps/webui/models/documents.py
View file

@ -1,15 +1,12 @@
from pydantic import BaseModel, ConfigDict
from typing import Optional
import time
import logging
from sqlalchemy import String, Column, BigInteger, Text
from apps.webui.internal.db import Base, get_db
import json
import logging
import time
from typing import Optional
from env import SRC_LOG_LEVELS
from open_webui.apps.webui.internal.db import Base, get_db
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -70,12 +67,10 @@ class DocumentForm(DocumentUpdateForm):
class DocumentsTable:
def insert_new_doc(
self, user_id: str, form_data: DocumentForm
) -> Optional[DocumentModel]:
with get_db() as db:
document = DocumentModel(
**{
**form_data.model_dump(),
@ -99,7 +94,6 @@ class DocumentsTable:
def get_doc_by_name(self, name: str) -> Optional[DocumentModel]:
try:
with get_db() as db:
document = db.query(Document).filter_by(name=name).first()
return DocumentModel.model_validate(document) if document else None
except Exception:
@ -107,7 +101,6 @@ class DocumentsTable:
def get_docs(self) -> list[DocumentModel]:
with get_db() as db:
return [
DocumentModel.model_validate(doc) for doc in db.query(Document).all()
]
@ -117,7 +110,6 @@ class DocumentsTable:
) -> Optional[DocumentModel]:
try:
with get_db() as db:
db.query(Document).filter_by(name=name).update(
{
"title": form_data.title,
@ -140,7 +132,6 @@ class DocumentsTable:
doc_content = {**doc_content, **updated}
with get_db() as db:
db.query(Document).filter_by(name=name).update(
{
"content": json.dumps(doc_content),
@ -156,7 +147,6 @@ class DocumentsTable:
def delete_doc_by_name(self, name: str) -> bool:
try:
with get_db() as db:
db.query(Document).filter_by(name=name).delete()
db.commit()
return True

24
backend/apps/webui/models/files.py → backend/open_webui/apps/webui/models/files.py
View file

@ -1,15 +1,11 @@
from pydantic import BaseModel, ConfigDict
from typing import Union, Optional
import time
import logging
import time
from typing import Optional
from sqlalchemy import Column, String, BigInteger, Text
from apps.webui.internal.db import JSONField, Base, get_db
import json
from env import SRC_LOG_LEVELS
from open_webui.apps.webui.internal.db import Base, JSONField, get_db
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -59,10 +55,8 @@ class FileForm(BaseModel):
class FilesTable:
def insert_new_file(self, user_id: str, form_data: FileForm) -> Optional[FileModel]:
with get_db() as db:
file = FileModel(
**{
**form_data.model_dump(),
@ -86,7 +80,6 @@ class FilesTable:
def get_file_by_id(self, id: str) -> Optional[FileModel]:
with get_db() as db:
try:
file = db.get(File, id)
return FileModel.model_validate(file)
@ -95,7 +88,6 @@ class FilesTable:
def get_files(self) -> list[FileModel]:
with get_db() as db:
return [FileModel.model_validate(file) for file in db.query(File).all()]
def get_files_by_user_id(self, user_id: str) -> list[FileModel]:
@ -106,9 +98,7 @@ class FilesTable:
]
def delete_file_by_id(self, id: str) -> bool:
with get_db() as db:
try:
db.query(File).filter_by(id=id).delete()
db.commit()
@ -118,9 +108,7 @@ class FilesTable:
return False
def delete_all_files(self) -> bool:
with get_db() as db:
try:
db.query(File).delete()
db.commit()

32
backend/apps/webui/models/functions.py → backend/open_webui/apps/webui/models/functions.py
View file

@ -1,18 +1,12 @@
from pydantic import BaseModel, ConfigDict
from typing import Union, Optional
import time
import logging
import time
from typing import Optional
from sqlalchemy import Column, String, Text, BigInteger, Boolean
from apps.webui.internal.db import JSONField, Base, get_db
from apps.webui.models.users import Users
import json
import copy
from env import SRC_LOG_LEVELS
from open_webui.apps.webui.internal.db import Base, JSONField, get_db
from open_webui.apps.webui.models.users import Users
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Boolean, Column, String, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -87,11 +81,9 @@ class FunctionValves(BaseModel):
class FunctionsTable:
def insert_new_function(
self, user_id: str, type: str, form_data: FunctionForm
) -> Optional[FunctionModel]:
function = FunctionModel(
**{
**form_data.model_dump(),
@ -119,7 +111,6 @@ class FunctionsTable:
def get_function_by_id(self, id: str) -> Optional[FunctionModel]:
try:
with get_db() as db:
function = db.get(Function, id)
return FunctionModel.model_validate(function)
except Exception:
@ -127,7 +118,6 @@ class FunctionsTable:
def get_functions(self, active_only=False) -> list[FunctionModel]:
with get_db() as db:
if active_only:
return [
FunctionModel.model_validate(function)
@ -143,7 +133,6 @@ class FunctionsTable:
self, type: str, active_only=False
) -> list[FunctionModel]:
with get_db() as db:
if active_only:
return [
FunctionModel.model_validate(function)
@ -159,7 +148,6 @@ class FunctionsTable:
def get_global_filter_functions(self) -> list[FunctionModel]:
with get_db() as db:
return [
FunctionModel.model_validate(function)
for function in db.query(Function)
@ -178,7 +166,6 @@ class FunctionsTable:
def get_function_valves_by_id(self, id: str) -> Optional[dict]:
with get_db() as db:
try:
function = db.get(Function, id)
return function.valves if function.valves else {}
@ -190,7 +177,6 @@ class FunctionsTable:
self, id: str, valves: dict
) -> Optional[FunctionValves]:
with get_db() as db:
try:
function = db.get(Function, id)
function.valves = valves
@ -204,7 +190,6 @@ class FunctionsTable:
def get_user_valves_by_id_and_user_id(
self, id: str, user_id: str
) -> Optional[dict]:
try:
user = Users.get_user_by_id(user_id)
user_settings = user.settings.model_dump() if user.settings else {}
@ -223,7 +208,6 @@ class FunctionsTable:
def update_user_valves_by_id_and_user_id(
self, id: str, user_id: str, valves: dict
) -> Optional[dict]:
try:
user = Users.get_user_by_id(user_id)
user_settings = user.settings.model_dump() if user.settings else {}
@ -246,7 +230,6 @@ class FunctionsTable:
def update_function_by_id(self, id: str, updated: dict) -> Optional[FunctionModel]:
with get_db() as db:
try:
db.query(Function).filter_by(id=id).update(
{
@ -261,7 +244,6 @@ class FunctionsTable:
def deactivate_all_functions(self) -> Optional[bool]:
with get_db() as db:
try:
db.query(Function).update(
{

21
backend/apps/webui/models/memories.py → backend/open_webui/apps/webui/models/memories.py
View file

@ -1,12 +1,10 @@
from pydantic import BaseModel, ConfigDict
from typing import Union, Optional
from sqlalchemy import Column, String, BigInteger, Text
from apps.webui.internal.db import Base, get_db
import time
import uuid
from typing import Optional
from open_webui.apps.webui.internal.db import Base, get_db
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
####################
# Memory DB Schema
@ -39,13 +37,11 @@ class MemoryModel(BaseModel):
class MemoriesTable:
def insert_new_memory(
self,
user_id: str,
content: str,
) -> Optional[MemoryModel]:
with get_db() as db:
id = str(uuid.uuid4())
@ -73,7 +69,6 @@ class MemoriesTable:
content: str,
) -> Optional[MemoryModel]:
with get_db() as db:
try:
db.query(Memory).filter_by(id=id).update(
{"content": content, "updated_at": int(time.time())}
@ -85,7 +80,6 @@ class MemoriesTable:
def get_memories(self) -> list[MemoryModel]:
with get_db() as db:
try:
memories = db.query(Memory).all()
return [MemoryModel.model_validate(memory) for memory in memories]
@ -94,7 +88,6 @@ class MemoriesTable:
def get_memories_by_user_id(self, user_id: str) -> list[MemoryModel]:
with get_db() as db:
try:
memories = db.query(Memory).filter_by(user_id=user_id).all()
return [MemoryModel.model_validate(memory) for memory in memories]
@ -103,7 +96,6 @@ class MemoriesTable:
def get_memory_by_id(self, id: str) -> Optional[MemoryModel]:
with get_db() as db:
try:
memory = db.get(Memory, id)
return MemoryModel.model_validate(memory)
@ -112,7 +104,6 @@ class MemoriesTable:
def delete_memory_by_id(self, id: str) -> bool:
with get_db() as db:
try:
db.query(Memory).filter_by(id=id).delete()
db.commit()
@ -124,7 +115,6 @@ class MemoriesTable:
def delete_memories_by_user_id(self, user_id: str) -> bool:
with get_db() as db:
try:
db.query(Memory).filter_by(user_id=user_id).delete()
db.commit()
@ -135,7 +125,6 @@ class MemoriesTable:
def delete_memory_by_id_and_user_id(self, id: str, user_id: str) -> bool:
with get_db() as db:
try:
db.query(Memory).filter_by(id=id, user_id=user_id).delete()
db.commit()

15
backend/apps/webui/models/models.py → backend/open_webui/apps/webui/models/models.py
View file

@ -1,14 +1,11 @@
import logging
from typing import Optional, List
from pydantic import BaseModel, ConfigDict
from sqlalchemy import Column, BigInteger, Text
from apps.webui.internal.db import Base, JSONField, get_db
from env import SRC_LOG_LEVELS
import time
from typing import Optional
from open_webui.apps.webui.internal.db import Base, JSONField, get_db
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])

19
backend/apps/webui/models/prompts.py → backend/open_webui/apps/webui/models/prompts.py
View file

@ -1,12 +1,9 @@
from pydantic import BaseModel, ConfigDict
from typing import Optional
import time
from typing import Optional
from sqlalchemy import String, Column, BigInteger, Text
from apps.webui.internal.db import Base, get_db
import json
from open_webui.apps.webui.internal.db import Base, get_db
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
####################
# Prompts DB Schema
@ -45,7 +42,6 @@ class PromptForm(BaseModel):
class PromptsTable:
def insert_new_prompt(
self, user_id: str, form_data: PromptForm
) -> Optional[PromptModel]:
@ -61,7 +57,6 @@ class PromptsTable:
try:
with get_db() as db:
result = Prompt(**prompt.dict())
db.add(result)
db.commit()
@ -70,13 +65,12 @@ class PromptsTable:
return PromptModel.model_validate(result)
else:
return None
except Exception as e:
except Exception:
return None
def get_prompt_by_command(self, command: str) -> Optional[PromptModel]:
try:
with get_db() as db:
prompt = db.query(Prompt).filter_by(command=command).first()
return PromptModel.model_validate(prompt)
except Exception:
@ -84,7 +78,6 @@ class PromptsTable:
def get_prompts(self) -> list[PromptModel]:
with get_db() as db:
return [
PromptModel.model_validate(prompt) for prompt in db.query(Prompt).all()
]
@ -94,7 +87,6 @@ class PromptsTable:
) -> Optional[PromptModel]:
try:
with get_db() as db:
prompt = db.query(Prompt).filter_by(command=command).first()
prompt.title = form_data.title
prompt.content = form_data.content
@ -107,7 +99,6 @@ class PromptsTable:
def delete_prompt_by_command(self, command: str) -> bool:
try:
with get_db() as db:
db.query(Prompt).filter_by(command=command).delete()
db.commit()

28
backend/apps/webui/models/tags.py → backend/open_webui/apps/webui/models/tags.py
View file

@ -1,16 +1,12 @@
from pydantic import BaseModel, ConfigDict
import logging
import time
import uuid
from typing import Optional
import json
import uuid
import time
import logging
from sqlalchemy import String, Column, BigInteger, Text
from apps.webui.internal.db import Base, get_db
from env import SRC_LOG_LEVELS
from open_webui.apps.webui.internal.db import Base, get_db
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -77,10 +73,8 @@ class ChatTagsResponse(BaseModel):
class TagTable:
def insert_new_tag(self, name: str, user_id: str) -> Optional[TagModel]:
with get_db() as db:
id = str(uuid.uuid4())
tag = TagModel(**{"id": id, "user_id": user_id, "name": name})
try:
@ -92,7 +86,7 @@ class TagTable:
return TagModel.model_validate(result)
else:
return None
except Exception as e:
except Exception:
return None
def get_tag_by_name_and_user_id(
@ -102,7 +96,7 @@ class TagTable:
with get_db() as db:
tag = db.query(Tag).filter_by(name=name, user_id=user_id).first()
return TagModel.model_validate(tag)
except Exception as e:
except Exception:
return None
def add_tag_to_chat(
@ -161,7 +155,6 @@ class TagTable:
self, chat_id: str, user_id: str
) -> list[TagModel]:
with get_db() as db:
tag_names = [
chat_id_tag.tag_name
for chat_id_tag in (
@ -186,7 +179,6 @@ class TagTable:
self, tag_name: str, user_id: str
) -> list[ChatIdTagModel]:
with get_db() as db:
return [
ChatIdTagModel.model_validate(chat_id_tag)
for chat_id_tag in (
@ -201,7 +193,6 @@ class TagTable:
self, tag_name: str, user_id: str
) -> int:
with get_db() as db:
return (
db.query(ChatIdTag)
.filter_by(tag_name=tag_name, user_id=user_id)
@ -236,7 +227,6 @@ class TagTable:
) -> bool:
try:
with get_db() as db:
res = (
db.query(ChatIdTag)
.filter_by(tag_name=tag_name, chat_id=chat_id, user_id=user_id)

25
backend/apps/webui/models/tools.py → backend/open_webui/apps/webui/models/tools.py
View file

@ -1,17 +1,12 @@
from pydantic import BaseModel, ConfigDict
from typing import Optional
import time
import logging
from sqlalchemy import String, Column, BigInteger, Text
import time
from typing import Optional
from apps.webui.internal.db import Base, JSONField, get_db
from apps.webui.models.users import Users
import json
import copy
from env import SRC_LOG_LEVELS
from open_webui.apps.webui.internal.db import Base, JSONField, get_db
from open_webui.apps.webui.models.users import Users
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -79,13 +74,10 @@ class ToolValves(BaseModel):
class ToolsTable:
def insert_new_tool(
self, user_id: str, form_data: ToolForm, specs: list[dict]
) -> Optional[ToolModel]:
with get_db() as db:
tool = ToolModel(
**{
**form_data.model_dump(),
@ -112,7 +104,6 @@ class ToolsTable:
def get_tool_by_id(self, id: str) -> Optional[ToolModel]:
try:
with get_db() as db:
tool = db.get(Tool, id)
return ToolModel.model_validate(tool)
except Exception:
@ -125,7 +116,6 @@ class ToolsTable:
def get_tool_valves_by_id(self, id: str) -> Optional[dict]:
try:
with get_db() as db:
tool = db.get(Tool, id)
return tool.valves if tool.valves else {}
except Exception as e:
@ -135,7 +125,6 @@ class ToolsTable:
def update_tool_valves_by_id(self, id: str, valves: dict) -> Optional[ToolValves]:
try:
with get_db() as db:
db.query(Tool).filter_by(id=id).update(
{"valves": valves, "updated_at": int(time.time())}
)

17
backend/apps/webui/models/users.py → backend/open_webui/apps/webui/models/users.py
View file

@ -1,11 +1,10 @@
from pydantic import BaseModel, ConfigDict
from typing import Optional
import time
from typing import Optional
from sqlalchemy import String, Column, BigInteger, Text
from apps.webui.internal.db import Base, JSONField, get_db
from apps.webui.models.chats import Chats
from open_webui.apps.webui.internal.db import Base, JSONField, get_db
from open_webui.apps.webui.models.chats import Chats
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, String, Text
####################
# User DB Schema
@ -113,7 +112,7 @@ class UsersTable:
with get_db() as db:
user = db.query(User).filter_by(id=id).first()
return UserModel.model_validate(user)
except Exception as e:
except Exception:
return None
def get_user_by_api_key(self, api_key: str) -> Optional[UserModel]:
@ -221,7 +220,7 @@ class UsersTable:
user = db.query(User).filter_by(id=id).first()
return UserModel.model_validate(user)
# return UserModel(**user.dict())
except Exception as e:
except Exception:
return None
def delete_user_by_id(self, id: str) -> bool:
@ -255,7 +254,7 @@ class UsersTable:
with get_db() as db:
user = db.query(User).filter_by(id=id).first()
return user.api_key
except Exception as e:
except Exception:
return None

57
backend/apps/webui/routers/auths.py → backend/open_webui/apps/webui/routers/auths.py
View file

@ -1,43 +1,36 @@
import logging
from fastapi import Request, UploadFile, File
from fastapi import Depends, HTTPException, status
from fastapi.responses import Response
from fastapi import APIRouter
from pydantic import BaseModel
import re
import uuid
import csv
from apps.webui.models.auths import (
SigninForm,
SignupForm,
from open_webui.apps.webui.models.auths import (
AddUserForm,
UpdateProfileForm,
UpdatePasswordForm,
UserResponse,
SigninResponse,
Auths,
ApiKey,
Auths,
SigninForm,
SigninResponse,
SignupForm,
UpdatePasswordForm,
UpdateProfileForm,
UserResponse,
)
from apps.webui.models.users import Users
from utils.utils import (
get_password_hash,
get_current_user,
get_admin_user,
create_token,
create_api_key,
)
from utils.misc import parse_duration, validate_email_format
from utils.webhook import post_webhook
from constants import ERROR_MESSAGES, WEBHOOK_MESSAGES
from config import (
WEBUI_AUTH,
from open_webui.apps.webui.models.users import Users
from open_webui.config import WEBUI_AUTH
from open_webui.constants import ERROR_MESSAGES, WEBHOOK_MESSAGES
from open_webui.env import (
WEBUI_AUTH_TRUSTED_EMAIL_HEADER,
WEBUI_AUTH_TRUSTED_NAME_HEADER,
)
from fastapi import APIRouter, Depends, HTTPException, Request, status
from fastapi.responses import Response
from pydantic import BaseModel
from open_webui.utils.misc import parse_duration, validate_email_format
from open_webui.utils.utils import (
create_api_key,
create_token,
get_admin_user,
get_current_user,
get_password_hash,
)
from open_webui.utils.webhook import post_webhook
router = APIRouter()
@ -273,7 +266,6 @@ async def signup(request: Request, response: Response, form_data: SignupForm):
@router.post("/add", response_model=SigninResponse)
async def add_user(form_data: AddUserForm, user=Depends(get_admin_user)):
if not validate_email_format(form_data.email.lower()):
raise HTTPException(
status.HTTP_400_BAD_REQUEST, detail=ERROR_MESSAGES.INVALID_EMAIL_FORMAT
@ -283,7 +275,6 @@ async def add_user(form_data: AddUserForm, user=Depends(get_admin_user)):
raise HTTPException(400, detail=ERROR_MESSAGES.EMAIL_TAKEN)
try:
print(form_data)
hashed = get_password_hash(form_data.password)
user = Auths.insert_new_auth(

39
backend/apps/webui/routers/chats.py → backend/open_webui/apps/webui/routers/chats.py
View file

@ -1,34 +1,25 @@
from fastapi import Depends, Request, HTTPException, status
from datetime import datetime, timedelta
from typing import Union, Optional
from utils.utils import get_verified_user, get_admin_user
from fastapi import APIRouter
from pydantic import BaseModel
import json
import logging
from typing import Optional
from apps.webui.models.users import Users
from apps.webui.models.chats import (
ChatModel,
ChatResponse,
ChatTitleForm,
from open_webui.apps.webui.models.chats import (
ChatForm,
ChatTitleIdResponse,
ChatResponse,
Chats,
ChatTitleIdResponse,
)
from apps.webui.models.tags import (
TagModel,
ChatIdTagModel,
from open_webui.apps.webui.models.tags import (
ChatIdTagForm,
ChatTagsResponse,
ChatIdTagModel,
TagModel,
Tags,
)
from constants import ERROR_MESSAGES
from config import SRC_LOG_LEVELS, ENABLE_ADMIN_EXPORT, ENABLE_ADMIN_CHAT_ACCESS
from open_webui.config import ENABLE_ADMIN_CHAT_ACCESS, ENABLE_ADMIN_EXPORT
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import BaseModel
from open_webui.utils.utils import get_admin_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -61,7 +52,6 @@ async def get_session_user_chat_list(
@router.delete("/", response_model=bool)
async def delete_all_user_chats(request: Request, user=Depends(get_verified_user)):
if (
user.role == "user"
and not request.app.state.config.USER_PERMISSIONS["chat"]["deletion"]
@ -220,7 +210,6 @@ class TagNameForm(BaseModel):
async def get_user_chat_list_by_tag_name(
form_data: TagNameForm, user=Depends(get_verified_user)
):
chat_ids = [
chat_id_tag.chat_id
for chat_id_tag in Tags.get_chat_ids_by_tag_name_and_user_id(
@ -299,7 +288,6 @@ async def update_chat_by_id(
@router.delete("/{id}", response_model=bool)
async def delete_chat_by_id(request: Request, id: str, user=Depends(get_verified_user)):
if user.role == "admin":
result = Chats.delete_chat_by_id(id)
return result
@ -323,7 +311,6 @@ async def delete_chat_by_id(request: Request, id: str, user=Depends(get_verified
async def clone_chat_by_id(id: str, user=Depends(get_verified_user)):
chat = Chats.get_chat_by_id_and_user_id(id, user.id)
if chat:
chat_body = json.loads(chat.chat)
updated_chat = {
**chat_body,

48
backend/apps/webui/routers/configs.py → backend/open_webui/apps/webui/routers/configs.py
View file

@ -1,29 +1,39 @@
from fastapi import Response, Request
from fastapi import Depends, FastAPI, HTTPException, status
from datetime import datetime, timedelta
from typing import Union
from fastapi import APIRouter
from open_webui.config import BannerModel
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel
import time
import uuid
from open_webui.utils.utils import get_admin_user, get_verified_user
from config import BannerModel
from apps.webui.models.users import Users
from utils.utils import (
get_password_hash,
get_verified_user,
get_admin_user,
create_token,
)
from utils.misc import get_gravatar_url, validate_email_format
from constants import ERROR_MESSAGES
from open_webui.config import get_config, save_config
router = APIRouter()
############################
# ImportConfig
############################
class ImportConfigForm(BaseModel):
config: dict
@router.post("/import", response_model=dict)
async def import_config(form_data: ImportConfigForm, user=Depends(get_admin_user)):
save_config(form_data.config)
return get_config()
############################
# ExportConfig
############################
@router.get("/export", response_model=dict)
async def export_config(user=Depends(get_admin_user)):
return get_config()
class SetDefaultModelsForm(BaseModel):
models: str

21
backend/apps/webui/routers/documents.py → backend/open_webui/apps/webui/routers/documents.py
View file

@ -1,21 +1,16 @@
from fastapi import Depends, FastAPI, HTTPException, status
from datetime import datetime, timedelta
from typing import Union, Optional
from fastapi import APIRouter
from pydantic import BaseModel
import json
from typing import Optional
from apps.webui.models.documents import (
Documents,
from open_webui.apps.webui.models.documents import (
DocumentForm,
DocumentUpdateForm,
DocumentModel,
DocumentResponse,
Documents,
DocumentUpdateForm,
)
from utils.utils import get_verified_user, get_admin_user
from constants import ERROR_MESSAGES
from open_webui.constants import ERROR_MESSAGES
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel
from open_webui.utils.utils import get_admin_user, get_verified_user
router = APIRouter()

47
backend/apps/webui/routers/files.py → backend/open_webui/apps/webui/routers/files.py
View file

@ -1,42 +1,17 @@
from fastapi import (
Depends,
FastAPI,
HTTPException,
status,
Request,
UploadFile,
File,
Form,
)
from datetime import datetime, timedelta
from typing import Union, Optional
from pathlib import Path
from fastapi import APIRouter
from fastapi.responses import StreamingResponse, JSONResponse, FileResponse
from pydantic import BaseModel
import json
from apps.webui.models.files import (
Files,
FileForm,
FileModel,
FileModelResponse,
)
from utils.utils import get_verified_user, get_admin_user
from constants import ERROR_MESSAGES
from importlib import util
import logging
import os
import shutil
import uuid
import os, shutil, logging, re
from config import SRC_LOG_LEVELS, UPLOAD_DIR
from pathlib import Path
from typing import Optional
from open_webui.apps.webui.models.files import FileForm, FileModel, Files
from open_webui.config import UPLOAD_DIR
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile, status
from fastapi.responses import FileResponse
from open_webui.utils.utils import get_admin_user, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])

30
backend/apps/webui/routers/functions.py → backend/open_webui/apps/webui/routers/functions.py
View file

@ -1,27 +1,18 @@
from fastapi import Depends, FastAPI, HTTPException, status, Request
from datetime import datetime, timedelta
from typing import Union, Optional
import os
from pathlib import Path
from typing import Optional
from fastapi import APIRouter
from pydantic import BaseModel
import json
from apps.webui.models.functions import (
Functions,
from open_webui.apps.webui.models.functions import (
FunctionForm,
FunctionModel,
FunctionResponse,
Functions,
)
from apps.webui.utils import load_function_module_by_id
from utils.utils import get_verified_user, get_admin_user
from constants import ERROR_MESSAGES
from importlib import util
import os
from pathlib import Path
from config import DATA_DIR, CACHE_DIR, FUNCTIONS_DIR
from open_webui.apps.webui.utils import load_function_module_by_id
from open_webui.config import CACHE_DIR, FUNCTIONS_DIR
from open_webui.constants import ERROR_MESSAGES
from fastapi import APIRouter, Depends, HTTPException, Request, status
from open_webui.utils.utils import get_admin_user, get_verified_user
router = APIRouter()
@ -304,7 +295,6 @@ async def update_function_valves_by_id(
):
function = Functions.get_function_by_id(id)
if function:
if id in request.app.state.FUNCTIONS:
function_module = request.app.state.FUNCTIONS[id]
else:

20
backend/apps/webui/routers/memories.py → backend/open_webui/apps/webui/routers/memories.py
View file

@ -1,18 +1,12 @@
from fastapi import Response, Request
from fastapi import Depends, FastAPI, HTTPException, status
from datetime import datetime, timedelta
from typing import Union, Optional
from fastapi import APIRouter
from pydantic import BaseModel
import logging
from typing import Optional
from apps.webui.models.memories import Memories, MemoryModel
from utils.utils import get_verified_user
from constants import ERROR_MESSAGES
from config import SRC_LOG_LEVELS, CHROMA_CLIENT
from open_webui.apps.webui.models.memories import Memories, MemoryModel
from open_webui.config import CHROMA_CLIENT
from open_webui.env import SRC_LOG_LEVELS
from fastapi import APIRouter, Depends, HTTPException, Request
from pydantic import BaseModel
from open_webui.utils.utils import get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])

21
backend/apps/webui/routers/models.py → backend/open_webui/apps/webui/routers/models.py
View file

@ -1,15 +1,14 @@
from fastapi import Depends, FastAPI, HTTPException, status, Request
from datetime import datetime, timedelta
from typing import Union, Optional
from typing import Optional
from fastapi import APIRouter
from pydantic import BaseModel
import json
from apps.webui.models.models import Models, ModelModel, ModelForm, ModelResponse
from utils.utils import get_verified_user, get_admin_user
from constants import ERROR_MESSAGES
from open_webui.apps.webui.models.models import (
ModelForm,
ModelModel,
ModelResponse,
Models,
)
from open_webui.constants import ERROR_MESSAGES
from fastapi import APIRouter, Depends, HTTPException, Request, status
from open_webui.utils.utils import get_admin_user, get_verified_user
router = APIRouter()

16
backend/apps/webui/routers/prompts.py → backend/open_webui/apps/webui/routers/prompts.py
View file

@ -1,15 +1,9 @@
from fastapi import Depends, FastAPI, HTTPException, status
from datetime import datetime, timedelta
from typing import Union, Optional
from typing import Optional
from fastapi import APIRouter
from pydantic import BaseModel
import json
from apps.webui.models.prompts import Prompts, PromptForm, PromptModel
from utils.utils import get_verified_user, get_admin_user
from constants import ERROR_MESSAGES
from open_webui.apps.webui.models.prompts import PromptForm, PromptModel, Prompts
from open_webui.constants import ERROR_MESSAGES
from fastapi import APIRouter, Depends, HTTPException, status
from open_webui.utils.utils import get_admin_user, get_verified_user
router = APIRouter()

22
backend/apps/webui/routers/tools.py → backend/open_webui/apps/webui/routers/tools.py
View file

@ -1,20 +1,14 @@
from fastapi import Depends, HTTPException, status, Request
from typing import Optional
from fastapi import APIRouter
from apps.webui.models.tools import Tools, ToolForm, ToolModel, ToolResponse
from apps.webui.utils import load_toolkit_module_by_id
from utils.utils import get_admin_user, get_verified_user
from utils.tools import get_tools_specs
from constants import ERROR_MESSAGES
import os
from pathlib import Path
from typing import Optional
from config import DATA_DIR, CACHE_DIR
from open_webui.apps.webui.models.tools import ToolForm, ToolModel, ToolResponse, Tools
from open_webui.apps.webui.utils import load_toolkit_module_by_id
from open_webui.config import CACHE_DIR, DATA_DIR
from open_webui.constants import ERROR_MESSAGES
from fastapi import APIRouter, Depends, HTTPException, Request, status
from open_webui.utils.tools import get_tools_specs
from open_webui.utils.utils import get_admin_user, get_verified_user
TOOLS_DIR = f"{DATA_DIR}/tools"
os.makedirs(TOOLS_DIR, exist_ok=True)

37
backend/apps/webui/routers/users.py → backend/open_webui/apps/webui/routers/users.py
View file

@ -1,33 +1,20 @@
from fastapi import Response, Request
from fastapi import Depends, FastAPI, HTTPException, status
from datetime import datetime, timedelta
from typing import Union, Optional
from fastapi import APIRouter
from pydantic import BaseModel
import time
import uuid
import logging
from typing import Optional
from apps.webui.models.users import (
from open_webui.apps.webui.models.auths import Auths
from open_webui.apps.webui.models.chats import Chats
from open_webui.apps.webui.models.users import (
UserModel,
UserUpdateForm,
UserRoleUpdateForm,
UserSettings,
Users,
UserSettings,
UserUpdateForm,
)
from apps.webui.models.auths import Auths
from apps.webui.models.chats import Chats
from utils.utils import (
get_verified_user,
get_password_hash,
get_current_user,
get_admin_user,
)
from constants import ERROR_MESSAGES
from config import SRC_LOG_LEVELS
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import SRC_LOG_LEVELS
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import BaseModel
from open_webui.utils.utils import get_admin_user, get_password_hash, get_verified_user
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
@ -69,7 +56,6 @@ async def update_user_permissions(
@router.post("/update/role", response_model=Optional[UserModel])
async def update_user_role(form_data: UserRoleUpdateForm, user=Depends(get_admin_user)):
if user.id != form_data.id and form_data.id != Users.get_first_user().id:
return Users.update_user_role_by_id(form_data.id, form_data.role)
@ -173,7 +159,6 @@ class UserResponse(BaseModel):
@router.get("/{user_id}", response_model=UserResponse)
async def get_user_by_id(user_id: str, user=Depends(get_verified_user)):
# Check if user_id is a shared chat
# If it is, get the user_id from the chat
if user_id.startswith("shared-"):

31
backend/apps/webui/routers/utils.py → backend/open_webui/apps/webui/routers/utils.py
View file

@ -1,23 +1,16 @@
from pathlib import Path
import site
from pathlib import Path
from fastapi import APIRouter, UploadFile, File, Response
from fastapi import Depends, HTTPException, status
from starlette.responses import StreamingResponse, FileResponse
from pydantic import BaseModel
from fpdf import FPDF
import markdown
import black
from utils.utils import get_admin_user
from utils.misc import calculate_sha256, get_gravatar_url
from config import OLLAMA_BASE_URLS, DATA_DIR, UPLOAD_DIR, ENABLE_ADMIN_EXPORT
from constants import ERROR_MESSAGES
import markdown
from open_webui.config import DATA_DIR, ENABLE_ADMIN_EXPORT
from open_webui.constants import ERROR_MESSAGES
from fastapi import APIRouter, Depends, HTTPException, Response, status
from fpdf import FPDF
from pydantic import BaseModel
from starlette.responses import FileResponse
from open_webui.utils.misc import get_gravatar_url
from open_webui.utils.utils import get_admin_user
router = APIRouter()
@ -115,7 +108,7 @@ async def download_chat_as_pdf(
return Response(
content=bytes(pdf_bytes),
media_type="application/pdf",
headers={"Content-Disposition": f"attachment;filename=chat.pdf"},
headers={"Content-Disposition": "attachment;filename=chat.pdf"},
)
@ -126,7 +119,7 @@ async def download_db(user=Depends(get_admin_user)):
status_code=status.HTTP_401_UNAUTHORIZED,
detail=ERROR_MESSAGES.ACCESS_PROHIBITED,
)
from apps.webui.internal.db import engine
from open_webui.apps.webui.internal.db import engine
if engine.name != "sqlite":
raise HTTPException(

11
backend/apps/webui/utils.py → backend/open_webui/apps/webui/utils.py
View file

@ -1,13 +1,12 @@
from importlib import util
import os
import re
import sys
import subprocess
import sys
from importlib import util
from apps.webui.models.tools import Tools
from apps.webui.models.functions import Functions
from config import TOOLS_DIR, FUNCTIONS_DIR
from open_webui.apps.webui.models.functions import Functions
from open_webui.apps.webui.models.tools import Tools
from open_webui.config import FUNCTIONS_DIR, TOOLS_DIR
def extract_frontmatter(file_path):

120
backend/config.py → backend/open_webui/config.py
View file

@ -1,58 +1,29 @@
from sqlalchemy import create_engine, Column, Integer, DateTime, JSON, func
from contextlib import contextmanager
import os
import sys
import json
import logging
import importlib.metadata
import pkgutil
from urllib.parse import urlparse
import os
import shutil
from datetime import datetime
from pathlib import Path
from typing import Generic, Optional, TypeVar
from urllib.parse import urlparse
import chromadb
from chromadb import Settings
from typing import TypeVar, Generic
from pydantic import BaseModel
from typing import Optional
from pathlib import Path
import json
import yaml
import requests
import shutil
from apps.webui.internal.db import Base, get_db
from constants import ERROR_MESSAGES
from env import (
ENV,
VERSION,
SAFE_MODE,
GLOBAL_LOG_LEVEL,
SRC_LOG_LEVELS,
BASE_DIR,
import yaml
from open_webui.apps.webui.internal.db import Base, get_db
from chromadb import Settings
from open_webui.env import (
OPEN_WEBUI_DIR,
DATA_DIR,
BACKEND_DIR,
ENV,
FRONTEND_BUILD_DIR,
WEBUI_NAME,
WEBUI_URL,
WEBUI_FAVICON_URL,
WEBUI_BUILD_HASH,
CONFIG_DATA,
DATABASE_URL,
CHANGELOG,
WEBUI_AUTH,
WEBUI_AUTH_TRUSTED_EMAIL_HEADER,
WEBUI_AUTH_TRUSTED_NAME_HEADER,
WEBUI_SECRET_KEY,
WEBUI_SESSION_COOKIE_SAME_SITE,
WEBUI_SESSION_COOKIE_SECURE,
WEBUI_FAVICON_URL,
WEBUI_NAME,
log,
)
from pydantic import BaseModel
from sqlalchemy import JSON, Column, DateTime, Integer, func
class EndpointFilter(logging.Filter):
@ -72,10 +43,15 @@ logging.getLogger("uvicorn.access").addFilter(EndpointFilter())
def run_migrations():
print("Running migrations")
try:
from alembic.config import Config
from alembic import command
from alembic.config import Config
alembic_cfg = Config(OPEN_WEBUI_DIR / "alembic.ini")
# Set the script location dynamically
migrations_path = OPEN_WEBUI_DIR / "migrations"
alembic_cfg.set_main_option("script_location", str(migrations_path))
alembic_cfg = Config("alembic.ini")
command.upgrade(alembic_cfg, "head")
except Exception as e:
print(f"Error: {e}")
@ -118,15 +94,6 @@ if os.path.exists(f"{DATA_DIR}/config.json"):
save_to_db(data)
os.rename(f"{DATA_DIR}/config.json", f"{DATA_DIR}/old_config.json")
def save_config():
try:
with open(f"{DATA_DIR}/config.json", "w") as f:
json.dump(CONFIG_DATA, f, indent="\t")
except Exception as e:
log.exception(e)
DEFAULT_CONFIG = {
"version": 0,
"ui": {
@ -200,6 +167,25 @@ def get_config_value(config_path: str):
return cur_config
PERSISTENT_CONFIG_REGISTRY = []
def save_config(config):
global CONFIG_DATA
global PERSISTENT_CONFIG_REGISTRY
try:
save_to_db(config)
CONFIG_DATA = config
# Trigger updates on all registered PersistentConfig entries
for config_item in PERSISTENT_CONFIG_REGISTRY:
config_item.update()
except Exception as e:
log.exception(e)
return False
return True
T = TypeVar("T")
@ -215,6 +201,8 @@ class PersistentConfig(Generic[T]):
else:
self.value = env_value
PERSISTENT_CONFIG_REGISTRY.append(self)
def __str__(self):
return str(self.value)
@ -231,6 +219,12 @@ class PersistentConfig(Generic[T]):
)
return super().__getattribute__(item)
def update(self):
new_value = get_config_value(self.config_path)
if new_value is not None:
self.value = new_value
log.info(f"Updated {self.env_name} to new value {self.value}")
def save(self):
log.info(f"Saving '{self.env_name}' to the database")
path_parts = self.config_path.split(".")
@ -441,7 +435,7 @@ load_oauth_providers()
# Static DIR
####################################
STATIC_DIR = Path(os.getenv("STATIC_DIR", BACKEND_DIR / "static")).resolve()
STATIC_DIR = Path(os.getenv("STATIC_DIR", OPEN_WEBUI_DIR / "static")).resolve()
frontend_favicon = FRONTEND_BUILD_DIR / "static" / "favicon.png"
@ -1237,6 +1231,18 @@ TAVILY_API_KEY = PersistentConfig(
os.getenv("TAVILY_API_KEY", ""),
)
SEARCHAPI_API_KEY = PersistentConfig(
"SEARCHAPI_API_KEY",
"rag.web.search.searchapi_api_key",
os.getenv("SEARCHAPI_API_KEY", ""),
)
SEARCHAPI_ENGINE = PersistentConfig(
"SEARCHAPI_ENGINE",
"rag.web.search.searchapi_engine",
os.getenv("SEARCHAPI_ENGINE", ""),
)
RAG_WEB_SEARCH_RESULT_COUNT = PersistentConfig(
"RAG_WEB_SEARCH_RESULT_COUNT",
"rag.web.search.result_count",

0
backend/constants.py → backend/open_webui/constants.py
View file

1
backend/open_webui/data/readme.txt Normal file
View file

@ -0,0 +1 @@
pip install dir for backend files (db, documents, etc.)

74
backend/env.py → backend/open_webui/env.py
View file

@ -1,32 +1,31 @@
from pathlib import Path
import os
import logging
import sys
import json
import importlib.metadata
import json
import logging
import os
import pkgutil
from urllib.parse import urlparse
from datetime import datetime
import sys
import shutil
from pathlib import Path
import markdown
from bs4 import BeautifulSoup
from constants import ERROR_MESSAGES
from open_webui.constants import ERROR_MESSAGES
####################################
# Load .env file
####################################
BACKEND_DIR = Path(__file__).parent # the path containing this file
OPEN_WEBUI_DIR = Path(__file__).parent # the path containing this file
print(OPEN_WEBUI_DIR)
BACKEND_DIR = OPEN_WEBUI_DIR.parent # the path containing this file
BASE_DIR = BACKEND_DIR.parent # the path containing the backend/
print(BACKEND_DIR)
print(BASE_DIR)
try:
from dotenv import load_dotenv, find_dotenv
from dotenv import find_dotenv, load_dotenv
load_dotenv(find_dotenv(str(BASE_DIR / ".env")))
except ImportError:
@ -89,14 +88,23 @@ WEBUI_FAVICON_URL = "https://openwebui.com/favicon.png"
ENV = os.environ.get("ENV", "dev")
PIP_INSTALL = False
try:
PACKAGE_DATA = json.loads((BASE_DIR / "package.json").read_text())
except Exception:
importlib.metadata.version("open-webui")
PIP_INSTALL = True
except importlib.metadata.PackageNotFoundError:
pass
if PIP_INSTALL:
PACKAGE_DATA = {"version": importlib.metadata.version("open-webui")}
else:
try:
PACKAGE_DATA = {"version": importlib.metadata.version("open-webui")}
except importlib.metadata.PackageNotFoundError:
PACKAGE_DATA = json.loads((BASE_DIR / "package.json").read_text())
except Exception:
PACKAGE_DATA = {"version": "0.0.0"}
VERSION = PACKAGE_DATA["version"]
@ -178,11 +186,35 @@ WEBUI_BUILD_HASH = os.environ.get("WEBUI_BUILD_HASH", "dev-build")
####################################
DATA_DIR = Path(os.getenv("DATA_DIR", BACKEND_DIR / "data")).resolve()
if PIP_INSTALL:
NEW_DATA_DIR = Path(os.getenv("DATA_DIR", OPEN_WEBUI_DIR / "data")).resolve()
NEW_DATA_DIR.mkdir(parents=True, exist_ok=True)
# Check if the data directory exists in the package directory
if DATA_DIR.exists():
log.info(f"Moving {DATA_DIR} to {NEW_DATA_DIR}")
for item in DATA_DIR.iterdir():
dest = NEW_DATA_DIR / item.name
if item.is_dir():
shutil.copytree(item, dest, dirs_exist_ok=True)
else:
shutil.copy2(item, dest)
DATA_DIR = OPEN_WEBUI_DIR / "data"
FRONTEND_BUILD_DIR = Path(os.getenv("FRONTEND_BUILD_DIR", BASE_DIR / "build")).resolve()
if PIP_INSTALL:
FRONTEND_BUILD_DIR = Path(
os.getenv("FRONTEND_BUILD_DIR", OPEN_WEBUI_DIR / "frontend")
).resolve()
RESET_CONFIG_ON_START = (
os.environ.get("RESET_CONFIG_ON_START", "False").lower() == "true"
)
if RESET_CONFIG_ON_START:
try:
os.remove(f"{DATA_DIR}/config.json")
@ -191,12 +223,6 @@ if RESET_CONFIG_ON_START:
except Exception:
pass
try:
CONFIG_DATA = json.loads((DATA_DIR / "config.json").read_text())
except Exception:
CONFIG_DATA = {}
####################################
# Database
####################################

300
backend/main.py → backend/open_webui/main.py
View file

@ -1,131 +1,138 @@
import base64
import inspect
import json
import logging
import mimetypes
import os
import shutil
import sys
import time
import uuid
from contextlib import asynccontextmanager
from authlib.integrations.starlette_client import OAuth
from authlib.oidc.core import UserInfo
import json
import time
import os
import sys
import logging
import aiohttp
import requests
import mimetypes
import shutil
import inspect
from typing import Optional
from fastapi import FastAPI, Request, Depends, status, UploadFile, File, Form
from fastapi.staticfiles import StaticFiles
from fastapi.responses import JSONResponse
from fastapi import HTTPException
import aiohttp
import requests
from open_webui.apps.audio.main import app as audio_app
from open_webui.apps.images.main import app as images_app
from open_webui.apps.ollama.main import app as ollama_app
from open_webui.apps.ollama.main import (
generate_openai_chat_completion as generate_ollama_chat_completion,
)
from open_webui.apps.ollama.main import get_all_models as get_ollama_models
from open_webui.apps.openai.main import app as openai_app
from open_webui.apps.openai.main import (
generate_chat_completion as generate_openai_chat_completion,
)
from open_webui.apps.openai.main import get_all_models as get_openai_models
from open_webui.apps.rag.main import app as rag_app
from open_webui.apps.rag.utils import get_rag_context, rag_template
from open_webui.apps.socket.main import app as socket_app
from open_webui.apps.socket.main import get_event_call, get_event_emitter
from open_webui.apps.webui.internal.db import Session
from open_webui.apps.webui.main import app as webui_app
from open_webui.apps.webui.main import (
generate_function_chat_completion,
get_pipe_models,
)
from open_webui.apps.webui.models.auths import Auths
from open_webui.apps.webui.models.functions import Functions
from open_webui.apps.webui.models.models import Models
from open_webui.apps.webui.models.users import UserModel, Users
from open_webui.apps.webui.utils import load_function_module_by_id
from authlib.integrations.starlette_client import OAuth
from authlib.oidc.core import UserInfo
from open_webui.config import (
CACHE_DIR,
CORS_ALLOW_ORIGIN,
DEFAULT_LOCALE,
ENABLE_ADMIN_CHAT_ACCESS,
ENABLE_ADMIN_EXPORT,
ENABLE_MODEL_FILTER,
ENABLE_OAUTH_SIGNUP,
ENABLE_OLLAMA_API,
ENABLE_OPENAI_API,
ENV,
FRONTEND_BUILD_DIR,
MODEL_FILTER_LIST,
OAUTH_MERGE_ACCOUNTS_BY_EMAIL,
OAUTH_PROVIDERS,
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
STATIC_DIR,
TASK_MODEL,
TASK_MODEL_EXTERNAL,
TITLE_GENERATION_PROMPT_TEMPLATE,
TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
WEBHOOK_URL,
WEBUI_AUTH,
WEBUI_NAME,
AppConfig,
run_migrations,
)
from open_webui.constants import ERROR_MESSAGES, TASKS, WEBHOOK_MESSAGES
from open_webui.env import (
CHANGELOG,
GLOBAL_LOG_LEVEL,
SAFE_MODE,
SRC_LOG_LEVELS,
VERSION,
WEBUI_BUILD_HASH,
WEBUI_SECRET_KEY,
WEBUI_SESSION_COOKIE_SAME_SITE,
WEBUI_SESSION_COOKIE_SECURE,
WEBUI_URL,
)
from fastapi import (
Depends,
FastAPI,
File,
Form,
HTTPException,
Request,
UploadFile,
status,
)
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel
from sqlalchemy import text
from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.middleware.sessions import SessionMiddleware
from starlette.responses import StreamingResponse, Response, RedirectResponse
from starlette.responses import RedirectResponse, Response, StreamingResponse
from apps.socket.main import app as socket_app, get_event_emitter, get_event_call
from apps.ollama.main import (
app as ollama_app,
get_all_models as get_ollama_models,
generate_openai_chat_completion as generate_ollama_chat_completion,
from open_webui.utils.misc import (
add_or_update_system_message,
get_last_user_message,
parse_duration,
prepend_to_first_user_message_content,
)
from apps.openai.main import (
app as openai_app,
get_all_models as get_openai_models,
generate_chat_completion as generate_openai_chat_completion,
from open_webui.utils.task import (
moa_response_generation_template,
search_query_generation_template,
title_generation_template,
tools_function_calling_generation_template,
)
from apps.audio.main import app as audio_app
from apps.images.main import app as images_app
from apps.rag.main import app as rag_app
from apps.webui.main import (
app as webui_app,
get_pipe_models,
generate_function_chat_completion,
)
from apps.webui.internal.db import Session
from pydantic import BaseModel
from apps.webui.models.auths import Auths
from apps.webui.models.models import Models
from apps.webui.models.functions import Functions
from apps.webui.models.users import Users, UserModel
from apps.webui.utils import load_function_module_by_id
from utils.utils import (
from open_webui.utils.tools import get_tools
from open_webui.utils.utils import (
create_token,
decode_token,
get_admin_user,
get_verified_user,
get_current_user,
get_http_authorization_cred,
get_password_hash,
create_token,
decode_token,
get_verified_user,
)
from utils.task import (
title_generation_template,
search_query_generation_template,
tools_function_calling_generation_template,
moa_response_generation_template,
)
from utils.tools import get_tools
from utils.misc import (
get_last_user_message,
add_or_update_system_message,
prepend_to_first_user_message_content,
parse_duration,
)
from apps.rag.utils import get_rag_context, rag_template
from config import (
run_migrations,
WEBUI_NAME,
WEBUI_URL,
WEBUI_AUTH,
ENV,
VERSION,
CHANGELOG,
FRONTEND_BUILD_DIR,
CACHE_DIR,
STATIC_DIR,
DEFAULT_LOCALE,
ENABLE_OPENAI_API,
ENABLE_OLLAMA_API,
ENABLE_MODEL_FILTER,
MODEL_FILTER_LIST,
GLOBAL_LOG_LEVEL,
SRC_LOG_LEVELS,
WEBHOOK_URL,
ENABLE_ADMIN_EXPORT,
WEBUI_BUILD_HASH,
TASK_MODEL,
TASK_MODEL_EXTERNAL,
TITLE_GENERATION_PROMPT_TEMPLATE,
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
SAFE_MODE,
OAUTH_PROVIDERS,
ENABLE_OAUTH_SIGNUP,
OAUTH_MERGE_ACCOUNTS_BY_EMAIL,
WEBUI_SECRET_KEY,
WEBUI_SESSION_COOKIE_SAME_SITE,
WEBUI_SESSION_COOKIE_SECURE,
ENABLE_ADMIN_CHAT_ACCESS,
AppConfig,
CORS_ALLOW_ORIGIN,
)
from constants import ERROR_MESSAGES, WEBHOOK_MESSAGES, TASKS
from utils.webhook import post_webhook
from open_webui.utils.webhook import post_webhook
if SAFE_MODE:
print("SAFE MODE ENABLED")
@ -628,7 +635,10 @@ class ChatCompletionMiddleware(BaseHTTPMiddleware):
async for data in original_generator:
yield data
return StreamingResponse(stream_wrapper(response.body_iterator, data_items))
return StreamingResponse(
stream_wrapper(response.body_iterator, data_items),
headers=dict(response.headers),
)
async def _receive(self, body: bytes):
return {"type": "http.request", "body": body, "more_body": False}
@ -729,10 +739,16 @@ class PipelineMiddleware(BaseHTTPMiddleware):
try:
data = filter_pipeline(data, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if len(e.args) > 1:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
else:
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content={"detail": str(e)},
)
modified_body_bytes = json.dumps(data).encode("utf-8")
# Replace the request body with the modified one
@ -1380,10 +1396,16 @@ async def generate_title(form_data: dict, user=Depends(get_verified_user)):
try:
payload = filter_pipeline(payload, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if len(e.args) > 1:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
else:
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content={"detail": str(e)},
)
if "chat_id" in payload:
del payload["chat_id"]
@ -1433,10 +1455,16 @@ async def generate_search_query(form_data: dict, user=Depends(get_verified_user)
try:
payload = filter_pipeline(payload, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if len(e.args) > 1:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
else:
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content={"detail": str(e)},
)
if "chat_id" in payload:
del payload["chat_id"]
@ -1490,10 +1518,16 @@ Message: """{{prompt}}"""
try:
payload = filter_pipeline(payload, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if len(e.args) > 1:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
else:
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content={"detail": str(e)},
)
if "chat_id" in payload:
del payload["chat_id"]
@ -1542,10 +1576,16 @@ Responses from models: {{responses}}"""
try:
payload = filter_pipeline(payload, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if len(e.args) > 1:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
else:
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content={"detail": str(e)},
)
if "chat_id" in payload:
del payload["chat_id"]

0
backend/migrations/README → backend/open_webui/migrations/README
View file

21
backend/migrations/env.py → backend/open_webui/migrations/env.py
View file

@ -1,24 +1,9 @@
import os
from logging.config import fileConfig
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from alembic import context
from apps.webui.models.auths import Auth
from apps.webui.models.chats import Chat
from apps.webui.models.documents import Document
from apps.webui.models.memories import Memory
from apps.webui.models.models import Model
from apps.webui.models.prompts import Prompt
from apps.webui.models.tags import Tag, ChatIdTag
from apps.webui.models.tools import Tool
from apps.webui.models.users import User
from apps.webui.models.files import File
from apps.webui.models.functions import Function
from env import DATABASE_URL
from open_webui.apps.webui.models.auths import Auth
from open_webui.env import DATABASE_URL
from sqlalchemy import engine_from_config, pool
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.

2
backend/migrations/script.py.mako → backend/open_webui/migrations/script.py.mako
View file

@ -9,7 +9,7 @@ from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
import apps.webui.internal.db
import open_webui.apps.webui.internal.db
${imports if imports else ""}
# revision identifiers, used by Alembic.

0
backend/migrations/util.py → backend/open_webui/migrations/util.py
View file

10
backend/migrations/versions/7e5b5dc7342b_init.py → backend/open_webui/migrations/versions/7e5b5dc7342b_init.py
View file

@ -1,17 +1,19 @@
"""init
Revision ID: 7e5b5dc7342b
Revises:
Revises:
Create Date: 2024-06-24 13:15:33.808998
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
import apps.webui.internal.db
from migrations.util import get_existing_tables
from alembic import op
import open_webui.apps.webui.internal.db
from open_webui.migrations.util import get_existing_tables
# revision identifiers, used by Alembic.
revision: str = "7e5b5dc7342b"

4
backend/migrations/versions/ca81bd47c050_add_config_table.py → backend/open_webui/migrations/versions/ca81bd47c050_add_config_table.py
View file

@ -8,10 +8,8 @@ Create Date: 2024-08-25 15:26:35.241684
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
import apps.webui.internal.db
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "ca81bd47c050"

0
backend/static/favicon.png → backend/open_webui/static/favicon.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

0
backend/static/fonts/NotoSans-Bold.ttf → backend/open_webui/static/fonts/NotoSans-Bold.ttf
View file

0
backend/static/fonts/NotoSans-Italic.ttf → backend/open_webui/static/fonts/NotoSans-Italic.ttf
View file

0
backend/static/fonts/NotoSans-Regular.ttf → backend/open_webui/static/fonts/NotoSans-Regular.ttf
View file

0
backend/static/fonts/NotoSansJP-Regular.ttf → backend/open_webui/static/fonts/NotoSansJP-Regular.ttf
View file

0
backend/static/fonts/NotoSansKR-Regular.ttf → backend/open_webui/static/fonts/NotoSansKR-Regular.ttf
View file

0
backend/static/fonts/NotoSansSC-Regular.ttf → backend/open_webui/static/fonts/NotoSansSC-Regular.ttf
View file

0
backend/static/logo.png → backend/open_webui/static/logo.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 6 KiB

After

Width:  |  Height:  |  Size: 6 KiB

0
backend/static/splash.png → backend/open_webui/static/splash.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 5.1 KiB

After

Width:  |  Height:  |  Size: 5.1 KiB

0
backend/static/user-import.csv → backend/open_webui/static/user-import.csv
View file

Some files were not shown because too many files have changed in this diff Show more