Commit graph

474 commits

Author SHA1 Message Date
Timothy Jaeryang Baek
7e6b8a9a71 refac/fix: docling auth 2025-12-06 08:06:42 -05:00
Timothy Jaeryang Baek
9a65ed2260 chore: format 2025-12-02 16:06:57 -05:00
Classic298
b29fdc2a0c
Update milvus_multitenancy.py (#19695) 2025-12-02 15:38:06 -05:00
Classic298
12f237ff80
fix: Update milvus.py (#19602)
* Update milvus.py

* Update milvus.py

* Update milvus.py

* Update milvus.py

* Update milvus.py

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
2025-12-02 15:30:31 -05:00
Henne
a7e614ca4c
feat: Adds document intelligence model configuration (#19692)
* Adds document intelligence model configuration

Enables the configuration of the Document Intelligence model to be used by the RAG pipeline.

This allows users to specify the model they want to use for document processing, providing flexibility and control over the extraction process.

* Added Titel to Document Intelligence Model Config

Added Titel to Document Intelligence Model Config
2025-12-02 14:41:09 -05:00
Timothy Jaeryang Baek
d19023288e feat/enh: kb files db migration 2025-12-02 10:53:32 -05:00
Poccia
6e531679f4
fix/adjust web search to properly block domains (#19670)
Co-authored-by: Tim Baek <tim@openwebui.com>
2025-12-02 04:17:32 -05:00
Classic298
0a14196afb
Update milvus_multitenancy.py (#19680) 2025-12-02 03:57:14 -05:00
Timothy Jaeryang Baek
743199f2d0 feat/enh: tool server function name filter list 2025-11-25 02:31:34 -05:00
Timothy Jaeryang Baek
2328dc284e feat/enh: async embedding processing setting
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-25 01:55:43 -05:00
Timothy Jaeryang Baek
9c19d0abd4 refac/breaking: docling params
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda126-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-slim-images (push) Blocked by required conditions
Python CI / Format Backend (push) Waiting to run
Frontend Build / Format & Build Frontend (push) Waiting to run
Frontend Build / Frontend Unit Tests (push) Waiting to run
2025-11-24 16:01:13 -05:00
Timothy Jaeryang Baek
06f0bfd9f5 fix 2025-11-24 05:58:22 -05:00
Timothy Jaeryang Baek
662a1fac47 fix: hybrid search 2025-11-24 05:52:18 -05:00
Timothy Jaeryang Baek
48d1e67e79 chore: format 2025-11-23 20:15:52 -05:00
Classic298
99f0fe7f32
chore: user header forward minimize code changes throughout codebase (#19392)
* Update external.py

* remove unused imports

* Update ollama.py

* Update ollama.py

* Update ollama.py

* Update openai.py
2025-11-23 16:48:03 -05:00
Classic298
60dbde7e19
chore (#19389) 2025-11-23 04:40:05 -05:00
Timothy Jaeryang Baek
f9c96d03ad refac 2025-11-22 22:57:27 -05:00
Timothy Jaeryang Baek
9bfc414d26 refac
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda126-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-slim-images (push) Blocked by required conditions
Python CI / Format Backend (push) Waiting to run
Frontend Build / Format & Build Frontend (push) Waiting to run
Frontend Build / Frontend Unit Tests (push) Waiting to run
2025-11-22 21:33:14 -05:00
Classic298
902c6cfbea
perf: 50x performance improvement for external embeddings (#19296)
* Update utils.py (#77)

Co-authored-by: Claude <noreply@anthropic.com>

* refactor: address code review feedback for embedding performance improvements (#92)

Co-authored-by: Claude <noreply@anthropic.com>

* fix: prevent sentence transformers from blocking async event loop (#95)

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-22 20:54:59 -05:00
Diwakar
b8728064d8
feat: add support for Weaviate vector database (#14747) 2025-11-20 19:23:46 -05:00
Timothy Jaeryang Baek
a1d09eae95 chore: format 2025-11-19 03:23:33 -05:00
Seth Argyle
720af637e6
fix: Use get_index() instead of list_indexes() in has_collection() to… (#19238)
* fix: Use get_index() instead of list_indexes() in has_collection() to handle pagination

Fixes #19233

  Replace list_indexes() pagination scan with direct get_index() lookup
  in has_collection() method. The previous implementation only checked
  the first ~1,000 indexes due to unhandled pagination, causing RAG
  queries to fail for indexes beyond the first page.

  Benefits:
  - Handles buckets with any number of indexes (no pagination needed)
  - ~8x faster (0.19s vs 1.53s in testing)
  - Proper exception handling for ResourceNotFoundException
  - Scales to millions of indexes

* Update s3vector.py

Unneeded exception handling removed to match original OWUI code
2025-11-19 00:19:10 -05:00
Jacob Leksan
07ef295a77
feat: Adding file metadata to hybrid search (#19095)
* Added metadata to hybrid search

* And config and env plus refac

* consistency

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
2025-11-18 15:29:07 -05:00
Timothy Jaeryang Baek
2af4c4b3c7 refac
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-slim-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda126-images (push) Blocked by required conditions
Python CI / Format Backend (push) Waiting to run
Frontend Build / Format & Build Frontend (push) Waiting to run
Frontend Build / Frontend Unit Tests (push) Waiting to run
2025-11-18 04:42:09 -05:00
Timothy Jaeryang Baek
02238d3113 feat/security: Add SSRF protection with configurable blocklist
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-18 04:40:55 -05:00
lazariv
6cdb13d5cb
feat: pgvector hnsw index type (#19158)
* Adding hnsw index type for pgvector, allowing vector dimensions larger than 2000

* remove some variable assignments

* Make USE_HALFVEC variable configurable

* Simplify USE_HALFVEC handling

* Raise runtime error if the index requires rebuilt

---------

Co-authored-by: Moritz <moritz.mueller2@tu-dresden.de>
2025-11-18 04:14:43 -05:00
Timothy Jaeryang Baek
ee10f372a0 refac/enh: web search domain allow/block filter 2025-11-16 13:52:09 -05:00
Sang Lê
64747f7f79
Add Azure Search (#19104)
Co-authored-by: Tim Baek <tim@openwebui.com>
2025-11-13 19:12:34 -05:00
Classic298
ad17d35ac4
feat: Add custom API endpoint and user info headers for Perplexity Search (#31) (#19147)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-12 22:53:54 -05:00
Timothy Jaeryang Baek
4673e120c4 refac/fix: mineru params
breaking change
2025-11-11 00:30:11 -05:00
Timothy Jaeryang Baek
bc739de024 refac: rerank 2025-11-09 21:33:50 -05:00
Tim Baek
284764e178
Merge pull request #19025 from krishna-medapati/fix-hybrid-search-17046-clean
fix: Handle AttributeError in hybrid search with reranking (#17046)
2025-11-09 20:43:52 -05:00
Timothy Jaeryang Baek
25c7f101f2 enh: optionally add user headers external websearch
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-09 16:09:29 -05:00
krishna-medapati
684324ae9e fix: Handle AttributeError in hybrid search with reranking (#17046)
- Split attribute existence checks from document content checks
- Added hasattr() check for metadatas attribute
- Prevents AttributeError when collection_result is missing attributes
- Maintains all original validation logic

Fixes #17046
2025-11-07 23:31:11 +05:30
Timothy Jaeryang Baek
7faf19dad9 refac 2025-11-06 15:21:06 -05:00
Timothy Jaeryang Baek
c38f878e1e fix: firecrawl import 2025-11-06 15:19:08 -05:00
Timothy Jaeryang Baek
e2b9942648 feat: Optionally forward user headers to external document loader
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-06 00:05:46 -05:00
Timothy Jaeryang Baek
415b93c7c3 enh: configurable mistral ocr base url 2025-11-05 23:25:51 -05:00
Timothy Jaeryang Baek
a4fd26b478 enh/fix: google pse referer header 2025-11-04 13:50:07 -05:00
Timothy Jaeryang Baek
d14329b285 refac 2025-11-02 18:47:41 -05:00
Timothy Jaeryang Baek
d98c539d89 refac 2025-11-01 18:48:11 -04:00
Tim Baek
c787070dc9
Merge pull request #18765 from mkhludnev/patch-2
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-slim-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / merge-cuda126-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda126-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/amd64, ubuntu-latest) (push) Waiting to run
Create and publish Docker images with specific build args / build-slim-image (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Python CI / Format Backend (push) Waiting to run
fix: Don't missguide Tika with mime-type
2025-10-31 14:18:43 -07:00
Mikhail Khludnev
24aeec9120
Don't missguide Tika with mime-type
Fix #18683 
* Tika is smart enough to detect content type. 
* Windows browsers just misguides Tika providing `application/ms-word` for .rtf files.
2025-10-31 23:12:23 +03:00
Omar Aburub
3bcf9a442a fix: make SSL verification async 2025-10-29 16:14:53 +03:00
Wang Weixuan
5e17882488
fix: use trusted env in web search loader
Signed-off-by: Wang Weixuan <wangweixvan@gmail.com>
2025-10-28 04:58:00 +08:00
Timothy Jaeryang Baek
a70bc52c34 chore: format 2025-10-26 19:33:39 -07:00
Timothy Jaeryang Baek
8197844ff7 refac 2025-10-26 17:22:23 -07:00
wei840222
7a3f4d85f6 refactor: replace requests with Firecrawl SDK in search and requests Firecrawl SDK in scrape rather than langchain_community FireCrawlLoader 2025-10-26 15:05:35 +08:00
Ivan Ostanin
7d29991fa5
fix: pass youtube_proxy as a GenericProxyConfig type object 2025-10-19 02:01:27 +02:00
Tim Jaeryang Baek
e8c1dbb2da
Merge pull request #18306 from palazski/main
feat: add mineru as document parser backend with support of both local and managed api
2025-10-16 10:49:29 -05:00