152 commits

Author	SHA1	Message	Date
Timothy Jaeryang Baek	6f635d8b7d	refac	2025-05-10 19:16:09 +04:00
Timothy Jaeryang Baek	be912f1529	refac	2025-05-10 18:29:04 +04:00
Timothy Jaeryang Baek	d5fd3b3600	feat: external reranker ... Co-Authored-By: Brendan Campbell <20541191+bcambs09@users.noreply.github.com>	2025-05-10 18:25:20 +04:00
Timothy Jaeryang Baek	34ec10a78c	refac: web search performance ... Co-Authored-By: Mabeck <64421281+mmabeck@users.noreply.github.com>	2025-05-10 17:54:41 +04:00
tth37	c95a65a4bd	fix: Duplicate web search urls	2025-05-09 20:06:35 +08:00
Timothy Jaeryang Baek	b50dcb1862	refac: remove duplicate urls	2025-05-07 22:25:18 +04:00
Athanasios Oikonomou	657162e96d	feat(ocr): add support for Docling OCR engine and language configuration ... This commit adds support for configuring the OCR engine and language(s) for Docling. Configuration can be set via the environment variables `DOCLING_OCR_ENGINE` and `DOCLING_OCR_LANG`, or through the UI. Fixes #13133	2025-05-03 00:32:06 +03:00
Tim Jaeryang Baek	e87f2669fa	Merge pull request #13191 from tth37/feat_firecrawl_search_engine ... feat: Add Firecrawl search engine	2025-04-29 08:38:28 -07:00
Tim Jaeryang Baek	7b863465a9	Merge pull request #13311 from stephen304/yacy-support ... feat: Yacy search support	2025-04-29 08:35:10 -07:00
Stephen Smith	240d91d38d	Add yacy config for user/pass, automatically add yacy json api path	2025-04-26 22:28:30 -04:00
Stephen Smith	0f73b96616	first pass at yacy support copied from searxng	2025-04-26 14:07:13 -04:00
tth37	92dbeb1939	feat: Add Firecrawl search engine	2025-04-24 14:57:28 +08:00
Timothy Jaeryang Baek	732d7aee70	enh: sentence transformers env vars ... Co-Authored-By: DrZoidberg09 <96449693+drzoidberg09@users.noreply.github.com>	2025-04-24 01:55:18 +09:00
Timothy Jaeryang Baek	09874ab83d	fix: FireCrawlLoader	2025-04-24 01:40:34 +09:00
Timothy Jaeryang Baek	43efff0fe6	refac	2025-04-22 23:22:50 +09:00
Tim Jaeryang Baek	87844a8042	Merge pull request #12822 from tth37/feat_external_search_loader ... feat: Support for Self-Hosted/External Web Search/Loader Engines	2025-04-18 23:51:27 -07:00
Youggls	9669cd3454	fix: use run_in_threadpool for search_web to prevent blocking ... Used fastapi's run_in_threadpool function to execute the search_web function, preventing the synchronous function from blocking the entire web search process.	2025-04-17 17:23:20 +08:00
tth37	85f8e91288	feat: Allow admin editing external search/loader settings	2025-04-14 18:19:26 +08:00
Timothy Jaeryang Baek	70718dda90	refac	2025-04-13 22:31:43 -07:00
tth37	839ba22c90	feat: Backend for Self-Hosted/External Web Search/Loader Engines	2025-04-14 01:49:05 +08:00
Timothy Jaeryang Baek	888b468576	fix	2025-04-12 23:00:34 -07:00
Timothy Jaeryang Baek	4dafbbccfc	fix: rag template display issue	2025-04-12 22:55:24 -07:00
tth37	8d53f1e770	fix: small bugs on updated web/rag settings	2025-04-13 12:55:50 +08:00
Timothy Jaeryang Baek	48a23ce3fe	refac: web/rag config	2025-04-12 16:33:36 -07:00
tth37	5eac5960ef	feat: Add frontend configuration for web loader	2025-04-12 17:13:30 +08:00
Youggls	3e2a6df1fb	feat: Add sougou web search API for backend, add config panel in for frontend.	2025-04-10 14:51:44 +08:00
Timothy Jaeryang Baek	914eb49767	chore: include accelerate dependency	2025-04-06 17:44:05 -07:00
Timothy Jaeryang Baek	cbe2056587	fix: audio file upload response issue	2025-04-06 17:31:50 -07:00
Timothy Jaeryang Baek	f243e523a6	refac	2025-04-06 15:52:38 -07:00
Timothy Jaeryang Baek	155dbd5a66	refac	2025-04-06 15:45:48 -07:00
Timothy Jaeryang Baek	9825d03602	Merge pull request #12507 from Ithanil/fix_web_result_collection_source_ids ... fix: fix web results all getting the same source id when using embedding and retrieval	2025-04-06 15:43:21 -07:00
Jan Kessler	a506a1a61e	only keep URLs as sources for which the content could actually be retrieved	2025-04-06 20:31:12 +02:00
Jan Kessler	4476060044	fix web results all getting the same source id when using embedding and retrieval	2025-04-06 15:51:05 +02:00
Marko Henning	3b2b6e183d	Added missing parameter for query_doc_with_hybrid_search.	2025-04-04 15:30:57 +02:00
Timothy Jaeryang Baek	94bf49440d	enh: unload hybrid model if set to False	2025-04-02 18:15:14 -07:00
Patrick Wachter	1ac6879268	Add Mistral OCR integration and configuration support	2025-04-01 14:24:33 +02:00
Timothy Jaeryang Baek	cafc5413f5	refac	2025-03-31 14:13:27 -07:00
Timothy Jaeryang Baek	d542881ee4	refac	2025-03-30 21:55:20 -07:00
Timothy Jaeryang Baek	433b5bddc1	Merge pull request #8594 from jayteaftw/main ... feat: Support for instruct/prefixing embeddings	2025-03-30 21:54:44 -07:00
Timothy Jaeryang Baek	4a79320253	chore: format	2025-03-27 01:40:28 -07:00
Timothy Jaeryang Baek	9d834a8e90	Merge branch 'dev' into k_reranker	2025-03-26 20:50:31 -07:00
Marko Henning	41a4cf7106	Added new k_reranker parameter	2025-03-06 10:47:57 +01:00
Fabio Polito	9aa407dbd2	feat: merge with main	2025-03-05 22:04:34 +00:00
Timothy Jaeryang Baek	efe8c4ca69	chore: format	2025-03-01 07:28:00 -08:00
Timothy Jaeryang Baek	d0ddb0637e	enh: web embed bypass embedding and retrieval support	2025-02-27 16:34:05 -08:00
Timothy Jaeryang Baek	1b56a8f3cb	Merge pull request #10864 from kurtdami/perplexity_integration ... feat: add perplexity integration to web search	2025-02-27 13:51:03 -08:00
kurtdami	b061775932	feat: add perplexity integration to web search	2025-02-27 00:30:48 -08:00
Timothy Jaeryang Baek	57010901e6	enh: bypass embedding and retrieval	2025-02-26 15:42:19 -08:00
Timothy Jaeryang Baek	78a8ef8e66	refac: audio file handling	2025-02-26 13:09:52 -08:00
Timothy Jaeryang Baek	3be5e3129b	Merge pull request #10752 from NovoNordisk-OpenSource/yvedeng/standardize-logging ... refactor: replace print statements with logging	2025-02-25 10:53:02 -08:00
Yifang Deng	0e5d5ecb81	refactor: replace print statements with logging for better error tracking	2025-02-25 15:53:55 +01:00
hurxxxx	4cc3102758	feat: onedrive file picker integration	2025-02-25 01:47:07 +09:00
Timothy Jaeryang Baek	b14e75dd6c	feat: added Trust Proxy Environment switch in Web Search admin settings tab. ... Co-Authored-By: harry zhou <67385896+harryzhou2000@users.noreply.github.com>	2025-02-21 13:40:11 -08:00
Timothy Jaeryang Baek	ab1b910d80	Merge pull request #10486 from Micca/feature/document_intelligence_support ... Feat: Adding Support for Azure AI Document Intelligence for Content Extraction (Revised)	2025-02-21 10:56:18 -08:00
Timothy Jaeryang Baek	81715f6553	enh: RAG full context mode	2025-02-18 21:14:58 -08:00
Rory	10e0c81de9	Merge remote-tracking branch 'upstream/dev' into playwright ... # Conflicts: # backend/open_webui/retrieval/web/utils.py # backend/open_webui/routers/retrieval.py	2025-02-17 21:53:39 -06:00
Timothy Jaeryang Baek	ba6cde8a87	fix: include_domain does NOT exist	2025-02-17 19:20:49 -08:00
Timothy Jaeryang Baek	ca0b7217d2	enh: full context web search	2025-02-17 18:14:26 -08:00
Rory	b1bab2ece8	Remove duplicate loader.alazy_load line from merge	2025-02-14 22:43:46 -06:00
Rory	4da220c513	Merge remote-tracking branch 'upstream/dev' into playwright ... # Conflicts: # backend/open_webui/config.py # backend/open_webui/main.py # backend/open_webui/retrieval/web/utils.py # backend/open_webui/routers/retrieval.py # backend/open_webui/utils/middleware.py # pyproject.toml	2025-02-14 20:48:22 -06:00
Guofeng Yi	b38acc8559	Merge branch 'dev' into feate-webloader-support-proxy	2025-02-15 09:50:02 +08:00
Fabio Polito	2419ef06a0	feat: docling support for document preprocessing	2025-02-14 12:08:03 +00:00
Yimi81	d3f71930f0	web loader support proxy	2025-02-14 07:15:09 +00:00
Yimi81	ceef600223	support async load for websearch	2025-02-14 07:05:10 +00:00
Timothy Jaeryang Baek	304aed0f13	chore: format	2025-02-13 22:54:45 -08:00
Timothy Jaeryang Baek	7b37cdcebb	Merge pull request #9980 from xring/web_search_serpapi ... feat: add web search via SerpApi	2025-02-13 22:51:14 -08:00
Timothy Jaeryang Baek	c9a8808b0d	refac	2025-02-13 21:45:29 -08:00
xring	27d395ba06	feat: add web search via SerpApi	2025-02-14 12:24:58 +08:00
Rory	40d4db97e6	Merge remote-tracking branch 'upstream/dev' into playwright	2025-02-12 22:32:44 -06:00
Timothy Jaeryang Baek	8906a2e260	Merge pull request #9803 from BochaAI/main ... add Bocha	2025-02-11 21:01:04 -08:00
luckyman-yan	31360fe991	add Bocha	2025-02-10 16:44:47 +08:00
Rory	2c711d8365	Merge remote-tracking branch 'upstream/dev' into playwright ... # Conflicts: # backend/requirements.txt	2025-02-09 23:52:21 -06:00
Timothy Jaeryang Baek	3c5ac4ace5	Merge pull request #9416 from abdalrohman/fix_filter_domains ... feat(ui): implement domain filter list for web search settings	2025-02-07 11:23:15 -08:00
Mazurek Michal	35f3824932	feat: Implement Document Intelligence as Content Extraction Engine	2025-02-07 13:44:47 +01:00
Rory	ec6fe9939b	Merge remote-tracking branch 'upstream/dev' into playwright	2025-02-05 17:47:58 -06:00
JT	40dea3fbe1	Merge branch 'dev' into main	2025-02-05 15:15:24 -08:00
M.Abdulrahman Alnaseer	68703951e8	feat(ui): implement domain filter list for web search settings	2025-02-05 19:14:40 +03:00
Timothy Jaeryang Baek	e41a2682f5	chore: format	2025-02-05 00:07:45 -08:00
Timothy Jaeryang Baek	f6f8c08cb0	Merge pull request #9068 from df-cgdm/main ... **feat** Add user related headers when calling an external embedding api	2025-02-05 00:05:44 -08:00
JT	81102f4be2	Merge branch 'open-webui:main' into main	2025-02-04 13:06:04 -08:00
jvinolus	7b8e5d4e7c	Fixed errors and added more support	2025-02-04 13:04:36 -08:00
M.Abdulrahman Alnaseer	2bb6b49f11	feat: implement Exa search engine integration	2025-02-04 21:13:16 +03:00
Rory	1b581b714f	Moving code out of playwright branch	2025-02-03 18:47:26 -06:00
Rory	77ae73e659	Adjust search event messages + translations	2025-01-30 23:18:11 -06:00
Didier FOURNOUT	6ca295ec59	Add user related headers when calling an external embedding api	2025-01-29 10:55:52 +00:00
Rory	4e8b390682	Add RAG_WEB_LOADER + Playwright mode + improve stability of search	2025-01-28 23:03:15 -06:00
jvinolus	47b8412695	Initialize support for prefixing embeddings	2025-01-15 17:05:04 -08:00
Timothy Jaeryang Baek	a0e63b08c3	fix: searchapi engine not showing	2025-01-06 10:10:25 -08:00
Gabriel Ecegi	ba2964cb01	fix: missing parameter	2024-12-30 17:36:34 +01:00
Taylor Wilsdon	909c94e983	Fix Google Drive integration	2024-12-26 14:41:58 -05:00
Timothy Jaeryang Baek	95da0734b6	refac	2024-12-24 17:56:46 -07:00
Timothy Jaeryang Baek	6b25139d4f	refac: web search	2024-12-24 17:52:57 -07:00
Timothy Jaeryang Baek	ef5a5be60d	fix	2024-12-19 20:16:24 -08:00
Timothy Jaeryang Baek	0f6d302760	refac	2024-12-18 18:04:56 -08:00
Taylor Wilsdon	0dc75363aa	Add configurable Google Drive toggle in the Documents admin section along with necessary config scaffolding	2024-12-18 13:25:57 -05:00
Timothy Jaeryang Baek	e500461dc0	refac	2024-12-17 18:40:50 -08:00
Gabriel Ecegi	46f2f0fbdb	Merge branch 'dev' of https://github.com/open-webui/open-webui into dev	2024-12-16 07:34:06 +01:00
Timothy Jaeryang Baek	4311bb7b99	wip	2024-12-12 20:22:17 -08:00
Timothy Jaeryang Baek	ccdf51588e	wip	2024-12-11 18:46:29 -08:00
Timothy Jaeryang Baek	b3987ad41e	wip	2024-12-11 18:08:55 -08:00