Commit Graph

34 Commits

Author SHA1 Message Date
ArindamRoy23
49dd5f936e Merge branch 'HKUDS:main' into main 2025-03-11 20:53:00 +05:30
Roy
8aa9d0e6ca Add optional ids filter to vector database query methods
- Updated query method signatures across multiple vector database implementations
- Added optional `ids` parameter to filter search results
- Consistent implementation across ChromaDB, Faiss, Milvus, MongoDB, NanoVectorDB, Oracle, Qdrant, and TiDB vector storage classes
2025-03-11 15:22:17 +00:00
zrguo
c26cb3a9ea fix merge bugs 2025-03-11 16:05:04 +08:00
zrguo
e822f35c89 Fix edit entity and relation bugs 2025-03-07 14:39:06 +08:00
yangdx
e8d0d065f3 fix: Improve async handling and FAISS storage reliability
- Add async context manager support
- Fix embedding data type conversion
- Improve error handling in FAISS ops
- Add multiprocess storage sync
2025-03-01 23:35:09 +08:00
yangdx
9aef112d51 Fix incorrect comment about update flag behavior in FAISS implementation 2025-03-01 22:27:12 +08:00
yangdx
e3a40c2fdb Fix linting 2025-03-01 16:23:34 +08:00
yangdx
35bcfca28f feat: add multi-process support for FAISS vector storage
• Add storage update flag and locks
• Support cross-process index reload
• Add async initialize method
2025-03-01 12:42:30 +08:00
yangdx
cd7648791a Fix linting 2025-02-28 01:25:59 +08:00
yangdx
291e0c1b14 revert vector and graph use local data(single process) 2025-02-28 01:14:25 +08:00
yangdx
438e4780a8 Refactor Faiss index access with helper method to improve code organization 2025-02-27 15:09:19 +08:00
yangdx
f007ebf006 Refactor initialization logic for vector, KV and graph storage implementations
• Add try_initialize_namespace check
• Move init code out of storage locks
• Reduce redundant init conditions
• Simplify initialization flow
• Make init thread-safer
2025-02-27 14:55:07 +08:00
yangdx
7436c06f6c Fix linting 2025-02-26 18:11:16 +08:00
yangdx
2c019dbc7b Refactor storage initialization to avoid redundant intitial data loads across processes, show init logs to first load only 2025-02-26 12:28:49 +08:00
yangdx
2752a764ae Refactor storage implementations to support both single and multi-process modes
• Add shared storage management module
• Support process/thread lock based on mode
2025-02-26 05:38:38 +08:00
yangdx
a642bb3190 refactor: use shared manager from main process for storage implementations. 2025-02-25 12:08:49 +08:00
yangdx
e22e014f22 feat(storage): Add shared memory support for FAISS 2025-02-25 11:25:06 +08:00
Yannick Stephan
48a1ad9b3b Merge pull request #883 from YanSte/fix-return-none
Optimised returns
2025-02-19 22:24:50 +01:00
Yannick Stephan
9277fe8c29 fixed return 2025-02-19 22:22:41 +01:00
Saifeddine ALOUI
8a0dbc97eb Update faiss_impl.py 2025-02-19 19:44:27 +01:00
Yannick Stephan
2524e02428 remove tqdm and cleaned readme and ollama 2025-02-18 19:58:03 +01:00
Yannick Stephan
2b2c81a722 added some comments 2025-02-16 16:04:07 +01:00
Yannick Stephan
a1607bbcb9 Merge remote-tracking branch 'origin/main' into make-clear-what-implemented-or-not
# Conflicts:
#	lightrag/base.py
#	lightrag/kg/json_doc_status_impl.py
#	lightrag/kg/mongo_impl.py
#	lightrag/kg/postgres_impl.py
2025-02-16 15:29:16 +01:00
Yannick Stephan
0e7aff96bb back to not making breaks 2025-02-16 15:08:50 +01:00
Yannick Stephan
a0844bca28 cleaned import 2025-02-16 14:45:45 +01:00
Yannick Stephan
3fef8201c6 added final, required methods and cleaned import 2025-02-16 14:38:09 +01:00
zrguo
2a0c7c0322 Merge pull request #785 from danielaskdd/improve-CORS-handling
improve CORS and streaming response headers
2025-02-16 20:31:33 +08:00
Yannick Stephan
3eba41aab6 updated clean of what implemented on BaseVectorStorage 2025-02-16 13:24:42 +01:00
Yannick Stephan
805da7b95b cleaned code 2025-02-15 00:02:24 +01:00
yangdx
2c56141bfd Standardize variable names with other vector database implementations (without functional modifications) 2025-02-14 12:34:26 +08:00
yangdx
ed73ea4076 Fix linting 2025-02-13 04:12:00 +08:00
yangdx
f01f57d0da refactor: make cosine similarity threshold a required config parameter
• Remove default threshold from env var
• Add validation for missing threshold
• Move default to lightrag.py config init
• Update all vector DB implementations
• Improve threshold validation consistency
2025-02-13 03:25:48 +08:00
Gurjot Singh
2894e8faf2 Fix linting errors 2025-01-31 19:05:47 +05:30
Gurjot Singh
8a624e198a Add faiss integration for storage 2025-01-31 19:00:36 +05:30