gardel/lightrag
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,405 Commits 2 Branches 0 Tags
49dd5f936e08508833ee34850b564e75cc4e5f1e
Commit Graph

47 Commits

Author SHA1 Message Date
yangdx
46610682ce Fix data persistence issue in single-process mode 
In single-process mode, data updates and persistence were not working properly because the update flags were not being correctly handled between different objects.
 2025-03-10 15:41:00 +08:00
yangdx
4065a7df92 Fix linting 2025-03-10 02:07:19 +08:00
yangdx
14e1b31d1c Improved logging clarity in storage operations 2025-03-10 02:05:55 +08:00
yangdx
6b0acce644 Avoid redundant llm cache updates 2025-03-10 01:45:58 +08:00
yangdx
d2708b966d Added update flag to avoid persistence if no data is changed for KV storage 2025-03-10 01:17:25 +08:00
yangdx
4977c718f1 Improve KV storage initialize logic 2025-03-10 00:12:35 +08:00
yangdx
c938989920 Fix llm cache save problem in json_kv storage 2025-03-09 23:33:03 +08:00
yangdx
e47883d872 Add atomic data initialization lock to prevent race conditions 2025-03-09 17:33:15 +08:00
yangdx
c854aabde0 Add process ID to log messages for better multi-process debugging clarity 
- Add PID to KV and Neo4j storage  logs
- Add PID to query context logs
- Improve KV data count logging for llm cache
 2025-03-09 15:25:10 +08:00
yangdx
90527875fd Fix async issues in namespace init 2025-03-09 15:22:06 +08:00
zrguo
fd9f71e0ee fix delete_by_doc_id 2025-03-04 13:22:33 +08:00
yangdx
fd76e00c6a Refactor storage initialization to separate object creation from data loading 
• Split __post_init__ and initialize()
• Move data loading to initialize()
• Add FastAPI lifespan integration
 2025-03-01 03:48:19 +08:00
yangdx
b3328542c7 refactor: migrate synchronous locks to async locks for improved concurrency 
• Add UnifiedLock wrapper class
• Convert with blocks to async with
 2025-03-01 02:22:35 +08:00
yangdx
cd7648791a Fix linting 2025-02-28 01:25:59 +08:00
yangdx
05cf029bcc fix: convert multiprocessing managed dict to normal dict before JSON dump 2025-02-27 20:16:53 +08:00
yangdx
64f22966a3 Fix linting 2025-02-27 19:05:51 +08:00
yangdx
f007ebf006 Refactor initialization logic for vector, KV and graph storage implementations 
• Add try_initialize_namespace check
• Move init code out of storage locks
• Reduce redundant init conditions
• Simplify initialization flow
• Make init thread-safer
 2025-02-27 14:55:07 +08:00
yangdx
7436c06f6c Fix linting 2025-02-26 18:11:16 +08:00
yangdx
2c019dbc7b Refactor storage initialization to avoid redundant intitial data loads across processes, show init logs to first load only 2025-02-26 12:28:49 +08:00
yangdx
2752a764ae Refactor storage implementations to support both single and multi-process modes 
• Add shared storage management module
• Support process/thread lock based on mode
 2025-02-26 05:38:38 +08:00
yangdx
a642bb3190 refactor: use shared manager from main process for storage implementations. 2025-02-25 12:08:49 +08:00
yangdx
087d5770b0 feat(storage): Add shared memory support for file-based storage implementations 
This commit adds multiprocessing shared memory support to file-based storage implementations:
- JsonDocStatusStorage
- JsonKVStorage
- NanoVectorDBStorage
- NetworkXStorage

Each storage module now uses module-level global variables with multiprocessing.Manager() to ensure data consistency across multiple uvicorn workers. All processes will see
updates immediately when data is modified through ainsert function.
 2025-02-25 11:10:13 +08:00
Yannick Stephan
9277fe8c29 fixed return 2025-02-19 22:22:41 +01:00
Yannick Stephan
0994d478f0 cleaned code 2025-02-18 10:21:54 +01:00
Yannick Stephan
fc0cf2934e fixed drop 2025-02-18 10:21:14 +01:00
Yannick Stephan
fc4b830036 fallback default drops 2025-02-18 08:43:23 +01:00
Yannick Stephan
66c4b01fdd remove drops unused 2025-02-17 23:16:23 +01:00
Yannick Stephan
0b16718f9f add missing final 2025-02-16 16:21:28 +01:00
Yannick Stephan
3fef8201c6 added final, required methods and cleaned import 2025-02-16 14:38:09 +01:00
Yannick Stephan
931c31fa8c cleaned code 2025-02-16 13:55:30 +01:00
Yannick Stephan
71a18d1de9 updated clean of what implemented on BaseKVStorage 2025-02-16 13:31:12 +01:00
zrguo
ef0e81315f fix linting 2025-02-16 19:53:28 +08:00
St1ve
edaba428ac Update json_kv_impl.py 
There is no delete function in chunks_vdb and text_chunks in lines 1300-1302 of the lightrag.py file：
    if chunk_ids:
                await self.chunks_vdb.delete(chunk_ids)
                await self.text_chunks.delete(chunk_ids)
 2025-02-16 19:33:59 +08:00
Yannick Stephan
9f6621454e fixed ids 2025-02-09 21:24:13 +01:00
Yannick Stephan
62115b836f cleaned set 2025-02-09 20:03:15 +01:00
Yannick Stephan
6480ddee5d cleaned code 2025-02-09 19:51:05 +01:00
Yannick Stephan
7d63898015 fixed bugs 2025-02-09 19:21:49 +01:00
Yannick Stephan
7116bd1872 cleaned code 2025-02-09 15:24:30 +01:00
Yannick Stephan
82481ecf28 cleaned code 2025-02-09 14:55:52 +01:00
Yannick Stephan
4cce14e65e cleaned import 2025-02-09 11:24:08 +01:00
Yannick Stephan
31fe96d74a cleaned optional not used 2025-02-09 10:33:15 +01:00
Yannick Stephan
50c7f26262 cleanup code 2025-02-08 23:58:15 +01:00
Yannick Stephan
5a082a0052 cleaned code 2025-02-08 23:20:37 +01:00
Yannick Stephan
cff415d91f implemented method and cleaned the mess 2025-02-08 23:18:12 +01:00
Yannick Stephan
fe3050adce updated json kv 2025-02-08 23:02:40 +01:00
zrguo
80451af839 fix linting errors 2025-01-27 23:21:34 +08:00
Saifeddine ALOUI
56e9c9f4d5 Moved the storages to kg folder 2025-01-27 09:59:26 +01:00