gardel/lightrag
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,250 Commits 2 Branches 0 Tags
694daf158aee44e2000e2d0178cc1bbfbbb31584
Commit Graph

2250 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yangdx
81f6f6e343 Fix lightrag logger initailization problem, fix gunicorn acccess log missing 2025-02-28 16:07:33 +08:00
yangdx
157ec862ae Enhance logging system with file rotation and unified configuration 
• Unify logging across Gunicorn and Uvicorn
• Add rotating file handlers
 2025-02-28 14:57:25 +08:00
yangdx
8cd45161f2 feat: add history_messages to track pipeline processing progress 
• Add shared history_messages list
• Track pipeline progress with messages
 2025-02-28 13:53:40 +08:00
yangdx
b090a22be7 Add concurrency check for auto scan task to prevent duplicate scans 
• Add pipeline status check before scan
• Add storage lock protection
• Add latest_message to status tracking
• Add helpful log message at startup
 2025-02-28 12:22:20 +08:00
yangdx
04bd5413c9 Add API endpoint to retrieve document indexing pipeline status 
• GET /pipeline_status endpoint added
• Returns current pipeline processing state
 2025-02-28 12:21:50 +08:00
yangdx
b2da69b7f1 Add pipeline status control for concurrent document indexing processes 
• Add shared pipeline status namespace
• Implement concurrent process control
• Add request queuing for pending jobs
 2025-02-28 11:52:42 +08:00
yangdx
feaa7ce69d Remove auto-scaling of workers based on CPU count in gunicorn config 2025-02-28 10:54:43 +08:00
yangdx
b4bcd76599 Remove useless scan progress tracking functionality and related code 2025-02-28 10:53:36 +08:00
yangdx
cd7648791a Fix linting 2025-02-28 01:25:59 +08:00
yangdx
3dcfa561d7 Remove debug logging 2025-02-28 01:15:12 +08:00
yangdx
291e0c1b14 revert vector and graph use local data(single process) 2025-02-28 01:14:25 +08:00
Huỳnh Triệu Vĩ
2f7fe5e4b6 feat: fix delete by document id 2025-02-27 23:34:57 +07:00
yangdx
db2a902fcb Rename get_scan_lock to get_storage_lock 2025-02-28 00:34:33 +08:00
yangdx
cf2f6b726c Add newline after Gunicorn initialization message for better readability 2025-02-27 20:26:12 +08:00
yangdx
05d03638ec Clean up logging output and remove redundant log messages 2025-02-27 20:17:28 +08:00
yangdx
05cf029bcc fix: convert multiprocessing managed dict to normal dict before JSON dump 2025-02-27 20:16:53 +08:00
yangdx
27500191b4 Standarize scan progress namespace initialization 2025-02-27 19:08:36 +08:00
yangdx
64f22966a3 Fix linting 2025-02-27 19:05:51 +08:00
yangdx
946095ef80 Fix multiprocess dict creation logic, add process safety locks for namespace creation. 2025-02-27 19:03:53 +08:00
yangdx
92ecb0da97 Refactor document scanning progress share variable initialization 2025-02-27 16:07:00 +08:00
yangdx
e881bc0709 simplify process state management by removing redundant multiprocess flag 2025-02-27 15:36:12 +08:00
yangdx
1699b10a25 Refactor direct client/graph access to reduce redundant get calls in vector/graph ops 2025-02-27 15:14:54 +08:00
yangdx
438e4780a8 Refactor Faiss index access with helper method to improve code organization 2025-02-27 15:09:19 +08:00
yangdx
f007ebf006 Refactor initialization logic for vector, KV and graph storage implementations 
• Add try_initialize_namespace check
• Move init code out of storage locks
• Reduce redundant init conditions
• Simplify initialization flow
• Make init thread-safer
 2025-02-27 14:55:07 +08:00
yangdx
03d05b094d Improve Gunicorn support and cleanup shared storage initialization 
• Move Gunicorn check before other startup
• Improve startup flow organization
 2025-02-27 14:13:42 +08:00
yangdx
7aec78833c Implement Gunicorn+Uvicorn integration for shared data preloading 
- Create run_with_gunicorn.py script to properly initialize shared data in the
  main process before forking worker processes
- Revert unvicorn to single process mode only, and let gunicorn do all the multi-process jobs
 2025-02-27 13:25:22 +08:00
yangdx
7c237920b1 Refactor shared storage to support both single and multi-process modes 
• Initialize storage based on worker count
• Remove redundant global variable checks
• Add explicit mutex initialization
• Centralize shared storage initialization
• Fix process/thread lock selection logic
 2025-02-27 08:48:33 +08:00
zrguo
781ce7dfe2 Merge pull request #953 from zhicwu/main 
Use consistent format as in examples
 2025-02-26 23:23:44 +08:00
Zhichun Wu
d79a9d7acc consistent format 2025-02-26 23:04:21 +08:00
Yannick Stephan
c785b4ba7d Merge pull request #952 from YanSte/version-2 
Version 1.2.2
 2025-02-26 12:13:52 +01:00
Yannick Stephan
8fa45556d0 updated version to 1.2.2 2025-02-26 12:12:08 +01:00
Yannick Stephan
3c9908b94a fixed lint 2025-02-26 12:11:28 +01:00
Yannick Stephan
2ba3447a56 Merge branch 'HKUDS:main' into main 2025-02-26 12:10:43 +01:00
Yannick Stephan
4963305dc3 Merge pull request #950 from cnjack/feat/custom_doc_ids 
add support for the single document and custom chunks method
 2025-02-26 12:08:02 +01:00
Yannick Stephan
3dca265b64 Merge pull request #940 from FeHuynhVI/FEHuynhVI-fix-run-async-safely 
fix this event loop is already running
 2025-02-26 12:06:58 +01:00
yangdx
7436c06f6c Fix linting 2025-02-26 18:11:16 +08:00
yangdx
7d12715f09 Refactor shared storage to safely handle multi-process initialization and data sharing 
• Add namespace initialization check
• Use atomic operations for shared data
 2025-02-26 18:11:02 +08:00
yangdx
4eb069d1d6 Initialize scan_progress with default values if not already set 2025-02-26 17:42:49 +08:00
yangdx
145bacc773 Add empty graph creation logging in NetworkXStorage 2025-02-26 17:42:30 +08:00
jack
fee90ddd9d add support for the single document and custom chunks method 2025-02-26 14:41:10 +08:00
yangdx
41f5d208a9 fix: shared data intitialization failed for multi-worker 2025-02-26 13:32:15 +08:00
yangdx
2c019dbc7b Refactor storage initialization to avoid redundant intitial data loads across processes, show init logs to first load only 2025-02-26 12:28:49 +08:00
yangdx
15a6a9cf7c fix: log filtering void when uvicorn wokers is greater than 1 
- Centralize logging setup
- Fix logger propagation issues
 2025-02-26 12:23:35 +08:00
yangdx
2752a764ae Refactor storage implementations to support both single and multi-process modes 
• Add shared storage management module
• Support process/thread lock based on mode
 2025-02-26 05:38:38 +08:00
Yannick Stephan
32b498b71c Merge pull request #946 from ArnoChenFx/dev-webui 
Enhance Graph Visualization with Configurable Depth and Layout Iterations
 2025-02-25 19:40:17 +01:00
ArnoChen
798c37d4fa build webui 2025-02-25 18:36:34 +08:00
ArnoChen
40a1a94a31 add graph depth and layout iteration settings 2025-02-25 18:32:53 +08:00
yangdx
8050b0f91b feat: automatically initialize API manager in single process mode 
- Add manager init check in __post_init__
- Call initialize_manager if needed
- Add info log message for init
- Ensure API manager is ready for use
 2025-02-25 12:09:30 +08:00
yangdx
a642bb3190 refactor: use shared manager from main process for storage implementations. 2025-02-25 12:08:49 +08:00
yangdx
e22e014f22 feat(storage): Add shared memory support for FAISS 2025-02-25 11:25:06 +08:00