Commit Graph

  • b07181ca39 Remove duplicated run_with_gunicorn.py from project root yangdx 2025-03-03 01:59:20 +08:00
  • e854313438 Merge branch 'main' into validate-content-before-enqueue yangdx 2025-03-03 01:34:55 +08:00
  • 11fdb60fe5 Remove Chinese comments yangdx 2025-03-03 01:30:41 +08:00
  • a9a1fae787 Merge branch 'main' into max-graph-node yangdx 2025-03-03 01:20:05 +08:00
  • 2d6d0be128 Merge pull request #969 from danielaskdd/add-multi-worker-support Yannick Stephan 2025-03-02 17:47:00 +01:00
  • aa5888042e Improved file handling and validation for document processing yangdx 2025-03-02 23:57:57 +08:00
  • 12fbc5b851 Updated env.example with better documentation and clarity yangdx 2025-03-02 19:12:11 +08:00
  • 644ed15d75 Merge branch 'add-multi-worker-support' into max-graph-node yangdx 2025-03-02 18:37:56 +08:00
  • fca6969b0b Update Gunicorn startup instructions in API documentation yangdx 2025-03-02 18:33:18 +08:00
  • 7a8bd15203 Merge branch 'add-multi-worker-support' into max-graph-node yangdx 2025-03-02 18:24:22 +08:00
  • fb5f11f594 Add Gunicorn support for production deployment of LightRAG server yangdx 2025-03-02 18:17:51 +08:00
  • 465737efed Fix linting yangdx 2025-03-02 17:32:25 +08:00
  • 68bf02abb6 refactor: improve graph querying with label substring matching and security fixes yangdx 2025-03-02 16:20:37 +08:00
  • 0f1eb42c8d Add node limit and prioritization for knowledge graph retrieval yangdx 2025-03-02 15:39:14 +08:00
  • 265fc2c87f fix linting zrguo 2025-03-02 14:24:54 +08:00
  • 5198240cf9 remove " and upper() zrguo 2025-03-02 14:23:06 +08:00
  • 87d0ee0127 Merge branch 'add-multi-worker-support' into max-graph-node yangdx 2025-03-02 12:53:05 +08:00
  • 1ca6837219 Add max nodes limit for graph retrieval of networkX yangdx 2025-03-02 12:52:25 +08:00
  • 396b8c3347 Add psutil to required dependencies for runtime monitoring yangdx 2025-03-02 12:44:58 +08:00
  • 7124845e55 Optimize document processing pipeline with better status tracking & batch handling yangdx 2025-03-02 11:09:32 +08:00
  • 1a5eb20003 Fix history_messages clearing in LightRAG pipeline status initialization yangdx 2025-03-02 04:43:41 +08:00
  • c0b22a8ae2 Merge branch 'main' into add-multi-worker-support yangdx 2025-03-02 02:54:57 +08:00
  • e20aeada92 docs: add gunicorn deployment guide and update server yangdx 2025-03-02 02:43:11 +08:00
  • 8fb3670ac7 Add additional log-related patterns to gitignore yangdx 2025-03-02 02:26:21 +08:00
  • 7cd25fe5ab Improve shared storage cleanup and clarify initialization in multi-worker setup yangdx 2025-03-02 01:00:27 +08:00
  • 8d6960f280 Fix linting yangdx 2025-03-02 00:13:11 +08:00
  • f76cf98dbd Add automatic dependency checking and installation for server startup yangdx 2025-03-02 00:04:59 +08:00
  • e8d0d065f3 fix: Improve async handling and FAISS storage reliability yangdx 2025-03-01 23:35:09 +08:00
  • 9aef112d51 Fix incorrect comment about update flag behavior in FAISS implementation yangdx 2025-03-01 22:27:12 +08:00
  • bf0ddc6450 making optional for ainsert Abyl Ikhsanov 2025-03-01 13:34:49 +01:00
  • baa505eeb0 adding full_doc_id to insert Abyl Ikhsanov 2025-03-01 13:26:02 +01:00
  • 9aa438cf79 fix linting zrguo 2025-03-01 18:35:12 +08:00
  • a8f4385c05 Add clear_cache zrguo 2025-03-01 18:30:58 +08:00
  • a923ee89b0 Update __init__.py zrguo 2025-03-01 17:47:53 +08:00
  • 4219454fab fix format zrguo 2025-03-01 17:45:06 +08:00
  • e3a40c2fdb Fix linting yangdx 2025-03-01 16:23:34 +08:00
  • 3507e894d9 Merge branch 'main' into add-multi-worker-support yangdx 2025-03-01 15:55:37 +08:00
  • d18eb52ccc Add type ignore comments for asyncpg imports to suppress mypy errors yangdx 2025-03-01 15:38:39 +08:00
  • 3511b9805c Add auto-installation of gunicorn if not present in environment yangdx 2025-03-01 15:34:01 +08:00
  • ab704aae47 Config log setting for very woker properly for Gunicorn mode. yangdx 2025-03-01 15:13:21 +08:00
  • 40e9e26edb feat: add update flags status to API health endpoint yangdx 2025-03-01 14:58:26 +08:00
  • 41eff2ca2f Fix data persistence issue in NanoVectorDBStorage yangdx 2025-03-01 13:35:00 +08:00
  • 48d9800573 Enhance gunicorn config handling with env vars and command line arg priority yangdx 2025-03-01 12:58:52 +08:00
  • 35bcfca28f feat: add multi-process support for FAISS vector storage yangdx 2025-03-01 12:42:30 +08:00
  • d4f6dcfd54 Improve multi-process data synchronization and persistence in storage implementations yangdx 2025-03-01 12:41:30 +08:00
  • c07a5039b7 Refactor shared storage locks to separate pipeline, storage and internal locks for deadlock preventing yangdx 2025-03-01 10:48:55 +08:00
  • d3de57c1e4 Add multi-process support for vector database and graph storage with lock flags yangdx 2025-03-01 10:37:05 +08:00
  • d704512139 Refactor shared storage module to improve async handling and naming consistency yangdx 2025-03-01 05:01:26 +08:00
  • fd76e00c6a Refactor storage initialization to separate object creation from data loading yangdx 2025-03-01 03:48:19 +08:00
  • 5bbe61a02d Merge pull request #960 from FeHuynhVI/fix-delete-by-doc-id Yannick Stephan 2025-02-28 19:44:05 +01:00
  • b3328542c7 refactor: migrate synchronous locks to async locks for improved concurrency yangdx 2025-03-01 02:22:35 +08:00
  • a721421bd8 Add async support and update flag mechanism for shared storage yangdx 2025-03-01 01:49:26 +08:00
  • aac1bdd9e6 feat: add configurable log rotation settings via environment variables yangdx 2025-02-28 23:21:14 +08:00
  • 731d820bcc Remove redundancy set_logger function and related calls yangdx 2025-02-28 21:46:45 +08:00
  • c973498c34 Fix linting yangdx 2025-02-28 21:35:04 +08:00
  • c37b1e8aa7 Align Gunicorn configuration with Uvicorn yangdx 2025-02-28 20:41:11 +08:00
  • ff549a3a9c Update Gunicorn config with logging filters and worker-specific configurations yangdx 2025-02-28 17:45:40 +08:00
  • f588cdc5df Optimize logging config & worker handling for different server modes yangdx 2025-02-28 16:50:54 +08:00
  • 81f6f6e343 Fix lightrag logger initailization problem, fix gunicorn acccess log missing yangdx 2025-02-28 16:07:33 +08:00
  • 157ec862ae Enhance logging system with file rotation and unified configuration yangdx 2025-02-28 14:57:25 +08:00
  • 8cd45161f2 feat: add history_messages to track pipeline processing progress yangdx 2025-02-28 13:53:40 +08:00
  • b090a22be7 Add concurrency check for auto scan task to prevent duplicate scans yangdx 2025-02-28 12:22:20 +08:00
  • 04bd5413c9 Add API endpoint to retrieve document indexing pipeline status yangdx 2025-02-28 12:21:50 +08:00
  • b2da69b7f1 Add pipeline status control for concurrent document indexing processes yangdx 2025-02-28 11:52:42 +08:00
  • feaa7ce69d Remove auto-scaling of workers based on CPU count in gunicorn config yangdx 2025-02-28 10:54:43 +08:00
  • b4bcd76599 Remove useless scan progress tracking functionality and related code yangdx 2025-02-28 10:53:36 +08:00
  • cd7648791a Fix linting yangdx 2025-02-28 01:25:59 +08:00
  • 3dcfa561d7 Remove debug logging yangdx 2025-02-28 01:15:12 +08:00
  • 291e0c1b14 revert vector and graph use local data(single process) yangdx 2025-02-28 01:14:25 +08:00
  • 2f7fe5e4b6 feat: fix delete by document id Huỳnh Triệu Vĩ 2025-02-27 23:34:57 +07:00
  • db2a902fcb Rename get_scan_lock to get_storage_lock yangdx 2025-02-28 00:34:33 +08:00
  • cf2f6b726c Add newline after Gunicorn initialization message for better readability yangdx 2025-02-27 20:26:12 +08:00
  • 05d03638ec Clean up logging output and remove redundant log messages yangdx 2025-02-27 20:17:28 +08:00
  • 05cf029bcc fix: convert multiprocessing managed dict to normal dict before JSON dump yangdx 2025-02-27 20:16:53 +08:00
  • 27500191b4 Standarize scan progress namespace initialization yangdx 2025-02-27 19:08:36 +08:00
  • 64f22966a3 Fix linting yangdx 2025-02-27 19:05:51 +08:00
  • 946095ef80 Fix multiprocess dict creation logic, add process safety locks for namespace creation. yangdx 2025-02-27 19:03:53 +08:00
  • 92ecb0da97 Refactor document scanning progress share variable initialization yangdx 2025-02-27 16:07:00 +08:00
  • e881bc0709 simplify process state management by removing redundant multiprocess flag yangdx 2025-02-27 15:36:12 +08:00
  • 1699b10a25 Refactor direct client/graph access to reduce redundant get calls in vector/graph ops yangdx 2025-02-27 15:14:54 +08:00
  • 438e4780a8 Refactor Faiss index access with helper method to improve code organization yangdx 2025-02-27 15:09:19 +08:00
  • f007ebf006 Refactor initialization logic for vector, KV and graph storage implementations yangdx 2025-02-27 14:55:07 +08:00
  • 03d05b094d Improve Gunicorn support and cleanup shared storage initialization yangdx 2025-02-27 14:13:42 +08:00
  • 7aec78833c Implement Gunicorn+Uvicorn integration for shared data preloading yangdx 2025-02-27 13:25:22 +08:00
  • 7c237920b1 Refactor shared storage to support both single and multi-process modes yangdx 2025-02-27 08:48:33 +08:00
  • 781ce7dfe2 Merge pull request #953 from zhicwu/main zrguo 2025-02-26 23:23:44 +08:00
  • d79a9d7acc consistent format Zhichun Wu 2025-02-26 23:04:21 +08:00
  • c785b4ba7d Merge pull request #952 from YanSte/version-2 Yannick Stephan 2025-02-26 12:13:52 +01:00
  • 8fa45556d0 updated version to 1.2.2 Yannick Stephan 2025-02-26 12:12:08 +01:00
  • 3c9908b94a fixed lint Yannick Stephan 2025-02-26 12:11:28 +01:00
  • 2ba3447a56 Merge branch 'HKUDS:main' into main Yannick Stephan 2025-02-26 12:10:43 +01:00
  • 4963305dc3 Merge pull request #950 from cnjack/feat/custom_doc_ids Yannick Stephan 2025-02-26 12:08:02 +01:00
  • 3dca265b64 Merge pull request #940 from FeHuynhVI/FEHuynhVI-fix-run-async-safely Yannick Stephan 2025-02-26 12:06:58 +01:00
  • 7436c06f6c Fix linting yangdx 2025-02-26 18:11:16 +08:00
  • 7d12715f09 Refactor shared storage to safely handle multi-process initialization and data sharing yangdx 2025-02-26 18:11:02 +08:00
  • 4eb069d1d6 Initialize scan_progress with default values if not already set yangdx 2025-02-26 17:42:49 +08:00
  • 145bacc773 Add empty graph creation logging in NetworkXStorage yangdx 2025-02-26 17:42:30 +08:00
  • fee90ddd9d add support for the single document and custom chunks method jack 2025-02-26 14:41:10 +08:00
  • 41f5d208a9 fix: shared data intitialization failed for multi-worker yangdx 2025-02-26 13:32:15 +08:00
  • 2c019dbc7b Refactor storage initialization to avoid redundant intitial data loads across processes, show init logs to first load only yangdx 2025-02-26 12:28:49 +08:00