Commit Graph

2372 Commits

Author SHA1 Message Date
yangdx
f1d312a2ce Update UI and refine node property display for compatible with legacy data 2025-03-11 15:54:41 +08:00
yangdx
30310c2845 Fix relation info in kownledge graph UI 2025-03-11 15:36:38 +08:00
yangdx
7fddabb441 Fix linting 2025-03-11 10:28:25 +08:00
yangdx
aefd596990 Refactor Neo4JStorage to use entity_id for node identification, use entity_type for node label 2025-03-11 09:59:42 +08:00
zrguo
7ed7cb4ecf Merge pull request #1036 from danielaskdd/neo4j-add-min-degree
Refactoring Neo4j implementation and fixing storage init problem for Gunicorn
2025-03-10 22:24:22 +08:00
zrguo
2bf40ab636 Merge branch 'main' into neo4j-add-min-degree 2025-03-10 22:24:06 +08:00
zrguo
2b230d403d Merge pull request #1011 from ParisNeo/main
Added docling option to load files
2025-03-10 22:14:47 +08:00
zrguo
ad13009cff fix edit_entity 2025-03-10 19:07:19 +08:00
yangdx
bbff3ed0ab Fix linting 2025-03-10 17:30:40 +08:00
yangdx
5d64f3b0a0 Improved auto-scan task initialization and status tracking.
- Added autoscan status tracking in pipeline
- Ensured auto-scan runs only once per startup
2025-03-10 17:14:14 +08:00
yangdx
3cca18c59c Refactor pipeline status updates and entity extraction.
- Let all parrallel jobs using one pipe_status objects
- Improved thread safety with pipeline_status_lock
- Only pipeline jobs can add message to pipe_status
- Marked insert_custom_chunks as deprecated
2025-03-10 16:48:59 +08:00
yangdx
57a41eedb8 Fix linting 2025-03-10 15:41:46 +08:00
yangdx
46610682ce Fix data persistence issue in single-process mode
In single-process mode, data updates and persistence were not working properly because the update flags were not being correctly handled between different objects.
2025-03-10 15:41:00 +08:00
yangdx
adca27fae9 Merge branch 'main' into neo4j-add-min-degree 2025-03-10 02:13:49 +08:00
yangdx
4065a7df92 Fix linting 2025-03-10 02:07:19 +08:00
yangdx
14e1b31d1c Improved logging clarity in storage operations 2025-03-10 02:05:55 +08:00
yangdx
6b0acce644 Avoid redundant llm cache updates 2025-03-10 01:45:58 +08:00
yangdx
d2708b966d Added update flag to avoid persistence if no data is changed for KV storage 2025-03-10 01:17:25 +08:00
yangdx
4977c718f1 Improve KV storage initialize logic 2025-03-10 00:12:35 +08:00
yangdx
c938989920 Fix llm cache save problem in json_kv storage 2025-03-09 23:33:03 +08:00
yangdx
bc42afe7b6 Unify llm_response_cache and hashing_kv, prevent creating an independent hashing_kv. 2025-03-09 22:15:26 +08:00
Saifeddine ALOUI
04862033d6 Made the defa&ult mode non docling 2025-03-09 13:14:39 +01:00
yangdx
e47883d872 Add atomic data initialization lock to prevent race conditions 2025-03-09 17:33:15 +08:00
yangdx
020a6b5ae0 Refactor LLM cache config to use argparse and add status display 2025-03-09 16:45:57 +08:00
yangdx
c854aabde0 Add process ID to log messages for better multi-process debugging clarity
- Add PID to KV and Neo4j storage  logs
- Add PID to query context logs
- Improve KV data count logging for llm cache
2025-03-09 15:25:10 +08:00
yangdx
90527875fd Fix async issues in namespace init 2025-03-09 15:22:06 +08:00
Saifeddine ALOUI
53cfb72db4 linted 2025-03-08 23:07:51 +01:00
Saifeddine ALOUI
4f42b6d933 Merge branch 'HKUDS:main' into main 2025-03-08 23:06:53 +01:00
zrguo
32a7d40650 Update lightrag_openai_neo4j_milvus_redis_demo.py 2025-03-09 02:11:31 +08:00
zrguo
fe34c50b43 Merge pull request #1030 from HKUDS/dev
Fixed and improved Iteration extraction
2025-03-09 01:29:49 +08:00
zrguo
c936aaf5c8 fix linting 2025-03-09 01:29:21 +08:00
zrguo
595d8bf372 Update prompt.py 2025-03-09 01:25:15 +08:00
zrguo
548f9a8234 Update prompts 2025-03-09 01:21:39 +08:00
yangdx
6a969e8de4 Disable logging for graph database lock acquisition and release 2025-03-09 01:14:24 +08:00
yangdx
c5d0962872 Fix linting 2025-03-09 01:00:42 +08:00
yangdx
3cf4268e7a Change logging level from INFO to DEBUG for cache hit/miss messages 2025-03-09 00:59:40 +08:00
zrguo
da59cc89d8 fix linting 2025-03-09 00:51:14 +08:00
zrguo
376b63b704 Merge pull request #1027 from dixyes/fix-qdrant-payload-id
Fix qdrant payload id
2025-03-09 00:28:17 +08:00
zrguo
3ae261dcd3 Merge pull request #1025 from danielaskdd/add-env-llm-cache-for-extract
Add LLM cache control env for API Server
2025-03-09 00:26:55 +08:00
yangdx
18c0770409 fix: duplicate nodes for same entity(label) problem in Neo4j
- Add entity_id field as key in Neo4j nodes
- Use  entity_id for nodes retrival and upsert
2025-03-09 00:24:55 +08:00
zrguo
c67e6aceca Merge branch 'main' into main 2025-03-09 00:23:06 +08:00
zrguo
38ebb16d63 Merge pull request #1021 from da-luggas/main
Fixed entites_section_list comma error
2025-03-09 00:19:41 +08:00
zrguo
906619fa93 Merge pull request #1024 from ShanGor/main
fix the postgres get all labels and get knowledge graph
2025-03-09 00:17:11 +08:00
yangdx
73452e63fa Add async lock for atomic graph database operations
• Introduced graph_db_lock mechanism
• Ensured atomic node/edge merge and insert operation
2025-03-08 22:48:12 +08:00
yangdx
95c06f1bde Add graph DB lock to shared storage system
• Introduced new graph_db_lock
• Added detailed lock debugging output
2025-03-08 22:36:41 +08:00
dixyes
458eafd714 Fix qdrant payload id
Qdrant now is using PointStruct.payload["id"], not PointStruct.id UUID.
This will fix id overwrite
2025-03-08 16:40:40 +08:00
Samuel Chan
b7f67eda21 fix the postgres get all labels and get knowledge graph 2025-03-08 11:45:59 +08:00
yangdx
fb4a4c736e Add duplicate edge upsert checking and logging 2025-03-08 11:36:24 +08:00
yangdx
22a93fb717 Limit neighbor nodes fetch to 1000 in Neo4JStorage. 2025-03-08 11:29:08 +08:00
yangdx
887f6ed81a Fix return empty list when no edges is found 2025-03-08 11:20:22 +08:00