Yannick Stephan
0b16718f9f
add missing final
2025-02-16 16:21:28 +01:00
Yannick Stephan
49bea486a7
cleaned code
2025-02-16 16:04:35 +01:00
Yannick Stephan
2b2c81a722
added some comments
2025-02-16 16:04:07 +01:00
Yannick Stephan
abad9f235c
make value on str enum
2025-02-16 15:54:54 +01:00
yangdx
9ec920661d
Improve token estimation accuracy by using tiktoken instead of regex-based approach
...
• Replace regex token estimation method
• Use tiktoken for more precise counting
• Support all types of text input
• Simplify code implementation
2025-02-16 22:53:58 +08:00
Yannick Stephan
2bf238396e
updated wrong status
2025-02-16 15:52:59 +01:00
yangdx
b450430109
Change log level from info to debug for token count logging
2025-02-16 22:42:53 +08:00
yangdx
8f6e9fcf50
Merge branch 'main' into add-env-settings
2025-02-16 22:34:39 +08:00
yangdx
cfb49fc013
Delete inline comment about config.ini
2025-02-16 22:31:12 +08:00
Yannick Stephan
a1607bbcb9
Merge remote-tracking branch 'origin/main' into make-clear-what-implemented-or-not
...
# Conflicts:
# lightrag/base.py
# lightrag/kg/json_doc_status_impl.py
# lightrag/kg/mongo_impl.py
# lightrag/kg/postgres_impl.py
2025-02-16 15:29:16 +01:00
Yannick Stephan
0c21442ca4
fixed default init
2025-02-16 15:20:46 +01:00
ArnoChen
801a980c03
Merge branch 'main' into light-webui
2025-02-16 22:09:18 +08:00
Yannick Stephan
0e7aff96bb
back to not making breaks
2025-02-16 15:08:50 +01:00
zrguo
200319f293
Merge pull request #791 from ArnoChenFx/refactor-server
...
Refactor File Indexing for Background Asynchronous Processing
2025-02-16 22:07:30 +08:00
ArnoChen
e5b89c5400
fix retrieval testing and query settings
2025-02-16 21:58:30 +08:00
Yannick Stephan
0a8c94a1e0
fix value
2025-02-16 14:51:24 +01:00
Yannick Stephan
9a5fbaaa5f
removed unused methods
2025-02-16 14:50:04 +01:00
Yannick Stephan
a0844bca28
cleaned import
2025-02-16 14:45:45 +01:00
ArnoChen
fc3ca2777f
add query settings
2025-02-16 21:43:14 +08:00
Yannick Stephan
3fef8201c6
added final, required methods and cleaned import
2025-02-16 14:38:09 +01:00
ArnoChen
893b645506
unify doc status retrieval with get_docs_by_status
2025-02-16 21:28:58 +08:00
yangdx
41cbecdbe6
Add token size limit truncation for node data retrieval
...
• Add truncate_list_by_token_size function
• Limit max tokens for local context
• Add logging for truncation info
• Apply truncation to node_datas list
2025-02-16 21:26:29 +08:00
ArnoChen
b580e47324
format
2025-02-16 21:20:43 +08:00
ArnoChen
1a6c6f30a1
Merge remote-tracking branch 'origin/main' into refactor-server
...
# Conflicts:
# lightrag/api/lightrag_server.py
2025-02-16 21:19:30 +08:00
Yannick Stephan
7848a38a45
added all abstractmethod
2025-02-16 14:11:19 +01:00
ArnoChen
bbe24ab7ce
enhance query and indexing with pipeline
2025-02-16 21:11:05 +08:00
Yannick Stephan
931c31fa8c
cleaned code
2025-02-16 13:55:30 +01:00
Yannick Stephan
882190a515
updated clean of what implemented on DocStatusStorage
2025-02-16 13:53:59 +01:00
zrguo
2a0c7c0322
Merge pull request #785 from danielaskdd/improve-CORS-handling
...
improve CORS and streaming response headers
2025-02-16 20:31:33 +08:00
Yannick Stephan
71a18d1de9
updated clean of what implemented on BaseKVStorage
2025-02-16 13:31:12 +01:00
Yannick Stephan
3eba41aab6
updated clean of what implemented on BaseVectorStorage
2025-02-16 13:24:42 +01:00
yangdx
7f13b779e1
Fix linting
2025-02-16 20:15:14 +08:00
yangdx
4e354451b7
Fix load_dotenv() missing problem for base.py
2025-02-16 20:05:45 +08:00
zrguo
ef0e81315f
fix linting
2025-02-16 19:53:28 +08:00
zrguo
517ceb16fa
Merge pull request #794 from St1veLiu/main
...
Update json_kv_impl.py
2025-02-16 19:51:43 +08:00
zrguo
31f14cf505
Merge pull request #793 from VeiledTee/patch-1
...
Update HuggingFace example in README.md
2025-02-16 19:49:05 +08:00
yangdx
36ff0bfa85
Improve logging message clarity by including max tokens info for truncation
2025-02-16 19:47:20 +08:00
zrguo
07178f6c2c
Merge pull request #782 from YanSte/code-cleaning
...
Code Cleanup & Maintenance Improvements
2025-02-16 19:46:42 +08:00
Yannick Stephan
1051ff402d
fixed lint
2025-02-16 12:45:27 +01:00
zrguo
f868c1da78
Merge branch 'main' into code-cleaning
2025-02-16 19:41:05 +08:00
zrguo
9ba3279424
Merge pull request #784 from YanSte/improve-docker
...
Improve .gitignore and .dockerignore
2025-02-16 19:37:41 +08:00
St1ve
edaba428ac
Update json_kv_impl.py
...
There is no delete function in chunks_vdb and text_chunks in lines 1300-1302 of the lightrag.py file：
if chunk_ids:
await self.chunks_vdb.delete(chunk_ids)
await self.text_chunks.delete(chunk_ids)
2025-02-16 19:33:59 +08:00
zrguo
5189a9438c
Merge pull request #780 from spo0nman/fix-bug-778
...
Enhance Neo4j graph storage with error handling and label validation
2025-02-16 19:30:32 +08:00
yangdx
601df31edf
feat: move query-related settings to env file for better configuration
...
• Add env vars for token and chunk settings
• Add token count logging for prompts
• Add token count logging for context
• Move hardcoded values to env variables
• Improve logging clarity and consistency
2025-02-16 19:26:57 +08:00
zrguo
292b65320a
Merge pull request #781 from ArnoChenFx/enhance-mongodb-backends
...
Add MongoDB VectorDB Support and Optimize Existing MongoDB Implementations
2025-02-16 19:26:35 +08:00
zrguo
9d21523541
Merge pull request #772 from hhxdestiny/chroma_support_local
...
chroma_impl supports local persistent client: PersistentClient
2025-02-16 19:21:53 +08:00
Yannick Stephan
3319db0dba
cleaned code
2025-02-15 22:37:32 +01:00
Yannick Stephan
eaf1d553d2
improved typing
2025-02-15 22:37:12 +01:00
Yannick Stephan
8d0d8b8279
remove unsused method
2025-02-15 22:24:49 +01:00
Yannick Stephan
49ce229af7
fix wrong type
2025-02-15 22:23:16 +01:00
