diff --git a/README.md b/README.md index 7670e3da..346a2d1b 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ pip install lightrag-hku ## Quick Start -* Set OpenAI API key in environment: `export OPENAI_API_KEY="sk-..."`. +* Set OpenAI API key in environment: `export OPENAI_API_KEY="sk-...".` * Download the demo text "A Christmas Carol by Charles Dickens" ``` curl https://raw.githubusercontent.com/gusye1234/nano-graphrag/main/tests/mock_data.txt > ./book.txt @@ -30,12 +30,7 @@ Use the below python snippet: ``` from lightrag import LightRAG, QueryParam -WORKING_DIR = "./dickens" - -if not os.path.exists(WORKING_DIR): - os.mkdir(WORKING_DIR) - -rag = LightRAG(working_dir=WORKING_DIR) +rag = LightRAG(working_dir="./dickens") with open("./book.txt") as f: rag.insert(f.read()) @@ -138,6 +133,30 @@ Output your evaluation in the following JSON format: }} }} ``` +### Overall Performance Table +| | **Agriculture** | | **CS** | | **Legal** | | **Mix** | | +|----------------------|-------------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------| +| | NaiveRAG | **LightRAG** | NaiveRAG | **LightRAG** | NaiveRAG | **LightRAG** | NaiveRAG | **LightRAG** | +| **Comprehensiveness** | 32.69% | 67.31% | 35.44% | 64.56% | 19.05% | 80.95% | 36.36% | 63.64% | +| **Diversity** | 24.09% | 75.91% | 35.24% | 64.76% | 10.98% | 89.02% | 30.76% | 69.24% | +| **Empowerment** | 31.35% | 68.65% | 35.48% | 64.52% | 17.59% | 82.41% | 40.95% | 59.05% | +| **Overall** | 33.30% | 66.70% | 34.76% | 65.24% | 17.46% | 82.54% | 37.59% | 62.40% | +| | RQ-RAG | **LightRAG** | RQ-RAG | **LightRAG** | RQ-RAG | **LightRAG** | RQ-RAG | **LightRAG** | +| **Comprehensiveness** | 32.05% | 67.95% | 39.30% | 60.70% | 18.57% | 81.43% | 38.89% | 61.11% | +| **Diversity** | 29.44% | 70.56% | 38.71% | 61.29% | 15.14% | 84.86% | 28.50% | 71.50% | +| **Empowerment** | 32.51% | 67.49% | 37.52% | 62.48% | 17.80% | 82.20% | 43.96% | 56.04% | +| **Overall** | 33.29% | 66.71% | 39.03% | 60.97% | 17.80% | 82.20% | 39.61% | 60.39% | +| | HyDE | **LightRAG** | HyDE | **LightRAG** | HyDE | **LightRAG** | HyDE | **LightRAG** | +| **Comprehensiveness** | 24.39% | 75.61% | 36.49% | 63.51% | 27.68% | 72.32% | 42.17% | 57.83% | +| **Diversity** | 24.96% | 75.34% | 37.41% | 62.59% | 18.79% | 81.21% | 30.88% | 69.12% | +| **Empowerment** | 24.89% | 75.11% | 34.99% | 65.01% | 26.99% | 73.01% | 45.61% |54.39% | +| **Overall** | 23.17% | 76.83% | 35.67% | 64.33% | 27.68% | 72.32% | 42.72% | 57.28% | +| | GraphRAG | **LightRAG** | GraphRAG | **LightRAG** | GraphRAG | **LightRAG** | GraphRAG | **LightRAG** | +| **Comprehensiveness** | 45.56% | 54.44% | 45.98% | 54.02% | 47.13% | 52.87% | 51.86% | 48.14% | +| **Diversity** | 19.65% | 80.35% | 39.64% | 60.36% | 25.55% | 74.45% | 35.87% | 64.13% | +| **Empowerment** | 36.69% | 63.31% | 45.09% | 54.91% | 42.81% | 57.19% | 52.94% | 47.06% | +| **Overall** | 43.62% | 56.38% | 45.98% | 54.02% | 45.70% | 54.30% | 51.86% | 48.14% | + ## Code Structure ```