From 54c11d773452799d887625393e5c9e18adcc9ee2 Mon Sep 17 00:00:00 2001 From: yangdx Date: Thu, 23 Jan 2025 01:17:21 +0800 Subject: [PATCH] Delete outdated doc (new version is lightrag/api/README.md) --- docs/LightRagAPI.md | 361 -------------------------------------------- 1 file changed, 361 deletions(-) delete mode 100644 docs/LightRagAPI.md diff --git a/docs/LightRagAPI.md b/docs/LightRagAPI.md deleted file mode 100644 index 6c82be66..00000000 --- a/docs/LightRagAPI.md +++ /dev/null @@ -1,361 +0,0 @@ -## Install with API Support - -LightRAG provides optional API support through FastAPI servers that add RAG capabilities to existing LLM services. You can install LightRAG with API support in two ways: - -### 1. Installation from PyPI - -```bash -pip install "lightrag-hku[api]" -``` - -### 2. Installation from Source (Development) - -```bash -# Clone the repository -git clone https://github.com/HKUDS/lightrag.git - -# Change to the repository directory -cd lightrag - -# Install in editable mode with API support -pip install -e ".[api]" -``` - -### Prerequisites - -Before running any of the servers, ensure you have the corresponding backend service running for both llm and embedding. -The new api allows you to mix different bindings for llm/embeddings. -For example, you have the possibility to use ollama for the embedding and openai for the llm. - -#### For LoLLMs Server -- LoLLMs must be running and accessible -- Default connection: http://localhost:9600 -- Configure using --llm-binding-host and/or --embedding-binding-host if running on a different host/port - -#### For Ollama Server -- Ollama must be running and accessible -- Default connection: http://localhost:11434 -- Configure using --ollama-host if running on a different host/port - -#### For OpenAI Server -- Requires valid OpenAI API credentials set in environment variables -- OPENAI_API_KEY must be set - -#### For Azure OpenAI Server -Azure OpenAI API can be created using the following commands in Azure CLI (you need to install Azure CLI first from [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)): -```bash -# Change the resource group name, location and OpenAI resource name as needed -RESOURCE_GROUP_NAME=LightRAG -LOCATION=swedencentral -RESOURCE_NAME=LightRAG-OpenAI - -az login -az group create --name $RESOURCE_GROUP_NAME --location $LOCATION -az cognitiveservices account create --name $RESOURCE_NAME --resource-group $RESOURCE_GROUP_NAME --kind OpenAI --sku S0 --location swedencentral -az cognitiveservices account deployment create --resource-group $RESOURCE_GROUP_NAME --model-format OpenAI --name $RESOURCE_NAME --deployment-name gpt-4o --model-name gpt-4o --model-version "2024-08-06" --sku-capacity 100 --sku-name "Standard" -az cognitiveservices account deployment create --resource-group $RESOURCE_GROUP_NAME --model-format OpenAI --name $RESOURCE_NAME --deployment-name text-embedding-3-large --model-name text-embedding-3-large --model-version "1" --sku-capacity 80 --sku-name "Standard" -az cognitiveservices account show --name $RESOURCE_NAME --resource-group $RESOURCE_GROUP_NAME --query "properties.endpoint" -az cognitiveservices account keys list --name $RESOURCE_NAME -g $RESOURCE_GROUP_NAME - -``` -The output of the last command will give you the endpoint and the key for the OpenAI API. You can use these values to set the environment variables in the `.env` file. - - -## Configuration - -LightRAG can be configured using either command-line arguments or environment variables. When both are provided, command-line arguments take precedence over environment variables. - -### Environment Variables - -You can configure LightRAG using environment variables by creating a `.env` file in your project root directory. Here's a complete example of available environment variables: - -```env -# Server Configuration -HOST=0.0.0.0 -PORT=9621 - -# Directory Configuration -WORKING_DIR=/app/data/rag_storage -INPUT_DIR=/app/data/inputs - -# LLM Configuration -LLM_BINDING=ollama -LLM_BINDING_HOST=http://localhost:11434 -LLM_MODEL=mistral-nemo:latest - -# Embedding Configuration -EMBEDDING_BINDING=ollama -EMBEDDING_BINDING_HOST=http://localhost:11434 -EMBEDDING_MODEL=bge-m3:latest - -# RAG Configuration -MAX_ASYNC=4 -MAX_TOKENS=32768 -EMBEDDING_DIM=1024 -MAX_EMBED_TOKENS=8192 - -# Security -LIGHTRAG_API_KEY= - -# Logging -LOG_LEVEL=INFO - -# Optional SSL Configuration -#SSL=true -#SSL_CERTFILE=/path/to/cert.pem -#SSL_KEYFILE=/path/to/key.pem - -# Optional Timeout -#TIMEOUT=30 -``` - -### Configuration Priority - -The configuration values are loaded in the following order (highest priority first): -1. Command-line arguments -2. Environment variables -3. Default values - -For example: -```bash -# This command-line argument will override both the environment variable and default value -python lightrag.py --port 8080 - -# The environment variable will override the default value but not the command-line argument -PORT=7000 python lightrag.py -``` - -#### LightRag Server Options - -| Parameter | Default | Description | -|-----------|---------|-------------| -| --host | 0.0.0.0 | Server host | -| --port | 9621 | Server port | -| --llm-binding | ollama | LLM binding to be used. Supported: lollms, ollama, openai | -| --llm-binding-host | (dynamic) | LLM server host URL. Defaults based on binding: http://localhost:11434 (ollama), http://localhost:9600 (lollms), https://api.openai.com/v1 (openai) | -| --llm-model | mistral-nemo:latest | LLM model name | -| --embedding-binding | ollama | Embedding binding to be used. Supported: lollms, ollama, openai | -| --embedding-binding-host | (dynamic) | Embedding server host URL. Defaults based on binding: http://localhost:11434 (ollama), http://localhost:9600 (lollms), https://api.openai.com/v1 (openai) | -| --embedding-model | bge-m3:latest | Embedding model name | -| --working-dir | ./rag_storage | Working directory for RAG storage | -| --input-dir | ./inputs | Directory containing input documents | -| --max-async | 4 | Maximum async operations | -| --max-tokens | 32768 | Maximum token size | -| --embedding-dim | 1024 | Embedding dimensions | -| --max-embed-tokens | 8192 | Maximum embedding token size | -| --timeout | None | Timeout in seconds (useful when using slow AI). Use None for infinite timeout | -| --log-level | INFO | Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) | -| --key | None | API key for authentication. Protects lightrag server against unauthorized access | -| --ssl | False | Enable HTTPS | -| --ssl-certfile | None | Path to SSL certificate file (required if --ssl is enabled) | -| --ssl-keyfile | None | Path to SSL private key file (required if --ssl is enabled) | - - - -For protecting the server using an authentication key, you can also use an environment variable named `LIGHTRAG_API_KEY`. -### Example Usage - -#### Running a Lightrag server with ollama default local server as llm and embedding backends - -Ollama is the default backend for both llm and embedding, so by default you can run lightrag-server with no parameters and the default ones will be used. Make sure ollama is installed and is running and default models are already installed on ollama. - -```bash -# Run lightrag with ollama, mistral-nemo:latest for llm, and bge-m3:latest for embedding -lightrag-server - -# Using specific models (ensure they are installed in your ollama instance) -lightrag-server --llm-model adrienbrault/nous-hermes2theta-llama3-8b:f16 --embedding-model nomic-embed-text --embedding-dim 1024 - -# Using an authentication key -lightrag-server --key my-key - -# Using lollms for llm and ollama for embedding -lightrag-server --llm-binding lollms -``` - -#### Running a Lightrag server with lollms default local server as llm and embedding backends - -```bash -# Run lightrag with lollms, mistral-nemo:latest for llm, and bge-m3:latest for embedding, use lollms for both llm and embedding -lightrag-server --llm-binding lollms --embedding-binding lollms - -# Using specific models (ensure they are installed in your ollama instance) -lightrag-server --llm-binding lollms --llm-model adrienbrault/nous-hermes2theta-llama3-8b:f16 --embedding-binding lollms --embedding-model nomic-embed-text --embedding-dim 1024 - -# Using an authentication key -lightrag-server --key my-key - -# Using lollms for llm and openai for embedding -lightrag-server --llm-binding lollms --embedding-binding openai --embedding-model text-embedding-3-small -``` - - -#### Running a Lightrag server with openai server as llm and embedding backends - -```bash -# Run lightrag with lollms, GPT-4o-mini for llm, and text-embedding-3-small for embedding, use openai for both llm and embedding -lightrag-server --llm-binding openai --llm-model GPT-4o-mini --embedding-binding openai --embedding-model text-embedding-3-small - -# Using an authentication key -lightrag-server --llm-binding openai --llm-model GPT-4o-mini --embedding-binding openai --embedding-model text-embedding-3-small --key my-key - -# Using lollms for llm and openai for embedding -lightrag-server --llm-binding lollms --embedding-binding openai --embedding-model text-embedding-3-small -``` - -#### Running a Lightrag server with azure openai server as llm and embedding backends - -```bash -# Run lightrag with lollms, GPT-4o-mini for llm, and text-embedding-3-small for embedding, use openai for both llm and embedding -lightrag-server --llm-binding azure_openai --llm-model GPT-4o-mini --embedding-binding openai --embedding-model text-embedding-3-small - -# Using an authentication key -lightrag-server --llm-binding azure_openai --llm-model GPT-4o-mini --embedding-binding azure_openai --embedding-model text-embedding-3-small --key my-key - -# Using lollms for llm and azure_openai for embedding -lightrag-server --llm-binding lollms --embedding-binding azure_openai --embedding-model text-embedding-3-small -``` - -**Important Notes:** -- For LoLLMs: Make sure the specified models are installed in your LoLLMs instance -- For Ollama: Make sure the specified models are installed in your Ollama instance -- For OpenAI: Ensure you have set up your OPENAI_API_KEY environment variable -- For Azure OpenAI: Build and configure your server as stated in the Prequisites section - -For help on any server, use the --help flag: -```bash -lightrag-server --help -``` - -Note: If you don't need the API functionality, you can install the base package without API support using: -```bash -pip install lightrag-hku -``` - -## API Endpoints - -All servers (LoLLMs, Ollama, OpenAI and Azure OpenAI) provide the same REST API endpoints for RAG functionality. - -### Query Endpoints - -#### POST /query -Query the RAG system with options for different search modes. - -```bash -curl -X POST "http://localhost:9621/query" \ - -H "Content-Type: application/json" \ - -d '{"query": "Your question here", "mode": "hybrid", ""}' -``` - -#### POST /query/stream -Stream responses from the RAG system. - -```bash -curl -X POST "http://localhost:9621/query/stream" \ - -H "Content-Type: application/json" \ - -d '{"query": "Your question here", "mode": "hybrid"}' -``` - -### Document Management Endpoints - -#### POST /documents/text -Insert text directly into the RAG system. - -```bash -curl -X POST "http://localhost:9621/documents/text" \ - -H "Content-Type: application/json" \ - -d '{"text": "Your text content here", "description": "Optional description"}' -``` - -#### POST /documents/file -Upload a single file to the RAG system. - -```bash -curl -X POST "http://localhost:9621/documents/file" \ - -F "file=@/path/to/your/document.txt" \ - -F "description=Optional description" -``` - -#### POST /documents/batch -Upload multiple files at once. - -```bash -curl -X POST "http://localhost:9621/documents/batch" \ - -F "files=@/path/to/doc1.txt" \ - -F "files=@/path/to/doc2.txt" -``` - -#### DELETE /documents -Clear all documents from the RAG system. - -```bash -curl -X DELETE "http://localhost:9621/documents" -``` - -### Utility Endpoints - -#### GET /health -Check server health and configuration. - -```bash -curl "http://localhost:9621/health" -``` - -## Development -Contribute to the project: [Guide](contributor-readme.MD) - -### Running in Development Mode - -For LoLLMs: -```bash -uvicorn lollms_lightrag_server:app --reload --port 9621 -``` - -For Ollama: -```bash -uvicorn ollama_lightrag_server:app --reload --port 9621 -``` - -For OpenAI: -```bash -uvicorn openai_lightrag_server:app --reload --port 9621 -``` -For Azure OpenAI: -```bash -uvicorn azure_openai_lightrag_server:app --reload --port 9621 -``` -### API Documentation - -When any server is running, visit: -- Swagger UI: http://localhost:9621/docs -- ReDoc: http://localhost:9621/redoc - -### Testing API Endpoints - -You can test the API endpoints using the provided curl commands or through the Swagger UI interface. Make sure to: -1. Start the appropriate backend service (LoLLMs, Ollama, or OpenAI) -2. Start the RAG server -3. Upload some documents using the document management endpoints -4. Query the system using the query endpoints - -### Important Features - -#### Automatic Document Vectorization -When starting any of the servers with the `--input-dir` parameter, the system will automatically: -1. Scan the specified directory for documents -2. Check for existing vectorized content in the database -3. Only vectorize new documents that aren't already in the database -4. Make all content immediately available for RAG queries - -This intelligent caching mechanism: -- Prevents unnecessary re-vectorization of existing documents -- Reduces startup time for subsequent runs -- Preserves system resources -- Maintains consistency across restarts - -**Important Notes:** -- The `--input-dir` parameter enables automatic document processing at startup -- Documents already in the database are not re-vectorized -- Only new documents in the input directory will be processed -- This optimization significantly reduces startup time for subsequent runs -- The working directory (`--working-dir`) stores the vectorized documents database