diff --git a/README.md b/README.md index d3ef268..c4ba4fd 100644 --- a/README.md +++ b/README.md @@ -2,39 +2,47 @@ ## Description -API to expose LLM backends via FastAPI and gRPC. +A Python API that exposes LLM backends, via FastAPI and gRPC, in the OpenAI API specification. + +## Usage + +See [instructions](#instructions) to get the API up and running. Then, go to http://localhost:8080/docs for the Swagger documentation on API usage. ## Instructions -Choose a LeapfrogAI model backend and get that running separately. Conisder: - * https://github.com/defenseunicorns/leapfrogai-backend-ctransformers - * https://github.com/defenseunicorns/leapfrogai-backend-whisper +The instructions in this section assume the following: + +1. Properly installed and configured Python 3.11.x, to include its development tools +2. The `config.yaml` is created based on the `config-example.yaml` +3. You have chosen a LeapfrogAI model backend and have that running. Some examples of existing backends: -#### Run Locally +- https://github.com/defenseunicorns/leapfrogai-backend-ctransformers +- https://github.com/defenseunicorns/leapfrogai-backend-whisper -1. Setup `config.yaml`, see `config-example.yaml` for common examples. +### Local Development -2. Setup and run the API +For cloning a model locally and running the development backend. ```bash # Setup Python Virtual Environment -python -m venv .venv -source .venv/bin/activate +make create-venv +make activate-venv make requirements-dev # Start Model Backend make dev ``` -### Docker Run +### Docker Container + +#### Image Build and Run + +For local image building and running. ```bash # Build the docker image docker build -t ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest . # Run the docker container -docker run -p 8080:8080 ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest +docker run -p 8080:8080 -v ./config.yaml:/leapfrogai/config.yaml ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest ``` - -1. Create `config.yaml`, see `config-example.yaml` for common examples. -2. Copy `config.yaml` into the work directory of the container while it is running.