From 4e5f7c4b4b96ded25b8ab8cd4834cfa0d6b626c0 Mon Sep 17 00:00:00 2001 From: justinthelaw Date: Fri, 8 Dec 2023 10:37:41 -0500 Subject: [PATCH 1/4] better README instructions and descriptions --- README.md | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index d3ef268..6a81301 100644 --- a/README.md +++ b/README.md @@ -2,19 +2,24 @@ ## Description -API to expose LLM backends via FastAPI and gRPC. +A Python API that exposes LLM backends, via FastAPI and gRPC, in the OpenAI API specification. + +## Usage + +See [instructions](#instructions) to get the API up and running. Then, go to http://localhost:8080/docs for the Swagger documentation on API usage. ## Instructions -Choose a LeapfrogAI model backend and get that running separately. Conisder: - * https://github.com/defenseunicorns/leapfrogai-backend-ctransformers - * https://github.com/defenseunicorns/leapfrogai-backend-whisper +Choose a LeapfrogAI model backend and get that running separately. Some examples of existing backends: + +- https://github.com/defenseunicorns/leapfrogai-backend-ctransformers +- https://github.com/defenseunicorns/leapfrogai-backend-whisper #### Run Locally -1. Setup `config.yaml`, see `config-example.yaml` for common examples. +1. Create `config.yaml`, see `config-example.yaml` for common examples. -2. Setup and run the API +2. Setup and run the API: ```bash # Setup Python Virtual Environment @@ -28,13 +33,14 @@ make dev ### Docker Run +1. Create `config.yaml`, see `config-example.yaml` for common examples. + +2. Copy `config.yaml` into the container during the run command as a volume: + ```bash # Build the docker image docker build -t ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest . # Run the docker container -docker run -p 8080:8080 ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest +docker run -p 8080:8080 -v ./config.yaml:/leapfrogai/config.yaml ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest ``` - -1. Create `config.yaml`, see `config-example.yaml` for common examples. -2. Copy `config.yaml` into the work directory of the container while it is running. From eec129e4f51f50d1367a57d725f102bda540aee6 Mon Sep 17 00:00:00 2001 From: justinthelaw Date: Fri, 8 Dec 2023 17:13:42 -0500 Subject: [PATCH 2/4] fixed up README and Makefile --- Makefile | 2 +- README.md | 21 +++++++++++++++------ 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index 988d3c7..5293f2e 100644 --- a/Makefile +++ b/Makefile @@ -10,7 +10,7 @@ requirements-dev: python -m pip install -r requirements-dev.txt requirements: - pip-sync requirements.txt requirements-dev.txt + python -m pip install -r requirements.txt build-requirements: pip-compile -o requirements.txt pyproject.toml diff --git a/README.md b/README.md index 6a81301..e88f88e 100644 --- a/README.md +++ b/README.md @@ -10,12 +10,17 @@ See [instructions](#instructions) to get the API up and running. Then, go to htt ## Instructions -Choose a LeapfrogAI model backend and get that running separately. Some examples of existing backends: +The instructions in this section assume the following: + +1. You have properly installed and configured Python 3.11.x, to include its development tools +2. You have chosen a LeapfrogAI model backend and have that running. Some examples of existing backends: - https://github.com/defenseunicorns/leapfrogai-backend-ctransformers - https://github.com/defenseunicorns/leapfrogai-backend-whisper -#### Run Locally +### Local Development + +For cloning a model locally and running the development backend. 1. Create `config.yaml`, see `config-example.yaml` for common examples. @@ -23,15 +28,19 @@ Choose a LeapfrogAI model backend and get that running separately. Some examples ```bash # Setup Python Virtual Environment -python -m venv .venv -source .venv/bin/activate +make create-venv +make activate-venv make requirements-dev # Start Model Backend make dev ``` -### Docker Run +### Docker Container + +#### Image Build and Run + +For local image building and running. 1. Create `config.yaml`, see `config-example.yaml` for common examples. @@ -43,4 +52,4 @@ docker build -t ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest . # Run the docker container docker run -p 8080:8080 -v ./config.yaml:/leapfrogai/config.yaml ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest -``` +``` \ No newline at end of file From 777ea6b75035f65e60c35563b557aa6fbf2dff6b Mon Sep 17 00:00:00 2001 From: justinthelaw Date: Fri, 8 Dec 2023 17:57:24 -0500 Subject: [PATCH 3/4] move config.yaml to assumptions at top --- README.md | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index e88f88e..c4ba4fd 100644 --- a/README.md +++ b/README.md @@ -12,8 +12,9 @@ See [instructions](#instructions) to get the API up and running. Then, go to htt The instructions in this section assume the following: -1. You have properly installed and configured Python 3.11.x, to include its development tools -2. You have chosen a LeapfrogAI model backend and have that running. Some examples of existing backends: +1. Properly installed and configured Python 3.11.x, to include its development tools +2. The `config.yaml` is created based on the `config-example.yaml` +3. You have chosen a LeapfrogAI model backend and have that running. Some examples of existing backends: - https://github.com/defenseunicorns/leapfrogai-backend-ctransformers - https://github.com/defenseunicorns/leapfrogai-backend-whisper @@ -22,10 +23,6 @@ The instructions in this section assume the following: For cloning a model locally and running the development backend. -1. Create `config.yaml`, see `config-example.yaml` for common examples. - -2. Setup and run the API: - ```bash # Setup Python Virtual Environment make create-venv @@ -42,14 +39,10 @@ make dev For local image building and running. -1. Create `config.yaml`, see `config-example.yaml` for common examples. - -2. Copy `config.yaml` into the container during the run command as a volume: - ```bash # Build the docker image docker build -t ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest . # Run the docker container docker run -p 8080:8080 -v ./config.yaml:/leapfrogai/config.yaml ghcr.io/defenseunicorns/leapfrogai/leapfrogai-api:latest -``` \ No newline at end of file +``` From df9476e9a5118f9bbfd680a6c44ba39acbd35bc1 Mon Sep 17 00:00:00 2001 From: justinthelaw Date: Wed, 13 Dec 2023 10:41:03 -0500 Subject: [PATCH 4/4] reintroduced pip-sync --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 5293f2e..988d3c7 100644 --- a/Makefile +++ b/Makefile @@ -10,7 +10,7 @@ requirements-dev: python -m pip install -r requirements-dev.txt requirements: - python -m pip install -r requirements.txt + pip-sync requirements.txt requirements-dev.txt build-requirements: pip-compile -o requirements.txt pyproject.toml