1. Prerequisites
- Python 3.13.x must be installed - download
- LM Studio v0.3.5 must be installed - release notes
2. Check that Python has been installed correctly
- Execute following command
python --version
Output should look like below
Python 3.13.0
3. Setting up the LMStudio
In order to use the LMStudio as a server navigate to Developer Mode.
Set up the port to 8080 (or other - in case that 8080 is used)
4. Check that LMStudio can be run from commandline
lms
__ __ ___ ______ ___ _______ ____
/ / / |/ / / __/ /___ _____/ (_)__ / ___/ / / _/
/ /__/ /|_/ / _\ \/ __/ // / _ / / _ \ / /__/ /___/ /
/____/_/ /_/ /___/\__/\_,_/\_,_/_/\___/ \___/____/___/
lms - LM Studio CLI - v0.0.28
GitHub: https://github.com/lmstudio-ai/lmstudio-cli
Usage
lms <subcommand>
where <subcommand> can be one of:
- status - Prints the status of LM Studio
- server - Commands for managing the local server
- ls - List all downloaded models
- ps - List all loaded models
- get - Searching and downloading a model from online.
- load - Load a model
- unload - Unload a model
- create - Create a new project with scaffolding
- log - Log operations. Currently only supports streaming logs from LM Studio via `lms log stream`
- import - Import a model file into LM Studio
- bootstrap - Bootstrap the CLI
- version - Prints the version of the CLI
For more help, try running `lms <subcommand> --help`
5. Clone the repository using git clone command:
git clone https://github.com/pwalaszkowski/msc_seminarium.git
$ git clone https://github.com/pwalaszkowski/msc_seminarium.git
Cloning into 'msc_seminarium'...
remote: Enumerating objects: 51, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (35/35), done.
remote: Total 51 (delta 22), reused 41 (delta 12), pack-reused 0 (from 0)
Receiving objects: 100% (51/51), 41.93 KiB | 1.50 MiB/s, done.
Resolving deltas: 100% (22/22), done.
or download
https://github.com/pwalaszkowski/msc_seminarium/archive/refs/heads/master.zip
6. Navigate to repository directory
1. Setup Python venv in current working directory
python -m venv env
2. Confirm that env is created (the directory env should exist)
$ ls -la env/
total 14
drwxr-xr-x 1 pwala 197609 0 Dec 17 18:35 ./
drwxr-xr-x 1 pwala 197609 0 Dec 17 18:35 ../
-rw-r--r-- 1 pwala 197609 71 Dec 17 18:35 .gitignore
drwxr-xr-x 1 pwala 197609 0 Dec 17 18:35 Include/
drwxr-xr-x 1 pwala 197609 0 Dec 17 18:35 Lib/
drwxr-xr-x 1 pwala 197609 0 Dec 17 18:35 Scripts/
-rw-r--r-- 1 pwala 197609 337 Dec 17 18:35 pyvenv.cfg
3. Navigate to directory env/Scripts
Activate virtual environment running the activate script
./activate.bat
Under Linux machine the script will be called
./activate
4. The user prompt should now start with (env)
(env) C:\Repositories\
7. Download required packages via pip
Navigate to main directory (msc_seminarium)
Confirm that you are in main directory
python -m pip install -r requirements.txt
1. Open LMStudio (do not run any prompts or download models) 2. Navigate to config.ini file to set the model, prompt and model paths under Windows
[DEFAULT]
MODEL = qwen2-0.5b-instruct
PROMPT = 'Create me a for loop in Python'
MODEL_PATH = C:\Users\pwala\.cache\lm-studio\models
- Run the lmstudio_wrapper.py
The wrapper will
python ./lmstudio_wrapper.py
- launch the server
- Download model
- Load the model into LMStudio
- Send prompt to LMStudio
- Return answer on console
- Write the answer into
output
directory with following formatMODEL_YYYY_MM_DD_HH_MM_SS
- The wrapper will eject the model
- The wrapper will remove the downloaded models
1. Navigate to directory env/Scripts
. Deactivate virtual environment running the deactivate script
./deactivate.bat
Under Linux machine the script will be called
./deactivate
2. The user prompt should now no longer start with (env)
C:\Repositories\
https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct