torchserve_on_win_native.md

TorchServe on Windows

Contents of this Document

• Prerequisites
• Install from binaries
• Install from source

Prerequisites

• At present, it has only been certified on windows server 2019 however should work fine on Windows 10.
• Make sure you are an admin user or have admin rights
• The instruction given here will use anaconda Powershell terminal to install torchserve
• Install Anaconda as given here
• Install Git as given here
• Install openjdk11
  • Download openjdk11
  • Unzip and edit/add environment variables i.e. PATH and JAVA_HOME e.g.
  • Using command line unzip openjdk-11*_bin.zip or using GUI interface
  • Edit system or user profile environment variable PATH value and append path <your-openjdk11-path>\bin to it

Install from binaries

NOTE At present, wheels for windows are not available on PyPi. However following steps can also be used if you have prebuilt torchserve wheel for windows.

• Start 'Anaconda Powershell Prompt' (APP) as Admin User i.e. By right click on APP and run following commands
• git clone https://github.com/pytorch/serve.git
• cd serve
• pip install -U -r requirements_cpu.txt
• For local wheel file
  • pip install <your-torchserve-wheel-file-name>.whl
• For PyPi package (N/A at present)
  • pip install torchserve torch-model-archiver
• Start torchserve torchserve.exe --start --model-store <path-to-model-store>
• For next steps refer Serving a model

Install from source

• Add system or user profile environment variable name JAVA_HOME and value as <your-openjdk11-path>

• Install Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019

  NOTE ensure that you have restarted system after install above Visual C++ components

• Start 'Anaconda Powershell Prompt' (APP) as Admin User i.e. By right click on APP and run following commands

• git clone https://github.com/pytorch/serve.git

• pip install click

• cd serve

• python .\scripts\install_from_src.py

• Refer Install torchserve for development

Troubleshooting

• If you are building from source then you may have to change the port number for inference, management and metrics apis as specified in frontend/server/src/test/resources/config.properties, all files in frontend/server/src/test/resources/snapshot/* and frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java
• If curl command fails to execute then run following command on APP (anaconda powershell promopt) Remove-item alias:curl Refer this SO answer for details.