Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs for version compatibility #692

Merged
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions src/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,25 @@ It hides the complexity of the generation process and minimizes the amount of co

## Install OpenVINO™ GenAI

> **NOTE**: Please make sure that you are following the versions compatibility rules, refer to the [OpenVINO™ GenAI Dependencies](#openvino-genai-dependencies) for more information.

The OpenVINO™ GenAI flavor is available for installation via Archive and PyPI distributions.
To install OpenVINO™ GenAI, refer to the [Install Guide](https://docs.openvino.ai/2024/get-started/install-openvino.html).

To build OpenVINO™ GenAI library from source, refer to the [Build Instructions](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/2/src/docs/BUILD.md).
To build OpenVINO™ GenAI library from source, refer to the [Build Instructions](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/src/docs/BUILD.md).

### OpenVINO™ GenAI Dependencies

OpenVINO™ GenAI depends on [OpenVINO](https://github.com/openvinotoolkit/openvino) and [OpenVINO Tokenizers](https://github.com/openvinotoolkit/openvino_tokenizers).

When installing OpenVINO™ GenAI from PyPi, the same versions of OpenVINO and OpenVINO Tokenizers are used (e.g. for `openvino-genai==2024.3.0` will be used `openvino==2024.3.0` and `openvino-tokenizers==2024.3.0.0`).
Wovchena marked this conversation as resolved.
Show resolved Hide resolved
yatarkan marked this conversation as resolved.
Show resolved Hide resolved
If you update one of the dependency packages (e.g. install `openvino-nightly`), versions might be incompatible due to different ABI and running OpenVINO GenAI can result to errors (e.g. `ImportError: libopenvino.so.2440: cannot open shared object file: No such file or directory`).
yatarkan marked this conversation as resolved.
Show resolved Hide resolved

`openvino::runtime` exports `_GLIBCXX_USE_CXX11_ABI=0` on CentOS7. It needs to be propagated to every library openvino_tokenizers links with. That prohibits linkage with prebuilt libraries because they aren't compiled with `_GLIBCXX_USE_CXX11_ABI=0`. OpenVINO GenAI package on PyPi is prebuilt yet it is compiled with `_GLIBCXX_USE_CXX11_ABI=0`.
Mixing up packages from different distribution channels is not recommended due to compatibility issues.
yatarkan marked this conversation as resolved.
Show resolved Hide resolved

If you want to try OpenVINO GenAI with different dependencies versions (**not** prebuilt packages as archives or python wheels), build OpenVINO GenAI library from source.
Having packages version in format `<MAJOR>.<MINOR>.<PATCH>.<REVISION>`, only `<REVISION>` part of the full version can be varied to ensure ABI compatibility, while changing `<MAJOR>`, `<MINOR>` or `<PATCH>` parts of the version might break ABI.

## Usage

Expand Down Expand Up @@ -198,8 +213,8 @@ int main(int argc, char* argv[]) {

## How It Works

For information on how OpenVINO™ GenAI works, refer to the [How It Works Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/2/src/docs/HOW_IT_WORKS.md).
For information on how OpenVINO™ GenAI works, refer to the [How It Works Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/src/docs/HOW_IT_WORKS.md).

## Supported Models

For a list of supported models, refer to the [Supported Models Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/2/src/docs/SUPPORTED_MODELS.md).
For a list of supported models, refer to the [Supported Models Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/src/docs/SUPPORTED_MODELS.md).
123 changes: 86 additions & 37 deletions src/docs/BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,41 @@
> **NOTE**: There is a known Python API issue with `ov::Tensor`. The issue is reproduced when building OpenVINO GenAI from sources while using OpenVINO from archives. Using `ov::Tensor` with OpenVINO GenAI fails. Possible errors: `TypeError: generate(): incompatible function arguments.`, `TypeError: __init__(): incompatible constructor arguments.`, `TypeError: Unregistered type : ov::Tensor`.
The preferred approach is to build both OpenVINO and OpenVINO GenAI from sources using the same build environment. Or to install prebuilt OpenVINO GenAI from [distribution channels](https://docs.openvino.ai/2024/get-started/install-openvino.html).

## Build for Linux Systems
## Software Requirements

### Software Requirements
### Linux

- [CMake](https://cmake.org/download/) 3.23 or higher
- GCC 7.5 or higher
- Python 3.8 or higher

### Build Instructions
### Windows

- [CMake](https://cmake.org/download/) 3.23 or higher
- Microsoft Visual Studio 2019 or higher, version 16.3 or later
- Python 3.8 or higher
- Git for Windows
Wovchena marked this conversation as resolved.
Show resolved Hide resolved

### macOS

- [CMake](https://cmake.org/download/) 3.23 or higher
- [brew](https://brew.sh/) package manager to install additional dependencies:
```sh
brew install coreutils scons
```
- Clang compiler and other command line tools from Xcode 10.1 or higher:
```sh
xcode-select --install
```
- Python 3.8 or higher


## Build Instructions

### Build OpenVINO and OpenVINO GenAI from sources
yatarkan marked this conversation as resolved.
Show resolved Hide resolved

1. Build and install OpenVINO from sources following the [instructions](https://github.com/openvinotoolkit/openvino/wiki#how-to-build).
The path to the openvino install directory is referred as <INSTALL_DIR> throughout the document.
The path to the OpenVINO install directory is referred as `<INSTALL_DIR>` throughout the document.
2. Clone OpenVINO GenAI repository and init submodules:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
Expand All @@ -25,63 +48,89 @@ The path to the openvino install directory is referred as <INSTALL_DIR> througho
source <INSTALL_DIR>/setupvars.sh
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ --config Release --target package -j
yatarkan marked this conversation as resolved.
Show resolved Hide resolved
cmake --install ./build/ --config Release --prefix ov
cmake --install ./build/ --config Release --prefix <INSTALL_DIR>
```
> **NOTE**: For running setupvars script on Windows, use command `call <INSTALL_DIR>\setupvars.bat`
Wovchena marked this conversation as resolved.
Show resolved Hide resolved

## Build for Windows Systems

### Software Requirements
To optimize the package size, you can reduce the ICU (International Components for Unicode) data size when building OpenVINO Tokenizers.
For more information please refer to the [OpenVINO Tokenizers instructions](https://github.com/openvinotoolkit/openvino_tokenizers?tab=readme-ov-file#reducing-the-icu-data-size).

- [CMake](https://cmake.org/download/) 3.23 or higher
- Microsoft Visual Studio 2019 or higher, version 16.3 or later
- Python 3.8 or higher
- Git for Windows
### Build OpenVINO GenAI only
Wovchena marked this conversation as resolved.
Show resolved Hide resolved

### Build Instructions
Assuming that you have OpenVINO installed at `<INSTALL_DIR>`:

1. Build and install OpenVINO from sources following the [instructions](https://github.com/openvinotoolkit/openvino/wiki#how-to-build)
The path to the openvino install directory is referred as <INSTALL_DIR> throughout the document.
2. Clone OpenVINO GenAI repository and init submodules:
1. Clone OpenVINO GenAI repository and init submodules:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
```
3. Build the project:
2. Build the project:
```sh
call <INSTALL_DIR>\setupvars.bat
export OpenVINO_DIR=<INSTALL_DIR>/runtime
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ --config Release --target package -j
cmake --install ./build/ --config Release --prefix ov
```
3. Set environment variables:
```sh
export PYTHONPATH=<INSTALL_DIR>/python:./build/:${PYTHONPATH}
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:${LD_LIBRARY_PATH}
```

## Build for macOS Systems

### Software Requirements
### Build OpenVINO GenAI Wheel

- [CMake](https://cmake.org/download/) 3.23 or higher
- [brew](https://brew.sh/) package manager to install additional dependencies:
1. Clone OpenVINO GenAI repository and init submodules:
```sh
brew install coreutils scons
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
```
- Clang compiler and other command line tools from Xcode 10.1 or higher:
2. Set up the environment:
- Option 1 - using OpenVINO `setupvars.sh` script:
```sh
source <INSTALL_DIR>/setupvars.sh
```
- Option 2 - setting environment variable manually:
```sh
export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:${PYTHONPATH}
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:${LD_LIBRARY_PATH}
```
3. Upgrade pip to ensure you have the latest version:
```sh
xcode-select --install
python -m pip install --upgrade pip
```
4. Build the wheel in the `dist` directory:
```sh
python -m pip wheel . -w dist/ --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release
```
- Python 3.8 or higher

### Build Instructions
### Install OpenVINO GenAI From Source

1. Build and install OpenVINO from sources following the [instructions](https://github.com/openvinotoolkit/openvino/wiki#how-to-build)
The path to the openvino install directory is referred as <INSTALL_DIR> throughout the document.
2. Clone OpenVINO GenAI repository and init submodules:
1. Clone OpenVINO GenAI repository and init submodules:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
```
3. Build the project:
2. Set up the environment:
- Option 1 - using OpenVINO `setupvars.sh` script:
```sh
source <INSTALL_DIR>/setupvars.sh
```
- Option 2 - setting environment variable manually:
```sh
export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:${PYTHONPATH}
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:${LD_LIBRARY_PATH}
```
3. Upgrade pip to ensure you have the latest version:
```sh
source <INSTALL_DIR>/setupvars.sh
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ --config Release --target package -j
cmake --install ./build/ --config Release --prefix ov
python -m pip install --upgrade pip
```
4. Install the package directly from source:
```sh
python -m pip install .
```
5. To verify the installation, run a simple Python script:
```python
import openvino_genai
print(openvino_genai.__version__)
```
Loading