Skip to content

Commit

Permalink
use c++ style of map initialization
Browse files Browse the repository at this point in the history
  • Loading branch information
pavel-esir committed Sep 26, 2024
1 parent f8b7d03 commit 33f2b3c
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/cpp/include/openvino/genai/tokenizer.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,15 @@ class OPENVINO_GENAI_EXPORTS Tokenizer {
/**
* @brief encode a single prompt
* @param prompt std::string with input prompt
* @param tokenization_params AnyMap with tokenization parameters, e.g. {'add_special_tokens': true}
* @param tokenization_params AnyMap with tokenization parameters, e.g. {'add_special_tokens', false}
* @return pair of [input_ids, attention_mask]
*/
TokenizedInputs encode(const std::string prompt, const ov::AnyMap& tokenization_params = {});

/**
* @brief encode batch of prompts. Left padding will be applied by default
* @param prompts vector storing batch of prompts
* @param tokenization_params AnyMap with tokenization parameters, e.g. {'add_special_tokens': false}
* @param tokenization_params AnyMap with tokenization parameters, e.g. {'add_special_tokens', false}
* @return pair of [input_ids, attention_mask]
*/
TokenizedInputs encode(std::vector<std::string>& prompt, const ov::AnyMap& tokenization_params = {});
Expand Down

0 comments on commit 33f2b3c

Please sign in to comment.