John Snow Labs releases LangTest 2.3.1: Critical Bug Fixes and Enhancements

[chakravarthik27]

Description

In this patch version, we've resolved several critical issues to enhance the functionality and bugs in the LangTest developed by JohnSnowLabs. Key fixes include correcting the NER task evaluation process to ensure that cases with empty expected results and non-empty predictions are appropriately flagged as failures. We've also addressed issues related to exceeding training dataset limits during test augmentation and uneven allocation of augmentation data across test cases. Enhancements include improved template generation using the OpenAI API, with added validation in the Pydantic model to ensure consistent and accurate outputs. Additionally, the integration of Azure OpenAI service for template-based augmentation has been initiated, and the issue with the Sphinx API documentation has been fixed to display the latest version correctly.

🐛 Fixes

• NER Task Evaluation Fixes:
  • Fixed an issue where NER evaluations passed incorrectly when expected results were empty, but actual results contained predictions. This should have failed. [chore: reapply transformations to NER task after importing test cases #1076]
  • Fixed an issue where NER predictions had differing lengths between expected and actual results. [chore: reapply transformations to NER task after importing test cases #1076]
• API Documentation Link Broken:
  • Fixed an issue where Sphinx API documentation wasn't showing the latest version docs. [updated the python api documentation with sphinx #1077]
• Training Dataset Limit Issue:
  • Fixed the issue where the maximum limit set on the training dataset was exceeded during test augmentation allocation. [fixed: consistent issues in augmentation #1085]
• Augmentation Data Allocation:
  • Fixed the uneven allocation of augmentation data, which resulted in some test cases not undergoing any transformations. [fixed: consistent issues in augmentation #1085]
• DataAugmenter Class Issues:
  • Fixed issues where export types were not functioning as expected after data augmentation. [fixed: consistent issues in augmentation #1085]
• Template Generation with OpenAI API:
  • Resolved issues with OpenAI API when generating different templates from user-provided ones, which led to invalid outputs like paragraphs or incorrect JSON. Implemented structured outputs to resolve this. [fixed: consistent issues in augmentation #1085]

⚡ Enhancements

• Pydantic Model Enhancements:
  • Added validation steps in the Pydantic model to ensure templates are generated as required. [fixed: consistent issues in augmentation #1085]
• Azure OpenAI Service Integration:
  • Implemented the template-based augmentation using Azure OpenAI service. [resolved/augmentation errors  #1090]
• Text Classification Support:
  • Support for multi-label classification in text classification tasks is added. [Feature/add support for the multi label classification model #1096]
• Data Augmentation:
  • Add JSON Output for NER Sample to Support Generative AI Lab[Add JSON Output for NER Sample to Support Generative AI Lab #1099][chore: update DataAugmenter to support generating JSON output in GEN AI LAB #1100]

What's Changed

• chore: reapply transformations to NER task after importing test cases by @chakravarthik27 in chore: reapply transformations to NER task after importing test cases #1076
• updated the python api documentation with sphinx by @chakravarthik27 in updated the python api documentation with sphinx #1077
• Patch/2.3.1 by @chakravarthik27 in Patch/2.3.1 #1078
• Bug/ner evaluation fix in is_pass() by @chakravarthik27 in Bug/ner evaluation fix in is_pass() #1080
• resolved: recovering the transformation object. by @chakravarthik27 in resolved: recovering the transformation object. #1081
• fixed: consistent issues in augmentation by @chakravarthik27 in fixed: consistent issues in augmentation #1085
• Chore: Add Option to Configure Number of Generated Templates in Templatic Augmentation by @chakravarthik27 in Chore: Add Option to Configure Number of Generated Templates in Templatic Augmentation #1089
• resolved/augmentation errors by @chakravarthik27 in resolved/augmentation errors  #1090
• Fix/augmentations by @chakravarthik27 in Fix/augmentations #1091
• Feature/add support for the multi label classification model by @chakravarthik27 in Feature/add support for the multi label classification model #1096
• Patch/2.3.1 by @chakravarthik27 in Patch/2.3.1 #1097
• chore: update pyproject.toml version to 2.3.1 by @chakravarthik27 in chore: update pyproject.toml version to 2.3.1 #1098
• chore: update DataAugmenter to support generating JSON output in GEN AI LAB by @chakravarthik27 in chore: update DataAugmenter to support generating JSON output in GEN AI LAB #1100
• Patch/2.3.1 by @chakravarthik27 in Patch/2.3.1 #1101
• implemented: basic version to handling document wise. by @chakravarthik27 in implemented: basic version to handling document wise. #1094
• Fix/module error with openai package by @chakravarthik27 in Fix/module error with openai package #1102
• Patch/2.3.1 by @chakravarthik27 in Patch/2.3.1 #1103

Full Changelog: 2.3.0...2.3.1

---

This discussion was created from the release John Snow Labs releases LangTest 2.3.1: Critical Bug Fixes and Enhancements.