Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Azure AI Vision from 3.2 to 4.0 #827

Open
1 task done
jeffpaul opened this issue Nov 21, 2024 · 2 comments
Open
1 task done

Update Azure AI Vision from 3.2 to 4.0 #827

jeffpaul opened this issue Nov 21, 2024 · 2 comments
Assignees
Labels
help wanted Extra attention is needed type:enhancement New feature or request.
Milestone

Comments

@jeffpaul
Copy link
Member

Is your enhancement related to a problem? Please describe.

Related to #826 and alongside adding OpenAI to alt text generation, let's look to update the Azure API version in ClassifAI.

While there are some differences from the features that are / are not available in 4.0 (which we'll want to validate ClassifAI features that are available in v4), let's look to update to 4.0 where that version includes coverage for specific ClassifAI image processing features using Azure.

Version 4.0 features available: Read text, Captions, Dense captions, Tags, Object detection, Custom image classification / object detection, People, Smart crop

Better models; use version 4.0 if it supports your use case.

Version 3.2 features available: Tags, Objects, Descriptions, Brands, Faces, Image type, Color scheme, Landmarks, Celebrities, Adult content, Smart crop

Wider range of features; use version 3.2 if your use case is not yet supported in version 4.0

Additional context from Azure:

We recommend you use the Image Analysis 4.0 API if it supports your use case. Use version 3.2 if your use case is not yet supported by 4.0.

You'll also need to use version 3.2 if you want to do image captioning and your Vision resource is outside the supported Azure regions. The image captioning feature in Image Analysis 4.0 is only supported in certain Azure regions. Image captioning in version 3.2 is available in all Azure AI Vision regions. See Region availability.

Designs

No response

Describe alternatives you've considered

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@jeffpaul jeffpaul added help wanted Extra attention is needed type:enhancement New feature or request. labels Nov 21, 2024
@jeffpaul jeffpaul added this to the 3.2.0 milestone Nov 21, 2024
@jeffpaul jeffpaul moved this from Incoming to To Do in Open Source Practice Nov 21, 2024
@dkotter
Copy link
Collaborator

dkotter commented Nov 21, 2024

Note a lot of research was done into this in #553, though some things have changed since then. From what I recall, the way the v4 API works is pretty different and will require changes to how we currently do things (seems there were asynchronous vs synchronous differences)

@dkotter dkotter self-assigned this Nov 22, 2024
@dkotter
Copy link
Collaborator

dkotter commented Nov 22, 2024

Status update (mostly to remind myself where I left off for next week):

Started work on this and have successfully migrated the Descriptive Text Generator, Image Tags Generator and Image Text Extraction Features over to the v4.0 API with no real challenges. Image Text Extraction required the most changes as we used to make two separate API requests and now that can be done in one.

Still left to fully look into are Image Cropping (which at a glance looks to be a fairly straight forward change) and PDF Text Extraction (which is a new API, Azure AI Document Intelligence, so will probably require more work)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed type:enhancement New feature or request.
Projects
Status: To Do
Development

No branches or pull requests

2 participants