You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #94 we discussed features that should be ported over from Cineast. One of these features was the VisualTextCoEmbedding feature that can be used to query image and video content using textual queries.
While the current implementation is arguably worse than OpenCLIP, is has one advantage: I uses information not just from the keyframe but also surrounding frames and may therefore spot things, other models might miss.
The goal of this issue is therefore to migrate the VisualTextCoEmbedding feature so that it can be used until we have a better alternative.
Dependencies
In light of #19, we ought to check, whether this should be part of the FES infrastructure or TorchServe.
Boundary Conditions
None
The text was updated successfully, but these errors were encountered:
Task Description
In #94 we discussed features that should be ported over from Cineast. One of these features was the
VisualTextCoEmbedding
feature that can be used to query image and video content using textual queries.While the current implementation is arguably worse than OpenCLIP, is has one advantage: I uses information not just from the keyframe but also surrounding frames and may therefore spot things, other models might miss.
The goal of this issue is therefore to migrate the
VisualTextCoEmbedding
feature so that it can be used until we have a better alternative.Dependencies
In light of #19, we ought to check, whether this should be part of the FES infrastructure or TorchServe.
Boundary Conditions
None
The text was updated successfully, but these errors were encountered: