As a dev, I want to be able to correctly analyze cylindrical containers.

[SamuelPelletierEvraire]

Description

The app currently fails to return correct data when analyzing round labels on cylindrical containers. This significantly affects the user experience as it leads to incomplete or missing data, which hinders the functionality of the app.

Bug Summary:
The app doesn't return the correct answer when analyzing round labels.

Steps to Reproduce:

1. Login into the app using valid credentials.
2. Navigate to the home page where the analysis functionality is available.
3. Attempt to add images of cylindrical containers with round labels for analysis (specific images will be added later).
4. Wait for the analysis process to complete.

Expected Behavior:

I expect the app to return data in the correct format, similar to how it processes other types of labels. The response should include all relevant information extracted from the labels and should be displayed on the frontend without any issues.

Actual Behavior:
In almost all occurrences, the backend fails to return bad response when analyzing round labels on cylindrical containers.

Environment:
Windows 10

Commit Hash:

OS Version: Windows 10

Browser Version: Chrome Version: 129.0.6668.59 (Build officiel) (64 bits)

Logs and Screenshots:
Not able to replicate as the 2024-09-24 the extraction work a little bit more than before.
Follow this link to access the data folder

Additional Context:
The issue might be related to how the app’s image processing algorithm handles non-flat surfaces. Cylindrical surfaces can present unique challenges due to perspective distortion and varying lighting conditions. Insights into how the app currently processes images can help pinpoint the exact point of failure.

Suggested Fix:

1. Implement User-Guidance System for Taking Photos:

   • Add a new page or overlay within the app that provides real-time feedback while taking photos. Use animations and guides similar to those in banking apps for check deposits.
   • Include instructions for optimal image capture conditions such as adequate lighting, proper distance from the object, and holding the device steady.
   • Develop an algorithm to detect cylinder edges and provide a frame to guide the user on aligning the label within the frame.

2. Post-Image Analysis with Blur Test System:

   • Develop a post-capture image quality assessment system. This system should analyze the image for parameters like sharpness, lighting, and focus before it’s sent to the backend for label analysis.
   • Implement a blur detection algorithm to ensure the image quality meets minimum standards. If an image is too blurry, return feedback to the user to retake the photo.
   • Incorporate machine learning techniques to enhance the detection and correction of image quality issues dynamically.

3. Enhance Image Processing Pipeline:

   • Review and refine the current image processing algorithms. Focus on the transformation techniques to flatten the curved surfaces of cylindrical objects for better label recognition.
   • Experiment with advanced image processing methods such as photogrammetry or 3D reconstruction to correct perspective distortions.
   • Conduct thorough testing under various conditions (e.g., different lighting scenarios, background complexities) to ensure robustness of the solution.

Here a one drive folder containing round label to test with:
https://136gc.sharepoint.com/:f:/s/CFIAAILabTeam-harvester/EngWgILnIJFGs5Ehabe0m_sBgxly265A2L9rTqLScGcnTA?e=dmhjYP

[SamuelPelletierEvraire]

Wasn't working during manual extraction but now seem to work way more but I created the issue to talk about the user-guidance into the app.

[Francois-Werbrouck]

This should be a as a user issue since it is the user who takes the picture of items 🖼️

[snakedye]

I also think this is moreso of a front-end issue issue. I think the solutions to enhance the readability of the image should be applied before the image even reaches the pipeline.

[k-allagbe]

Transferring to fertiscan-frontend.