[feature request ]Track individual objects

[Minglarn]

Hi!
First a BIG thanks for this HASS-Deepstack add-on...
Great work and keep up the good job!
Been using for a couple of days now and realized that the object detection works great but it reacts on same objects all the time.

Let's say that my parking lot have 0 car parked.
1 car arrives and parks next to the already parked car -> Object detected and sends a trigger with boundary box
Then 2 car arrives and parks next to the already parked car --> Object detected and sends a trigger with BOTH cars with boundary boxes on both cars.

Object detection reacts on BOTH cars which is kinda overkill as we already detected the first car earlier.

Same thing happens if a bird flies bye and triggers my camera. Now object detection triggers with 2 cars in snapshot where in reality it's not new objects.. There have been there all time..

I would like to have some kind of threshold value so it compares if there really are a NEW object. If possible. :)

PS: Sorry for my bad english...

[koldogut]

A quick solution maybe to control it with a helper setting the number of cars with the latest detection and comparing with the previous value.

[robmarkcole]

@Minglarn events are fired with all info for each object, from these you can craft any behaviour you desire. Can you refine the feature request?

[Minglarn]

A quick solution maybe to control it with a helper setting the number of cars with the latest detection and comparing with the previous value.

That could help but it would not be accurate.
If the object changes quickly (probably not) it will still count as 3 cars.
I could be thinking wrong about this (just not good at programming at the moment 😕)..

If every image with objects have a "score" value its would bee easier to se if the objects are new or the same.
With that score you could compare the "possibilities" for new objects.

[robmarkcole]

Each object is given a 'confidence', but for a given object this might change between captures (e.g. due to changing lighting) so cannot be relied on to identify an object. I understand that you basically want to track individual objects but this is currently outside the scope of this integration. Given the event data you can try out your own solutions

[Minglarn]

A quick solution maybe to control it with a helper setting the number of cars with the latest detection and comparing with the previous value.

Am pretty new with helpers (with HA actually). How would you accomplish a "comparison" helper?

[Minglarn]

Each object is given a 'confidence', but for a given object this might change between captures (e.g. due to changing lighting) so cannot be relied on to identify an object. I understand that you basically want to track individual objects but this is currently outside the scope of this integration. Given the event data you can try out your own solutions

Yes, you're right. It's kinda out of the scope :)
A possible solution would be a total score for the whole image?
Let's say that frame_1 have a score of 100
and frame_2 have a score of 92
The diff is 8.

Example:
frame_1 = 100
frame_2 = 92

if (frame_1 - frame_2) =< 10 then 
 ... do not send new notification. Frames are to equal
else
 ... send a notification. Frames are to not equal.

Hope I could explain this...

[robmarkcole]

@Minglarn thanks for putting thought into this. What would really assist the discussion is a demonstration of this algorithm on real data. I propose extracting a sequence of say 10 images, and applying this algorithm to them. The goal is to both demonstrate the algorithm and expose any edge cases early on. For this study you could make use of https://github.com/robmarkcole/deepstack-analytics

[koldogut]

A quick solution maybe to control it with a helper setting the number of cars with the latest detection and comparing with the previous value.

Am pretty new with helpers (with HA actually). How would you accomplish a "comparison" helper?

More or less this way:

1. • First, create a helper (Configuration - helpers - add - numeric helper)
2. • Create an automation based on event (deepstack.object_detected, object: car)
3. • Add a condition to be launched if new value is different from previous value using templating (find some examples in the forums)
4. • Add an action to do whatever you need (notification, call script, service, fire alarm, order a pizza...)
5. • Add an action at the end to update the helper with the new value (calling service input_number.set_value)

(Writen from handheld, please excuse any inaccuracy)

[pops106]

This might be quite helpful actually, I have a similar thing 2 cars on the drive hardly ever move.

I believe when the image processing happens and it draws the boxes it has a centre X for a coordinate, could we use that to say if it hasn't moved or say within 5% it is an already detected object.