TTObjectBase: problem of refcounting that never reach 0 when passing our self as observer

[theod]

First email from @theod :

I've managed to find the reason why some TTObject instances was still alive after the deletion of a patcher.
the reason is not related to a bug of Foundation but more to a user case we maybe didn't figured out until now.

Many Modular TTObjects use to pass themself into a baton of a callback for directory observations.
For example in TTContainer class, when we bind on an address (in order to watch all the content below), we do :

mObserver = TTObject("callback");
baton = TTValue(TTObject(this), aContext);
mObserver.set(kTTSym_baton, baton);
mObserver.set(kTTSym_function, TTPtr(&TTContainerDirectoryCallback));

accessApplicationLocalDirectory->addObserverForNotifications(mAddress, mObserver); // ask for notification for addresses below

But the problem is that until now I used to remove the observation into the ~TTContainer function which was never called as there is still an instance refered by the observer baton …

A solution would be to call a method to clear this observation mechanism but this sounds not user friendly to have to call something before deleting things.
Another solution would be to register this as a pointer into the baton with the risk to have unconsistant refcounting.

What do you think ?
is this a typical case you already resolved by some magic tricks ?

[theod]

answer from @tap :

Did you ever do anything with regards to this pattern? It is common in e.g. Max objects to detach from observing in an objects free method. However it is also common that when you have attached to object to observe it that you also pay attention and get the "free" notification from that object so know to stop observing it. Maybe this doesn't help, or maybe we need to make a change so this is all much easier?

[theod]

then @theod said :

not sure if there is a solution to this problem … maybe it's like a dog chasing its tail as, if I understand well, the free notification will be sent when no more instance is referenced but there is still one reference stored into the observer that the object manage itself … so the only solution I've found is to have a way to clear observers myself before.

maybe we can introduce the notion of "main" instance so that when the TTObject handling the "main" instance is deleted it tells the other TTObjects refering to none "main" instances to forget … but it sounds like opening hell doors no ?

[theod]

and finally @tap said :

This seems like a pretty important issue to get straight. Before trying to put a bandage on it I'd like to make certain we understand it thoroughly and have that documented. I think the best way to do this is have unit tests for all of the refcounting scenarios we envisage ourselves using. I think there is a basic one already which seems to pass. So we can extend that.

Beyond that I'm probably going to need a picture with arrows showing the flow of events going on. Might be something we can create using some web app during a Skype call?

[tap]

This is highly relevant to #353. We need to rewrite this to use std::shared_ptr and std::weak_pointer -- and to eliminate much of our pointer use in general.