Figure out how to deal with user deletions with multiple flavours

[john-gom]

Just thinking about user deletions and the different off flavours. At the moment if we delete a user then a minion job will be created for the particular flavour which will anonymise contributions for that flavour, but presumably it won't deal with other flavours. With Keycloak we generate a Redis event when the user is deleted which will then trigger the minion job. We could get all flavours to listen to this event, but they will each generate a different anonymous user id for that user's contributions. I'm thinking that we should maybe switch to generating the anonymous id in Keycloak so that it is included in the Redis event - that way all flavours can receive the event and redact their own products using the same user id

Alternatively we have one Redis listener which iterates over all of the MongoDB databases to anonymise contributions

[hangy]

MongoDB databases

That would have to be the product directories with .sto files, though. The MongoDB databases are just derivates of the sto files, IIRC.

I'm thinking that we should maybe switch to generating the anonymous id in Keycloak so that it is included in the Redis event

Sounds like a good idea!

[john-gom]

Based on discussions in the Product Owner meeting we will have each flavour deal with their own updates. This means that each flavour / product type will have their own Minion queue and be listening for Redis events. We will need to make sure that only one process is dealing with global events, like user registration and the deletion of the user.sto file on user deletion.

[john-gom]

I have committed a suggested change to the Keycloak branch: openfoodfacts/openfoodfacts-server@d14c9d1

@stephanegigandet , @alexgarel , @hangy

Note I still need to make the change to pass in the new anonymous user id from Keycloak so that all flavours use the same anonymous id when redacting edits for the deleted user.

[hangy]

Note I still need to make the change to pass in the new anonymous user id from Keycloak so that all flavours use the same anonymous id when redacting edits for the deleted user.

9b6f9eb might help with that

[john-gom]

Thanks @hangy . I'd already started on that too. I'll pull your changes

[john-gom]

@hangy , I've extended it to save the new user id in the deleted_user table. Not 100% on the way I have done the mocking for the tests if you could maybe take a look at that

[alexgarel]

I have committed a suggested change to the Keycloak branch: openfoodfacts/openfoodfacts-server@d14c9d1

@john-gom I would like more comments in _read_user_streams because it's really not obvious to understand 😬