How to create a MixedMemory class that combines preloaded and new experiences in skrl?

[Ginga-Kennis]

Hello,

I'm working on a reinforcement learning project using the skrl library, and I need some help customizing the replay buffer.

I want to create a MixedMemory class that does the following:

• Loads pre-saved experiences (states, actions, rewards) from existing data.

• Uses half of the memory capacity for these preloaded experiences.

• Uses the other half for new experiences collected during interaction with the environment (simulation).

I found the CustomMemory class in the skrl documentation as a starting point．
However, I'm unsure how to modify this class to implement the MixedMemory functionality as described. Specifically, I need guidance on:

Loading pre-saved experiences into the memory.
Properly sampling from both halves during training.
Could someone please provide an example or point me in the right direction on how to achieve this?

Thank you in advance for your help!

[Toni-SM]

Hi @Ginga-Kennis

This is a good feature that, if I have time I am going to include in next release... Or you can just open a new PR with the implementation 😅

Meanwhile, the solution is to play with the memory_index property of the Memory base class and define the default value the memory index will be assigned (currently 0, which means start overwriting from the beginning) when maximum capacity has been reached

https://github.com/Toni-SM/skrl/blob/main/skrl%2Fmemories%2Ftorch%2Fbase.py#L305