Storehouse is taking more than 10min to load due to its internal cache

[italovalcy]

As part of the storehouse loading process, it creates a in-memory cache for all boxes and all namespaces stored (https://github.com/kytos-ng/storehouse/blob/master/main.py#L99-L110). This is done inline into the loading process, which depending on the amount of data stored, can lead to a long delay to boot up Kytos.

This problem became evident after the introduction of the AmLight Napp FlowStats (https://github.com/amlight/flow_stats/blob/master/main.py#L596-L603).

In a scenario with 300 EVCs (created using mef_eline), 6 switches, and 12 links, we had more than 160k boxes:

find /var/tmp/kytos/storehouse/ -type f | wc -l
164727

The lead Kytos to delay its startup process by 12min:

2021-11-11 14:37:15,318 - INFO [kytos.core.logs] (MainThread) Logging config file "/etc/kytos/logging.ini" loaded successfully.
2021-11-11 14:37:15,319 - INFO [kytos.core.controller] (MainThread) /var/run/kytos
2021-11-11 14:37:15,319 - INFO [kytos.core.controller] (MainThread) Starting Kytos - Kytos Controller
2021-11-11 14:37:15,319 - INFO [kytos.core.controller] (MainThread) Starting TCP server: <kytos.core.atcp_server.KytosServer object at 0x7f3b10e56278>
2021-11-11 14:37:15,320 - INFO [kytos.core.atcp_server] (MainThread) Kytos listening at 0.0.0.0:6653
2021-11-11 14:37:15,320 - INFO [kytos.core.controller] (MainThread) ThreadPool started: <concurrent.futures.thread.ThreadPoolExecutor object at 0x7f3b11c7c940>
2021-11-11 14:37:15,320 - INFO [kytos.core.controller] (MainThread) Loading Kytos NApps...
2021-11-11 14:37:15,322 - INFO [kytos.core.napps.napp_dir_listener] (MainThread) NAppDirListener Started...
2021-11-11 14:37:15,326 - INFO [kytos.core.controller] (MainThread) Loading NApp kytos/storehouse
2021-11-11 14:37:15,414 - INFO [kytos.napps.kytos/storehouse] (MainThread) Loading 'filesystem' backend...
## --->>> long delay
2021-11-11 14:49:33,500 - INFO [kytos.napps.kytos/storehouse] (MainThread) Storehouse NApp started.

[viniarck]

For future readers, I'm setting this as wontfix since this NApp has been deprecated see PR #15