Lower memory usage for ICL

[dakinggg]

What does this PR do?

What issue(s) does this change relate to?

Before submitting

• Have you read the contributor guidelines?
• Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
• Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
• Did you update any related docs and document your change?
• Did you update any related tests and add any new tests related to your change? (see testing)
• Did you run the tests locally to make sure they pass?
• Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

[bmosaicml]

Do you have any idea how this is affecting the overall run time? Writing to/from a cache file may be slow.

One alternative design I was considering would be writing the encoded_dataset to instead be a generator pointing to a prep_example function that uses a yield statement to lazily execute. Not sure if this would work with the rest of composer though since it seems datasets require support of indexing?

[dakinggg]

Closing in favor of #2713

[dakinggg]

Closing in favor of #2713