Create a function to read and sort a list of urls of car splits / validate car file splits

[linuskendall]

1. HEAD request to get size
2. Read header
3. Figure out CID for subset
4. Read a chunk at the end to load the subset

This will give us:

• start and end slot
• plus ordering
• validation that the remote car file is what we expect
• validation that we have all the slots for the epoch

This can also be done when loading split car files from the file system so we can validate that we have every slot in the epoch.

[anjor]

@linuskendall just to confirm this is basically a url version of #166 right?

[linuskendall]

Yep!

…

On Thu, Oct 10, 2024 at 6:25 PM Anjor Kanekar ***@***.***> wrote: @linuskendall <https://github.com/linuskendall> just to confirm this is basically a url version of #166 <#166> right? — Reply to this email directly, view it on GitHub <#169 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHOYRNX3IG3QB2QGVJF76DZ2Z2LXAVCNFSM6AAAAABPUKMY3SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBVGAYTEOBQGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[anjor]

This is now done.