Issue with timestamps on GEDI plugin

[gravinamike]

I was attempting to get data through the GEDI plugin, based on the example notebook here. The resulting timestamps, however, are clearly incorrect. The first timestamp in the resulting GeoDataFrame is 2124-08-12 02:11:06.989444730, and the rest of the timestamps are also far-future.

I'm not sure what's causing this issue! I'll post my full code here for reference:

from sliderule import gedi

region = [
    {"lat": 40.615989, "lon": -97.795012},
    {"lat": 40.615989, "lon": -97.695012}, 
    {"lat": 40.515989, "lon": -97.695012},
    {"lat": 40.515989, "lon": -97.795012},
]

parms = {
    "poly": region,
    "degrade_flag": 0,
    "l2_quality_flag": 1,
    "beam": 0
}

gedi.init("slideruleearth.io", verbose=True)

gdf = gedi.gedi04ap(parms)

gdf.head()

[jpswinski]

@gravinamike thanks for reaching out and posting this issue. I took a look into what you are seeing and it looks like you've found a bug in the server-side code. I've created a GitHub issue for this: #218 where you can track our progress on fixing this issue.

Just as a heads up - the GEDI plugin for SlideRule is currently in a feature-preview state and is not yet officially supported. We are targeting a mid-April release of the first version of SlideRule that will officially support GEDI APIs. Please feel free to continue using the GEDI APIs, but there may be other gotchas that you will find.

Once officially supported, you will see GEDI documentation appear on our documentation website (https://slideruleearth.io/rtd). You can also check the latest release notes for any changes in GEDI functionality here: https://slideruleearth.io/rtd/release_notes/release_notes.html. The release notes should include a list of new GEDI features and accompanying resources to help you use them. Also, feel free to reach out directly through a discussion if you have any other questions.

[gravinamike]

Thanks for the info, @jpswinski . I figured it was something along those lines. We'll be watching for updates in the coming months!

[jpswinski]

@gravinamike Our latest release (v3.1.0) should resolve the issues with the GEDI timestamps. Thanks again for bringing it to our attention. With this release, we now support subsetting GEDI L1B, L2A, and L4A, and we support sampling L3 and L4B. Here is a link to our documentation: https://slideruleearth.io/web/rtd/user_guide/GEDI.html. For some examples on how to use the access the data, please see the examples in this repo (they are probably the best place to start anyway).

Lastly, note that the LP DAAC has not yet pushed all of the L1 and L2 data to the cloud, so we have staged a very small number of granules that intersect with the Grand Mesa region in Colorado just to demonstrate our capability to subset the data. Once the LP DAAC uploads all of the data to S3, we will have access to all of it.

[gravinamike]

This is fantastic. Thank you so much for implementing this and for notifying me - we'll definitely be making good use of this!

[gravinamike]

Hello again - I know that the LP DAAC is still pushing L1 and L2 data to the cloud. Do you happen to know when that is expected to be complete? Thanks again for any info you can provide!

[jpswinski]

Unfortunately, we also don't know when they are going to be done.

[gravinamike]

I figured that might be the case. Good to know - thank you!

[dshean]

Hey @gravinamike. Do you have priority study site(s) you are using for testing? We can potentially stage relevant GEDI granules in our S3 bucket to enable further testing until the full archive is available on S3 via LPDAAC. Let us know!

[gravinamike]

Thank you for getting back to me about this, David! Yes, we do have priority sites. Specifically, we're looking at 6 meteor craters across the globe. If granules could be staged for a roughly 100 x 100 km square around each site, that would take care of our needs. These are the coordinates our sites are centered on: * Bigach 48° 36' N, 82° 0' E * Logancha - 65° 31' N 95° 56' E * Bosomtwe - 6° 30' N 1° 24' W * Pantasma - 13° 22' N 85° 57' W * KaraKul - 39° 2' N 73° 25' E * Barringer - 35° 1' N, 111° 1' W Thank you enormously for your help with this! Mike Gravina (508) 667-6162 ***@***.***

…

________________________________ From: David Shean ***@***.***> Sent: Friday, June 16, 2023 8:19 PM To: ICESat2-SlideRule/sliderule-python ***@***.***> Cc: Gravina, Michael ***@***.***>; Mention ***@***.***> Subject: Re: [ICESat2-SlideRule/sliderule-python] Issue with timestamps on GEDI plugin (Discussion ICESat2-SlideRule/sliderule-python#131) Hey @gravinamike. Do you have priority study site(s) you are using for testing? We can potentially stage relevant GEDI granules in our S3 bucket to enable further testing until the full archive is available on S3 via LPDAAC. Let us know! —Reply Hey @gravinamike<https://urldefense.com/v3/__https://github.com/gravinamike__;!!KGKeukY!1fqzaHzbRzLF9iTUZTSWzuM1uDksH34N6dUMzwwyA0kTWfCw7QQpX94he6aB-s2ghivpBqpELXiovjDWIG9IaBryeQ$>. Do you have priority study site(s) you are using for testing? We can potentially stage relevant GEDI granules in our S3 bucket to enable further testing until the full archive is available on S3 via LPDAAC. Let us know! — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/ICESat2-SlideRule/sliderule-python/discussions/131*discussioncomment-6201984__;Iw!!KGKeukY!1fqzaHzbRzLF9iTUZTSWzuM1uDksH34N6dUMzwwyA0kTWfCw7QQpX94he6aB-s2ghivpBqpELXiovjDWIG9moHjM4w$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AFJHJW3ZV6Z3RWSFRVUPIQTXLTZYFANCNFSM6AAAAAAWEIIUPM__;!!KGKeukY!1fqzaHzbRzLF9iTUZTSWzuM1uDksH34N6dUMzwwyA0kTWfCw7QQpX94he6aB-s2ghivpBqpELXiovjDWIG8FPVhbTw$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jpswinski]

Hi @gravinamike, we are in the process of staging the L2A data for the sites you've listed above. Things are on track for finishing sometime at the start of next week. But as of now we have Bigach, Bosomtwe, and Barringer staged. If you have any questions or issues accessing the data from those sites, please let us know. I will post again when the rest of the sites have been completed.

For convenience, here is a link to the docs on the API that accesses these datasets: https://slideruleearth.io/web/rtd/api_reference/gedi.html#gedi02ap

[gravinamike]

Excellent! Thank you very much for the update! Mike Gravina (508) 667-6162 ***@***.***

…

________________________________ From: JP Swinski ***@***.***> Sent: Tuesday, August 1, 2023 2:29 PM To: ICESat2-SlideRule/sliderule-python ***@***.***> Cc: Gravina, Michael ***@***.***>; Mention ***@***.***> Subject: Re: [ICESat2-SlideRule/sliderule-python] Issue with timestamps on GEDI plugin (Discussion ICESat2-SlideRule/sliderule-python#131) Hi @gravinamike, we are in the process of staging the L2A data for the sites you've listed above. Things are on track for finishing sometime at the start of next week. But as of now we have Bigach, Bosomtwe, and Barringer staged. If you have Hi @gravinamike<https://urldefense.com/v3/__https://github.com/gravinamike__;!!KGKeukY!x0eD2flmdvylj6GZaliPKZwaZgG8BhQD5dxsE_f3-nxLsLPcuEGR_gVAwP5C3WUr_2FNKb62fsOQsCLdypnQh6QZUQ$>, we are in the process of staging the L2A data for the sites you've listed above. Things are on track for finishing sometime at the start of next week. But as of now we have Bigach, Bosomtwe, and Barringer staged. If you have any questions or issues accessing the data from those sites, please let us know. I will post again when the rest of the sites have been completed. For convenience, here is a link to the docs on the API that accesses these datasets: https://slideruleearth.io/web/rtd/api_reference/gedi.html#gedi02ap<https://urldefense.com/v3/__https://slideruleearth.io/web/rtd/api_reference/gedi.html*gedi02ap__;Iw!!KGKeukY!x0eD2flmdvylj6GZaliPKZwaZgG8BhQD5dxsE_f3-nxLsLPcuEGR_gVAwP5C3WUr_2FNKb62fsOQsCLdypm-3IJ8-A$> — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/ICESat2-SlideRule/sliderule-python/discussions/131*discussioncomment-6608925__;Iw!!KGKeukY!x0eD2flmdvylj6GZaliPKZwaZgG8BhQD5dxsE_f3-nxLsLPcuEGR_gVAwP5C3WUr_2FNKb62fsOQsCLdypmVb78vYA$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AFJHJW77UERDGF727R5JYT3XTFDKHANCNFSM6AAAAAAWEIIUPM__;!!KGKeukY!x0eD2flmdvylj6GZaliPKZwaZgG8BhQD5dxsE_f3-nxLsLPcuEGR_gVAwP5C3WUr_2FNKb62fsOQsCLdypnggbJbcQ$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jpswinski]

@gravinamike - all of the GEDI L2A data for the areas of interest listed above have been staged, except for Logancha. Queries to CMR for GEDI L2A data for the Logacha area are producing no granules for me, which has me puzzled. I need to investigate more, but I wanted to update you now that you can start using the other areas.

[gravinamike]

Thank you for your continued work on this. We can start to use the currently-uploaded materials for now - that will give us plenty to work with until the Logancha data becomes available. This is a big help! Mike Gravina (508) 667-6162 ***@***.***

…

________________________________ From: JP Swinski ***@***.***> Sent: Monday, August 7, 2023 7:42 AM To: ICESat2-SlideRule/sliderule-python ***@***.***> Cc: Gravina, Michael ***@***.***>; Mention ***@***.***> Subject: Re: [ICESat2-SlideRule/sliderule-python] Issue with timestamps on GEDI plugin (Discussion ICESat2-SlideRule/sliderule-python#131) @gravinamike - all of the GEDI L2A data for the areas of interest listed above have been staged, except for Logancha. Queries to CMR for GEDI L2A data for the Logacha area are producing no granules for me, which has me puzzled. I need to investigate @gravinamike<https://urldefense.com/v3/__https://github.com/gravinamike__;!!KGKeukY!x8zuHBikqMW3drbaYN_UQEGAvbwQEUzFvNmRtW0Fpgg0pTNl6Na6EeVVRAYi84Ue46EzSeGWZBgzT8VTCD2PNkyTyg$> - all of the GEDI L2A data for the areas of interest listed above have been staged, except for Logancha. Queries to CMR for GEDI L2A data for the Logacha area are producing no granules for me, which has me puzzled. I need to investigate more, but I wanted to update you now that you can start using the other areas. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/ICESat2-SlideRule/sliderule-python/discussions/131*discussioncomment-6657720__;Iw!!KGKeukY!x8zuHBikqMW3drbaYN_UQEGAvbwQEUzFvNmRtW0Fpgg0pTNl6Na6EeVVRAYi84Ue46EzSeGWZBgzT8VTCD3s6j-kbQ$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AFJHJW2VITOOE7HG565BLBDXUDIA7ANCNFSM6AAAAAAWEIIUPM__;!!KGKeukY!x8zuHBikqMW3drbaYN_UQEGAvbwQEUzFvNmRtW0Fpgg0pTNl6Na6EeVVRAYi84Ue46EzSeGWZBgzT8VTCD1aqrL3TQ$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jpswinski]

@gravinamike In an effort to consolidate our points of contact with the community, we are transferring this discussion to our primary repository - https://github.com/ICESat2-SlideRule/sliderule. Hopefully, this will make it it easier to monitor and respond to future discussion posts.