Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/nikki-t/earthaccess into fe…
Browse files Browse the repository at this point in the history
…ature/issue-447
  • Loading branch information
nikki-t committed Sep 10, 2024
2 parents 79319c8 + c27b502 commit 01bad9c
Show file tree
Hide file tree
Showing 21 changed files with 522 additions and 371 deletions.
67 changes: 67 additions & 0 deletions .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: 🐞 Bug
description: >
A bug is an issue that differs from documentation or has unexpected behavior.
title: "[BUG] {{ title }}"
labels:
- Bug
- Needs Triage
body:
- type: checkboxes
attributes:
label: Is this issue already tracked somewhere, or is this a new report?
description: Check if a similar issue has already been reported before submitting a new one.
options:
- label: I've reviewed existing issues and couldn't find a duplicate for this problem.
required: true

- type: textarea
attributes:
label: Current Behavior
description: A concise description of what you're experiencing.
validations:
required: true

- type: textarea
attributes:
label: Expected Behavior
description: A concise description of what you expected to happen.
validations:
required: true

- type: textarea
attributes:
label: Steps To Reproduce
description: |
Steps to Reproduce
1. In this environment...
2. With this config...
3. Run '...'
4. See error...
validations:
required: true

- type: textarea
attributes:
label: Environment
description: |
Examples:
- **OS**: Ubuntu 20.04
- **Python**: 3.11.2
value: |
- OS:
- Python:
render: markdown
validations:
required: true

- type: textarea
attributes:
label: Additional Context
description: |
Any additional information that might be helpful in resolving this issue, such as:
- Error messages or stack traces
- Relevant configuration files or code snippets
Tip: You can attach files or images by clicking this area to highlight it and then dragging files in.
validations:
required: false
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: 🌟 Community support
url: https://github.com/nsidc/earthaccess/discussions/categories/q-a
about: Explore this section to ask questions, share insights, and connect with others about Earthdata access and usage.
- name: 💡 Suggest a new feature
url: https://github.com/nsidc/earthaccess/discussions/new?category=ideas
about: Got a brilliant idea? Share your innovative suggestions to help us make amazing improvements!
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ repos:
- id: trailing-whitespace

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.5.0
rev: v0.6.3
hooks:
- id: ruff
args: ["--fix", "--exit-non-zero-on-fix"]
Expand Down
12 changes: 11 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
# Changelog

## Unreleased
### Unreleased

- Automatically refresh EDL token and deprecate the `Auth.refresh_tokens` method with no replacement, as there is no longer a need to explicitly refresh ([#484](https://github.com/nsidc/earthaccess/issues/484)) ([**@fwfichtner**](https://github.com/fwfichtner))
- Deprecate `earthaccess.get_s3fs_session` and `Store.get_s3fs_session`. Use
`earthaccess.get_s3_filesystem` and `Store.get_s3_filesystem`, respectively,
instead ([#766](https://github.com/nsidc/earthaccess/issues/766))([**@Sherwin-14**](https://github.com/Sherwin-14))
- Added Issue Templates([#281](https://github.com/nsidc/earthaccess/issues/281))([**@Sherwin-14**](https://github.com/Sherwin-14))
- Removed Broken Link "Introduction to NASA earthaccess" ([#779](https://github.com/nsidc/earthaccess/issues/779))([**@Sherwin-14**](https://github.com/Sherwin-14))



## [v0.10.0] 2024-07-19

Expand All @@ -18,6 +27,7 @@
([**@danielfromearth**](https://github.com/danielfromearth),[**@chuckwondo**](https://github.com/chuckwondo),
[**@jhkennedy**](https://github.com/jhkennedy),[**@mfisher87**](https://github.com/mfisher87))


### Added

- Enable queries to Earthdata User Acceptance Testing (UAT) system for authenticated accounts
Expand Down
25 changes: 24 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,30 @@ To install `earthaccess` go to your terminal and install it using `pip`:
pip install earthaccess
```

Visit [our quick start guide](https://earthaccess.readthedocs.io/en/latest/quick-start/) to learn how to install and see a simple example of using `earthaccess`.

### How to access NASA Earth Science data

With _earthaccess_, data is 3 steps away!

```python
import earthaccess

# 1. Login
earthaccess.login()

# 2. Search
results = earthaccess.search_data(
short_name='ATL06', # ATLAS/ICESat-2 L3A Land Ice Height
bounding_box=(-10, 20, 10, 50), # Only include files in area of interest...
temporal=("1999-02", "2019-03"), # ...and time period of interest.
count=10
)

# 3. Access
files = earthaccess.download(results, "/tmp/my-download-folder")
```

Visit [our quick start guide](https://earthaccess.readthedocs.io/en/latest/quick-start/) for more details.


## Compatibility
Expand Down
2 changes: 1 addition & 1 deletion docs/howto/search-granules.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ results = earthaccess.search_data(
### **Searching for data**

Once we have selected our dataset we can search for the data granules using *doi*, *short_name* or *concept_id*.
If we are not sure or we don't know how to search for a particular dataset, we can start with the ["Introducing NASA earthaccess"](https://nsidc.github.io/earthaccess/tutorials/demo/#querying-for-datasets) tutorial or through the [NASA Earthdata Search portal](https://search.earthdata.nasa.gov/). For a complete list of search parameters we can use visit the extended [API documentation](https://earthaccess.readthedocs.io/en/latest/user-reference/api/api/).
If we are not sure or we don't know how to search for a particular dataset, we can start with the [NASA Earthdata Search portal](https://search.earthdata.nasa.gov/). For a complete list of search parameters we can use visit the extended [API documentation](https://earthaccess.readthedocs.io/en/latest/user-reference/api/api/).

```python

Expand Down
5 changes: 3 additions & 2 deletions docs/user_guide/access.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@

We are reorganizing and updating the documentation, so not all pages are complete. If you are looking for information about accessing data using earthaccess see the
HOW-TO pages below.
[How-to download data](/howto/onprem.md)
[Getting Started Tutorial](/tutorials/getting-started.ipynb)

* [Quick start](../../quick-start/)
* [How-to download data](../howto/onprem.md)

## Downloading data

Expand Down
5 changes: 3 additions & 2 deletions docs/user_guide/authenticate.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ Introduces the `earthaccess.login` method for managing Earthdata Login and cloud

We are reorganizing and updating the documentation, so not all pages are complete. If you are looking for information about authenticating using earthaccess see the
How-Tos and Tutorials in links below.
[How-To Authenticate with earthaccess](/howto/authenticate.md)
[Getting Started Tutorial](/tutorials/getting-started.ipynb)

* [Quick start](../../quick-start/)
* [How-To Authenticate with earthaccess](../howto/authenticate.md)
5 changes: 3 additions & 2 deletions docs/user_guide/search.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@

We are reorganizing and updating the documentation, so not all pages are complete. If you are looking for information about authenticating using earthaccess see the
How-Tos and Tutorials in links below.
[How-To Access Data](/howto/access-data.md)
[Getting Started Tutorial](/tutorials/getting-started.ipynb)

* [Quick start](../../quick-start/)
* [How-To Access Data](../howto/access-data.md)

## `search_datasets`

Expand Down
2 changes: 2 additions & 0 deletions earthaccess/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
get_fsspec_https_session,
get_requests_https_session,
get_s3_credentials,
get_s3_filesystem,
get_s3fs_session,
granule_query,
login,
Expand All @@ -34,6 +35,7 @@
"get_fsspec_https_session",
"get_s3fs_session",
"get_s3_credentials",
"get_s3_filesystem",
"get_edl_token",
"granule_query",
"collection_query",
Expand Down
27 changes: 24 additions & 3 deletions earthaccess/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import requests
import s3fs
from fsspec import AbstractFileSystem
from typing_extensions import Any, Dict, List, Optional, Union
from typing_extensions import Any, Dict, List, Optional, Union, deprecated

import earthaccess

Expand Down Expand Up @@ -325,13 +325,34 @@ def get_requests_https_session() -> requests.Session:
return session


@deprecated("Use get_s3_filesystem instead")
def get_s3fs_session(
daac: Optional[str] = None,
provider: Optional[str] = None,
results: Optional[DataGranule] = None,
) -> s3fs.S3FileSystem:
"""Returns a fsspec s3fs file session for direct access when we are in us-west-2.
Parameters:
daac: Any DAAC short name e.g. NSIDC, GES_DISC
provider: Each DAAC can have a cloud provider.
If the DAAC is specified, there is no need to use provider.
results: A list of results from search_data().
`earthaccess` will use the metadata from CMR to obtain the S3 Endpoint.
Returns:
An `s3fs.S3FileSystem` authenticated for reading in-region in us-west-2 for 1 hour.
"""
return get_s3_filesystem(daac, provider, results)


def get_s3_filesystem(
daac: Optional[str] = None,
provider: Optional[str] = None,
results: Optional[DataGranule] = None,
) -> s3fs.S3FileSystem:
"""Return an `s3fs.S3FileSystem` for direct access when running within the AWS us-west-2 region.
Parameters:
daac: Any DAAC short name e.g. NSIDC, GES_DISC
provider: Each DAAC can have a cloud provider.
Expand All @@ -347,9 +368,9 @@ def get_s3fs_session(
if results is not None:
endpoint = results[0].get_s3_credentials_endpoint()
if endpoint is not None:
session = earthaccess.__store__.get_s3fs_session(endpoint=endpoint)
session = earthaccess.__store__.get_s3_filesystem(endpoint=endpoint)
return session
session = earthaccess.__store__.get_s3fs_session(daac=daac, provider=provider)
session = earthaccess.__store__.get_s3_filesystem(daac=daac, provider=provider)
return session


Expand Down
Loading

0 comments on commit 01bad9c

Please sign in to comment.