diff --git a/.nojekyll b/.nojekyll index dbccca54..041113fc 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -f10095fc \ No newline at end of file +9120d8eb \ No newline at end of file diff --git a/how-tos/access-data/access-command-line.html b/how-tos/access-data/access-command-line.html index 1e514920..c2961db1 100644 --- a/how-tos/access-data/access-command-line.html +++ b/how-tos/access-data/access-command-line.html @@ -570,13 +570,25 @@

How do I access data stored in Earthdata Cloud with cURL and W

How To Access Data With cURL And Wget provides an step-by-step instruction on how to bulk download data from NASA EOSDIS DAACs using wget and cURL from the command line.

-

This section provides additional recommendations relevant to bulk downloading data using wget/cURL. The existing documentation will be updated with these additional notes in future. - You need to install wget or install cURL before diving into this documentation. - View installing curl instructions and Frequently Asked Questions About Downloading GNU Wget for more details. - You also need to save download link(s) for your data as a text file using Nasa Earthdata Search or Common Metadata Repository (CMR) API. - Navigate to the directory you want to save the data using cd Insert_Your_Directory. - To bulk download multiple files using cURL try: text xargs -n 1 curl -O -b ~/.urs_cookies -c ~/.urs_cookies -L -n < Insert_Text_File - If you experience authentication issues working with wget, try creating .wgetrc file in your home directory/user profile as suggested in Troubleshooting wget.

-

To Create a .wgetrc file in the command line, enter the following in Terminal.
-- Windows
-text NUL >> .wgetrc - MacOS or Linux
-text touch .wgetrc | chmod og-rw .wgetrc To insert your NASA Earthdata Login username and password into the file, enter the following in the Command Prompt and replace your username and password.

-
echo http-user=Insert_Your_Username >> .wgetrc | echo http-password=Insert_Your_Password >> .wgetrc
-

After creating .wgetrc file in your home directory/user profile, for downloading a single file try: text wget Insert_the_Download_Link For downloading multiple files try: text wget -i Insert_Text_File

+

This section provides additional recommendations relevant to bulk downloading data using wget/cURL. The existing documentation will be updated with these additional notes in future.

+

You need to install wget or install cURL before diving into this documentation. View installing curl instructions and Frequently Asked Questions About Downloading GNU Wget for more details.

+

You also need to save download link(s) for your data as a text file using Nasa Earthdata Search or Common Metadata Repository (CMR) API.

+

Navigate to the directory you want to save the data using cd YOUR_DIRECTORY_NAME.

+

To bulk download multiple files using cURL try:

+
xargs -n 1 curl -O -b ~/.urs_cookies -c ~/.urs_cookies -L -n < YOUR_FILENAME
+

If you experience authentication issues working with wget, try creating .wgetrc file in your home directory/user profile as suggested in Troubleshooting wget.

+

To create a .wgetrc file in the command line, enter the following in Terminal.

+

Windows

+
NUL >> .wgetrc  
+

MacOS or Linux

+
touch .wgetrc | chmod og-rw .wgetrc
+

To insert your NASA Earthdata Login username and password into the file, enter the following in the Command Prompt and replace your username and password.

+
echo http-user=YOUR_USERNAME >> .wgetrc | echo
+http-password=YOUR_PASSWORD >> .wgetrc
+

After creating .wgetrc file in your home directory/user profile, for downloading a single file try:

+
wget YOUR_DOWNLOAD_LINK
+

For downloading multiple files try:

+
wget -i YOUR_FILENAME
diff --git a/search.json b/search.json index 05976622..d26da268 100644 --- a/search.json +++ b/search.json @@ -2741,7 +2741,7 @@ "href": "how-tos/access-data/access-command-line.html", "title": "How do I access data stored in Earthdata Cloud with cURL and Wget?", "section": "", - "text": "How To Access Data With cURL And Wget provides an step-by-step instruction on how to bulk download data from NASA EOSDIS DAACs using wget and cURL from the command line.\nThis section provides additional recommendations relevant to bulk downloading data using wget/cURL. The existing documentation will be updated with these additional notes in future. - You need to install wget or install cURL before diving into this documentation. - View installing curl instructions and Frequently Asked Questions About Downloading GNU Wget for more details. - You also need to save download link(s) for your data as a text file using Nasa Earthdata Search or Common Metadata Repository (CMR) API. - Navigate to the directory you want to save the data using cd Insert_Your_Directory. - To bulk download multiple files using cURL try: text xargs -n 1 curl -O -b ~/.urs_cookies -c ~/.urs_cookies -L -n < Insert_Text_File - If you experience authentication issues working with wget, try creating .wgetrc file in your home directory/user profile as suggested in Troubleshooting wget.\nTo Create a .wgetrc file in the command line, enter the following in Terminal.\n- Windows\ntext NUL >> .wgetrc - MacOS or Linux\ntext touch .wgetrc | chmod og-rw .wgetrc To insert your NASA Earthdata Login username and password into the file, enter the following in the Command Prompt and replace your username and password.\necho http-user=Insert_Your_Username >> .wgetrc | echo http-password=Insert_Your_Password >> .wgetrc\nAfter creating .wgetrc file in your home directory/user profile, for downloading a single file try: text wget Insert_the_Download_Link For downloading multiple files try: text wget -i Insert_Text_File" + "text": "How To Access Data With cURL And Wget provides an step-by-step instruction on how to bulk download data from NASA EOSDIS DAACs using wget and cURL from the command line.\nThis section provides additional recommendations relevant to bulk downloading data using wget/cURL. The existing documentation will be updated with these additional notes in future.\nYou need to install wget or install cURL before diving into this documentation. View installing curl instructions and Frequently Asked Questions About Downloading GNU Wget for more details.\nYou also need to save download link(s) for your data as a text file using Nasa Earthdata Search or Common Metadata Repository (CMR) API.\nNavigate to the directory you want to save the data using cd YOUR_DIRECTORY_NAME.\nTo bulk download multiple files using cURL try:\nxargs -n 1 curl -O -b ~/.urs_cookies -c ~/.urs_cookies -L -n < YOUR_FILENAME\nIf you experience authentication issues working with wget, try creating .wgetrc file in your home directory/user profile as suggested in Troubleshooting wget.\nTo create a .wgetrc file in the command line, enter the following in Terminal.\nWindows\nNUL >> .wgetrc \nMacOS or Linux\ntouch .wgetrc | chmod og-rw .wgetrc\nTo insert your NASA Earthdata Login username and password into the file, enter the following in the Command Prompt and replace your username and password.\necho http-user=YOUR_USERNAME >> .wgetrc | echo\nhttp-password=YOUR_PASSWORD >> .wgetrc\nAfter creating .wgetrc file in your home directory/user profile, for downloading a single file try:\nwget YOUR_DOWNLOAD_LINK\nFor downloading multiple files try:\nwget -i YOUR_FILENAME" }, { "objectID": "how-tos/find-data/earthdata_search.html", diff --git a/sitemap.xml b/sitemap.xml index 24eb372a..be50727d 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,330 +2,330 @@ https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/GESDISC/GESDISC_MERRA2_tavg1_2d_flx_Nx__Kerchunk.html - 2024-06-05T16:12:51.990Z + 2024-06-05T16:35:10.625Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/LPDAAC/Find_and_Access_HLS_PointBuffer.html - 2024-06-05T16:12:38.870Z + 2024-06-05T16:34:58.161Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/USGS_Landsat/Landsat_C2_Search_Access.html - 2024-06-05T16:12:35.942Z + 2024-06-05T16:34:55.169Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/NSIDC/ICESat2-CMR-OnPrem-vs-Cloud.html - 2024-06-05T16:12:33.934Z + 2024-06-05T16:34:53.109Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/ORNL/Data_Access__Direct_S3_Access__ORNL_DAYMET.html - 2024-06-05T16:12:33.086Z + 2024-06-05T16:34:52.257Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/PODAAC/PODAAC_ECCO_SSH__Kerchunk.html - 2024-06-05T16:12:30.390Z + 2024-06-05T16:34:49.577Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/workshops/setup.html - 2024-06-05T16:12:26.914Z + 2024-06-05T16:34:46.165Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/data_discovery_cmr-stac_api.html - 2024-06-05T16:12:25.450Z + 2024-06-05T16:34:44.841Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/data_discovery_cmr.html - 2024-06-05T16:12:23.630Z + 2024-06-05T16:34:43.045Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/nasa_earthdata_authentication.html - 2024-06-05T16:12:22.114Z + 2024-06-05T16:34:41.601Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/xarray.html - 2024-06-05T16:12:20.638Z + 2024-06-05T16:34:40.221Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/data_access_direct_S3.html - 2024-06-05T16:12:19.534Z + 2024-06-05T16:34:39.165Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/use_apis.html - 2024-06-05T16:12:16.102Z + 2024-06-05T16:34:35.781Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/find-data/find-python.html - 2024-06-05T16:12:15.202Z + 2024-06-05T16:34:34.913Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/find-data/find-r.html - 2024-06-05T16:12:14.322Z + 2024-06-05T16:34:34.029Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/reproject_regrid.html - 2024-06-05T16:12:13.138Z + 2024-06-05T16:34:32.865Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-cloud-python.html - 2024-06-05T16:12:12.202Z + 2024-06-05T16:34:32.161Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-local-python.html - 2024-06-05T16:12:11.050Z + 2024-06-05T16:34:31.029Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/read_data.html - 2024-06-05T16:12:09.886Z + 2024-06-05T16:34:29.901Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/Earthdata_Cloud__Open-Science-Tutorial.html - 2024-06-05T16:12:08.642Z + 2024-06-05T16:34:28.649Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/using-s3-storage.html - 2024-06-05T16:12:01.082Z + 2024-06-05T16:34:20.961Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/CMR-STAC-Search.html - 2024-06-05T16:11:59.918Z + 2024-06-05T16:34:19.797Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/index.html - 2024-06-05T16:11:58.374Z + 2024-06-05T16:34:18.269Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/when-to-cloud.html - 2024-06-05T16:11:57.414Z + 2024-06-05T16:34:17.301Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/our-cookbook.html - 2024-06-05T16:11:56.538Z + 2024-06-05T16:34:16.449Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/troubleshooting.html - 2024-06-05T16:11:55.714Z + 2024-06-05T16:34:15.637Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/quarto-website-tutorial.html - 2024-06-05T16:11:54.034Z + 2024-06-05T16:34:13.981Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/setup.html - 2024-06-05T16:11:53.342Z + 2024-06-05T16:34:13.305Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/community.html - 2024-06-05T16:11:52.510Z + 2024-06-05T16:34:12.505Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/index.html - 2024-06-05T16:11:51.706Z + 2024-06-05T16:34:11.717Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/leading-workshops/index.html - 2024-06-05T16:11:50.602Z + 2024-06-05T16:34:10.649Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/appendix/appendix-alternate-in-cloud-access/Earthdata_Cloud__Single_File__Direct_S3_Access_NetCDF4_Example.html - 2024-06-05T16:11:49.770Z + 2024-06-05T16:34:09.833Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/appendix/appendix-alternate-in-cloud-access/Earthdata_Cloud__Single_File__HTTPS_Access_COG_Example.html - 2024-06-05T16:11:48.650Z + 2024-06-05T16:34:08.685Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/appendix/index.html - 2024-06-05T16:11:47.554Z + 2024-06-05T16:34:07.557Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/glossary.html - 2024-06-05T16:11:46.534Z + 2024-06-05T16:34:06.549Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Above_Ground_Biomass_Density.html - 2024-06-05T16:11:45.662Z + 2024-06-05T16:34:05.697Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/earthaccess-demo.html - 2024-06-05T16:11:44.486Z + 2024-06-05T16:34:04.541Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Earthdata_Cloud__Data_Access_OPeNDAP_Example.html - 2024-06-05T16:11:43.566Z + 2024-06-05T16:34:03.589Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Sea_Level_Rise.html - 2024-06-05T16:11:42.598Z + 2024-06-05T16:34:02.633Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/fair-workflow-geoweaver-demo.html - 2024-06-05T16:11:37.710Z + 2024-06-05T16:33:57.745Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/04_On-Prem_Cloud.html - 2024-06-05T16:11:35.026Z + 2024-06-05T16:33:55.137Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/matlab.html - 2024-06-05T16:11:33.598Z + 2024-06-05T16:33:53.733Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Observing_Seasonal_Ag_Changes.html - 2024-06-05T16:11:36.510Z + 2024-06-05T16:33:56.609Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Earthdata-cloud-clinic.html - 2024-06-05T16:11:38.470Z + 2024-06-05T16:33:58.497Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/index.html - 2024-06-05T16:11:42.998Z + 2024-06-05T16:34:03.053Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Data_stories_template.html - 2024-06-05T16:11:43.914Z + 2024-06-05T16:34:03.961Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/Hurricanes_Wind_and_Sea_Surface_Temperature.html - 2024-06-05T16:11:45.054Z + 2024-06-05T16:34:05.077Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/policies-usage/index.html - 2024-06-05T16:11:46.046Z + 2024-06-05T16:34:06.057Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/appendix/authentication.html - 2024-06-05T16:11:47.106Z + 2024-06-05T16:34:07.121Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/appendix/appendix-alternate-in-cloud-access/Multi-File_Direct_S3_Access_NetCDF_Example.html - 2024-06-05T16:11:47.994Z + 2024-06-05T16:34:08.037Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/appendix/appendix-alternate-in-cloud-access/Access_COG.html - 2024-06-05T16:11:49.334Z + 2024-06-05T16:34:09.401Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/index.html - 2024-06-05T16:11:50.242Z + 2024-06-05T16:34:10.289Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/leading-workshops/add-folks-to-2i2c-github-teams.html - 2024-06-05T16:11:51.338Z + 2024-06-05T16:34:11.357Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/onboarding.html - 2024-06-05T16:11:52.182Z + 2024-06-05T16:34:12.189Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/calendar-project-management.html - 2024-06-05T16:11:52.898Z + 2024-06-05T16:34:12.893Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/environment.html - 2024-06-05T16:11:53.706Z + 2024-06-05T16:34:13.669Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/workflow.html - 2024-06-05T16:11:55.322Z + 2024-06-05T16:34:15.273Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/LICENSE.html - 2024-06-05T16:11:56.114Z + 2024-06-05T16:34:16.025Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/environment-setup/index.html - 2024-06-05T16:11:56.922Z + 2024-06-05T16:34:16.833Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/matlab.html - 2024-06-05T16:11:58.050Z + 2024-06-05T16:34:17.941Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/in-development/earthdata-python-r-handoff.html - 2024-06-05T16:11:59.026Z + 2024-06-05T16:34:18.917Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/aggregate.html - 2024-06-05T16:12:00.242Z + 2024-06-05T16:34:20.109Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/reformat.html - 2024-06-05T16:12:01.414Z + 2024-06-05T16:34:21.269Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/plot.html - 2024-06-05T16:12:09.038Z + 2024-06-05T16:34:29.053Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/how_to_guide_parking_lot.html - 2024-06-05T16:12:10.238Z + 2024-06-05T16:34:30.245Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-local-opendap.html - 2024-06-05T16:12:11.390Z + 2024-06-05T16:34:31.373Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-command-line.html - 2024-06-05T16:12:12.786Z + 2024-06-05T16:34:32.521Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/find-data/earthdata_search.html - 2024-06-05T16:12:13.662Z + 2024-06-05T16:34:33.381Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/find-data/programmatic.html - 2024-06-05T16:12:14.858Z + 2024-06-05T16:34:34.549Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/subset.html - 2024-06-05T16:12:15.698Z + 2024-06-05T16:34:35.401Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/on-prem_cloud.html - 2024-06-05T16:12:16.914Z + 2024-06-05T16:34:36.565Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/cof-zarr-reformat.html - 2024-06-05T16:12:20.082Z + 2024-06-05T16:34:39.697Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/sentinel-6_opendap_access_gridding.html - 2024-06-05T16:12:21.502Z + 2024-06-05T16:34:41.025Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/harmony_subsetting.html - 2024-06-05T16:12:22.974Z + 2024-06-05T16:34:42.401Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/zarr-eosdis-store.html - 2024-06-05T16:12:24.010Z + 2024-06-05T16:34:43.417Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/external/appeears_csv_cloud_access.html - 2024-06-05T16:12:26.386Z + 2024-06-05T16:34:45.685Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/workshops/index.html - 2024-06-05T16:12:27.414Z + 2024-06-05T16:34:46.629Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/ORNL/ORNL_DAYMET__Kerchunk.html - 2024-06-05T16:12:30.762Z + 2024-06-05T16:34:49.937Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/index.html - 2024-06-05T16:12:33.422Z + 2024-06-05T16:34:52.609Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/NSIDC/ICESat2-CMR-AWS-S3.html - 2024-06-05T16:12:34.614Z + 2024-06-05T16:34:53.777Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/Transform-workflow.html - 2024-06-05T16:12:37.678Z + 2024-06-05T16:34:56.953Z https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/LPDAAC/LPDAAC_ECOSTRESS_LSTE__Kerchunk.html - 2024-06-05T16:12:40.074Z + 2024-06-05T16:34:59.317Z