First Pre-Release

Brief

dataio aims to simplify the process of programmatic access to files and folders on AWS S3, with the DSIH ARTPARK. This is currently not stable, and is an internal private alpha release. The internal aspects of the function can be customised to any server, but is not server agnostic at the moment and written solely for AWS.

Functionality

The current version of dataio supports 3 main functions:

1. To download an entire dataset, or part of it, using download_dataset_v2.
2. To fetch the data documentation of a given dataset.
3. To fetch a specific file based on the S3 URI

from dataio.download import fetch_data_documentation, download_dataset_v2

# Example usage of fetch_data_documentation to get metadata
metadata, datadict = fetch_data_documentation(dsid="DSID01")

# Example usage of download_dataset_v2 to download your dataset with ID "DSID01" only for files for the year "2023"
# This assumes the files for that year contain the year in the name, or are in a folder with the same name
download_dataset_v2(dsid="DSID01", data_state="standardised", contains = "2023")