thredds-catalog-crawler

A package for crawling THREDDS catalogs, including nested catalogs.

Install

npm install thredds-catalog-crawler --save

Basic usage

Using async/await & modules

import threddsCrawler from 'thredds-catalog-crawler'

async function crawlThredds() {
    const catalog = await threddsCrawler('http://something/thredds/catalog/my/catalog.xml')
    await catalog.loadAllNestedCatalogs()
    const datasets = catalog.getAllChildDatasets()
}
crawlThredds()

Using traditional promises & modules

const threddsCrawler = require('thredds-catalog-crawler')

threddsCrawler('http://something/thredds/catalog/my/catalog.xml')
.then(function (catalog) {
    catalog.loadAllNestedCatalogs()
})

Dependencies

This package uses the Fetch API. If you're using this library in NodeJS you'll need a fetch polyfill like isomorphic-fetch or if you want to support older browsers we'd suggest unfetch. You may also need a URL API polyfill such as url-polyfill for older browsers.

import 'isomorphic-fetch'
import 'url-polyfill'
import threddsCrawler from 'thredds-catalog-crawler'

API

Catalog

A Catalog represents the root object in a THREDDS heirarchy. Catalogs can contain datasets, as well as references to other catalogs.

Properties

Property	Type	Description
url	String	Url of the catalog
name	String	Name of the catalog
isLoaded	Boolean	Indicates whether the endpoint has been loaded. If true then datasets and services will be populated, and limited catalog information. If false then most information will be unavailable.
datasets	Array	An array of datasets found in the catalog
catalogs	Array	An array of nested catalogs found in the catalog
services	Object	An object containing the available services for a catalog, organised by service type
hasDatasets	Boolean	Indicates whether the catalog contains datasets directly
hasNestedCatalogs	Boolean	Indicates whether the catalog contains nested catalogs
wmsBase	String	The base url for a WMS service. If no WMS support is available then null
supportsWms	Boolean	Does a catalog have a WMS service for datasets

Methods

Method	Returns	Description
loadAllNestedCatalogs	N/A	Loads all nested catalogs recurrsively
loadNestedCatalogById(id)	Catalog	Crawls a single nested catalog based on a id string
getAllChildDatasets	Array[Dataset1, Dataset2...]	Returns a flattened array of datasets from all nested catalogs which have been loaded.

Dataset

A Dataset can contain datasets, as well as references to other catalogs.

Properties

Property	Type	Description
id	String	An id of the dataset
name	String	Name of the dataset
parent	Object	A catalog or another dataset
urlPath	String	Partial url path of the dataset
isParentDataset	Boolean	Indicates whether this dataset contains other datasets
datasets	Array[Dataset]	An array of nested datasets
wmsUrl	String	A url for the GetCapabilities endpoint for the dataset. Constructed from the dataset and its parent catalog item
parentCatalog	Catalog	A reference to the datasets parent catalog
supportsWms	Boolean	Does this dataset have a WMS available

Methods

Method	Returns	Description
loadAllNestedCatalogs	N/A	Loads all nested catalogs recurrsively