Waste Data Tool

This repository contains the source code and usage instructions for the Waste Data Tool (WDT). The WDT provides functionality for:

reading open data about waste in Scotland
re-working this data so that it is easier to use, understand, consume & parse
writing the re-worked data as simple CSV and JSON files, to a single public repository.

The WDT is an output of the Data Commons Scotland research project.

Warning

This is software is very rough’n’ready - it is at a proof-of-concept level.

Note	Information about the mothballed Wikibase feature, can be found here.

Use

Prerequisite software:

Git 2.24.0 (-ish)
JDK 11.0.5 (-ish)
Clojure 1.10.1 (-ish)
Cojure Tools 1.10.1.754 (-ish)
lib/csvw-validator-cli-app-1.0.0-SNAPSHOT.jar (performs extra CSVW related validation; used by gen-turtle; get from Vojtech’s repo)
lib/csv2rdf-0.4.7-SNAPSHOT-standalone.jar (performs the actual Turtle generation; used by gen-turtle; get from Swirrl’s repo)
lib/blazegraph.jar 2.1.6 (ish) (provides an RDF store and a SPARQL service; used by run-blaze; get from Blazegraph’s repo)

C:\>git clone https://github.com/data-commons-scotland/dcs-wdt.git

Create ingestable local files from SPARQL queries:

clj -X:sparql

Create a database from ingestable local files then describe the database on the console:

clj -X:db

Create a database from ingestable local files then generate CSV & JSON files for general use:

clj -X:general

Validate the generated CSV files against their CSVW specifications, then generate their equivalent Turtle files:

gen-turtle

Create ZIP bundles of the CSV, JSON, Turtle & CSVW files:

zip-files

Load the generated Turtle files into a Blazegraph DB file, and then run a Blazegraph service:

run-blaze