Waste Data Tool

This repository contains the source code and usage instructions for the Waste Data Tool (WDT). The WDT provides functionality for:

• reading open data about waste in Scotland

• re-working this data so that it is easier to use, understand, consume & parse

• writing the re-worked data as simple CSV and JSON files, to a single public repository.

The WDT is an output of the Data Commons Scotland research project.

Warning

This is software is very rough’n’ready - it is at a proof-of-concept level.

Note	Information about the mothballed Wikibase feature, can be found here.

---

Use

Setting up the tools, environment, etc.

Prerequisite software:

• Git 2.24.0 (-ish)

• JDK 11.0.5 (-ish)

• Clojure 1.10.1 (-ish)

• Cojure Tools 1.10.1.754 (-ish)

• lib/csvw-validator-cli-app-1.0.0-SNAPSHOT.jar (performs extra CSVW related validation; used by gen-turtle; get from Vojtech’s repo)

• lib/csv2rdf-0.4.7-SNAPSHOT-standalone.jar (performs the actual Turtle generation; used by gen-turtle; get from Swirrl’s repo)

• lib/blazegraph.jar 2.1.6 (ish) (provides an RDF store and a SPARQL service; used by run-blaze; get from Blazegraph’s repo)

Get the software

C:\>git clone https://github.com/data-commons-scotland/dcs-wdt.git

Run

Create ingestable local files from SPARQL queries:

clj -X:sparql

Create a database from ingestable local files then describe the database on the console:

clj -X:db

Create a database from ingestable local files then generate CSV & JSON files for general use:

clj -X:general

Validate the generated CSV files against their CSVW specifications, then generate their equivalent Turtle files:

gen-turtle

Create ZIP bundles of the CSV, JSON, Turtle & CSVW files:

zip-files

Load the generated Turtle files into a Blazegraph DB file, and then run a Blazegraph service:

run-blaze