Skip to content

Releases: mwvgroup/Pitt-Google-Broker

v0.5

09 Aug 00:35
3092915
Compare
Choose a tag to compare

Adds an auto-scheduler
Reconfigures ReadTheDocs
Adds tutorials
Adds a stream looper

See #62

v0.4

30 Jul 22:57
d9e7566
Compare
Choose a tag to compare

Major Updates and New Features:

  • Accommodate generic survey: (1) Renames resources. (2) Adds a "survey" keyword. (3) Adds a schema_maps module and configures all components to use it.
  • Creates a broker_utils python package and configures broker components to use it. Includes modules: beam_transforms, consumer_sim, data_utils, gcp_utils, schema_maps
  • Listen for Kafka topics: Consumer VM now polls for available Kafka topics and begins consuming once its assigned topic becomes active. Prior to this, the broker could not be started before the topic was active.
  • Documentation: updates for above changes, and reorganization in preparation for integration with ReadtheDocs.

v0.3

25 Feb 18:49
69a62ce
Compare
Choose a tag to compare

Implements the following new items:

  • Consumer: VM running a Kafka -> Pub/Sub connector
  • BQ sink: Dataflow job storing alerts to BigQuery
  • Avro storage sink: Cloud Function storing alerts to Cloud Storage as Avro files
  • Value-added pipeline: Dataflow job running Salt2 fits on likely extragalactic events
  • Night Conductor: VM running a set of scripts that start/stop the broker each night/morning
  • Consumer Simulator: Python module allowing user to manually simulate the Consumer by publishing alert packets to a Pub/Sub topic
  • Broker Testing Instances: tutorial plus code integrated with all items above that allows users (PGB members) to create and use a testing/development instance of the broker

v0.2

11 Feb 16:27
aea85f0
Compare
Choose a tag to compare
Merge pull request #45 from mwvgroup/development

Development

v0.1

28 Jan 15:27
58af663
Compare
Choose a tag to compare
Merge pull request #36 from mwvgroup/schedule_consumers

Deploys preliminary broker