datavault4dbt by Scalefree International GmbH

---

Included Macros

• Staging Area (For Hashing, prejoins and ghost records)
• Hubs, Links & Satellites (allowing multiple deltas)
• Non-Historized Links and Satellites
• Multi-Active Satellites
• Virtualized End-Dating (in Satellites)
• PIT Tables
  • Hook for Cleaning up PITs
• Snapshot Control

Features

With datavault4dbt you will get a lot of awesome features, including:

• A Data Vault 2.0 implementation congruent to the original Data Vault 2.0 definition by Dan Linstedt
• Ready for both Persistent Staging Areas and Transient Staging Areas, due to the allowance of multiple deltas in all macros, without loosing any intermediate changes- Enforcing standards in naming conventions by implementing global variables for technical columns
• A fully auditable solution for a Data Warehouse
• Creating a centralized, snapshot-based Business interface by using a centralized snapshot table supporting logarithmic logic
• A modern insert-only approach that avoids updating data
• Optimizing incremental loads by implemting a high-water-mark that also works for entities that are loaded from multiple sources
• A straight-forward, standardized approach to conduct agile datawarehouse development cycles

Requirements

To use the macros efficiently, there are a few prerequisites you need to provide:

• Flat & Wide source data, available within your target database
• Load Date column that represents the arriving time in the source data storage
• Record Source column that gives information about where the source data is coming from (e.g. the file location inside a Data Lake)

Resources:

• Learn more about dbt in the docs
• Check out Discourse for commonly asked questions and answers
• Join the chat on Slack for live discussions and support
• Check out the blog for the latest news on dbt's development and best practices
• Find dbt events near you
• Check out the Scalefree-Blog
  • Data-Vault 2.0 with dbt #1
  • Data-Vault 2.0 with dbt #2
  • Using Multi-Active-Satellites #1
  • Using Multi-Active-Satellites #2
  • Non-Historized Links
  • Bridge Tables
  • PIT Tables
  • Hash Keys in Data-Vault

Supported platforms:

Currently supported platforms are:

• Google Bigquery
• Exasol
• Snowflake

We are working continuously at high pressure to adapt the package for large variety of different platforms. In the future, the package will hopefully be available for SQL Server, Oracle and many more.

---

Installation

Datavault4dbt is not yet listed on dbt Hub, but we are working hard to get it there. Until then, you can still experience datavault4dbt by installing via Git. Just add the following lines to your packages.yml:

  packages:
    - git: "https://github.com/ScalefreeCOM/datavault4dbt.git"
      revision: 1.0.0

For further information on how to install packages in dbt, please visit the following link: https://docs.getdbt.com/docs/building-a-dbt-project/package-management

Global variables

datavault4dbt is highly customizable by using many global variables. Since they are applied on multiple levels, a high rate of standardization across your data vault 2.0 solution is guaranteed. The default values of those variables are set inside the packages dbt_project.yml and should be copied to your own dbt_project.yml. For an explanation of all global variables see the wiki.

---

Usage

The datavault4dbt package provides macros for Staging and Creation of all DataVault-Entities you need, to build your own DataVault2.0 solution. The usage of the macros is well-explained in the documentation: https://github.com/ScalefreeCOM/datavault4dbt/wiki

---

Contributing

View our contribution guidelines

---

License

Apache 2.0

Contact

For questions, feedback, etc. reach out to us via [email protected]!