DBT-693: Pre-commit checks for code quality and syntax, with python 3…

….7 (#171)

support

Co-authored-by: Nitesh Yadav <[email protected]>

.pre-commit-config.yaml

@@ -0,0 +1,61 @@
+default_language_version:
+  python: python3
+
+exclude: ^(./.venv/|./build/)
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+      # Identify invalid files
+      - id: check-ast
+      - id: check-yaml
+      - id: check-json
+      - id: check-toml
+
+      # git checks
+      - id: check-merge-conflict
+      - id: check-added-large-files
+      - id: detect-private-key
+      - id: check-case-conflict
+
+      # Python checks
+      - id: check-docstring-first
+      - id: debug-statements
+      - id: requirements-txt-fixer
+      - id: fix-byte-order-marker
+
+      # General quality checks
+      - id: mixed-line-ending
+      - id: trailing-whitespace
+        args: [--markdown-linebreak-ext=md]
+      - id: end-of-file-fixer
+
+  - repo: https://github.com/PyCQA/autoflake
+    rev: v2.0.1
+    hooks:
+      - id: autoflake
+
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v3.3.1
+    hooks:
+      - id: pyupgrade
+        args:
+          - '--py37-plus'
+
+  - repo: https://github.com/psf/black
+    rev: 23.1.0
+    hooks:
+    - id: black
+      additional_dependencies: ['click~=8.1']
+      args:
+      - "--line-length=99"
+      - "--target-version=py37"
+    - id: black
+      alias: black-check
+      stages: [manual]
+      additional_dependencies: ['click~=8.1']
+      args:
+      - "--line-length=99"
+      - "--target-version=py37"
+      - "--check"
+      - "--diff"

CONTRIBUTING.md

@@ -3,6 +3,7 @@
 1. [About this document](#about-this-document)
 2. [Getting the code](#getting-the-code)
 3. [Running `dbt-impala` in development](#running-dbt-impala-in-development)
+4. [Testing](#testing)
 4. [Submitting a Pull Request](#submitting-a-pull-request)
 
 ## About this document
@@ -49,6 +50,8 @@ If you are a member of the `Cloudera` GitHub organization, you will have push ac
 
 4. Install `dbt-impala` and development dependencies in the active `virtualenv`. Run `pip install -e . -r dev-requirements.txt`.
 
+5. Add the pre-commit hook. Run `pre-commit install`
+
 When `dbt-impala` is installed this way, any changes you make to the `dbt-impala` source code will be reflected immediately (i.e. in your next local dbt invocation against a Impala target).
 
 ## Testing
@@ -76,6 +79,9 @@ You may run a specific test or group of tests using `pytest` directly. Activate
 ```sh
 # Note: replace $strings with valid names
 
+# run full tests suite against an environment/endpoint
+python -m pytest --profile dwx_endpoint
+
 # run all impala functional tests in a directory
 python -m pytest tests/functional/$test_directory --profile dwx_endpoint
 python -m pytest tests/functional/adapter/test_basic.py --profile dwx_endpoint

README.md

@@ -13,7 +13,7 @@ The `dbt-impala` adapter allows you to use [dbt](https://www.getdbt.com/) along
 Current version of dbt-impala work only with dbt-core 1.3 but not with dbt-core >= 1.4.
 We are actively working on next release 1.4 which will work with dbt-core 1.4
 
-Python >= 3.8
+Python >= 3.7
 dbt-core == 1.3.*
 
 For development/testing or contribution to the dbt-impala, please follow [Contributing](CONTRIBUTING.md) guidelines.

dev-requirements.txt

@@ -1,3 +1,5 @@
 dbt-tests-adapter==1.3.*
+pre-commit~=2.21;python_version=="3.7"
+pre-commit~=3.2;python_version>="3.8"
 pytest
 pytest-dotenv

setup.py

@@ -74,6 +74,7 @@ def _get_dbt_core_version():
         "Operating System :: MacOS :: MacOS X",
         "Operating System :: Microsoft :: Windows",
         "Operating System :: POSIX :: Linux",
+        "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "License :: OSI Approved :: Apache Software License",