Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/initial pr #8

Merged
merged 30 commits into from
Oct 4, 2023
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7d68063
Introduces the palace-quicksight script with three operations:
dbernstein Sep 1, 2023
8a0e6d7
Fix formatting on README.
dbernstein Sep 1, 2023
4c7d55d
Add link to palace-quicksight-resources project.
dbernstein Sep 1, 2023
843001b
Fix docstring on import operation.
dbernstein Sep 2, 2023
18621e7
Add instructions to readme for running the script.
dbernstein Sep 2, 2023
b74d794
Simplify command names.
dbernstein Sep 2, 2023
b4e863a
Add a couple of tests, tox configuration, github ci configuration.
dbernstein Sep 6, 2023
806b96d
Fix github workflow file errors (remove references to non-existent ci…
dbernstein Sep 6, 2023
365973f
Update poetry lock file.
dbernstein Sep 6, 2023
25da5b3
Initial pre-commit formatting of all files.
dbernstein Sep 6, 2023
5dbf298
Add ci group back into the toml.
dbernstein Sep 6, 2023
834fe19
Add pymarkdown config file and fix README.md.
dbernstein Sep 6, 2023
76167a5
Formatting fix.
dbernstein Sep 6, 2023
cdb0dca
Disable isort in pre commit config.
dbernstein Sep 6, 2023
4a6a6ff
Remove python 3.8 from tox config
dbernstein Sep 6, 2023
f9e33c7
Fix ci test issue
dbernstein Sep 6, 2023
3279884
Set minimum python version to 3.9
dbernstein Sep 6, 2023
6ae3e84
Fix formatting.
dbernstein Sep 6, 2023
ffd9f9f
Fix mypy issues; re-enable isort plugin.
dbernstein Sep 6, 2023
3915ea7
Add prereq and installation sections to README.md.
dbernstein Sep 7, 2023
d97004d
Add import test.
dbernstein Sep 19, 2023
6a7335a
Add publish test.
dbernstein Sep 19, 2023
448f42c
Improve output for cli operations.
dbernstein Sep 19, 2023
8168668
Use log.info rather than click.echo. Also correct variable style.
dbernstein Sep 25, 2023
add9334
Use dataclass instead of tuple for return values.
dbernstein Sep 25, 2023
0a360cc
Remove existing templates and data sets before creating to ensure tha…
dbernstein Sep 25, 2023
0baa75f
Fix typo and use tempfile to generated test output file.
dbernstein Sep 25, 2023
b770c8e
Remove extraneous comment.
dbernstein Sep 25, 2023
60111c2
Use standard approach to passing aws profiles to boto3.
dbernstein Sep 25, 2023
1960ba5
Improve logging.
dbernstein Oct 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/actions/poetry/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: 'Poetry'
description: 'Install python poetry'

inputs:
version:
description: "Version of poetry to install"
required: false
default: "1.5.1"

outputs:
version:
description: "Installed version"
value: ${{ steps.poetry-version.outputs.version }}

runs:
using: "composite"
steps:
- run: |
curl -sSL https://install.python-poetry.org | python - --yes --version ${{ inputs.version }}
echo "$HOME/.local/bin" >> $GITHUB_PATH
shell: bash

- id: poetry-version
run: echo "version=$(poetry --version)" >> $GITHUB_OUTPUT
shell: bash
15 changes: 15 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "daily"
versioning-strategy: increase-if-necessary

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
22 changes: 22 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## Description

<!--- Describe your changes -->

## Motivation and Context

<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->

## How Has This Been Tested?

<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

- [ ] I have updated the documentation accordingly.
- [ ] All new and existing tests passed.
23 changes: 23 additions & 0 deletions .github/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
changelog:
exclude:
labels:
- ignore for notes
categories:
- title: Incompatible Changes
labels:
- incompatible changes
- title: Features
labels:
- feature
- title: Bugfixes
labels:
- bug
- title: Other Changes
labels:
- "*"
exclude:
labels:
- dependencies
- title: Dependency Updates
labels:
- dependencies
38 changes: 38 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Lint
on: [push, pull_request]
env:
PYTHON_VERSION: 3.9

jobs:
lint:
name: Lint
runs-on: ubuntu-latest
permissions:
contents: read

steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Install Poetry
uses: ./.github/actions/poetry

- name: Install Pre-commit
run: |
poetry install --only ci
env:
POETRY_VIRTUALENVS_CREATE: false

- name: Restore pre-commit cache
uses: actions/cache@v3
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ runner.os }}-py${{ env.PYTHON_VERSION }}-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: pre-commit-${{ runner.os }}-py${{ env.PYTHON_VERSION }}

- name: Lint
run: pre-commit run --all-files --show-diff-on-failure
32 changes: 32 additions & 0 deletions .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Mypy (Type check)
on: [push, pull_request]
env:
PYTHON_VERSION: 3.9

jobs:
mypy:
runs-on: ubuntu-latest
permissions:
contents: read

steps:
- uses: actions/checkout@v3

- name: Set up Python 🐍
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Install Poetry 🎸
uses: ./.github/actions/poetry

- name: Install OS Packages 🧰
run: |
sudo apt-get update
sudo apt-get install --yes libxmlsec1-dev libxml2-dev

- name: Install Python Packages 📦
run: poetry install --without ci

- name: Run MyPy 🪄
run: poetry run mypy
55 changes: 55 additions & 0 deletions .github/workflows/test-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: Test & Build
on: [push, pull_request]

concurrency:
group: test-build-${{ github.ref_name }}-${{ github.event_name }}
cancel-in-progress: true

jobs:
test:
name: ${{ matrix.module }} Tests (Py ${{ matrix.python-version }})
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:
contents: read
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11"]
module: [Core]

# We want to run on external PRs, but not on our own internal PRs as they'll be run
# by the push to the branch. This prevents duplicated runs on internal PRs.
# Some discussion of this here:
# https://github.community/t/duplicate-checks-on-push-and-pull-request-simultaneous-event/18012
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository

steps:
- uses: actions/checkout@v3

# See comment here: https://github.com/actions/runner-images/issues/1187#issuecomment-686735760
- name: Disable network offload
run: sudo ethtool -K eth0 tx off rx off

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install Apt Packages
run: |
sudo apt-get update

- name: Install Poetry
uses: ./.github/actions/poetry

- name: Install Tox
run: |
poetry install --only ci
env:
POETRY_VIRTUALENVS_CREATE: false

- name: Run Tests
run: tox
env:
MODULE: ${{ matrix.module }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
__pycache__
.idea
.tox
71 changes: 71 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-json
- id: check-ast
- id: check-toml
- id: check-shebang-scripts-are-executable
- id: check-executables-have-shebangs
- id: check-merge-conflict
- id: check-added-large-files
- id: mixed-line-ending

- repo: https://github.com/asottile/pyupgrade
rev: v3.3.2
hooks:
- id: pyupgrade
args:
- --py38-plus
- --keep-runtime-typing

- repo: https://github.com/myint/autoflake
rev: v2.1.1
hooks:
- id: autoflake
args:
- --in-place
- --remove-all-unused-imports
- --ignore-init-module-imports

- repo: https://github.com/psf/black
rev: 22.10.0
hooks:
- id: black
name: Run black

- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort
name: Run isort

- repo: https://github.com/sirosen/check-jsonschema
rev: 0.22.0
hooks:
- id: check-github-workflows
- id: check-github-actions

- repo: https://github.com/pappasam/toml-sort
rev: v0.23.0
hooks:
- id: toml-sort
args: []
files: pyproject.toml

- repo: https://github.com/jackdewinter/pymarkdown
rev: v0.9.9
hooks:
- id: pymarkdown
args:
- --config
- .pymarkdown.config.json
- scan

# Exclude test files, since they may be intentionally malformed
exclude: ^tests/(core)/files/
21 changes: 21 additions & 0 deletions .pymarkdown.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"plugins" : {
"line-length" : {
"line_length" : 120,
"heading_line_length": 120,
"code_block_line_length": 120
},
"no-duplicate-heading" : {
"allow_different_nesting": true
},
"ul-indent" : {
"indent" : 4
},
"blanks-around-fences" : {
"list_items": false
},
"first-line-heading" : {
"enabled": false
}
}
}
32 changes: 30 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,30 @@
# palace-quicksight
AWS QuickSight Resources for Palace
# Palace Quicksight Tools

A suite of command line operations for exporting and importing quicksight dashboards from and to AWS accounts.
Exported resources can be found [here](https://github.com/ThePalaceProject/palace-quicksight-resources).

## Prerequisites

Install Poetry: [Installation instructions here](https://python-poetry.org/docs/).

## Installation

```shell
poetry install
poetry run ./bin/palace-quicksight --help
```

## Usage

```shell
./bin/palace-quicksight --help
Usage: palace-quicksight [OPTIONS] COMMAND [ARGS]...

Options:
--help Show this message and exit.

Commands:
export-analysis Creates a template from the analysis and exports at...
import-template Import template and datasource files from json
publish-dashboard Create/Update a dashboard from a template
```
12 changes: 12 additions & 0 deletions bin/palace-quicksight
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/env python
"""The entry point for Palace Quicksight Tools"""
import os
import sys

from core.cli import cli

bin_dir = os.path.split(__file__)[0]
package_dir = os.path.join(bin_dir, "..")
sys.path.append(os.path.abspath(package_dir))

cli()
Empty file added core/__init__.py
Empty file.
Loading