Skip to content

Commit

Permalink
add support for tox; fix yamllint issues
Browse files Browse the repository at this point in the history
sync with latest template, which adds support for tox
and a couple of new tests, including yamllint

fix the yamllint issues
  • Loading branch information
richm committed Apr 8, 2020
1 parent 488f359 commit 2287f92
Show file tree
Hide file tree
Showing 46 changed files with 2,023 additions and 254 deletions.
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
passes.yml
vault.yml
*.pyc
*.retry
/tests/.coverage
/tests/htmlcov*
/.tox
/venv*/
/.venv/
.vscode/
artifacts/
__pycache__/
*~
.pytest_cache/
5 changes: 5 additions & 0 deletions .lgtm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
extraction:
python:
python_setup:
version: 2
30 changes: 24 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,30 @@
# SPDX-License-Identifier: MIT
---
dist: xenial
language: python
env:
global:
- LSR_ANSIBLES='ansible==2.7.* ansible==2.8.* ansible==2.9.*'
- LSR_MSCENARIOS='default'
matrix:
include:
- python: 2.6
dist: trusty
- python: 2.7
- python: 3.5
- python: 3.6
- python: 3.7
- python: 3.8
- python: 3.8-dev

services:
- docker

before_install:
- ./.travis/preinstall

install:
- pip install 'molecule<3' docker
- pip install tox tox-travis

script:
- molecule --version
- ansible --version
- molecule lint
- molecule syntax
- molecule test
- ./.travis/runtox
49 changes: 49 additions & 0 deletions .travis/config.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# SPDX-License-Identifier: MIT
#
# Use this file to specify custom configuration for a project. Generally, this
# involves the modification of the content of LSR_* environment variables, see
#
# * .travis/preinstall:
#
# - LSR_EXTRA_PACKAGES
#
# * .travis/runtox:
#
# - LSR_ANSIBLES
# - LSR_MSCENARIOS
#
# * .travis/runcoveralls.sh:
#
# - LSR_PUBLISH_COVERAGE
# - LSR_TESTSDIR
# - function lsr_runcoveralls_hook
#
# Environment variables that not start with LSR_* but have influence on CI
# process:
#
# * .travis/runpylint.sh:
#
# - RUN_PYLINT_INCLUDE
# - RUN_PYLINT_EXCLUDE
# - RUN_PYLINT_DISABLED
# - RUN_PYLINT_SETUP_MODULE_UTILS
#
# * .travis/runblack.sh:
#
# - RUN_BLACK_INCLUDE
# - RUN_BLACK_EXCLUDE
# - RUN_BLACK_DISABLED
# - RUN_BLACK_EXTRA_ARGS
#
# * .travis/runflake8.sh:
#
# - RUN_FLAKE8_DISABLED
# - RUN_FLAKE8_EXTRA_ARGS
#
# * .travis/runsyspycmd.sh:
#
# - function lsr_runsyspycmd_hook
#
# * .travis/runpytest.sh:
#
# - RUN_PYTEST_SETUP_MODULE_UTILS
12 changes: 12 additions & 0 deletions .travis/custom.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/bin/bash
# SPDX-License-Identifier: MIT

set -e

ME=$(basename $0)
SCRIPTDIR=$(readlink -f $(dirname $0))

. ${SCRIPTDIR}/utils.sh
. ${SCRIPTDIR}/config.sh

# Write your custom commands here that should be run when `tox -e custom`:
171 changes: 171 additions & 0 deletions .travis/custom_pylint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# -*- coding: utf-8 -*-
# SPDX-License-Identifier: MIT
#
# Copyright (c) 2019-2020 Red Hat, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
"""
Probe directory tree for python files and pass them to pylint.
Usage: python custom_pylint.py ARGUMENTS
Run pylint with ARGUMENTS followed by the list of python files contained in the
working directory and its subdirectories. As a python file is recognized a
file that match INCPAT. Files and directories that match EXPAT are skipped.
Symbolic links are also skipped. It is assumed that files to be pylinted are
specified only with INCPAT and EXPAT.
There are several cases when argument from ARGUMENTS is not passed to pylint
but it is handled by run_pylint.py instead:
1. if -h or --help is contained in ARGUMENTS, this help screen is printed to
the standard output and run_pylint.py exits with 0;
2. if --include followed by a PATTERN is contained in ARGUMENTS, the PATTERN
is used instead of INCPAT to recognize whether the file is a python file
or not;
3. if --exclude followed by a PATTERN is contained in ARGUMENTS, the PATTERN
is used instead of EXPAT to recognize whether the file or directory should
be skipped.
Exclusion takes a priority over inclusion, i.e. if a file or directory can be
both included and excluded, it is excluded.
The default value of INCPAT is .*\\.py[iw]?$. For EXPAT, it is ^\\..*.
Environment variables:
RUN_PYLINT_INCLUDE
overrides default value of INCPAT;
RUN_PYLINT_EXCLUDE
overrides default value of EXPAT;
RUN_PYLINT_DISABLED
if set to an arbitrary non-empty value, pylint will be not executed
"""

import os
import re
import sys

from colorama import Fore
from pylint.lint import Run


def blue(s):
"""
Return string `s` colorized to blue.
"""

return "%s%s%s" % (Fore.BLUE, s, Fore.RESET)


def print_line(s):
"""
Write `s` followed by the line feed character to the standard output.
"""

sys.stdout.write("%s\n" % s)


def probe_args():
"""
Analyze the command line arguments and return a tuple containing a list of
pylint arguments, pattern string to recognize files to be included, and
pattern string to recognize files and directories to be skipped.
Default values of pattern strings are taken from RUN_PYLINT_INCLUDE and
RUN_PYLINT_EXCLUDE environment variables. In the case they are not defined,
.*\\.py[iw]?$ and ^\\..* are used, respectively.
"""

args = []
include_pattern = os.getenv("RUN_PYLINT_INCLUDE", r".*\.py[iw]?$")
exclude_pattern = os.getenv("RUN_PYLINT_EXCLUDE", r"^\..*")
i, nargs = 1, len(sys.argv)
while i < nargs:
arg = sys.argv[i]
if arg == "--include":
i += 1
assert i < nargs, "--include: missing PATTERN"
include_pattern = sys.argv[i]
elif arg == "--exclude":
i += 1
assert i < nargs, "--exclude: missing PATTERN"
exclude_pattern = sys.argv[i]
else:
args.append(arg)
i += 1
return args, include_pattern, exclude_pattern


def probe_dir(path, include_re, exclude_re):
"""
Recursively go through directory structure starting at `path`, collect
files that match `include_re`, skip files and directories that are either
symbolic links or match `exclude_re`. Return the list of collected files.
"""

files = []
for direntry in os.listdir(path):
fullpath = os.path.join(path, direntry)
if os.path.islink(fullpath) or exclude_re.match(direntry):
continue
elif os.path.isdir(fullpath):
files.extend(probe_dir(fullpath, include_re, exclude_re))
elif os.path.isfile(fullpath) and include_re.match(direntry):
files.append(fullpath)
return files


def show_files(files):
"""
Print `files` to the standard output, one item per line, in a blue color.
"""

print_line(blue("%s: files to be checked:" % sys.argv[0]))
for f in files:
print_line(blue(" %s" % f))


def main():
"""
Script entry point. Return exit code.
"""

args, include_pattern, exclude_pattern = probe_args()
if "-h" in args or "--help" in args:
print_line(__doc__)
return 0
if os.getenv("RUN_PYLINT_DISABLED", "") != "":
return 0
files = probe_dir(
os.getcwd(), re.compile(include_pattern), re.compile(exclude_pattern)
)
if not files:
return 0
show_files(files)
args.extend(files)
sys.argv[0] = "pylint"
return Run(args, None, False).linter.msg_status


if __name__ == "__main__":
sys.exit(main())
35 changes: 35 additions & 0 deletions .travis/preinstall
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash
# SPDX-License-Identifier: MIT

# Install package specified by user in LSR_EXTRA_PACKAGES. Executed by Travis
# during before_install phase.
#
# LSR_EXTRA_PACKAGES, set by user in .travis/config.sh, is a space separated
# list of packages to be installed on the Travis build environment (Ubuntu).

set -e

SCRIPTDIR=$(readlink -f $(dirname $0))

. ${SCRIPTDIR}/utils.sh

# Add python3-selinux package (needed by Molecule on selinux enabled systems,
# because Molecule is using `copy` and `file` Ansible modules to setup the
# container).
if lsr_venv_python_matches_system_python; then
LSR_EXTRA_PACKAGES='python3-selinux'
fi

# extra packages needed with python 2.6 and ansible 2.6
if lsr_check_python_version python -lt 2.7 ; then
LSR_EXTRA_PACKAGES="$LSR_EXTRA_PACKAGES libffi-dev libssl-dev"
fi

. ${SCRIPTDIR}/config.sh

# Install extra dependencies.
if [[ "${LSR_EXTRA_PACKAGES}" ]]; then
set -x
sudo apt-get update
sudo apt-get install -y ${LSR_EXTRA_PACKAGES}
fi
68 changes: 68 additions & 0 deletions .travis/runblack.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/bin/bash
# SPDX-License-Identifier: MIT

# A shell wrapper around black (Python formatter). The purpose of this wrapper
# is to get a user the opportunity to control black from config.sh via setting
# environment variables.

# The given command line arguments are passed to black.

# Environment variables:
#
# RUN_BLACK_INCLUDE
# a regular expression specifying files to be included; can be overridden
# from command line by --include;
#
# RUN_BLACK_EXCLUDE
# a regular expression specifying files to be excluded; can be overridden
# from command line by --exclude;
#
# RUN_BLACK_DISABLED
# if set to an arbitrary non-empty value, black will be not executed
#
# RUN_BLACK_EXTRA_ARGS
# extra cmd line args to pass to black

set -e

ME=$(basename $0)
SCRIPTDIR=$(readlink -f $(dirname $0))

. ${SCRIPTDIR}/utils.sh
. ${SCRIPTDIR}/config.sh

if [[ "${RUN_BLACK_DISABLED}" ]]; then
lsr_info "${ME}: black is disabled. Skipping."
exit 0
fi

DEFAULT_INCLUDE='^[^.].*\.py$'
DEFAULT_EXCLUDE='/(\.[^.].*|tests/roles)/'

INCLUDE_ARG=""
EXCLUDE_ARG=""
OTHER_ARGS=()

while [[ $# -gt 0 ]]; do
case "$1" in
--include)
shift
INCLUDE_ARG="$1"
;;
--exclude)
shift
EXCLUDE_ARG="$1"
;;
*)
OTHER_ARGS+=( "$1" )
;;
esac
shift
done

set -x
python -m black \
--include "${INCLUDE_ARG:-${RUN_BLACK_INCLUDE:-${DEFAULT_INCLUDE}}}" \
--exclude "${EXCLUDE_ARG:-${RUN_BLACK_EXCLUDE:-${DEFAULT_EXCLUDE}}}" \
${RUN_BLACK_EXTRA_ARGS:-} \
"${OTHER_ARGS[@]}"
Loading

0 comments on commit 2287f92

Please sign in to comment.