feat: implement flag for error on ignore migration

[zanebclark]

This is a resurrection of PR 222 with a few modifications. This pull request started out as a feature addition. It ended up being a bit of a repo refactor.

• Deprecate verbse cli argument. Opt for log-level instead
• Replace print statements and if verbose checks with a structlog logger and log levels.
• Removed SecretManager in favor of a structlog processor that implements very similar logic.
• Centralize config-related logic and input checking into a Config object. For example, there was logic for checking the validity of a directory and the presence of environment variables. Centralizing this logic is conceptually easier for me to wrap my head around and makes testing much easier.
• Generally, moved a bunch of class definitions out of cli.py and into separate files
• Stored script details in a Script class to clarify the API
• Removed a bunch of credential-related logic from SnowflakeClient and created a Credential factory.
• Generally expanded test case coverage.
• Added two CLI arguments:
  • raise-exception-on-ignored-version-script: At my shop, we're using epoch time as our version number. If you create a script with an epoch time and your colleague merges their changes in before you get a chance to, the max_published_version will cause schemachange will skip your script without complaint. To address this, I handle versioned scripts almost identically to repeatable scripts. I store the metadata on the script's execution in a dictionary and compare that to the scripts in the script directory. If a script is "stale", hasn't been applied, and the flag is true: I raise an exception.
  • version-number-validation-regex: Again, we're using epoch time as the version number in my shop. If someone deploys a script prefixed with Vspam__, future scripts versioned with the current epoch time will be skipped. The max_published_version is now spam and cannot be trumped by a number. To address this, I've added a version_number_validation_regex CLI argument that will reject spam as a version number if I've specificed \d{10}

[zanebclark]

@sfc-gh-tmathew , I've stripped out the poetry package management from this pull request

[zanebclark]

@sfc-gh-tmathew , I've also restored the METADATA database behavior. Existing tests are still passing. I'd appreciate your confirmation.

[zanebclark]

I found the following notes while reviewing the old pull request:

1. A dry run without a change_history table will result in a failure. I didn't properly mimic the previous logic to handle this case.
2. When I configured the regex pattern as \d{9}, it failed to raise an exception when a microsecond epoch time was used as the version number.

I'll confirm this behavior shortly.

[sfc-gh-tmathew]

@zanebclark Thank you for your contribution. Will need to understand the changes, some of them seem straight forward.

@MACKAT05 started #157 on similar ability to reorganize code and improve developer experience with schemachange.

Can we collaborate on the PRs that will increment the changes?

Can we separate the PRs into the following categories and raise separate PRs for each of these?

• Code reorg (No functionality changes)
• New flags
• Replace old flags ( This I believe should be done to support verbose and the log level at the same time to be backward compatible)

Lastly, #205 is already address in 3.7.0

[zanebclark]

@sfc-gh-tmathew, I get nervous when I read "no functionality changes" 😬. Does that cover everything but the flag changes? Or, do we want to reorganize and retain SecretsManager and the print logging?

[zanebclark]

@sfc-gh-tmathew , I'll start working on this weekend on the lesser of the two potential asks here. I'll create a different PR for the flag addition and flag deprecation and leave the rest alone.

@sfc-gh-tmathew, I get nervous when I read "no functionality changes" 😬. Does that cover everything but the flag changes? Or, do we want to reorganize and retain SecretsManager and the print logging?

[zanebclark]

@sfc-gh-tmathew , I've created PR 1 targeting the code organization branch with the feature flag branch. It looks like I'm not able to cause that pull request to show up in this repo until there's a representation of the code-reorganization branch in this repo.

[zanebclark]

@sfc-gh-tmathew , I've created PR 2 for deprecating the verbose flag.

[zanebclark]

Per @sfc-gh-tmathew 's ask, closing this pull request with three separate pull requests with the same content:

• PR 285: feat: code reorganization
• PR 1: feat: flag for error on ignored versioned migration and version number regex
• PR 2: feat: deprecate verbose flag