Skip to content

Commit

Permalink
feat: allow more kinds of authentication, clarify usage
Browse files Browse the repository at this point in the history
  • Loading branch information
Justintime50 committed Aug 24, 2023
1 parent d229b6f commit c26b7ab
Show file tree
Hide file tree
Showing 10 changed files with 61 additions and 24 deletions.
4 changes: 3 additions & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
[run]
omit = github_archive/cli.py
omit =
github_archive/cli.py
github_archive/_version.py
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# CHANGELOG

## v6.1.0 (2023-06-24)

- Allows the tool to be run without passing any authentication flags (previously, to use unauthenticated, you'd have to at least pass the `--https` flag)
- Removes constraint that required `--token` and `--https` to be mutually exclusive (you can now authenticate with other tools such as Git Credential Manager instead of only SSH)

## v6.0.0 (2023-06-30)

- Drops support for Python 3.7
Expand Down
24 changes: 19 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,23 +60,37 @@ Options:
The log level used for the tool. Default: info
```

### Automating SSH Passphrase Prompt (Recommended)
### Authentication

To allow the script to run continuosly without requiring your SSH passphrase, you'll need to add your passphrase to the SSH agent. **NOTE:** Your SSH passphrase will be unloaded upon logout.
There are three methods of authentication with this tool.

#### Unauthenticated

You can run a command similar to `github-archive --users justintime50 --clone` which would only clone public repositories. GitHub has a hard limit of `60 requests per hour` - not authenticating may quickly burn through that limit if you have a large GitHub instance to archive.

#### SSH

To allow the script to run continuosly without requiring passwords for every repo, you can add your SSH passphrase to the SSH agent:

```bash
# This assumes you've saved your SSH keys to the default location
ssh-add
```

You can then run a command similar to `github-archive --users justintime50 --clone --token 123` where the token is your GitHub API token. This will authenticate you with the GitHub API via the `token` and with GitHub via `ssh`.

#### Git Credential Manager

Alternatively, you can use a tool like [Git Credential Manager](https://github.com/git-ecosystem/git-credential-manager) to populate your Git credentials under the hood. When not using SSH, we'll clone/pull from the git URLs instead of the SSH URLs. To trigger this behavior, you must pass the `--https` flag.

You can then run a command similar to `github-archive --users justintime50 --clone --token 123 --https` where the token is your GitHub API token. This will authenticate you with the GitHub API via the `token` and with GitHub via your Git credentials via `GCM`.

### Notes

**SSH Key:** By default, you must have an SSH key generated on your local machine and added to your GitHub account as this tool uses the `ssh_url` to clone/pull. If you'd like to instead use the `git_url` to clone/pull, you can pass the `--https` flag which currently requires no authentication. Without using a token/SSH, you will not be able to interact with private git assets. Additionally, GitHub has a hard limit of 60 requests per hour - using the `--https` option may quickly burn through that unauthenticated limit if you have a large GitHub instance to archive.
**Access**: GitHub Archive can only clone or pull git assets that the authenticated user has access to. This means that private repos from another user or org that you don't have access to will not be able to be cloned or pulled. Additionally without using a token and SSH/CGM, you will not be able to interact with private git assets.

**Merge Conflicts:** Be aware that using GitHub Archive could lead to merge conflicts if you do not commit or stash your changes if using these repos as active development repos instead of simply an archive or one-time clone.

**Access**: GitHub Archive can only clone or pull git assets that the authenticated user has access to. This means that private repos from another user or org that you don't have access to will not be able to be cloned or pulled.

## Development

```bash
Expand Down
1 change: 1 addition & 0 deletions github_archive/_version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__version__ = "6.1.0"
12 changes: 6 additions & 6 deletions github_archive/archive.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

import woodchips
from github import (
Auth,
Gist,
Github,
Repository,
Expand Down Expand Up @@ -88,7 +89,11 @@ def __init__(
self.log_level = log_level

# Internal variables
self.github_instance = Github(login_or_token=self.token, base_url=self.base_url)
self.github_instance = (
Github(auth=Auth.Token(self.token), base_url=self.base_url)
if self.token
else Github(base_url=self.base_url)
)
self.authenticated_user = self.github_instance.get_user() if self.token else None
self.authenticated_username = self.authenticated_user.login.lower() if self.token else None

Expand Down Expand Up @@ -256,11 +261,6 @@ def initialize_project(self):
logger=logger,
message='The include and exclude flags are mutually exclusive. Only one can be used on each run.',
)
elif self.token and self.use_https:
log_and_raise_value_error(
logger=logger,
message='Use only one of `token` or `https` flags to authenticate.',
)

def authenticated_user_in_users(self) -> bool:
"""Returns True if the authenticated user is in the list of users."""
Expand Down
6 changes: 6 additions & 0 deletions github_archive/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from typing import get_args

from github_archive import GithubArchive
from github_archive._version import __version__
from github_archive.constants import (
DEFAULT_BASE_URL,
DEFAULT_LOCATION,
Expand Down Expand Up @@ -161,6 +162,11 @@ def __init__(self):
choices=set(get_args(LOG_LEVEL_CHOICES)),
help=f'The log level used for the tool. Default: {DEFAULT_LOG_LEVEL}',
)
parser.add_argument(
'--version',
action='version',
version=f'%(prog)s {__version__}',
)
parser.parse_args(namespace=self)

def run(self):
Expand Down
4 changes: 3 additions & 1 deletion github_archive/repos.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,11 @@ def _archive_repo(
PULL_OPERATION: ['git', '-C', repo_path, 'pull', '--rebase'],
}

if github_archive.use_https:
if github_archive.use_https or not github_archive.token:
# Will be used for unauthenticated requests or with items like GCM
commands.update({CLONE_OPERATION: ['git', 'clone', repo.html_url, repo_path]})
else:
# Will be used for SSH authenticated requests
commands.update({CLONE_OPERATION: ['git', 'clone', repo.ssh_url, repo_path]})

git_command = commands[operation]
Expand Down
20 changes: 15 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,22 @@
import re

import setuptools


with open('README.md', 'r') as fh:
long_description = fh.read()
with open('README.md', 'r') as readme_file:
long_description = readme_file.read()

# Inspiration: https://stackoverflow.com/a/7071358/6064135
with open('github_archive/_version.py', 'r') as version_file:
version_groups = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file.read(), re.M)
if version_groups:
version = version_groups.group(1)
else:
raise RuntimeError('Unable to find version string!')

REQUIREMENTS = [
'PyGithub == 1.*',
'woodchips == 0.2.*',
'woodchips == 1.*',
]

DEV_REQUIREMENTS = [
Expand All @@ -15,15 +25,15 @@
'build == 0.10.*',
'flake8 == 6.*',
'isort == 5.*',
'mypy == 1.3.*',
'mypy == 1.5.*',
'pytest == 7.*',
'pytest-cov == 4.*',
'twine == 4.*',
]

setuptools.setup(
name='github-archive',
version='6.0.0',
version=version,
description=(
'A powerful tool to concurrently clone, pull, or fork user and org repos and gists to create a GitHub archive.'
),
Expand Down
4 changes: 0 additions & 4 deletions test/unit/test_archive.py
Original file line number Diff line number Diff line change
Expand Up @@ -306,10 +306,6 @@ def test_initialize_project(mock_make_dirs, mock_dir_exist, mock_logger):
{'users': 'justintime50', 'clone': True, 'include': 'mock-repo', 'exclude': 'another-mock-repo'},
'The include and exclude flags are mutually exclusive. Only one can be used on each run.',
),
(
{'token': '123', 'use_https': True, 'users': 'justintime50', 'clone': True},
'Use only one of `token` or `https` flags to authenticate.',
),
],
)
@patch('github_archive.archive.Github.get_user')
Expand Down
5 changes: 3 additions & 2 deletions test/unit/test_repos.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,13 @@ def test_view_repos(mock_logger, mock_git_asset):
mock_logger.assert_called_with('mock_username/mock-asset-name')


@patch('github_archive.archive.Github')
@patch('subprocess.check_output')
@patch('logging.Logger.info')
def test_archive_repo_success(mock_logger, mock_subprocess, mock_git_asset):
def test_archive_repo_success(mock_logger, mock_subprocess, mock_github, mock_git_asset):
operation = CLONE_OPERATION
message = f'Repo: {mock_git_asset.owner.login}/{mock_git_asset.name} {operation} success!'
github_archive = GithubArchive()
github_archive = GithubArchive(token='123')
_archive_repo(github_archive, mock_git_asset, 'mock/path', operation)

mock_subprocess.assert_called_once_with(
Expand Down

0 comments on commit c26b7ab

Please sign in to comment.