Skip to content
This repository has been archived by the owner on May 31, 2024. It is now read-only.

Commit

Permalink
Upload files to s3 (#347)
Browse files Browse the repository at this point in the history
* fix: removing github

* fix: add s3

* fix: removing git references

* fix: remove none general

* fix: use label framework

* fix: upload

* fix: upload url

* fix: unable to upload

* fix: mic component zip

* fix: if fails ends

* fix: wrong path

* fix: wrong key

* fix: first version install.sh

* fix: install

* fix: install script

* fix: improving start

* fix: r must lower

* fix: remove autodetect parameters

* fix: file with windows encoding

* fix: add install script

* Revert "fix: remove autodetect parameters"

This reverts commit a3c784a.

* fix: message

* fix: old mic with wrong reference

* fix: remove references github from test

* fix: remove github references docs

* fix: remove deprecated docs

* fix: remove some tests
  • Loading branch information
mosoriob authored Jun 1, 2021
1 parent 3f80c19 commit ed79ed9
Show file tree
Hide file tree
Showing 36 changed files with 517 additions and 900 deletions.
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"python.linting.pylintArgs": [
"--enable=W0614"
],
"python.pythonPath": "/home/mosorio/.pyenv/versions/mic/bin/python",
"python.pythonPath": "/home/mosorio/.pyenv/versions/mic_delete_git/bin/python",
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true
Expand Down
32 changes: 1 addition & 31 deletions docs/commands/configure.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ mic credentials [-p | --profile] [--server] [--username] [--password] [--name]

## Description

MIC uses several APIs (GitHub, DockerHub and MINT Model catalog) to upload model components. The MINT Model Catalog requires credentials for adding and modifying contents in the catalog. You can use the `credentials` command to configure a username and password for the [Model Catalog API](https://model-catalog-python-api-client.readthedocs.io/en/latest/endpoints/), GitHub and DockerHub. For ease of use, this command can also be used with no parameters, it will prompt the user to enter any required field not given.
MIC uses several APIs (DockerHub and MINT Model catalog) to upload model components. The MINT Model Catalog requires credentials for adding and modifying contents in the catalog. You can use the `credentials` command to configure a username and password for the [Model Catalog API](https://model-catalog-python-api-client.readthedocs.io/en/latest/endpoints/) and DockerHub. For ease of use, this command can also be used with no parameters, it will prompt the user to enter any required field not given.

## Options

Expand All @@ -31,39 +31,11 @@ Password for Model Catalog - [required]

Full name of the author - [required]

`--git_username <GitHub Username>`

Author's Github username - [required]

`--git_token <GitHub API Token>`

Authors's GitHub API Token. More information can be found in the [setting up GitHub credentials](#GitHubCreds) section below - [required]

`--dockerhub_username <Username>`

Username for dockerhub


## <a name="GitHubCreds">Setting up GitHub credentials</a>

!!! info
If you are transforming IPython Notebook, you can skip this section

The `GitHub Username` field is the users GitHub username. If unknown the username can be found at [GitHub.com](https://github.com/). Once logged, in at the top right dropdown menu there will be a "signed in as **[username]**"

The `GitHub Token` is the user's [personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line). To create a personal access token click [here](https://github.com/settings/tokens/new), or go to GitHub.com -> Settings -> Developer settings -> personal access token. Click **Generate new token** this will open the new personal access token page. The following options must be checked:

- [x] repo: | Full control of private repositories
- [x] write:packages | Upload packages to github package registry
- [x] read:packages | Download packages from github package registry

Writing "mic access token" under notes is also recommended

When done click **Generate token** at the bottom of the page. Once the token is generated be sure to copy and save it in a secure location. Enter this key in the `GitHub API token` field when prompted.

!!! warning
If this token is lost there is no way to recover it without generating a new one.

## Setting up DockerHub credentials

MIC will prompt you to add your user in [DockerHub](hub.docker.com/), a repository used for publishing Docker images. MIC will help you publish the computational dependencies of your model as a virtual image, giving you full control over the result.
Expand All @@ -77,8 +49,6 @@ Username [[email protected]]:
Password:
Name:
Email:
GitHub Username:
GitHub API token:
Docker Username:
```

Expand Down
3 changes: 2 additions & 1 deletion docs/model_configuration/02-Preparing your executable.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ If you would like to proceed with this option, the next step is to [prepare your

If the software is not written in Python or cannot be practically encapsulated in a notebook, using the command line version of MIC might be easier. To do so:

1. Place your software code in a local directory so it can be invoked from a command line along with any input data and configuration files if applicable. MIC will test that the code can be executed and will do so three times during the encapsulation process. Therefore, we recommend that your execution can be completed in a manner of minutes, for example, by reducing the size of the dataset (e.g., only considering a few months instead of a year). Because of GitHub limitations, files should not exceed **100 MB** each.
1. Place your software code in a local directory so it can be invoked from a command line along with any input data and configuration files if applicable. MIC will test that the code can be executed and will do so three times during the encapsulation process. Therefore, we recommend that your execution can be completed in a manner of minutes, for example, by reducing the size of the dataset (e.g., only considering a few months instead of a year).

2. The code should not contain any hardcoded paths or values for the input files/variables that you wish to expose. We recommend making them explicit in a configuration file or as parameters from the command line execution.

### Parameters
Expand Down
8 changes: 0 additions & 8 deletions docs/model_configuration/03b-step1.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,19 +45,11 @@ Successfully built 4950fcaa2d0d
Successfully tagged test_192:latest

You are in a Linux environment Debian distribution
We detect the following dependencies.

- If you install new dependencies using `apt` or `apt-get`, remember to add them in Dockerfile mic\docker\Dockerfile
- If you install new dependencies using python. Before the step `upload` run:

pip freeze > mic/docker/requirements.txt

Please, run your Model Component.
```
As can be seen in the message above, MIC is creating an execution environment to make sure we capture the minimum set of dependencies needed for execution. Since we had a java executable, MIC already selected a Java environment. If we had python files, MIC would have promted us to select which version of Python to start from.

!!! warning
This command must **NOT** be executed on a folder already tracked by GitHub.

### Creating your own image

Expand Down
12 changes: 0 additions & 12 deletions docs/model_configuration/deprecated/03-create-skeleton.md

This file was deleted.

67 changes: 0 additions & 67 deletions docs/model_configuration/deprecated/03-step1.md

This file was deleted.

77 changes: 0 additions & 77 deletions docs/model_configuration/deprecated/04-copy-your-data.md

This file was deleted.

61 changes: 0 additions & 61 deletions docs/model_configuration/deprecated/05-write-invocation.md

This file was deleted.

Loading

0 comments on commit ed79ed9

Please sign in to comment.