Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add public key generation functionality #186

Merged
merged 1 commit into from
Aug 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions client-docs/cli/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,14 @@ Once you've populated a YAML file with your desired parameters, configure |platf

Generating Keys
---------------
If you don't already have a keypair and/or a symmetric key, you'll want to generate them so that you can interact with |platform| cloud compute services in a cryptographically secure manner. |platform| uses your certificate and private key to authenticate you to |platform| compute services, and uses your symmetric key to encrypt your data to ensure that the cloud doesn't see it in plaintext..
If you don't already have a keypair and/or a symmetric key, you'll want to generate them so that you can interact with |platform| cloud compute services in a cryptographically secure manner. |platform| uses your keypair to authenticate you to |platform| compute services, and uses your symmetric key to encrypt your data to ensure that the cloud doesn't see it in plaintext..

You can generate a certificate and corresponding private key, and a symmetric key, through the CLI. You should have specified paths for your certificate, private key, and symmetric key during configuration. If something already exists at either the certificate or private key path, |platform| Client will skip generating the certificate and private key. If something already exists at the symmetric key path, |platform| Client will skip generating the symmetric key.
You can generate a keypair, corresponding certificate, and a symmetric key through the CLI. You should have specified paths for your private key, public key, certificate, and symmetric key during configuration. If something already exists at either the private key, public key, or certificate path, |platform| Client will skip generating the keypair and corresponding certificate. If something already exists at the symmetric key path, |platform| Client will skip generating the symmetric key.

.. code-block:: bash
:substitutions:

# Generate a certificate, corresponding private key, and symmetric key
# Generate a keypair, corresponding certificate, and symmetric key
# If something exists at any paths specified in the config,
# |platform| Client will skip generation.
$ |cmd| init
Expand Down
22 changes: 12 additions & 10 deletions client-docs/config/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,25 @@ Global Configuration
====================
.. _conf:

Before using MC\ :sup:`2` Client, you'll need to perform some configuration by specifying parameters in a YAML file. An example YAML file can be found `here <https://github.com/mc2-project/mc2/blob/master/demo/config.yaml>`_, and has also been copied at the bottom of this page. We describe the various parameters below.
Before using |platform| Client, you'll need to perform some configuration by specifying parameters in a YAML file. An example YAML file can be found `here <https://github.com/mc2-project/mc2/blob/master/demo/config.yaml>`_, and has also been copied at the bottom of this page. We describe the various parameters below.

User Configuration
------------------
We'll need to perform some configuration for the user in the ``user`` section of the YAML file. Parameters are:

- ``username`` : your username. This username will be used for certificate generation and authentication purposes.

- ``symmetric_key`` : path to your symmetric key. If you don't yet have a symmetric key, you can ask MC\ :sup:`2` Client to generate a key for you (see :ref:`Generating Keys`). MC\ :sup:`2` Client will look to this path for your key when encrypting and decrypting your data.
- ``symmetric_key`` : path to your symmetric key. If you don't yet have a symmetric key, you can ask |platform| Client to generate a key for you (see :ref:`Generating Keys`). |platform| Client will look to this path for your key when encrypting and decrypting your data.

- ``private_key`` : path to your private key. If you don't yet have a private key, you can ask MC\ :sup:`2` Client to generate a private key/certificate for you (see :ref:`Generating Keys`). MC\ :sup:`2` Client will use your private key to sign messages sent to the cloud.
- ``private_key`` : path to your private key. If you don't yet have a private key, you can ask |platform| Client to generate a keypair/certificate for you (see :ref:`Generating Keys`). |platform| Client will use your private key to sign messages sent to the cloud.

- ``certificate`` : path to your certificate. if you don't yet have a certificate, you can ask MC\ :sup:`2` Client to generate a certificate/private key for you (see :ref:`Generating Keys`). MC\ :sup:`2` Client will use your certificate to authenticate you to the cloud.
- ``public_key`` : path to your public key. If you don't yet have a public key, you can ask |platform| Client to generate a keypair/certificate for you (see :ref:`Generating Keys`). |platform| Client will use your public key to authenticate you to the cloud.

- ``root_private_key`` : path to the Certificate Authority's private key. MC\ :sup:`2` Client uses the CA's private key to generate a certificate for you. The MC\ :sup:`2` compute service should also be aware of the CA private key.
- ``certificate`` : path to your certificate. If you don't yet have a certificate, you can ask |platform| Client to generate a keypair/certificate for you (see :ref:`Generating Keys`). |platform| Client will use your certificate to authenticate you to the cloud.

- ``root_certificate`` : path to the Certificate Authority's certificate. MC\ :sup:`2` Client uses the CA's certificate to generate a certificate for you. The MC\ :sup:`2` compute service should also be aware of the CA certificate.
- ``root_private_key`` : path to the Certificate Authority's private key. |platform| Client uses the CA's private key to generate a certificate for you. The |platform| compute service should also be aware of the CA private key.

- ``root_certificate`` : path to the Certificate Authority's certificate. |platform| Client uses the CA's certificate to generate a certificate for you. The |platform| compute service should also be aware of the CA certificate.

Launch
------
Expand Down Expand Up @@ -55,9 +57,9 @@ Upload
------
In this section, you can specify what data you want to encrypt and upload, how you want to upload the data, and what encryption format you want to use.

- ``storage`` : options are ``blob`` or ``disk``. If ``blob``, MC\ :sup:`2` Client will upload your data to the Azure storage container you create. If ``disk``, MC\ :sup:`2` Client will ``scp`` your data to each launched VM.
- ``storage`` : options are ``blob`` or ``disk``. If ``blob``, |platform| Client will upload your data to the Azure storage container you create. If ``disk``, |platform| Client will ``scp`` your data to each launched VM.

- ``format`` : options are ``xgb`` and ``sql``. If ``xgb``, MC\ :sup:`2` Client will encrypt your data in a format compatible with Secure XGBoost. If ``sql``, MC\ :sup:`2` Client will encrypt your data in a format compatible with Opaque SQL.
- ``format`` : options are ``xgb`` and ``sql``. If ``xgb``, |platform| Client will encrypt your data in a format compatible with Secure XGBoost. If ``sql``, |platform| Client will encrypt your data in a format compatible with Opaque SQL.

- ``src`` : a list of files to encrypt and upload

Expand Down Expand Up @@ -94,9 +96,9 @@ Download
In this section, you can specify what you want to download and decrypt, how you want to download the data, and what decryption format you want to use.


- ``storage`` : options are ``blob`` or ``disk``. If ``blob``, MC\ :sup:`2` Client will upload your data to the Azure storage container you create. If ``disk``, MC\ :sup:`2` Client will ``scp`` your data to each launched VM.
- ``storage`` : options are ``blob`` or ``disk``. If ``blob``, |platform| Client will upload your data to the Azure storage container you create. If ``disk``, |platform| Client will ``scp`` your data to each launched VM.

- ``format`` : options are ``xgb`` and ``sql``. If ``xgb``, MC\ :sup:`2` Client will decrypt your data in a format compatible with Secure XGBoost. If ``sql``, MC\ :sup:`2` Client will decrypt your data in a format compatible with Opaque SQL.
- ``format`` : options are ``xgb`` and ``sql``. If ``xgb``, |platform| Client will decrypt your data in a format compatible with Secure XGBoost. If ``sql``, |platform| Client will decrypt your data in a format compatible with Opaque SQL.

- ``src`` : a list of files to download.

Expand Down
5 changes: 3 additions & 2 deletions client-docs/opaquesql_usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -139,12 +139,13 @@ All together, the configuration file should look something like the following wh
# `|platform| init` will not overwrite anything at this path
symmetric_key: ${|platform_uppercase|_CLIENT_HOME}/quickstart/keys/user1_sym.key

# Path to your private key and certificate
# If you don't have a private key / certificate, specify paths here
# Path to your keypair and certificate
# If you don't have a keypair / certificate, specify paths here
# and run `|platform| init` to generate a keypair
#
# `|platform| init` will not overwrite anything at this path
private_key: ${|platform_uppercase|_CLIENT_HOME}/quickstart/keys/user1.pem
public_key: ${|platform_uppercase|_CLIENT_HOME}/quickstart/keys/user1.pub
certificate: ${|platform_uppercase|_CLIENT_HOME}/quickstart/keys/user1.crt

# Path to CA certificate and private key
Expand Down
6 changes: 3 additions & 3 deletions client-docs/python/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,14 @@ Once you've populated a YAML file with your desired parameters, set the path to

Key Generation
--------------
If you don't already have a keypair and/or a symmetric key, you'll want to generate them so that you can interact with |platform| cloud compute services in a cryptographically secure manner. |platform| uses your certificate and private key to authenticate you to |platform| compute services, and uses your symmetric key to encrypt your data to ensure that the cloud doesn't see it in plaintext..
If you don't already have a keypair and/or a symmetric key, you'll want to generate them so that you can interact with |platform| cloud compute services in a cryptographically secure manner. |platform| uses your keypair and certificate to authenticate you to |platform| compute services, and uses your symmetric key to encrypt your data to ensure that the cloud doesn't see it in plaintext..

|platform| Client provides a function to generate a certificate and corresponding private key, and a function to generate a symmetric key. You should have specified paths for your certificate, private key, and symmetric key during configuration. These functions will output the certificate, private key, and symmetric key to these paths.
|platform| Client provides a function to generate a keypair and corresponding certificate as well as a function to generate a symmetric key. You should have specified paths for your private key, public key, certificate, and symmetric key during configuration. These functions will output the private key, public key, certificate, and symmetric key to these paths.

.. code-block:: python
:substitutions:

# Generate a certificate and corresponding private key
# Generate a keypair and corresponding certificate
|python-package-short|.generate_keypair()

# Generate a symmetric key
Expand Down
4 changes: 3 additions & 1 deletion demo/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@ user:
# `mc2 init` will not overwrite anything at this path
symmetric_key: ${MC2_CLIENT_HOME}/demo/keys/user1_sym.key

# Path to your private key and certificate

# Path to your keypair and certificate.
# If you don't have a private key / certificate, specify paths here
# and run `mc2 init` to generate a keypair
#
# `mc2 init` will not overwrite anything at this path
private_key: ${MC2_CLIENT_HOME}/demo/keys/user1.pem
public_key: ${MC2_CLIENT_HOME}/demo/keys/user1.pub
certificate: ${MC2_CLIENT_HOME}/demo/keys/user1.crt

# Path to CA certificate and private key
Expand Down
16 changes: 16 additions & 0 deletions python-package/mc2client/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -586,6 +586,7 @@ def generate_keypair(expiration=10 * 365 * 24 * 60 * 60):

username = user_config["username"]
private_key_path = user_config["private_key"]
public_key_path = user_config["public_key"]
cert_path = user_config["certificate"]

root_cert_path = user_config["root_certificate"]
Expand All @@ -599,6 +600,14 @@ def generate_keypair(expiration=10 * 365 * 24 * 60 * 60):
)
return

if os.path.exists(public_key_path):
logger.warning(
"Skipping keypair generation - public key already exists at {}".format(
public_key_path
)
)
return

if os.path.exists(cert_path):
logger.warning(
"Skipping keypair generation - certificate already exists at {}".format(
Expand All @@ -618,6 +627,13 @@ def generate_keypair(expiration=10 * 365 * 24 * 60 * 60):
"Generated private key and outputted to {}".format(private_key_path)
)

with open(public_key_path, "wb") as public_key_file:
public_key_file.write(crypto.dump_publickey(crypto.FILETYPE_PEM, key))

logger.info(
"Generated public key and outputted to {}".format(public_key_path)
)

# Generate the certificate signing request
ca_cert = crypto.load_certificate(
crypto.FILETYPE_PEM, open(root_cert_path).read()
Expand Down
1 change: 1 addition & 0 deletions python-package/tests/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ user:
#
# `mc2 init` will not overwrite anything at this path
private_key: keys/user1.pem
public_key: keys/user1.pub
certificate: keys/user1.crt

# Path to CA certificate and private key
Expand Down
4 changes: 4 additions & 0 deletions python-package/tests/test_crypto.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@ def config(tmp_path):

test_cert = os.path.join(tmp_path, "test.crt")
test_priv_key = os.path.join(tmp_path, "test.pem")
test_pub_key = os.path.join(tmp_path, "test.pub")
test_symm_key = os.path.join(tmp_path, "test_sym.key")

# Rewrite config YAML with test paths
config = EnvYAML(original_config_path)
config["user"]["certificate"] = test_cert
config["user"]["private_key"] = test_priv_key
config["user"]["public_key"] = test_pub_key
config["user"]["symmetric_key"] = test_symm_key

# Point to root certificate
Expand Down Expand Up @@ -63,10 +65,12 @@ def schema(config):
def test_key_generation(keys, tmp_path):
test_cert = os.path.join(tmp_path, "test.crt")
test_priv_key = os.path.join(tmp_path, "test.pem")
test_pub_key = os.path.join(tmp_path, "test.pub")
test_symm_key = os.path.join(tmp_path, "test_sym.key")

assert os.path.exists(test_cert)
assert os.path.exists(test_priv_key)
assert os.path.exists(test_pub_key)
assert os.path.exists(test_symm_key)


Expand Down
3 changes: 2 additions & 1 deletion quickstart/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,13 @@ user:
# `mc2 init` will not overwrite anything at this path
symmetric_key: ${MC2_CLIENT_HOME}/playground/keys/user1_sym.key

# Path to your private key and certificate
# Path to your keypair and certificate.
# If you don't have a private key / certificate, specify paths here
# and run `mc2 init` to generate a keypair
#
# `mc2 init` will not overwrite anything at this path
private_key: ${MC2_CLIENT_HOME}/playground/keys/user1.pem
public_key: ${MC2_CLIENT_HOME}/playground/keys/user1.pub
certificate: ${MC2_CLIENT_HOME}/playground/keys/user1.crt

# Path to CA certificate and private key
Expand Down