Customer Managed Keys (CMK) Migration Scanner

---

• Customer Managed Keys (CMK) Migration Scanner
  • Overview
  • Project Structure
  • Getting started
    • Clone the source code repository
    • Run the program
      • Using environment variables
        • Usage example with environment variables
      • Using arguments
        • Usage example with arguments
  • Help

Overview

Azure Cosmos DB allows a second layer of encryption with keys managed by customers. This feature is called Customer-Managed Keys (CMK).

In order to allow an existing Cosmos DB account to use to CMK, a scan needs to be done to ensure that the account doesn't have "big ids". A "big id" is a document id that exceeds 990 characters length. This scan is mandatory for the CMK migration and it is done by Microsoft automatically/ However, customers can replicate the scan result with this project. More info: https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-setup-customer-managed-keys-existing-accounts

Project Structure

The CMK Migration Tool project is a C# command-line executable. The tool consists on the main file, with an Utility files for validations, that scans by using one of the two available scanners (depending the APIs):

• For Mongo uses REST.
• For the Gremlin, Table and SQL uses SDK: Cosmos Client.

Getting started

Clone the source code repository

From a command prompt, execute the following command in an empty working folder that will house the source code.

git clone https://github.com/AzureCosmosDB/Cosmos-DB-Non-CMK-to-CMK-Migration-Scanner.git

Run the program

Move to the CMK_Migration_Scanner folder, the project files are located there. Before running the program, you need to select the authorization method to connect to your Azure Cosmos DB Account. You can choose up to three different authorization methods, which depends on the API type of your account. The options are shown as:

• If you are using MongoDB: Account Keys.
• If you are using Table: AAD or Account Keys.
• If you are using SQL or Gremlin: AAD, Account Keys or Connection String.

The next step is to provide your account details. Depending on the authorization method selected, the program could require some of this information:

• Hostname: Is the URI for the account.
• Account Key: Is the master key of your account. More info: Primary/secondary keys.
• Connection String: Is a string combination of the hostname and accounts keys. It is different depending on the API of the Cosmos DB account. More info: Retrieve Connection Strings using CLI.
• Tenant ID: Microsoft Entra ID entity in which you have your resources. More info: Get Tenant ID.
• Client ID: Idenitifier of your App Registration. If you want to use AAD authorization, you will need to have a service prinicipal. Please follow the steps here, but only to Create a Service Principal.
• Client Secret: A client secret from the App Registration identified with your Client ID. More info: How to Create a Secret.

The script allows to run using environment variables or arguments. Once you understood the information mentioned earlier, move to the the desired run method and check which are your required details.

There is an optional value of exponential-base-value, which will be used if your account is facing issues with 429s http errors. This value helps to create batches of documents to query, depending on the amount of documents per container and the retry attempts. The value can be from 2 to 10. The default value is 2.

Using environment variables

You will always need to set the variables:

CMK_AUTH_TYPE CMK_API_TYPE

Then, set the following variables depending on your authorization method as it follows:

• If using Account Keys: CMK_HOSTNAME, CMK_ACCOUNT_KEY.
• If using Connection String: CMK_CONNECTION_STRING.
• If using AAD: CMK_TENANT_ID, CMK_CLIENT_ID, CMK_CLIENT_SECRET.

Finally, just run:

dotnet run Env

Usage example with environment variables

$env:CMK_CONNECTION_STRING = "AccountEndpoint=<uri>;AccountKey=<key>;"
$env:CMK_AUTH_TYPE = "ConnectionString"
$env:CMK_API_TYPE = "SQL"
dotnet run Env

Using arguments

You will always need to:

1. Choose auth method: AAD, AccountKey or ConnectionString.
2. Add --api-type: Could be MongoDB, SQL, Gremlin or Table.

• If using Account Keys: Add options: --host-name and --account-key.
• If using Connection String: Add option: --connection-string.
• If using AAD: Add options: --host-name, --tenant-id, client-id and client-secret.

Finally, just run:

dotnet run <auth-method> <options>

Usage example with arguments

dotnet run AAD --api-type Gremlin --host-name $hostname --tenant-id $tenantId --client-id $clientAppId --client-secret $clientSecretValue

Help

For more help, you can use the command --h or -h. An example will be to run:

dotnet run -- -h