Getting BadCiphertextException: Invalid version

[Smaz2024]

I am creating a POC where I have created a public and private key pair. The private key is stored in KMS. The public key is stored in Secrets Manager. I am using the following code for envelope encryption.

Java Code

package com.poc.envelope.encryption;

import java.io.StringWriter;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.security.KeyFactory;
import java.security.PublicKey;
import java.security.spec.X509EncodedKeySpec;
import java.util.Base64;
import java.util.Collections;
import java.util.Map;

import org.bouncycastle.util.io.pem.PemObject;
import org.bouncycastle.util.io.pem.PemWriter;

import com.amazonaws.encryptionsdk.AwsCrypto;
import com.amazonaws.encryptionsdk.CommitmentPolicy;
import com.amazonaws.encryptionsdk.CryptoAlgorithm;
import com.amazonaws.encryptionsdk.CryptoResult;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.LambdaLogger;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;

import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.kms.KmsClient;
import software.amazon.awssdk.services.kms.model.EncryptionAlgorithmSpec;
import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueResponse;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsRsaKeyringInput;
import software.amazon.cryptography.materialproviders.model.MaterialProvidersConfig;

//https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/java-example-code.html
public class HandleRequestForEnvelopeEncryption implements RequestHandler<Map<String, String>, String> {
LambdaLogger logger;

@Override
/**
 * 
 */
public String handleRequest(Map<String, String> event, Context context) {
	logger = context.getLogger();
	try {
		String secretName = ""; // Name of the secret in AWS Secrets Manager
		String secretKey = "";
                    String rsaKeyArn = // removed the value
		
		String plaintext = "Some would say that literature has been the foundation of life. It has the ability to emphasize worldly issues and human cataclysm. These are written in paragraphs that makes our minds imagine things based on what the context is showing us. It enables every individual to see what others may perceive and even make other living being like animals and plants to be characters of a particular piece. Literature provided each one a chance to catch a lesson about life experiences from the tragic stories to the happiest one.";

		// Fetch the public key from AWS Secrets Manager
		String publicKeyPem = fetchPublicKeyFromSecretsManager(secretName, logger, secretKey);

		logger.log("Public Key : " + publicKeyPem);

		// Convert PEM to PublicKey object
		ByteBuffer publicKeyByteBuffer = getPublicKeyFromPem(publicKeyPem);

		logger.log("Public Key Byte Buffer : " + publicKeyByteBuffer);

		

		final CreateAwsKmsRsaKeyringInput kmsKeyRing = CreateAwsKmsRsaKeyringInput.builder()
				.kmsClient(KmsClient.create()).kmsKeyId(rsaKeyArn).publicKey(publicKeyByteBuffer)
				.encryptionAlgorithm(EncryptionAlgorithmSpec.RSAES_OAEP_SHA_256).build();


		logger.log("Create Raw Rsa Keyring Input .........................................................");

		final MaterialProviders matProv = MaterialProviders.builder()
				.MaterialProvidersConfig(MaterialProvidersConfig.builder().build()).build();

		logger.log("Material Providers ....................................................................");

		
		IKeyring awsKmsRsaKeyring = matProv.CreateAwsKmsRsaKeyring(kmsKeyRing);

		logger.log("Raw Pub Rsa Keyring ....................................................................");

		// Encrypt the message
		// String ciphertext = encryptMessage(plaintext, rawPubRsaKeyring);
		String ciphertext = encryptMessage(plaintext, awsKmsRsaKeyring);
		String decryptedtext = decryptMessage(ciphertext, awsKmsRsaKeyring);

		logger.log("Final Output in encrypted format: " + ciphertext);
		logger.log("Final Output in decrypted text: " + decryptedtext);

		if (plaintext.equals(decryptedtext))
			logger.log("Is encrypt decrypt success : ");

		return ciphertext;
	} catch (Exception e) {
		e.printStackTrace();
		logger.log(e.getLocalizedMessage());
		return "Error: " + e.getMessage();
	}
}

/**
 * 
 * @param secretName
 * @param logger
 * @param secretKey  TODO
 * @return
 */
private String fetchPublicKeyFromSecretsManager(String secretName, LambdaLogger logger, String secretKey) {
	Region region = Region.of("");

	logger.log("Secrets Manager Client ......................................................");
	// Create a Secrets Manager client
	SecretsManagerClient client = SecretsManagerClient.builder().region(region).build();

	logger.log("Get Secret Value Request .................................................... ");
	GetSecretValueRequest getSecretValueRequest = GetSecretValueRequest.builder().secretId(secretName).build();

	GetSecretValueResponse getSecretValueResponse;
	try {
		getSecretValueResponse = client.getSecretValue(getSecretValueRequest);

		logger.log("Get Secret Value Response " + getSecretValueResponse.secretString());

	} catch (Exception e) {
		logger.log(e.toString());
		throw e;
	}

	JsonObject jsonObject = JsonParser.parseString(getSecretValueResponse.secretString()).getAsJsonObject();

	// printing the values
	logger.log(jsonObject.get(secretKey).getAsString());

	return jsonObject.get(secretKey).getAsString();
}

/**
 * 
 * @param pem
 * @return
 * @throws Exception
 */
private ByteBuffer getPublicKeyFromPem(String pem) throws Exception {
	String publicKeyPEM = pem.replace("-----BEGIN PUBLIC KEY-----", "").replace("-----END PUBLIC KEY-----", "")
			.replaceAll("\\s", "");
	byte[] encoded = Base64.getDecoder().decode(publicKeyPEM);

	logger.log("Encoded publicKeyPEM ...... " + encoded);

	

	StringWriter publicKeyStringWriter = new StringWriter();
	PemWriter publicKeyPemWriter = new PemWriter(publicKeyStringWriter);
	try {
		logger.log("Public Key Encoded .......  " + encoded);
		publicKeyPemWriter.writeObject(new PemObject("PUBLIC KEY", encoded));
		publicKeyPemWriter.close();
	} catch (Exception e) {
		throw new RuntimeException("Exception while writing public key PEM", e);
	}
	return StandardCharsets.UTF_8.encode(publicKeyStringWriter.toString());
	
	
}

/**
 * 
 * @param plaintext
 * @param keyring
 * @return
 */
private String encryptMessage(String plaintext, IKeyring keyring) {
	// Instantiate the SDK
	final AwsCrypto crypto = AwsCrypto.builder().withCommitmentPolicy(CommitmentPolicy.ForbidEncryptAllowDecrypt)
			.withEncryptionAlgorithm(CryptoAlgorithm. ALG_AES_256_GCM_IV12_TAG16_NO_KDF).build();

	logger.log(
			"Aws Crypto - Encrypt .................................................................................. ");
	// Create an encryption context
	final Map<String, String> encryptionContext = Collections.singletonMap("ExampleContextKey",
			"ExampleContextValue");

	// Encrypt the data
	final CryptoResult<byte[], ?> encryptResult = crypto.encryptData(keyring,
			plaintext.getBytes(StandardCharsets.UTF_8), encryptionContext);

	logger.log("Crypto Result post encryption..... " + encryptResult.getResult());

	return Base64.getEncoder().encodeToString(encryptResult.getResult());
}

/**
 * 
 * @param plaintext
 * @param keyring
 * @return
 */
private String decryptMessage(String ciphertext, IKeyring keyring) {
	// Instantiate the SDK
	final AwsCrypto crypto = AwsCrypto.builder().withCommitmentPolicy(CommitmentPolicy.ForbidEncryptAllowDecrypt)
			.withEncryptionAlgorithm(CryptoAlgorithm. ALG_AES_256_GCM_IV12_TAG16_NO_KDF).build();
									 
	logger.log("Aws Crypto - Decrypt ...... " + ciphertext);
	// Create an encryption context
	final Map<String, String> encryptionContext = Collections.singletonMap("ExampleContextKey",
			"ExampleContextValue");

	// 5. Decrypt the data
	final CryptoResult<byte[], ?> decryptResult = crypto.decryptData(keyring,
			ciphertext.getBytes(StandardCharsets.UTF_8),
			// Verify that the encryption context in the result contains the
			// encryption context supplied to the encryptData method
			encryptionContext);
	logger.log("Crypto Result post decryption..... " + decryptResult.getResult());

	return Base64.getEncoder().encodeToString(decryptResult.getResult());
}

}

pom.xml

4.0.0
aws.envelope.encryption
aws.envelope.encryption
0.0.1-SNAPSHOT

<dependencyManagement>
	<dependencies>
		<dependency>
			<groupId>software.amazon.awssdk</groupId>
			<artifactId>bom</artifactId>
			<version>2.26.12</version> <!-- Use the latest version available -->
			<type>pom</type>
			<scope>import</scope>
		</dependency>
	</dependencies>
</dependencyManagement>

<build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-shade-plugin</artifactId>
			<version>3.2.2</version>
			<configuration>
				<createDependencyReducedPom>false</createDependencyReducedPom>
				<transformers>
					<transformer
						implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
						<mainClass>
							com.poc.envelope.encryption.HandleRequestForEnvelopeEncryption</mainClass>
					</transformer>
				</transformers>
			</configuration>
			<executions>
				<execution>
					<phase>package</phase>
					<goals>
						<goal>shade</goal>
					</goals>
				</execution>
			</executions>
		</plugin>
	</plugins>
</build>

<dependencies>
	<!-- AWS Lambda Java Core dependency -->
	<dependency>
		<groupId>com.amazonaws</groupId>
		<artifactId>aws-lambda-java-core</artifactId>
		<version>1.2.1</version>
	</dependency>

	<!-- AWS Lambda Java Events dependency -->
	<dependency>
		<groupId>com.amazonaws</groupId>
		<artifactId>aws-lambda-java-events</artifactId>
		<version>3.9.0</version>
	</dependency>
	<!-- AWS SDK for Secrets Manager -->
	<dependency>
		<groupId>software.amazon.awssdk</groupId>
		<artifactId>secretsmanager</artifactId>

	</dependency>


	<dependency>
		<groupId>com.amazonaws</groupId>
		<artifactId>aws-encryption-sdk-java</artifactId>
		<version>3.0.1</version>
	</dependency>
	<!-- AWS Encryption SDK (Material Providers) -->
	<dependency>
		<groupId>software.amazon.cryptography</groupId>
		<artifactId>aws-cryptographic-material-providers</artifactId>
		<version>1.5.0</version>
	</dependency>
	<dependency>
		<groupId>com.fasterxml.jackson.core</groupId>
		<artifactId>jackson-databind</artifactId>
		<version>2.13.1</version>
	</dependency>
	<dependency>
		<groupId>org.slf4j</groupId>
		<artifactId>slf4j-api</artifactId>
		<version>1.7.30</version>
	</dependency>

	<!-- Logback (for logging) -->
	<dependency>
		<groupId>ch.qos.logback</groupId>
		<artifactId>logback-classic</artifactId>
		<version>1.2.3</version>
	</dependency>
	<dependency>
		<groupId>org.bouncycastle</groupId>
		<artifactId>bcprov-jdk15on</artifactId>
		<version>1.70</version>
	</dependency>
	<dependency>
		<groupId>org.bouncycastle</groupId>
		<artifactId>bcpkix-jdk15on</artifactId>
		<version>1.70</version>
	</dependency>

	<dependency>
		<groupId>software.amazon.awssdk</groupId>
		<artifactId>kms</artifactId>
		<version>2.26.12</version>
	</dependency>
	<dependency>
		<groupId>software.amazon.awssdk</groupId>
		<artifactId>aws-sdk-java</artifactId>
		<version>2.26.12</version>
		<scope>provided</scope>
	</dependency>
	<dependency>
		<groupId>com.google.code.gson</groupId>
		<artifactId>gson</artifactId>
		<version>2.10.1</version>
	</dependency>

</dependencies>

=================

Error I am getting

com.amazonaws.encryptionsdk.exception.BadCiphertextException: Invalid version
at com.amazonaws.encryptionsdk.model.CiphertextHeaders.deserialize(CiphertextHeaders.java:588)
at com.amazonaws.encryptionsdk.ParsedCiphertext.(ParsedCiphertext.java:42)
at com.amazonaws.encryptionsdk.AwsCrypto.decryptData(AwsCrypto.java:752)

I have tried several options but it did not help. PLease share inputs

[Smaz2024]

I have removed several AWS related names and arns here for confidentiality.

[josecorella]

Hey @Smaz2024,
Thank you for cutting the issue!
It looks like you are running into a Base64 encoding problem between your encrypt and decrypt functions.

/**
 * 
 * @param plaintext
 * @param keyring
 * @return
 */
private String encryptMessage(String plaintext, IKeyring keyring) {
	// Instantiate the SDK
	final AwsCrypto crypto = AwsCrypto.builder().withCommitmentPolicy(CommitmentPolicy.ForbidEncryptAllowDecrypt)
			.withEncryptionAlgorithm(CryptoAlgorithm. ALG_AES_256_GCM_IV12_TAG16_NO_KDF).build();

	logger.log(
			"Aws Crypto - Encrypt .................................................................................. ");
	// Create an encryption context
	final Map<String, String> encryptionContext = Collections.singletonMap("ExampleContextKey",
			"ExampleContextValue");

	// Encrypt the data
	final CryptoResult<byte[], ?> encryptResult = crypto.encryptData(keyring,
			plaintext.getBytes(StandardCharsets.UTF_8), encryptionContext);

	logger.log("Crypto Result post encryption..... " + encryptResult.getResult());

	return Base64.getEncoder().encodeToString(encryptResult.getResult());
}

/**
 * 
 * @param plaintext
 * @param keyring
 * @return
 */
private String decryptMessage(String ciphertext, IKeyring keyring) {
	// Instantiate the SDK
	final AwsCrypto crypto = AwsCrypto.builder().withCommitmentPolicy(CommitmentPolicy.ForbidEncryptAllowDecrypt)
			.withEncryptionAlgorithm(CryptoAlgorithm. ALG_AES_256_GCM_IV12_TAG16_NO_KDF).build();
									 
	logger.log("Aws Crypto - Decrypt ...... " + ciphertext);
	// Create an encryption context
	final Map<String, String> encryptionContext = Collections.singletonMap("ExampleContextKey",
			"ExampleContextValue");

	// 5. Decrypt the data
	final CryptoResult<byte[], ?> decryptResult = crypto.decryptData(keyring,
			ciphertext.getBytes(StandardCharsets.UTF_8),
			// Verify that the encryption context in the result contains the
			// encryption context supplied to the encryptData method
			encryptionContext);
	logger.log("Crypto Result post decryption..... " + decryptResult.getResult());

	return Base64.getEncoder().encodeToString(decryptResult.getResult());
}

It looks like you are Base64 encoding the ciphertext after you successfully encrypting it but not decoding it back to bytes before you decrypt it. If you base64 decode it before you pass it to the decryptData method the com.amazonaws.encryptionsdk.exception.BadCiphertextException: Invalid version exception should go away.

Please let us know if you run into more issues. Thanks!

[Smaz2024]

Hi @josecorella : Thanks a lot for your reply and helped. The solution worked for me . However I was experimenting more on the same codebase.

1. Tried saving public key of the pair in RAWRSA Keyring in Java and the corresponding private key in KMSKeyRing. Tried to encrypt the text with RAW RSA Keyring and decrypt the ciphertext using KMSKeyring. Ideally since both are part of same pair so should work, but it was failing.

Code

String publicKeyPEM = publicKeyPem.replace("-----BEGIN PUBLIC KEY-----", "")
                    .replace("-----END PUBLIC KEY-----", "")
                    .replaceAll("\\s+", ""); 

			byte[] publicKeyBytes = Base64.getDecoder().decode(publicKeyPEM);
			X509EncodedKeySpec spec = new X509EncodedKeySpec(publicKeyBytes);
			
			KeyFactory kf = KeyFactory.getInstance("RSA");
			RSAPublicKey generatePublic = (RSAPublicKey) kf.generatePublic(spec);
			

			final AwsCrypto crypto = AwsCrypto.builder()
					.withEncryptionAlgorithm(CryptoAlgorithm.ALG_AES_256_GCM_IV12_TAG16_HKDF_SHA256)
					.withCommitmentPolicy(CommitmentPolicy.ForbidEncryptAllowDecrypt).build();


			final MaterialProviders matProv = MaterialProviders.builder()
					.MaterialProvidersConfig(MaterialProvidersConfig.builder().build()).build();


			/*
			 * final CreateRawRsaKeyringInput encryptingKeyringInput =
			 * CreateRawRsaKeyringInput.builder()
			 * .keyName("rsa-key").keyNamespace("rsa-keyring").paddingScheme(PaddingScheme.
			 * PKCS1) .publicKey(publicKeyByteBuffer).build();
			 */
			JceMasterKey jcemasterKey = JceMasterKey.getInstance(

					generatePublic, null, "", "",
					// "RSA/ECB/PKCS1Padding"
					"RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
			// "RSA/ECB/PKCS1Padding"
			

			
			final Map<String, String> encryptionContext = Collections.singletonMap("ExampleContextKey",
					"ExampleContextValue");

			
			final CryptoResult<byte[], ?> encryptResult = crypto.encryptData(jcemasterKey,
					plaintext.getBytes(StandardCharsets.UTF_8), encryptionContext);
			final byte[] ciphertext = encryptResult.getResult();


			final CreateAwsKmsRsaKeyringInput kmsKeyRing = CreateAwsKmsRsaKeyringInput.builder()
					.kmsClient(KmsClient.create()).kmsKeyId(rsaKeyArn)
					// .publicKey(publicKeyByteBuffer)
					.encryptionAlgorithm(EncryptionAlgorithmSpec.RSAES_OAEP_SHA_256).build();

			IKeyring awsKmsRsaKeyring = matProv.CreateAwsKmsRsaKeyring(kmsKeyRing);


			final CryptoResult<byte[], ?> decryptResult = crypto.decryptData(awsKmsRsaKeyring, ciphertext,
					// Verify that the encryption context in the result contains the
					// encryption context supplied to the encryptData method
					encryptionContext);

			logger.log(
					"Final Output in decrypted text: " + new String(decryptResult.getResult(), StandardCharsets.UTF_8));

Error :

Unable to decrypt data key: No Encrypted Data Keys found to match.
Expected:
KeyProviderId: , KeyProviderInfo:

2. Next I tried with 2 separate KMSKeyRings but again it failed. (Created 2 separate key pairs and secrets)

Code

final CreateAwsKmsRsaKeyringInput kmsKeyRingEncryption = CreateAwsKmsRsaKeyringInput.builder()
					.kmsClient(KmsClient.create()).kmsKeyId(rsaKeyArn).publicKey(publicKeyByteBuffer2)
					.encryptionAlgorithm(EncryptionAlgorithmSpec.RSAES_OAEP_SHA_256).build();
			
final CreateAwsKmsRsaKeyringInput kmsKeyRingDecryption = CreateAwsKmsRsaKeyringInput.builder()
					.kmsClient(KmsClient.create()).kmsKeyId(rsaKeyArn2).publicKey(publicKeyByteBuffer)
					.encryptionAlgorithm(EncryptionAlgorithmSpec.RSAES_OAEP_SHA_256).build();

logger.log("Material Providers ....................................................................");

			IKeyring awsKmsRsaKeyringEncryption = matProv.CreateAwsKmsRsaKeyring(kmsKeyRingEncryption);
			IKeyring awsKmsRsaKeyringDecryption = matProv.CreateAwsKmsRsaKeyring(kmsKeyRingDecryption);

			logger.log("Raw Pub Rsa Keyring ....................................................................");

			// Encrypt the message
			String ciphertext = encryptMessage(plaintext, awsKmsRsaKeyringEncryption);
			String decryptedtext = decryptMessage(ciphertext, awsKmsRsaKeyringDecryption);

Error:

Unable to decrypt data key: No Encrypted Data Keys found to match.
Expected:
KeyProviderId: aws-kms-rsa, KeyProviderInfo: arn:aws:kms:ap-south-1:732077235871:key/f2cbec0a-1ccd-4869-bc6b-bd51caadb6ca

The reason I am trying to do so because the encrypting and decrypting parties can be in different cloud/on-prem environment or different account in AWS. So separate key rings can be used.

Please help

[texastony]

@Smaz2024 ,

I am sorry Crypto Tools did not respond to your latest post sooner.

When a Keyring wraps a data key,
it serializes information that describes the Keyring,
such that at decryption,
the decryptor knows which Keyring to use to decrypt it.

Because you are using different Keyrings at encrypt and decrypt,
the description created at Encrypt
does not match the Decrypt expectation.

Your options:

1. Use a Raw RSA Keyring on both sides
2. Use a KMS RSA Keyring on both sides
3. Write a custom Keyring and use that on both sides

But they all boil down to a consistent description of what encrypted the data key.

Think of a regular physical key chain.
You need to know which Key opens which lock.

The same principle applies here.