Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Remove build golang lambda #535

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f3f6aa2
chore: try crane
wchaws Feb 20, 2024
f13bbcf
chore: try crane wip
wchaws Feb 20, 2024
0eff792
chore: try crane wip
wchaws Feb 20, 2024
96c307b
chore: projen update
wchaws Feb 20, 2024
8fd1025
chore: crane support read creds from secrets manager
wchaws Feb 20, 2024
7b43ef5
chore: self mutation
invalid-email-address Feb 20, 2024
a0f0d52
chore: remove golang build
wchaws Feb 21, 2024
43efd4f
Merge branch 'dev' of https://github.com/cdklabs/cdk-ecr-deployment i…
wchaws Feb 21, 2024
02d7f83
fxi: build error
wchaws Feb 21, 2024
c4531a1
fix: build error
wchaws Feb 21, 2024
f90e103
Merge branch 'main' into dev
wchaws Feb 21, 2024
7c11e1b
chore: remove build lambda
wchaws Feb 21, 2024
af1c525
chore: add unit test for python
wchaws Feb 21, 2024
c8e6192
chore: enable lambda test
wchaws Feb 21, 2024
bf66e8e
chore: self mutation
invalid-email-address Feb 21, 2024
d8438ae
chore: add python unit test
wchaws Feb 21, 2024
5c63612
Merge branch 'dev' of https://github.com/cdklabs/cdk-ecr-deployment i…
wchaws Feb 21, 2024
13462b1
chore: move pyproject.toml -> lambda2/pyproject.toml
wchaws Feb 21, 2024
1939c90
Merge branch 'main' into dev
wchaws Mar 3, 2024
b1b8fc5
chore: add more python ut
wchaws Mar 3, 2024
8013833
Merge branch 'dev' of https://github.com/cdklabs/cdk-ecr-deployment i…
wchaws Mar 3, 2024
620fb87
chore: remove golang lambda
wchaws Mar 3, 2024
20386dc
chore: add comments
wchaws Mar 3, 2024
58a4878
chore: update README.md
wchaws Mar 3, 2024
963dedf
chore: update REAMDE.md
wchaws Mar 3, 2024
1614181
chore: update README
wchaws Mar 3, 2024
8e62c49
chore: update README
wchaws Mar 3, 2024
7b80c20
Merge branch 'main' into dev
wchaws Mar 11, 2024
24865dd
Merge branch 'main' into dev
wchaws Mar 16, 2024
e58665a
Merge branch 'main' into dev
wchaws Mar 19, 2024
aee18b7
Merge branch 'main' into dev
wchaws Apr 5, 2024
b6d829c
Merge branch 'main' into dev
wchaws Jun 13, 2024
5d7249c
Merge branch 'main' into dev
wchaws Jun 14, 2024
0f0ff9e
chore: set major version to 4
wchaws Jun 14, 2024
41431b3
Merge branch 'main' of github.com:cdklabs/cdk-ecr-deployment into dev
wchaws Jun 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 0 additions & 26 deletions .github/workflows/build-lambda.yml

This file was deleted.

23 changes: 0 additions & 23 deletions .github/workflows/release.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions .gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 10 additions & 2 deletions .projen/tasks.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

56 changes: 9 additions & 47 deletions .projenrc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
// SPDX-License-Identifier: Apache-2.0

import { CdklabsConstructLibrary } from 'cdklabs-projen-project-types';
import { github } from 'projen';

const project = new CdklabsConstructLibrary({
setNodeEngineVersion: false,
Expand All @@ -13,7 +12,7 @@ const project = new CdklabsConstructLibrary({
cdkVersion: '2.0.0',
cdkVersionPinning: false,
defaultReleaseBranch: 'main',
majorVersion: 3,
majorVersion: 4,
enablePRAutoMerge: true,
name: 'cdk-ecr-deployment',
projenrcTs: true,
Expand All @@ -36,63 +35,26 @@ const project = new CdklabsConstructLibrary({
repositoryUrl: 'https://github.com/cdklabs/cdk-ecr-deployment', /* The repository is the location where the actual code for your package lives. */
gitignore: [
'cdk.out/',
'*.tar.gz',
'*.zip',
'/layer/crane/crane',
'__pycache__/',
'.pytest_cache',
], /* Additional entries to .gitignore. */
npmignore: [
'/cdk.out',
], /* Additional entries to .npmignore. */
});

project.preCompileTask.exec('layer/build.sh');
project.testTask.exec('test/lambda/test.sh');

project.package.addField('jsiiRosetta', {
exampleDependencies: {
'@types/node': '^18',
},
});

project.release?.addJobs({
release_prebuilt_lambda: {
runsOn: ['ubuntu-latest'],
name: 'Publish Lambda to GitHub Releases',
needs: ['release'],
permissions: {
contents: github.workflows.JobPermission.WRITE,
},
steps: [
{
name: 'Checkout',
uses: 'actions/checkout@v2',
with: {
'fetch-depth': 0,
},
},
{
name: 'Download build artifacts',
uses: 'actions/download-artifact@v4',
with: {
name: 'build-artifact',
path: '.repo',
},
},
{
name: 'Build lambda',
run: [
'docker build -t cdk-ecr-deployment-lambda --build-arg GOPROXY="https://goproxy.io|https://goproxy.cn|direct" lambda',
'docker run -v $PWD/lambda:/out cdk-ecr-deployment-lambda cp /asset/bootstrap /out',
'echo $(sha256sum lambda/bootstrap | awk \'{ print $1 }\') > lambda/bootstrap.sha256',
].join(' && '),
},
{
name: 'Release lambda',
// For some reason, need '--clobber' otherwise we always get errors that these files already exist. They're probably
// uploaded elsewhere but TBH I don't know where so just add this flag to make it not fail.
run: 'gh release upload --clobber -R $GITHUB_REPOSITORY v$(cat .repo/dist/version.txt) lambda/bootstrap lambda/bootstrap.sha256 ',
env: {
GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}',
GITHUB_REPOSITORY: '${{ github.repository }}',
},
},
],
},
});

project.package.addField('resolutions', {
'trim-newlines': '3.0.1',
Expand Down
5 changes: 5 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"python.analysis.extraPaths": [
"./lambda"
]
}
36 changes: 1 addition & 35 deletions API.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 12 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,6 @@ stage. Please use only ^2.0.0.
## Features

- Copy image from ECR/external registry to (another) ECR/external registry
- Copy an archive tarball image from s3 to ECR/external registry

## Environment variables

Enable flags: `true`, `1`. e.g. `export CI=1`

- `CI` indicate if it's CI environment. This flag will enable building lambda from scratch.
- `NO_PREBUILT_LAMBDA` disable using prebuilt lambda.
- `FORCE_PREBUILT_LAMBDA` force using prebuilt lambda.

⚠️ If you want to force using prebuilt lambda in CI environment to reduce build time. Try `export FORCE_PREBUILT_LAMBDA=1`.

⚠️ The above flags are only available in cdk-ecr-deployment 2.x and 3.x.

## Examples

Expand All @@ -52,9 +39,9 @@ new ecrdeploy.ECRDeployment(this, 'DeployDockerImage2', {
// Copy from private docker registry to ECR.
// The format of secret in aws secrets manager must be plain text! e.g. <username>:<password>
new ecrdeploy.ECRDeployment(this, 'DeployDockerImage3', {
src: new ecrdeploy.DockerImageName('javacs3/nginx:latest', 'username:password'),
// src: new ecrdeploy.DockerImageName('javacs3/nginx:latest', 'aws-secrets-manager-secret-name'),
// src: new ecrdeploy.DockerImageName('javacs3/nginx:latest', 'arn:aws:secretsmanager:us-west-2:000000000000:secret:id'),
src: new ecrdeploy.DockerImageName('your-private-docker-registry/nginx:latest', 'username:password'),
// src: new ecrdeploy.DockerImageName('your-private-docker-registry/nginx:latest', 'aws-secrets-manager-secret-name'),
// src: new ecrdeploy.DockerImageName('your-private-docker-registry/nginx:latest', 'arn:aws:secretsmanager:us-west-2:000000000000:secret:id'),
dest: new ecrdeploy.DockerImageName(`${cdk.Aws.ACCOUNT_ID}.dkr.ecr.us-west-2.amazonaws.com/my-nginx3:latest`),
}).addToPrincipalPolicy(new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
Expand All @@ -68,17 +55,19 @@ new ecrdeploy.ECRDeployment(this, 'DeployDockerImage3', {
## Sample: [test/example.ecr-deployment.ts](./test/example.ecr-deployment.ts)

```shell
# Generate crane lambda layer firstly.
./layer/build.sh
Comment on lines 55 to +59
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section needs some clarification that these instructions only apply if you're developing on the repository, not if you're a user of the construct.

README.md is typically consumer-targeted, and the section title says "Sample", so it should (mostly) only contain instructions for consumers.

These instructions would make more sense in CONTRIBUTING.md in a section title "How to run tests".

If you want to keep them here, clearly label the instructions as intended for developers, not for consumers.


# Run the following command to try the sample.
NO_PREBUILT_LAMBDA=1 npx cdk deploy -a "npx ts-node -P tsconfig.dev.json --prefer-ts-exts test/example.ecr-deployment.ts"
npx cdk deploy -a "npx ts-node -P tsconfig.dev.json --prefer-ts-exts test/example.ecr-deployment.ts"

# To run python unit test.
pytest -c lambda/pyproject.toml

```

## [API](./API.md)

## Tech Details & Contribution

The core of this project relies on [containers/image](https://github.com/containers/image) which is used by [Skopeo](https://github.com/containers/skopeo).
Please take a look at those projects before contribution.

To support a new docker image source(like docker tarball in s3), you need to implement [image transport interface](https://github.com/containers/image/blob/master/types/types.go). You could take a look at [docker-archive](https://github.com/containers/image/blob/ccb87a8d0f45cf28846e307eb0ec2b9d38a458c2/docker/archive/transport.go) transport for a good start.

To test the `lambda` folder, `make test`.
The underlying implementation depends on the https://github.com/google/go-containerregistry.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean?

9 changes: 0 additions & 9 deletions lambda/.dockerignore

This file was deleted.

26 changes: 0 additions & 26 deletions lambda/Dockerfile

This file was deleted.

33 changes: 0 additions & 33 deletions lambda/Makefile

This file was deleted.

Loading
Loading