Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(lerna): aggressive patch bump #1163

Open
wants to merge 20 commits into
base: release-v10
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
80e999f
fix(lerna): aggressive patch bump
SudilHasithaCognite Oct 7, 2024
b62a9ca
fix(lerna): aggressive patch bump keeping constant version
SudilHasithaCognite Oct 8, 2024
74d6b15
fix(lerna): aggressive patch bump keeping constant version
SudilHasithaCognite Oct 8, 2024
1dcedda
fix(lerna): unpredictable master branch versioning
SudilHasithaCognite Oct 9, 2024
d77da16
chore(fix): switch to patch from conventional commit
SudilHasithaCognite Oct 10, 2024
75af492
chore(doc): update the docs
SudilHasithaCognite Oct 25, 2024
aca7bab
Merge branch 'release-v10' into constant-package-version
SudilHasithaCognite Oct 25, 2024
41796cb
chore(contribution.md): fix documentation
SudilHasithaCognite Nov 4, 2024
54e46c0
fix(release): add custom script to remove rc tag
SudilHasithaCognite Nov 7, 2024
643a8bd
fix(scripts): lint errors
SudilHasithaCognite Nov 7, 2024
2533ec9
Merge branch 'release-v10' into constant-package-version
SudilHasithaCognite Nov 7, 2024
1c96d11
fix(scripts): lint errors
SudilHasithaCognite Nov 7, 2024
97b7a2c
fix(scripts): lint errors
SudilHasithaCognite Nov 7, 2024
bed92df
fix(scripts): lint errors
SudilHasithaCognite Nov 7, 2024
114243c
fix(scripts): lint errors
SudilHasithaCognite Nov 7, 2024
5528e00
feat(release): add rc tag removal script run
SudilHasithaCognite Nov 8, 2024
a0b1adb
fix(versioning): modify rc tag removing script and release.yaml
SudilHasithaCognite Nov 20, 2024
61c0fb9
fix(linting): modify rc tag removing script
SudilHasithaCognite Nov 20, 2024
981a74e
fix(contributing): add the mistakenly removed doc
SudilHasithaCognite Nov 26, 2024
8815be7
fix(contributing): add the mistakenly removed doc
SudilHasithaCognite Nov 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 47 additions & 11 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,18 @@ jobs:
run: yarn build

publish:
if: |-
${{
!contains(github.event.commits[0].message, '[skip-ci]') &&
(
startsWith(github.event.commits[0].message, 'fix:') ||
startsWith(github.event.commits[0].message, 'fix(') ||
startsWith(github.event.commits[0].message, 'feat:') ||
startsWith(github.event.commits[0].message, 'feat(') ||
contains(github.event.commits[0].message, '[release]')
if: ${{ !contains(github.event.head_commit.message, '[skip-ci]') &&
(
(github.ref == 'refs/heads/master' && contains(github.event.head_commit.message, '[release]')) ||
(startsWith(github.ref, 'refs/heads/release-') &&
(startsWith(github.event.head_commit.message, 'fix:') ||
startsWith(github.event.head_commit.message, 'fix(') ||
startsWith(github.event.head_commit.message, 'feat:') ||
startsWith(github.event.head_commit.message, 'feat(') ||
contains(github.event.head_commit.message, '[release]')
)
)
)
}}
runs-on: ubuntu-latest
environment: CD
Expand Down Expand Up @@ -81,17 +83,51 @@ jobs:
- name: Build doc
run: yarn docs:bundle

# Remove rc tag in master branch when it is a [release] commit
- name: Remove RC tags
if: ${{ github.ref == 'refs/heads/master' && contains(github.event.head_commit.message, 'Merge pull request') && contains(github.event.head_commit.message, 'release-') && contains(github.event.head_commit.message, '[release]') }}
run: |
node scripts/remove-rc-tags.js
git config --global user.email "[email protected]"
git config --global user.name "Cognite CICD"
git add packages/*/package.json
git commit -m "chore: remove rc tags [skip-ci]"
git push origin master

- name: Lerna version
env:
GH_TOKEN: ${{ secrets.GH_LERNA_TOKEN }} # Temporary fix to make lerna able to push the new versions commit to master
GITHUB_TOKEN: ${{ secrets.GH_LERNA_TOKEN }} # Temporary fix to make lerna able to push the new versions commit to master
run: yarn lerna version --conventional-commits --conventional-prerelease --preid rc --create-release github --no-private --yes
run: |
if [[ "${GITHUB_REF}" == "refs/heads/master" ]]; then
EVENT_PATH_CONTENT=$(cat ${GITHUB_EVENT_PATH} | jq -r '.')
HEAD_COMMIT_MSG=$(echo $EVENT_PATH_CONTENT | jq -r '.head_commit.message')
MERGE_COMMIT_MSG=$(echo $EVENT_PATH_CONTENT | jq -r '.commits[-1].message')

if [[ "$HEAD_COMMIT_MSG" == *"Merge"* && "$HEAD_COMMIT_MSG" == *"release-"* && "$HEAD_COMMIT_MSG" == *"[release]"* ]] ||
[[ "$MERGE_COMMIT_MSG" == *"Merge"* && "$MERGE_COMMIT_MSG" == *"release-"* && "$MERGE_COMMIT_MSG" == *"[release]"* ]]; then
Comment on lines +102 to +108
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These checks are a bit hard to read. What's the intention?

echo "Running major version bump for release merge"
yarn lerna version major --conventional-commits --create-release github --no-private --yes --force-publish
else
echo "Running conventional commit versioning on master"
yarn lerna version --conventional-commits --create-release github --no-private --yes
SudilHasithaCognite marked this conversation as resolved.
Show resolved Hide resolved
fi
elif [[ "${GITHUB_REF}" == refs/heads/release-* ]]; then
echo "Running pre-release versioning on release branch"
yarn lerna version --conventional-commits --conventional-prerelease --preid rc --create-release github --no-private --yes
fi

- name: Lerna Publish
if: contains(github.event.head_commit.message, '[release]')
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_PUBLISH_TOKEN }}
run: yarn lerna publish from-package --pre-dist-tag rc --yes --no-git-reset --no-verify-access
run: |
if [[ "${GITHUB_REF}" == "refs/heads/master" ]]; then
yarn lerna publish from-package --yes --no-git-reset --no-verify-access
elif [[ "${GITHUB_REF}" == refs/heads/release-* ]]; then
yarn lerna publish from-package --pre-dist-tag rc --yes --no-git-reset --no-verify-access
fi

# - name: Publish documentation
# uses: JamesIves/github-pages-deploy-action@releases/v3
Expand Down
92 changes: 15 additions & 77 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,22 +69,22 @@ More details are documented in the [codegen README](packages/codegen/README.md).

## Pull request

Make a pull request from your branch to the main branch. When merging the pull request,
Make a pull request from your branch to the master branch. When merging the pull request,
only use squashing if the resulting squash commit can accurately describe the change as a single conventional commit.
Once the change is pushed to the main branch, it is time for a release.
Once the change is pushed to the master branch, it is time for a release.

## Releases & Versioning

Releases are done from the main branch, so when a pull request is merged,
Releases are done from the master branch, so when a pull request is merged,
CI/CD will run tests, and if successful, do deploys.
Documentation is built and deployed, and code snippets
are exported to the service contract repo as a pull request.

Updating and uploading npm packages only happens if the HEAD commit of the main branch
contains `[release]` in its description or the PR title starts with `feat` or `fix`.
Updating and uploading npm packages only happens if the HEAD commit of the master branch
contains `[release]` in its description and the PR title starts with `feat` or `fix`.
When CI/CD sees this, it will use lerna to update
package versions of changed packages based on commit messages, and add the
changes to the changelogs. The changes are comitted to the main branch
changes to the changelogs. The changes are comitted to the master branch
with the new versions as git tags, and the new package versions are uploaded to npm.

We restrict new npm releases to `[release]`-tagged commits because lerna is
Expand All @@ -93,7 +93,7 @@ cause a PATCH bump. Markdown files and tests are ignored, but changing anything
like a comment in a source file, will trigger a new version,
irrespective of conventional commits.

This does _not_ mean you should store unfinished work on the main branch.
This does _not_ mean you should store unfinished work on the master branch.
Another package may be ready for release, and once a `[release]`
commit is pushed, all changed packages are updated.
Repository administrators should be in control of `[release]` commits.
Expand All @@ -111,9 +111,15 @@ If you want to push the empty commit to master via a pull request,
use a squash merge (not rebase+ff). Otherwise GitHub will ignore the empty PR.

Also, keep in mind that the `[release]` commit has to be the HEAD of
main, and Github Action only runs on the HEAD. If HEAD has changed by the time
master, and Github Action only runs on the HEAD. If HEAD has changed by the time
the versioning happens, Github Action will fail.

In order to perform a major release by merging a release candidate branch to master by keeping the same major version in release candidate version.

- Create a PR from release-* to master
- Make a PR title and the message when merging to be ` chore(): details [release]`
- The release tag is to trigger the release of the package.

## Patching older major versions

If you need to backport a fix to a previous MAJOR version of a package,
Expand Down Expand Up @@ -166,72 +172,4 @@ git push && git push --tags
```

This will make a commit with the updated `package.json`, create a new git tag, and publish to npm.
Make sure you are logged in to npm, talk to a maintainer.

## Code overview
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why the "Code overview" section is removed in the PR?


### HTTP Client

The core of the SDK is the HTTP client. The HTTP client is divided into multiple layers:

1. [BasicHttpClient](./packages/core/src/httpClient/basicHttpClient.ts)
2. [RetryableHttpClient](./packages/core/src/httpClient/retryableHttpClient.ts)
3. [CDFHttpClient](./packages/core/src/httpClient/cdfHttpClient.ts)

See each file for a description of what they do.

### Pagination

We have multiple utilities to easy pagination handling. The first entrypoint is [cursorBasedEndpoint](./packages/core/src/baseResourceApi.ts) which adds a `next()` function on the response to fetch the next page of result. Then we use [makeAutoPaginationMethods](./packages/core/src/autoPagination.ts) to add the following methods:

- autoPagingToArray
```ts
const assets = await client.assets.list().autoPagingToArray({ limit: 100 });
```
- autoPagingEach
```ts
for await (const asset of client.assets.list().autoPagingEach({ limit: Infinity })) {
// ...
}
```

### Date parser

Some API responses includes DateTime responses represented as UNIX timestamps. We have utility class to automatically translate the number response into a Javascript [Date instance](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date).

See the [DateParser class](./packages/core/src/dateParser.ts).

### Metadata

We offer users to get hold of the HTTP response status code and headers through the [MetadataMap class](./packages/core/src/metadata.ts).

### Core utilities

- [promiseAllWithData](./packages/core/src/utils.ts)
- [promiseCache](./packages/core/src/utils.ts)
- [topologicalSort](./packages/core/src/graphUtils.ts)
- [RevertableArraySorter](./packages/core/src/revertableArraySorter.ts)

### Cognite Clients

There is a Cognite Client per SDK package:
- [Core](./packages/core/src/baseCogniteClient.ts)
- [Stable](./packages/stable/src/cogniteClient.ts)
- [Beta](./packages/beta/src/cogniteClient.ts)
- [Alpha](./packages/alpha/src/cogniteClient.ts)
- [Playground](./packages/playground/src/cogniteClientPlayground.ts)
- [Template](./packages/template/src/cogniteClient.ts)

The Core one is the base, meaning the others extends from it.


#### Authentication

The authentication logic lives in the [core BaseCogniteClient](./packages/core/src/baseCogniteClient.ts).
The client constructor offer the field `oidcTokenProvider` (formely called `getToken`) where the SDK user will provide a valid access token.

The SDK will call this method when:

- The user calls `authenticate` on the client.
- The SDK receives a 401 from the API.
When multiple requests receives a 401, then only a single call to `oidcTokenProvider` will be invoked. All requests will wait for `oidcTokenProvider` to resolve/reject. If it's resolved, then all the requests will retry before returning the response to the SDK caller. However, if the resolved access token matches the original access token, then no retry will be performed.
Make sure you are logged in to npm, talk to a maintainer.
2 changes: 1 addition & 1 deletion lerna.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"**/*.test.ts"
],
"message": "chore(release): publish new package versions [skip ci]",
"allowBranch": "release-v10"
"allowBranch": ["master", "release-*"]
}
}
}
24 changes: 24 additions & 0 deletions scripts/remove-rc-tags.js
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// Remove -rc tag from release branch when it merge to master
const fs = require('node:fs');
const glob = require('glob');

// Retrive all pacakage.json file paths
const packageFilesPaths = glob.sync('packages/*/package.json');

// loop through each and remove rc tag
for (const packageFilePath of packageFilesPaths) {
//get the package content to json file
const packageJson = JSON.parse(fs.readFileSync(packageFilePath, 'utf-8'));

// check for rc, remove it, reassign the version and rewrite
if (packageJson.version?.includes('-rc')) {
// 6.0.0-rc.1 --> [6.0.0, -rc.1]
// 5.0.0 which then convert to 6.0.0 by release.yaml
majorVersion = Number.parseInt(packageJson.version.split('-rc')[0]) - 1;
packageJson.version = `${majorVersion}.0.0`;
fs.writeFileSync(
packageFilePath,
`${JSON.stringify(packageJson, null, 2)}\n`
);
}
}
Loading