Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(planner): subsequent work for logical planning rcte #16673

Closed
wants to merge 90 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
e93db36
chore(deps): Bump rustls 0.21.{10,11} and 0.22.{3,4} (#16463)
xiangjinwu Apr 26, 2024
a692ed0
test(sink): add bigquery-sink compatible test (#16497)
xuefengze Apr 26, 2024
2dcd0f6
feat(sink): support bigquery sink upsert (#15780)
xxhZs Apr 26, 2024
c08aa2b
feat(metric): add metric sink_chunk_buffer_size (#16493)
zwang28 Apr 26, 2024
c94445e
fix(docker): use `single_node` as dockerfile entrypoint (#16484)
kwannoel Apr 26, 2024
6329101
feat(cdc): amortize snapshot read to multiple barriers (#16349)
StrikeW Apr 26, 2024
9e6528a
feat(meta): support sql meta store backup and restoration (#16384)
zwang28 Apr 26, 2024
b82e5f9
doc(meta): add more concrete examples of `prometheus_endpoint` usage …
kwannoel Apr 26, 2024
a4a9110
feat: reject some param for creating cdc table (#16513)
st1page Apr 26, 2024
4ba2e66
feat(connector): add request.required.acks for kafka connector produc…
yuhao-su Apr 26, 2024
9996e75
doc: add comments for source executor's state table (#16499)
xxchan Apr 27, 2024
b213243
feat(cdc): introduce with option to configure cdc snapshot (#16426)
StrikeW Apr 28, 2024
6b5b873
feat: do not valid rw's generated column on source (#16522)
st1page Apr 28, 2024
1b4f7ed
feat(risedev): support docker-backed mysql service (#16514)
BugenZhao Apr 28, 2024
9be74ae
refactor(cdc source): simplify CDC transaction and heartbeat messages…
stdrc Apr 28, 2024
82d847c
feat(batch): Support log iter batch for table or mv (#16487)
xxhZs Apr 28, 2024
1e55914
fix(cdc): allow null in list in pg-cdc backfill (#16457)
KeXiangWang Apr 29, 2024
c13caf6
fix: Revert "feat: try to remove parallel unit mapping in frontend" (…
shanicky Apr 29, 2024
8ee413b
fix(config): revert some compaction config due to perf degradation (#…
Li0k Apr 29, 2024
03e6283
feat(frontend): support alter rate limit in mv, source, table with so…
kwannoel Apr 29, 2024
68cf636
feat(storage): limit trivial move loop count (#16479)
Li0k Apr 29, 2024
db00d49
chore(deps): Bump serde_with from 3.7.0 to 3.8.0 (#16502)
dependabot[bot] Apr 29, 2024
8265424
chore(deps): Bump google-cloud-bigquery from 0.7.0 to 0.8.0 in the go…
dependabot[bot] Apr 29, 2024
da899ed
chore(ci): fix integration test docker clean (#16533)
huangjw806 Apr 29, 2024
a051c25
fix(state table): use `value_checker` in `MemTable::delete` (#16539)
stdrc Apr 29, 2024
45e9f77
fix(storage): correctly update table stats in SkipWatermarkIterator (…
zwang28 Apr 29, 2024
fbc3a6c
chore(bench): add plain_parser and json_parser comparison bench (#16526)
xuefengze Apr 29, 2024
4cfcbcd
fix(storage): fix unexpected task picked by sublevel picker (#16454)
Li0k Apr 29, 2024
6726ba7
chore(ci): fix integration test docker logs (#16548)
huangjw806 Apr 30, 2024
201f903
fix(Sink): fix redis sink Cluster error (#16405)
xxhZs Apr 30, 2024
43b418d
fix(cdc): don't wait epoch commit for cdc connectors except postgres …
StrikeW Apr 30, 2024
bc85958
fix(ci): fix madsim unit test and backfill test failure in main-cron …
wenym1 Apr 30, 2024
b9e96a5
feat(log-store): add monitoring sink buffer with metrics and await tr…
wenym1 Apr 30, 2024
1c65460
fix(error): fix compiling of lints (#16560)
BugenZhao Apr 30, 2024
5b40139
feat(storage): add metrics to monitor old value size (#16559)
wenym1 Apr 30, 2024
5789529
fix(risedev): don't kill tmux session unnecessarily (#16518)
xxchan May 1, 2024
943da74
fix(test): make kafka/shared_source.slt less flaky (#16565)
xxchan May 1, 2024
3579aae
fix(error): do not directly format `BoxedError` (#16558)
BugenZhao May 1, 2024
d7eec09
chore(deps): Bump enum-iterator from 2.0.0 to 2.1.0 (#16571)
dependabot[bot] May 2, 2024
2895f48
chore(deps): Bump base64-url from 2.0.0 to 3.0.0 (#16570)
dependabot[bot] May 2, 2024
19f1452
fix: name mis-align for `kinesis-s3-source` in main cron (#16569)
tabVersion May 2, 2024
7b6cc24
feat(mysql-cdc): support `ssl.mode` to allow configure the ssl behavi…
StrikeW May 6, 2024
e362226
feat(cdc-backfill): support uuid type as pk for cdc backfill (#16380)
StrikeW May 6, 2024
abcf9ca
refactor(optimizer): add plan phase to plan root (#16510)
chenzl25 May 6, 2024
4fe1894
fix(storage): include change log sst in object size map (#16561)
wenym1 May 6, 2024
222b291
feat(meta): support inject barrier to enable and disable writing old …
wenym1 May 6, 2024
b86ffb2
fix: change full version delta column type for mysql meta backend (#1…
yezizp2012 May 6, 2024
16dbae3
refactor(sink): DRY `SinkFormatterImpl::new` (#16575)
xiangjinwu May 6, 2024
656ad33
chore(risedev): provide option to disable backtrace (#16586)
wenym1 May 6, 2024
4fea7a5
fix(metric): filter out NaN due to divide by zero in Grafana (#16590)
hzxa21 May 6, 2024
3a1e259
feat: support inspecting local barrier worker state (#16562)
wenym1 May 6, 2024
7df9465
feat(meta): refactor subscription and cursor with Partial ckpt (#16563)
xxhZs May 7, 2024
b541764
doc: remove invalid markdown alert (#16595)
varunpv May 7, 2024
b4ecd45
chore: update cherry pick bot to release-1.9 (#16596)
lmatz May 7, 2024
0422eab
chore(deps): Bump async-nats from 0.34.0 to 0.35.0 (#16597)
dependabot[bot] May 7, 2024
746faae
fix(meta): Delete first and then rebuild for subscription modle v2 ta…
xxhZs May 7, 2024
4030ddc
feat(sink): add metrics to monitor sink back pressure (#16593)
wenym1 May 7, 2024
2db8e42
fix(test): install sqlite in ci script (#16604)
zwang28 May 7, 2024
293db73
fix(source): make s3 source compatible with minio (#16521)
wcy-fdu May 7, 2024
f7d9ff7
refactor(test): use ci-provisioned message-queue service in backwards…
BugenZhao May 7, 2024
b9fd56e
feat(metrics): add metrics for prefetch memory usage (#16582)
Little-Wallace May 7, 2024
cfedc2a
fix(storage): fix unexpected dropping change log on compaction versio…
wenym1 May 7, 2024
c429309
fix: fix the issue where the sql election follower hangs (#16614)
shanicky May 7, 2024
ea1ecc4
chore: unify show cluster result with system catalog (#16609)
yezizp2012 May 7, 2024
39491e6
feat(risedev): deprecate zookeeper & use kraft mode for kafka service…
BugenZhao May 7, 2024
b64a0ac
feat(cdc): add a parameter to control timeout of cdc source waiting t…
KeXiangWang May 7, 2024
a4adc5e
feat(sink): add Google Pub/Sub support (#16363)
jetjinser May 7, 2024
4b8786c
fix(test): set missing env var for ci test script (#16627)
zwang28 May 8, 2024
0504754
refactor(arrow): improve the arrow conversion trait (#16567)
wangrunji0408 May 8, 2024
8a6b46c
feat: add database and schema in dashboard (#16610)
fuyufjh May 8, 2024
1bda190
fix: fix user manager refcount on startup (#16632)
xxchan May 8, 2024
9d1b594
feat: support memory limit for batch queries (#16616)
chenzl25 May 8, 2024
94d9e1b
refactor(batch): remove unused mem code (#16641)
chenzl25 May 8, 2024
2929fb8
refactor(object_store): refactor timeout and retry of object store in…
Li0k May 8, 2024
926ba1b
feat: impl `key encode` clause for sink (#16377)
tabVersion May 8, 2024
299f582
chore(deps): Bump rustls-pki-types from 1.5.0 to 1.7.0 (#16647)
dependabot[bot] May 9, 2024
11db115
chore(deps): Bump pulsar from 6.1.0 to 6.2.0 (#16648)
dependabot[bot] May 9, 2024
464f8ef
chore: refactor bind connector props for create source and create tab…
st1page May 9, 2024
1b5e7d8
feat: specify column when sink into table (#16587)
st1page May 9, 2024
d472238
feat: add pipeline auto-update Helm Charts and Operator on New Releas…
Stab123 May 9, 2024
cb8a92a
fix: fix the inconsistency issue with the two-stage write to the meta…
shanicky May 9, 2024
c9a2083
fix: fix dependent table id column type in the model (#16660)
yezizp2012 May 9, 2024
dcec31a
refactor(parser): simplify ARRAY expression parsing (#16639)
TennyZhuang May 9, 2024
f78eb21
chore: a typo (#16663)
st1page May 9, 2024
164e4a3
feat(connector): MSK connection by AWS IAM (#16625)
yuhao-su May 9, 2024
91b7ee2
fix(object_store): fix delete objects timeout config (#16661)
Li0k May 9, 2024
4d5d790
Merge branch 'risingwavelabs:main' into xzhseh/plan_rcte
xzhseh May 9, 2024
defe677
update rcte cache plan for prune_col
xzhseh May 9, 2024
c52cb9c
chore(standalone): add monitoring configuration files (#16594)
kwannoel May 10, 2024
27ed103
Merge branch 'main' into xzhseh/plan-rcte
xzhseh May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Update Helm Charts and Risingwave Operator on New Release

on:
release:
types: [published]
workflow_dispatch:
inputs:
version:
description: 'release version'
required: true

env:
NEW_APP_VERSION: ${{ github.event.inputs.version || github.event.release.tag_name }}

jobs:
update-helm-charts:
runs-on: ubuntu-latest
steps:
- name: Checkout Helm Charts Repository
uses: actions/checkout@v3
with:
repository: 'risingwavelabs/helm-charts'
token: ${{ secrets.GITHUB_TOKEN }}
path: 'helm-charts'

- name: Update values.yaml
run: |
sed -i "s/^ tag:.*/ tag: \"${{ env.NEW_APP_VERSION }}\"/" helm-charts/charts/risingwave/values.yaml

- name: Update Chart.yaml
run: |
cd helm-charts/charts/risingwave
CURRENT_VERSION=$(grep 'version:' Chart.yaml | awk '{print $2}' | head -n 1)
NEW_VERSION=$(echo $CURRENT_VERSION | awk -F. -v OFS='.' '{$NF++; print}')
sed -i "/type: application/,/version:/!b; /version:/s/version: .*/version: $NEW_VERSION/" Chart.yaml
sed -i "s/^appVersion: .*/appVersion: \"${{ env.NEW_APP_VERSION }}\"/" Chart.yaml
echo "NEW_CHART_VERSION=$NEW_VERSION" >> $GITHUB_ENV

- name: Create Pull Request
uses: peter-evans/create-pull-request@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: 'chore: bump risingwave to ${{ env.NEW_APP_VERSION }}, release chart ${{ env.NEW_CHART_VERSION }}'
title: 'chore: bump risingwave to ${{ env.NEW_APP_VERSION }}, release chart ${{ env.NEW_CHART_VERSION }}'
body: 'This is an automated pull request to update the chart versions'
branch: 'auto-update-${{ env.NEW_APP_VERSION }}'

update-risingwave-operator:
runs-on: ubuntu-latest
steps:
- name: Checkout Risingwave Operator Repository
uses: actions/checkout@v3
with:
repository: 'risingwavelabs/risingwave-operator'
token: ${{ secrets.GITHUB_TOKEN }}
path: 'risingwave-operator'

- name: Update risingwave-operator image tags
run: |
cd risingwave-operator
PREV_VERSION=$(grep -roh "risingwavelabs/risingwave:v[0-9\.]*" * | head -n 1 | cut -d':' -f2)
grep -rl "risingwavelabs/risingwave:$PREV_VERSION" . | xargs sed -i "s|risingwavelabs/risingwave:$PREV_VERSION|risingwavelabs/risingwave:${{ env.NEW_APP_VERSION }}|g"

- name: Create Pull Request for risingwave-operator
uses: peter-evans/create-pull-request@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: 'chore: bump risingwave image tags to ${{ env.NEW_APP_VERSION }}'
title: 'chore: bump risingwave image tags to ${{ env.NEW_APP_VERSION }}'
body: 'This is an automated pull request to update the risingwave image tags'
branch: 'auto-update-${{ env.NEW_APP_VERSION }}'
31 changes: 8 additions & 23 deletions .github/workflows/cherry-pick-to-release-branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ on:
types: ["closed", "labeled"]

jobs:
release_pull_request_1_8:
if: "contains(github.event.pull_request.labels.*.name, 'need-cherry-pick-release-1.8') && github.event.pull_request.merged == true"
release_pull_request_1_9:
if: "contains(github.event.pull_request.labels.*.name, 'need-cherry-pick-release-1.9') && github.event.pull_request.merged == true"
runs-on: ubuntu-latest
name: release_pull_request
steps:
Expand All @@ -16,14 +16,14 @@ jobs:
- name: Create PR to branch
uses: risingwavelabs/github-action-cherry-pick@master
with:
pr_branch: 'release-1.8'
pr_branch: 'release-1.9'
pr_labels: 'cherry-pick'
pr_body: ${{ format('Cherry picking \#{0} onto branch release-1.8', github.event.number) }}
pr_body: ${{ format('Cherry picking \#{0} onto branch release-1.9', github.event.number) }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

release_pull_request_1_7_standalone:
if: "contains(github.event.pull_request.labels.*.name, 'need-cherry-pick-release-1.7-standalone') && github.event.pull_request.merged == true"
release_pull_request_1_8:
if: "contains(github.event.pull_request.labels.*.name, 'need-cherry-pick-release-1.8') && github.event.pull_request.merged == true"
runs-on: ubuntu-latest
name: release_pull_request
steps:
Expand All @@ -32,24 +32,9 @@ jobs:
- name: Create PR to branch
uses: risingwavelabs/github-action-cherry-pick@master
with:
pr_branch: 'release-1.7.0-standalone'
pr_branch: 'release-1.8'
pr_labels: 'cherry-pick'
pr_body: ${{ format('Cherry picking \#{0} onto branch release-1.7.0-standalone', github.event.number) }}
pr_body: ${{ format('Cherry picking \#{0} onto branch release-1.8', github.event.number) }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

release_pull_request_1_7:
if: "contains(github.event.pull_request.labels.*.name, 'need-cherry-pick-release-1.7') && github.event.pull_request.merged == true"
runs-on: ubuntu-latest
name: release_pull_request
steps:
- name: checkout
uses: actions/checkout@v1
- name: Create PR to branch
uses: risingwavelabs/github-action-cherry-pick@master
with:
pr_branch: 'release-1.7'
pr_labels: 'cherry-pick'
pr_body: ${{ format('Cherry picking \#{0} onto branch release-1.7', github.event.number) }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Loading
Loading