Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(storage): refactor ReportTask event to reduce the network packet size #11711

Closed
wants to merge 306 commits into from
Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
306 commits
Select commit Hold shift + click to select a range
cf96c85
chore: refine kinesis error & sleep when shard iter is None (#11847)
tabVersion Aug 23, 2023
d1ee0e8
fix(compaction): do not split creating table (#11826)
Little-Wallace Aug 23, 2023
faf2a75
fix(json parser): fill None when parse error (#11845)
tabVersion Aug 23, 2023
b072774
chore(deps): bump dashmap from 5.5.0 to 5.5.1 (#11841)
dependabot[bot] Aug 23, 2023
bdf3a7b
feat(frontend): change index default distributed by columns (#11865)
chenzl25 Aug 24, 2023
21b7fb5
feat: support alter source add column (#11350)
wugouzi Aug 24, 2023
15017b9
feat: support show and cancel jobs syntax (#11854)
yezizp2012 Aug 24, 2023
12d4645
chore(check): `egrep` -> `grep -E` (#11870)
stdrc Aug 24, 2023
490ad60
chore(deps): bump simd-json from 0.10.5 to 0.10.6 (#11857)
dependabot[bot] Aug 24, 2023
0fa5e0f
chore(ci): bump flamegraph gen timeout (#11886)
kwannoel Aug 25, 2023
754b503
fix(watermark): return error when watermark return data type does not…
yuhao-su Aug 25, 2023
fd165a7
chore(deps): bump petgraph from 0.6.3 to 0.6.4 (#11816)
dependabot[bot] Aug 25, 2023
31b20f8
fix(watermark): correct data type in transform_with_expr (#11887)
yuhao-su Aug 25, 2023
4bf45a3
feat: add streamChunkMut (#11808)
st1page Aug 25, 2023
1060a6d
fix(cdc): fix postgres privilege check for AWS RDS (#11893)
StrikeW Aug 25, 2023
0f5b460
feat(parser): parse lambda function (take 2) (#11888)
TennyZhuang Aug 25, 2023
34489e3
feat(ci): test standalone cluster in main-cron, with e2e test (#11871)
kwannoel Aug 25, 2023
d770363
feat(compaction): compact sstable with many tombstones periodic (#11740)
Little-Wallace Aug 25, 2023
82d5509
feat(frontend): support lateral subqueries (#11780)
chenzl25 Aug 25, 2023
eeef495
refactor(agg): decouple aggregate function and state (#11690)
wangrunji0408 Aug 25, 2023
bbb8607
chore(deps): bump bytesize from 1.2.0 to 1.3.0 (#11901)
dependabot[bot] Aug 26, 2023
e8377d9
chore(deps): bump time from 0.3.25 to 0.3.27 (#11900)
dependabot[bot] Aug 26, 2023
540a4c2
chore(deps): bump google-cloud apis and pubsub (#11889)
yuhao-su Aug 26, 2023
3c4c460
feat: port kafka native metrics to Prometheus (#11346)
tabVersion Aug 26, 2023
b543431
fix: change Chinese quotation marks into English (#11902)
SYaoJun Aug 27, 2023
0932d48
chore(deps): bump nix from 0.25.1 to 0.26.2 (#11031)
dependabot[bot] Aug 27, 2023
5262e04
refactor(storage): checked u32 conversion during block building (#11734)
zwang28 Aug 28, 2023
2208e07
refactor(metric): slim down storage metric size (#11843)
zwang28 Aug 28, 2023
6a36b57
refactor(metrics): store all metrics and metrics registry as global v…
wenym1 Aug 28, 2023
e40a25f
feat: add new props `client.id` for kafka (#11911)
tabVersion Aug 28, 2023
f4d2fcf
feat(integration_tests): Add Clickhouse sink integration_tests (#11786)
xxhZs Aug 28, 2023
ad1c633
fix(cdc): fix mysql privilege check and add test (#11916)
StrikeW Aug 28, 2023
797a7c5
feat: ReplaceTable supports replacing downstream (#11913)
shanicky Aug 28, 2023
2decdc8
fix(test): fix unclean metrics state in deterministic unit test (#11918)
wenym1 Aug 28, 2023
9e34dcd
fix(ci): wait for standalone cluster to restart (#11907)
kwannoel Aug 28, 2023
8b068c9
fix(optimizer): common sub-expression extraction on special exprs (#1…
xiangjinwu Aug 28, 2023
c7392ba
chore(deps): bump pin-project-lite from 0.2.12 to 0.2.13 (#11920)
dependabot[bot] Aug 29, 2023
5344504
feat(source): support json schema (#11797)
wugouzi Aug 29, 2023
927ac44
refactor(metric): support storage metric level (#11895)
zwang28 Aug 29, 2023
77341f2
chore: bump foyer (#11930)
MrCroxx Aug 29, 2023
75ba89b
refactor(object storage): do not pass root to object storage (#11757)
wcy-fdu Aug 29, 2023
a6d9519
chore(config): set default visibility_mode to checkpoint for frontend…
zwang28 Aug 29, 2023
bf34d8e
refactor(state-table): Avoid exposing vnode-prefixed key in state tab…
hzxa21 Aug 29, 2023
e42f250
feat(streaming): flow control executor (#11919)
kwannoel Aug 29, 2023
bd016cf
chore(deps): bump memchr from 2.5.0 to 2.6.1 (#11947)
dependabot[bot] Aug 30, 2023
89c286e
chore(deps): bump serde from 1.0.185 to 1.0.188 (#11946)
dependabot[bot] Aug 30, 2023
d1b11cb
test(streaming): test hdr approach for `approx_percentile` (#11931)
kwannoel Aug 30, 2023
396e270
fix(binder): disallow set-returning function in JoinOn/Having/Filter/…
xiangjinwu Aug 30, 2023
390f556
feat: add `initialized_at` to ddl_progress (#11940)
shanicky Aug 30, 2023
4f73bb3
chore(deps): bump time from 0.3.27 to 0.3.28 (#11922)
dependabot[bot] Aug 30, 2023
d744afb
feat(expr): add `regexp_replace` (#11819)
xzhseh Aug 30, 2023
989ad3e
chore(deps): bump openssl-sys from 0.9.91 to 0.9.92 (#11923)
dependabot[bot] Aug 30, 2023
164daa4
chore(udf): update README and migrate `setup.py` to `pyproject.toml` …
wangrunji0408 Aug 30, 2023
63262e4
feat(metrics): support monitor network connection of S3 and grpc (#11…
wenym1 Aug 30, 2023
c8fdb9d
feat(expr): Implement lambda function and array_transform (#11937)
TennyZhuang Aug 30, 2023
35b9848
chore: fix typo in avro source (#11938)
fuyufjh Aug 30, 2023
0c46b56
chore(config): disable storage prefetch (#11953)
zwang28 Aug 30, 2023
29a0fd4
doc: Understanding RisingWave Macros (#11942)
kwannoel Aug 30, 2023
2fedc13
chore: Upgrade icelake version (#11962)
liurenjie1024 Aug 30, 2023
7601abb
chore: add code owners for Cargo.lock (#11965)
BugenZhao Aug 30, 2023
18ea4e8
feat(cdc): in-memory backfill for single mysql cdc table (#11707)
StrikeW Aug 30, 2023
9f14c3c
doc: add dash doc feed & fix rustdoc warnings (#11963)
xxchan Aug 30, 2023
6d35f78
chore(deps): bump regex from 1.9.3 to 1.9.4 (#11969)
dependabot[bot] Aug 31, 2023
0e5e98e
chore(deps): bump dashmap from 5.5.1 to 5.5.3 (#11971)
dependabot[bot] Aug 31, 2023
107e3d3
chore(deps): bump clap from 4.3.22 to 4.4.1 (#11973)
dependabot[bot] Aug 31, 2023
746d2cb
fix(ci): fix rustdoc site (#11976)
xxchan Aug 31, 2023
c31409b
chore(deps): bump ethnum from 1.3.2 to 1.4.0 (#11972)
dependabot[bot] Aug 31, 2023
e42bd84
refactor(compaction): extract reporting full scan task logic (#11958)
wcy-fdu Aug 31, 2023
aa89640
refactor(log_store): make delete range end excluded (#11978)
zwang28 Aug 31, 2023
abcea2a
fix(ci): fix unit test deterministic test (#11985)
StrikeW Aug 31, 2023
f1c6fbb
feat(optimizer): support expand to project rule (#11933)
chenzl25 Aug 31, 2023
d97e951
feat(doc): add guidance for docker compose xxx (#11749)
wcy-fdu Aug 31, 2023
73fa58d
refactor(stream common): split `StreamChunkBuilder` and `JoinStreamCh…
stdrc Aug 31, 2023
d513c50
feat(sink): Support Nats sink (append_only) (#11924)
yufansong Aug 31, 2023
fefb114
fix(test): fix test failure caused by shared metric instance (#11992)
zwang28 Aug 31, 2023
8a08329
chore: set default value of cdc_backfill to false (#11987)
StrikeW Aug 31, 2023
5edcbe6
fix: ban scale operations when meta is in recovery state. (#11997)
shanicky Aug 31, 2023
df24a7e
fix(cdc-backfill): also persist the finish flag even though snapshot …
StrikeW Aug 31, 2023
c2fc67c
fix: use match to handle non queue full error instead of assert (#11998)
tabVersion Aug 31, 2023
1cf2570
feat(compaction): dynamically determine the type of sstable_object_id…
wcy-fdu Aug 31, 2023
a1153fb
refactor(simulation): make it compile without `cfg(madsim)` (#11960)
BugenZhao Aug 31, 2023
0ebf17a
chore: cherry pick action for v1.2 (#12004)
lmatz Aug 31, 2023
460f104
chore(doc): update doc site and cloud site links in README (#11926)
hengm3467 Aug 31, 2023
9fab242
chore(deps): bump url from 2.4.0 to 2.4.1 (#12011)
dependabot[bot] Sep 1, 2023
7cc2af3
chore(deps): bump base64 from 0.21.2 to 0.21.3 (#12009)
dependabot[bot] Sep 1, 2023
90e2e19
chore(deps): bump clap from 4.4.1 to 4.4.2 (#12007)
dependabot[bot] Sep 1, 2023
fe6d12d
chore(deps): bump openssl from 0.10.56 to 0.10.57 (#12006)
dependabot[bot] Sep 1, 2023
3a260f3
feat(compaction): dynamically determine the type filter_key_extracto…
wcy-fdu Sep 1, 2023
b0bd2e6
feat(expr): add `regexp_count` (#11975)
xzhseh Sep 1, 2023
1ba63a3
test: Remove busy loop tests (#12017)
liurenjie1024 Sep 1, 2023
5bb43da
chore(deps): bump prost-reflect from 0.11.4 to 0.11.5 (#12012)
dependabot[bot] Sep 1, 2023
55e13e3
chore(deps): bump reqwest from 0.11.18 to 0.11.20 (#12008)
dependabot[bot] Sep 1, 2023
34afd39
feat(meta): add system parameter `pause_on_next_bootstrap` (#11936)
BugenZhao Sep 1, 2023
f79e4a2
fix(cdc-backfill): fix bug when handle upstream chunk (#12027)
StrikeW Sep 1, 2023
b26f7fa
fix(storage): fix opendal object store error handle (#12031)
Li0k Sep 1, 2023
675a28e
chore(ci): optimize trailing space check (#12019)
stdrc Sep 1, 2023
43ce1a1
chore(deps): bump redis from 0.23.2 to 0.23.3 (#12038)
dependabot[bot] Sep 1, 2023
4dc489f
chore(deps): bump sysinfo from 0.29.8 to 0.29.9 (#12034)
dependabot[bot] Sep 1, 2023
0c3be50
chore(deps): bump memchr from 2.6.1 to 2.6.2 (#12039)
dependabot[bot] Sep 2, 2023
c49e07a
fix: fill `source` and `name` fields correctly for Debezium (#12018)
wugouzi Sep 4, 2023
e84bff5
refactor(common): Use safe interfaces for `DataChunkBuilder` (#12014)
kwannoel Sep 4, 2023
8a1017d
chore(deps): bump tower-http from 0.4.3 to 0.4.4 (#12036)
dependabot[bot] Sep 4, 2023
e54f466
chore(deps): bump aho-corasick from 1.0.4 to 1.0.5 (#12035)
dependabot[bot] Sep 4, 2023
0c44c62
perf(over window): locality-based range cache for general over window…
stdrc Sep 4, 2023
e6fb726
fix: clickhouse https url (#12041)
lmatz Sep 4, 2023
a205ac8
feat(stream): first version of nats source (#12015)
yufansong Sep 4, 2023
51aec3d
fix: reject kinesis source if the start up mode is not sequence numbe…
tabVersion Sep 4, 2023
c9a3d8f
feat: support for migration by new parallel units (#11892)
yezizp2012 Sep 4, 2023
3827dd2
chore: refine error message on unhandled error in kinesis (#12055)
tabVersion Sep 4, 2023
14ea28c
feat(hummock): refill cache before apply version delta (#11950)
MrCroxx Sep 4, 2023
3e56e7d
feat: add `target_parallelism_per_worker` field for scaling (#11945)
shanicky Sep 4, 2023
11bed1c
feat: auto rebuild source reader in executor (#11993)
wugouzi Sep 4, 2023
5e79dd9
feat: support information_schema.views (#12045)
neverchanje Sep 5, 2023
381a27f
chore(deps): bump memchr from 2.6.2 to 2.6.3 (#12068)
dependabot[bot] Sep 5, 2023
dbd8e54
chore(deps): bump syn from 2.0.29 to 2.0.31 (#12067)
dependabot[bot] Sep 5, 2023
a0ca755
feat(storage): refactor `LocalStateStore::init` and `StateTable::init…
kwannoel Sep 5, 2023
eba6583
chore(deps): bump google-cloud-pubsub from 0.19.0 to 0.20.0 (#12066)
dependabot[bot] Sep 5, 2023
f3c2b61
chore(deps): bump tracing-opentelemetry from 0.20.0 to 0.21.0 (#12010)
dependabot[bot] Sep 5, 2023
a32441b
refactor(streaming): better traces for development (#12024)
BugenZhao Sep 5, 2023
1b9a45d
feat(grafana): report local barrier manager progress (#12001)
BugenZhao Sep 5, 2023
e2b4cb5
chore(deps): bump async-recursion from 1.0.4 to 1.0.5 (#12069)
dependabot[bot] Sep 5, 2023
a2aa604
fix(jdbc-sink): quote identifier to handle case-sensitive case (#12078)
StrikeW Sep 5, 2023
bffe0ec
feat: HA for schema registry (#11982)
tabVersion Sep 5, 2023
cf0cc81
feat: modify the scale command to support both horizontal and vertica…
shanicky Sep 5, 2023
74efab4
refactor(source): implement the common split reader as a generic func…
wenym1 Sep 5, 2023
9ce33ac
chore: add over window executor cache metrics (#12092)
stdrc Sep 5, 2023
1d4e98e
perf(over window): fix over window performance degradation after #115…
stdrc Sep 5, 2023
30e2f46
refactor(metrics): support streaming metrics level (#11994)
fuyufjh Sep 5, 2023
af6f418
chore(rpc): set message size to max for stream client (#12082)
zwang28 Sep 5, 2023
897e72a
chore(deps): bump deranged from 0.3.7 to 0.3.8 (#12102)
dependabot[bot] Sep 6, 2023
df7fc2d
chore(deps): bump walkdir from 2.3.3 to 2.4.0 (#12101)
dependabot[bot] Sep 6, 2023
60db789
chore(deps): bump openssl-sys from 0.9.92 to 0.9.93 (#12106)
dependabot[bot] Sep 6, 2023
f568ed9
refactor(compaction): remove meta client from compactor context (#12020)
wcy-fdu Sep 6, 2023
415329c
fix(streaming): handle missing `cache_policy` field in `OverWindow` (…
kwannoel Sep 6, 2023
b78b02d
refactor(java_binding): extract java_binding to jni_core (#12108)
chenzl25 Sep 6, 2023
de88d5f
chore(storage): remove compactor workload (#12005)
Li0k Sep 6, 2023
aac14a7
fix: temporarily remove the assert checking for split migration (#12117)
shanicky Sep 6, 2023
2c54ada
chore(deps): bump flate2 from 1.0.26 to 1.0.27 (#12103)
dependabot[bot] Sep 6, 2023
23ac976
fix(stream): remove useless stream in nats sink (#12107)
yufansong Sep 6, 2023
9f02036
refactor(meta): introduce meta store ref and avoid pass generic param…
wenym1 Sep 6, 2023
2c6afd8
feat(log-store): refine log store trait and add update vnode bitmap i…
wenym1 Sep 6, 2023
16e40c6
refactor: refactor file cache metrics build, bump foyer (#12121)
MrCroxx Sep 6, 2023
72a3056
feat(stream): Add recovery feature for nats source (#12073)
yufansong Sep 6, 2023
8e3117c
test: Add nexmark test, kafka test for backwards compatibility testin…
kwannoel Sep 6, 2023
e8bfe42
feat(expr): add `array_min` (#12071)
xzhseh Sep 6, 2023
36a6633
fix(ci): fix backwards compat with `ci-release` profile (#12139)
kwannoel Sep 7, 2023
49339bf
chore(deps): Bump sysinfo from 0.29.9 to 0.29.10 (#12135)
dependabot[bot] Sep 7, 2023
e69353f
chore(deps): Bump num-bigint from 0.4.3 to 0.4.4 (#12137)
dependabot[bot] Sep 7, 2023
85156d3
fix(expr): parse `timestamptz` without seconds but with offset (#12084)
xiangjinwu Sep 7, 2023
bfc0d61
refactor(meta): refine `pause`/`resume` response (#12079)
BugenZhao Sep 7, 2023
ee48a45
fix(parser): more error reporting with locations (#12129)
BugenZhao Sep 7, 2023
2c93266
refactor(storage): seperate validator logic from picker (#11984)
Li0k Sep 7, 2023
2453f20
chore(deps): Bump regex-automata from 0.3.7 to 0.3.8 (#12134)
dependabot[bot] Sep 7, 2023
4e2520e
chore(deps): Bump chrono from 0.4.26 to 0.4.29 (#12136)
dependabot[bot] Sep 7, 2023
751d7bf
feat(expr): add `array_max` (#12100)
xzhseh Sep 7, 2023
8d30045
fix(eowc): fix SortBuffer when there are multiple records having same…
stdrc Sep 7, 2023
38998de
feat(storage): sepearte intra picker (#12147)
Li0k Sep 7, 2023
825641b
refactor(metrics): unify storage_metric_level & streaming_metric_leve…
fuyufjh Sep 7, 2023
00b7afe
feat(storage): reduce group deltas for l0 task (#12085)
Li0k Sep 7, 2023
e4cf738
feat(watermark): handle watermark in project_set (#12128)
yuhao-su Sep 7, 2023
b1c57ee
feat(sink): Support cassandra sink (#11878)
xxhZs Sep 7, 2023
f54319a
refactor(source): specify cdc generic type parameter for different cd…
wenym1 Sep 7, 2023
2f4771f
feat(stream): add multiple server url feature into nats, delete usele…
yufansong Sep 7, 2023
e01bf51
chore(deps): Bump xxhash-rust from 0.8.6 to 0.8.7 (#12159)
dependabot[bot] Sep 8, 2023
c125ec4
chore(object store): change streaming write buffer size to 16MB for g…
wcy-fdu Sep 8, 2023
1a548fe
fix: mysql-sink integration test (#12161)
StrikeW Sep 8, 2023
fb9eac4
fix(risedev): typo fixing (#12163)
xzhseh Sep 8, 2023
25699f3
fix: protobuf fixed64 was incorrectly parsed as int64 (#12126)
neverchanje Sep 8, 2023
01500df
chore: trigger cherry-pick when the label is added (#12171)
lmatz Sep 8, 2023
e213ae6
chore(deps): Bump rust_decimal from 1.31.0 to 1.32.0 (#12160)
dependabot[bot] Sep 8, 2023
d52c34d
fix(batch): fallback for node mask (#12177)
zwang28 Sep 8, 2023
0b228a4
feat: risectl hummock validate-version (#11737)
hzxa21 Sep 8, 2023
41eaf4f
chore(metric): fix table id filter and some panel reorgs (#12148)
hzxa21 Sep 8, 2023
ae89bd3
feat(demo): Add Feature Store demo (#11695)
xxhZs Sep 8, 2023
ab25efa
ci: hide nextest progress bar (#11974)
xxchan Sep 8, 2023
f6425c3
refactor(config): remove system.telemetry_enabled (#12156)
fuyufjh Sep 9, 2023
ce2149c
build: bump toolchain to 2023-09-09 (#11809)
xxchan Sep 9, 2023
97c0597
fix(risedev): Reset risedev.yml (#12179)
xxhZs Sep 9, 2023
dd35845
ci: fix disk out of space for GHA (#12192)
xxchan Sep 11, 2023
69ea900
feat: auto heap dump by default if `MALLOC_CONF=prof:true` (#12186)
fuyufjh Sep 11, 2023
73fb63a
chore(deps): regular cargo update after v1.2.0 (#12190)
xxchan Sep 11, 2023
2aab341
fix: ban pk comprising generated column (#12181)
yuhao-su Sep 11, 2023
b5789b2
chore(test): disable prometheus in some tests (#12208)
zwang28 Sep 11, 2023
4b34dbd
fix(udf): check udf schema fields num and total records (#12206)
KveinAxel Sep 11, 2023
1b9c185
feat(common): impl `Row` for `Either` (#12211)
BugenZhao Sep 11, 2023
de4d387
chore(deps): Bump syn from 2.0.31 to 2.0.32 (#12217)
dependabot[bot] Sep 11, 2023
5a05a16
chore(deps): Bump serde_json from 1.0.105 to 1.0.106 (#12214)
dependabot[bot] Sep 12, 2023
f453ce8
build: downgrade to openssl@1 temporarily (#12219)
xxchan Sep 12, 2023
afc1c8e
feat(expr): add `array_sort` (#12189)
xzhseh Sep 12, 2023
508de61
chore: bump foyer (#12133)
MrCroxx Sep 12, 2023
f4b8e9b
feat: make `enable.auto.commit` configurable (#12223)
tabVersion Sep 12, 2023
8267947
fix: ban update modify columns referenced by generated pk (#12212)
yuhao-su Sep 12, 2023
22b1ab9
feat: alter column for table with connector (#12164)
wugouzi Sep 12, 2023
9f2eb0f
refactor(docker/doc): apply docker file with new release version (#12…
wcy-fdu Sep 12, 2023
b3b8f78
fix(udf): check the data type returned from UDF server (#12202)
wangrunji0408 Sep 12, 2023
5400909
chore(udf): prepare for publishing java udf to maven (#11981)
wangrunji0408 Sep 12, 2023
aff2769
feat: disable some two phase agg (#12185)
st1page Sep 12, 2023
f6543ce
fix(test): set correct row per second (#12213)
zwang28 Sep 12, 2023
14e32b7
chore: remove ci main workflow (#12232)
huangjw806 Sep 12, 2023
fb0622d
feat(connector): init embedded connector node (#12122)
chenzl25 Sep 12, 2023
5b7d720
refactor: use usize range instead of BlockLocation to read obj (#12225)
MrCroxx Sep 12, 2023
a73e886
feat(binder): support implicit table function lateral subquery (#12153)
chenzl25 Sep 12, 2023
8b77936
refactor: ignore unrecogized/deprecated system params (#12229)
fuyufjh Sep 13, 2023
6afb299
chore(ci): prune tests which should run occasionally (#12233)
kwannoel Sep 13, 2023
39ebaaf
chore(ci): bump timeouts for bulid-release and unit test and e2e-sink…
kwannoel Sep 13, 2023
d52b037
feat(risedev): core dump on panics for development (#12235)
BugenZhao Sep 13, 2023
62550bf
chore(deps): Bump bytes from 1.4.0 to 1.5.0 (#12158)
dependabot[bot] Sep 13, 2023
f957f90
refactor(source): bind prop, split, reader, enumerator via trait (#12…
wenym1 Sep 13, 2023
2bbe5bf
refactor(kafka-sink): change sequential await to group await when com…
xzhseh Sep 13, 2023
76dc109
chore(deps): Bump vergen from 8.2.4 to 8.2.5 (#12244)
dependabot[bot] Sep 13, 2023
fd95898
feat(stream): compact stream chunk by dropping useless rows (#11070)
xx01cyx Sep 13, 2023
0e801b2
test: add an slt test for select with eowc (#12236)
fuyufjh Sep 13, 2023
78da15d
feat: allow kinesis source start with timestamp (#12241)
tabVersion Sep 13, 2023
c0e649d
refactor: replace more GAT-based async trait with RPITIT (#12271)
BugenZhao Sep 13, 2023
a82f328
chore: bump `jsonschema-transpiler` version (#12267)
wugouzi Sep 13, 2023
f6758e5
feat(catalog): add pg_type.typinput and support dummy casting to regp…
TennyZhuang Sep 13, 2023
eb45a0b
refactor(expr): merge expression templates into `#[function]` macros …
wangrunji0408 Sep 13, 2023
877ae0f
feat(meta): expose hummock pinned versions and snapshots via rw_catal…
zwang28 Sep 13, 2023
e016361
chore(deps): Bump syn from 2.0.32 to 2.0.33 (#12290)
dependabot[bot] Sep 14, 2023
94306bf
chore(deps): Bump libc from 0.2.147 to 0.2.148 (#12288)
dependabot[bot] Sep 14, 2023
b0412d1
fix(integration_tests): datagen timestamp and timestamptz accordingly…
xiangjinwu Sep 14, 2023
7002fcd
feat(agg): support streaming `bool_and` and `bool_or` agg (#11956)
wangrunji0408 Sep 14, 2023
9a873a2
fix(ci): fix regress test download of `locales` (#12296)
kwannoel Sep 14, 2023
56341aa
refactor(sink): impl RowEncoder for JsonEncoder (#12264)
xiangjinwu Sep 14, 2023
d8c3b6b
feat(frontend): implement `WITH ORDINALITY` clause (#12273)
xxchan Sep 14, 2023
9d58b90
feat(sink): support elasticsearch 8 sink (#12269)
hzxa21 Sep 14, 2023
798a28a
fix(storage): remove directly convert of tier compaction picker (#12280)
Li0k Sep 14, 2023
79c660f
fix: panic when dumping memory profile (#12276)
fuyufjh Sep 14, 2023
01a2843
refactor(connector): replace validate source rpc with jni (#12270)
chenzl25 Sep 14, 2023
6c90f0b
feat(stream): support source throttling (#12295)
kwannoel Sep 14, 2023
639acc2
feat(expr): add `pg_sleep` function (#12294)
wangrunji0408 Sep 14, 2023
b320ddf
chore: use cfg! to instead of #cfg[] for jemalloc control policy (#12…
yuhao-su Sep 14, 2023
3aa1a47
feat(refill): fetch whole sst file when refilling (#12265)
MrCroxx Sep 14, 2023
a7c83ea
chore(deps): Bump clap from 4.4.2 to 4.4.3 (#12245)
dependabot[bot] Sep 14, 2023
1d59c5f
refactor(risedev): split the steps for building and running playgroun…
BugenZhao Sep 14, 2023
bac62be
feat(stream): handling watermark in temporal join (#12302)
chenzl25 Sep 14, 2023
9aa8b73
fix(optimizer): relax scan predicate pull up mapping inverse restrict…
chenzl25 Sep 14, 2023
c4b58b5
refactor(connector): migrate cdc source metric from connector to comp…
chenzl25 Sep 14, 2023
b8d9ad5
refactor(jdbc-sink): execute statements in batch and set isolation le…
StrikeW Sep 14, 2023
9b498dd
chore(ci): install locales in prebuilt image (#12311)
huangjw806 Sep 14, 2023
95715da
feat(meta): add hummock version relevant tables to rw_catalog (#12309)
zwang28 Sep 14, 2023
b743ec4
fix(ci): exclude tikv-jemalloc-sys in hakari check (#12320)
MrCroxx Sep 14, 2023
0a11104
feat(expr): add `array_sum` (#12162)
xzhseh Sep 14, 2023
0b5cbd2
fix(regexp): substraction overflow when incorrectly speicifying `star…
xzhseh Sep 15, 2023
efe5785
chore(deps): Bump serde_json from 1.0.106 to 1.0.107 (#12322)
dependabot[bot] Sep 15, 2023
d3a8c7a
chore: fix comment and metrics (#12331)
MrCroxx Sep 15, 2023
e594a29
refactor(storage): refactor
Li0k Sep 14, 2023
bc078d9
Merge branch 'main' of https://github.com/risingwavelabs/risingwave i…
Li0k Sep 15, 2023
d336e4d
fix(storage): fix ut
Li0k Sep 15, 2023
7ac4ffa
Merge branch 'main' of https://github.com/risingwavelabs/risingwave i…
Li0k Sep 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion proto/hummock.proto
Original file line number Diff line number Diff line change
Expand Up @@ -391,8 +391,13 @@ message SubscribeCompactionEventRequest {

// ReportTask provides the compact task to report to the meta.
message ReportTask {
CompactTask compact_task = 2;
reserved 2;
reserved "compact_task";
map<uint32, TableStats> table_stats_change = 3;

uint64 task_id = 4;
CompactTask.TaskStatus task_status = 5;
repeated SstableInfo sorted_output_ssts = 6;
}

// HeartBeat provides the progress status of all tasks on the Compactor.
Expand Down
12 changes: 9 additions & 3 deletions src/meta/src/hummock/manager/compaction_group_manager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -595,6 +595,7 @@ impl<S: MetaStore> HummockManager<S> {
};
let mut current_version = versioning.current_version.clone();
let sst_split_info = current_version.apply_version_delta(&new_version_delta);

let mut branched_ssts = BTreeMapTransaction::new(&mut versioning.branched_ssts);
let mut trx = Transaction::default();
new_version_delta.apply_to_txn(&mut trx)?;
Expand Down Expand Up @@ -652,10 +653,15 @@ impl<S: MetaStore> HummockManager<S> {
}
}
}
for mut task in canceled_tasks {
task.set_task_status(TaskStatus::ManualCanceled);
for task in canceled_tasks {
if !self
.report_compact_task_impl(&mut task, &mut compaction_guard, None)
.report_compact_task_impl(
task.task_id,
TaskStatus::ManualCanceled,
vec![],
&mut compaction_guard,
None,
)
.await
.unwrap_or(false)
{
Expand Down
Loading