Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bad interval can cause infinite and conflict outputs in streaming generate_series #17786

Open
stdrc opened this issue Jul 23, 2024 · 1 comment
Open

bad interval can cause infinite and conflict outputs in streaming generate_series #17786

stdrc opened this issue Jul 23, 2024 · 1 comment
Labels
no-issue-activity type/bug Something isn't working

Comments

@stdrc
Copy link
Member

stdrc commented Jul 23, 2024 •
edited
Loading

Describe the bug

As noted by @xiangjinwu in #17461 (comment), the following query will panic compute node because the given interval makes streaming generate_series keep outputting the same timestamp 2024-01-31T00:00:00Z, and thus cause conflict in Materialize.

create materialized view mv as
select * from generate_series(
  '2024-01-31T00:00:00Z'::timestamptz,
  now(),
  interval '1 month -29 days'
);

Maybe we should ban such intervals.

Error message/log

2024-07-23T15:43:00.013619+08:00  INFO  rw-standalone-frontend handle_query{mode="simple query" session_id=0 sql=CREATE MATERIALIZED VIEW time_mv AS SELECT * FROM generate_series(CAST('2024-01-31T00:00:00Z' AS timestamptz), now(), INTERVAL '1 month -29 days')}: pgwire_query_log: status="ok" time=171ms
thread 'rw-streaming' panicked at src/stream/src/common/table/state_table.rs:929:17:
mem-table operation inconsistent! table_id: 1, vnode: 0, key: OwnedRow([Some(Timestamptz(Timestamptz(1706659200000000)))]), prev: Insert(Ok([Some(Timestamptz(Timestamptz(1706659200000000)))])), new: Insert(Ok([Some(Timestamptz(Timestamptz(1706659200000000)))]))
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/panicking.rs:72:14
   2: risingwave_stream::common::table::state_table::StateTableInner<S,SD,_,W,_>::handle_mem_table_error
             at ./src/stream/src/common/table/state_table.rs:929:17
   3: risingwave_stream::common::table::state_table::StateTableInner<S,SD,_,W,_>::insert_inner::{{closure}}
             at ./src/stream/src/common/table/state_table.rs:955:33
   4: core::result::Result<T,E>::unwrap_or_else
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/result.rs:1431:23
   5: risingwave_stream::common::table::state_table::StateTableInner<S,SD,_,W,_>::insert_inner
             at ./src/stream/src/common/table/state_table.rs:953:9
   6: risingwave_stream::common::table::state_table::StateTableInner<S,SD,_,W,_>::write_chunk
             at ./src/stream/src/common/table/state_table.rs:1114:25
   7: risingwave_stream::executor::mview::materialize::MaterializeExecutor<S,SD>::execute_inner::{{closure}}
             at ./src/stream/src/executor/mview/materialize.rs:260:29
   8: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
   9: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  10: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  11: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/stream/mod.rs:1638:9
  12: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/stream/next.rs:32:9
  13: <await_tree::future::Instrumented<F,_> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/await-tree-0.2.1/src/future.rs:119:15
  14: risingwave_stream::executor::wrapper::trace::instrument_await_tree::{{closure}}
             at ./src/stream/src/executor/wrapper/trace.rs:116:10
  15: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  16: risingwave_stream::executor::wrapper::schema_check::schema_check::{{closure}}
             at ./src/stream/src/executor/wrapper/schema_check.rs:24:1
  17: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  18: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  19: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/stream/mod.rs:1638:9
  20: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/stream/next.rs:32:9
  21: risingwave_stream::executor::wrapper::epoch_check::epoch_check::{{closure}}
             at ./src/stream/src/executor/wrapper/epoch_check.rs:31:44
  22: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  23: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  24: <S as futures_core::stream::TryStream>::try_poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:196:9
  25: futures_util::stream::try_stream::TryStreamExt::try_poll_next_unpin
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/try_stream/mod.rs:1131:9
  26: <futures_util::stream::try_stream::try_next::TryNext<St> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/try_stream/try_next.rs:32:9
  27: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:391:31
  28: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:217:19
  29: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:387:19
  30: risingwave_common::util::epoch::task_local::scope::{{closure}}
             at ./src/common/src/util/epoch.rs:249:47
  31: risingwave_stream::executor::wrapper::epoch_provide::epoch_provide::{{closure}}
             at ./src/stream/src/executor/wrapper/epoch_provide.rs:31:59
  32: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  33: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  34: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/stream/mod.rs:1638:9
  35: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/stream/stream/next.rs:32:9
  36: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/instrument.rs:321:9
  37: risingwave_stream::executor::wrapper::trace::trace::{{closure}}
             at ./src/stream/src/executor/wrapper/trace.rs:53:69
  38: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  39: risingwave_stream::executor::wrapper::update_check::update_check::{{closure}}
             at ./src/stream/src/executor/wrapper/update_check.rs:27:1
  40: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  41: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  42: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  43: <risingwave_stream::executor::dispatch::DispatchExecutor as risingwave_stream::executor::StreamConsumer>::execute::{{closure}}
             at ./src/stream/src/executor/dispatch.rs:390:9
  44: <futures_async_stream::try_stream::GenTryStream<G> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-async-stream-0.2.9/src/lib.rs:506:33
  45: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:120:9
  46: <&mut S as futures_core::stream::Stream>::poll_next
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/stream.rs:104:9
  47: <tokio_stream::stream_ext::next::Next<St> as core::future::future::Future>::poll
             at /Users/richard/.cargo/git/checkouts/tokio-968c02b7a1a41bea-shallow/fe39bb8/tokio-stream/src/stream_ext/next.rs:42:9
  48: <tokio_stream::stream_ext::try_next::TryNext<St> as core::future::future::Future>::poll
             at /Users/richard/.cargo/git/checkouts/tokio-968c02b7a1a41bea-shallow/fe39bb8/tokio-stream/src/stream_ext/try_next.rs:43:9
  49: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/instrument.rs:321:9
  50: <await_tree::future::Instrumented<F,_> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/await-tree-0.2.1/src/future.rs:119:15
  51: risingwave_stream::executor::actor::Actor<C>::run_consumer::{{closure}}
             at ./src/stream/src/executor/actor.rs:214:18
  52: <tokio::future::maybe_done::MaybeDone<Fut> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/future/maybe_done.rs:62:56
  53: risingwave_stream::executor::actor::Actor<C>::run::{{closure}}::{{closure}}::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/macros/join.rs:126:24
  54: <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/future/poll_fn.rs:58:9
  55: risingwave_stream::executor::actor::Actor<C>::run::{{closure}}::{{closure}}
             at ./src/stream/src/executor/actor.rs:170:17
  56: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:391:31
  57: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:217:19
  58: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:387:19
  59: risingwave_expr::expr_context::expr_context_scope::{{closure}}
             at ./src/expr/core/src/expr_context.rs:35:65
  60: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:391:31
  61: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:217:19
  62: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:387:19
  63: risingwave_stream::executor::actor::Actor<C>::run::{{closure}}
             at ./src/stream/src/executor/actor.rs:178:10
  64: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/map.rs:55:37
  65: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/lib.rs:91:13
  66: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:391:31
  67: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:217:19
  68: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:387:19
  69: await_tree::root::TreeRoot::instrument::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/await-tree-0.2.1/src/root.rs:43:34
  70: <futures_util::future::either::Either<A,B> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/either.rs:109:32
  71: core::ops::function::FnOnce::call_once
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/ops/function.rs:250:5
  72: tokio_metrics::task::instrument_poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-metrics-0.3.1/src/task.rs:2530:15
  73: <tokio_metrics::task::Instrumented<T> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-metrics-0.3.1/src/task.rs:2430:9
  74: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:391:31
  75: tokio::task::task_local::LocalKey<T>::scope_inner
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:217:19
  76: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/task/task_local.rs:387:19
  77: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/instrument.rs:321:9
  78: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/core.rs:328:17
  79: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/loom/std/unsafe_cell.rs:16:9
  80: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/core.rs:317:13
  81: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/richard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:485:19
  82: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/panic/unwind_safe.rs:272:9
  83: std::panicking::try::do_call
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panicking.rs:552:40
  84: ___rust_try
  85: std::panicking::try
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panicking.rs:516:19
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


*** await tree context of current task ***

Actor 1: `CREATE MATERIALIZED VIEW time_mv AS SELECT * FROM generate_series(CAST('2024-01-31T00:00:00Z' AS timestamptz), now(), INTERVAL '1 month -29 days')` [3.671s]
  Epoch 6847553863548928 [3.659s]
    Materialize 100000001 [3.659s]  <== current

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response
@stdrc stdrc added the type/bug Something isn't working label Jul 23, 2024
@github-actions github-actions bot added this to the release-1.11 milestone Jul 23, 2024
@stdrc stdrc removed this from the release-1.11 milestone Jul 23, 2024
@stdrc stdrc mentioned this issue Jul 23, 2024
Closed
@stdrc stdrc changed the title bad interval can cause conflict outputs in streaming generate_series bad interval can cause infinite and conflict outputs in streaming generate_series Jul 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

This issue has been open for 60 days with no activity.

If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.

You can also confidently close this issue as not planned to keep our backlog clean.
Don't worry if you think the issue is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
no-issue-activity type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@stdrc