Skip to content

Releases: hydro-project/hydro

hydro_std v0.11.0

23 Dec 22:06
@hydro-project-bot hydro-project-bot
hydro_std-v0.11.0
b58dccc
Compare
Choose a tag to compare
hydro_std v0.11.0

Chore

  • bump versions manually for renamed crates, per RELEASING.md
  • Rename HydroflowPlus to Hydro

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • cleanups for the rename, fixing links

Commit Statistics

Commit Details

view details
  • #1501
  • #1617
    • Rename HydroflowPlus to Hydro (162e49c)
  • #1624
    • Cleanups for the rename, fixing links (6ab6252)
  • #1627
    • Bump versions manually for renamed crates, per RELEASING.md (03b3a34)
  • Uncategorized
    • Release dfir_lang v0.11.0, dfir_datalog_core v0.11.0, dfir_datalog v0.11.0, dfir_macro v0.11.0, hydroflow_deploy_integration v0.11.0, lattices_macro v0.5.8, variadics v0.0.8, variadics_macro v0.5.6, lattices v0.5.9, multiplatform_test v0.4.0, pusherator v0.0.10, dfir_rs v0.11.0, hydro_deploy v0.11.0, stageleft_macro v0.5.0, stageleft v0.6.0, stageleft_tool v0.5.0, hydro_lang v0.11.0, hydro_std v0.11.0, hydro_cli v0.11.0, safety bump 6 crates (9a7e486)
Assets 2
Loading

hydro_lang v0.11.0

23 Dec 22:06
@hydro-project-bot hydro-project-bot
hydro_lang-v0.11.0
b58dccc
Compare
Choose a tag to compare
hydro_lang v0.11.0

Chore

  • use same hashing library everywhere

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

  • cleanups for the rename, fixing links

Chore

  • bump versions manually for renamed crates, per RELEASING.md
  • cleanup snapshots
  • Rename Hydroflow -> DFIR
    Work In Progress:
    • hydroflow_macro
    • hydroflow_datalog_core
    • hydroflow_datalog
    • hydroflow_lang
    • hydroflow
  • Rename HydroflowPlus to Hydro

New Features

  • add round_robin helpers for networking
    Also fixes compiler crashes when using .enumerate() on an un-batched
    stream.

Bug Fixes

  • drop nightly feature flag from trybuild codegen
  • restrict lifetime parameters to be actually invariant
    Our lifetimes were accidentally made covariant when the lifetime 'a
    was removed from the process/cluster tag type. This fixes that typing
    hole, and also loosens some restrictions on the lifetime of deploy
    environments.

Refactor

  • use match_box macro to compile on stable
  • generalize quorum logic

New Features (BREAKING)

  • minimize dependencies pulled into trybuild builds
    We don't need hydroflow_lang in the runtime builds, do some feature
    flagging to avoid it.

  • mark non-deterministic operators as unsafe and introduce timestamped streams
    Big PR.

    First big change is we introduce a Timestamped location. This is a bit
    of a hybrid between top-level locations and Tick locations. The idea
    is that you choose where timestamps are generated, and then have a
    guarantee that everything after that will be atomically computed (useful
    for making sure we add payloads to the log before ack-ing).

    The contract is that an operator or module that takes a Timestamped
    input must still be deterministic regardless of the stamps on messages
    (which are hidden unless you tick_batch). But unlike a top-level
    stream (which has the same constraints), you have the atomicity
    guarantee. Right now the guarantee is trivial since we have one global
    tick for everything. But in the future when we want to apply
    @davidchuyaya's optimizations this will be helpful to know when there
    are causal dependencies on when data can be sent to others.

    Second change is we mark every non-deterministic operator (modulo
    explicit annotations such as NoOrder) with Rust's unsafe keyword.
    This makes it super clear where non-determinism is taking place.

    I've used this to put unsafe blocks throughout our example code and
    add SAFETY annotations that argue why the non-determinism is safe (or
    point out that we've explicitly documented / expect non-determinism). I
    also added #![warn(unsafe_op_in_unsafe_fn)] to the examples and the
    template, since this forces good hygiene of annotating sources of
    non-determinism even inside a module that is intentionally
    non-deterministic.

    Paxos changes are mostly refactors, and I verified that the performance
    is the same as before.

  • allow runtime context to be referenced as a global constant

  • allow cluster self ID to be referenced as a global constant
    This eliminates the need to store cluster.self_id() in a local
    variable first, instead you can directly reference CLUSTER_SELF_ID.

  • introduce an unordered variant of streams to strengthen determinism guarantees
    Previously, sending data from a Cluster would return a stream assumed
    to have deterministic contents and ordering, which is false. This
    introduces another type parameter for Stream which tracks whether
    element ordering is expected to be deterministic, and restricts
    operators such as fold and reduce to commutative aggregations
    accordingly.

Bug Fixes (BREAKING)

  • rename union to chain and restrict LHS to be bounded
    Returning a Stream from union on unbounded streams was unsound,
    since the order of outputs is not deterministic.

Refactor (BREAKING)

  • use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit
    Previous PR (#1587) website build did not work because panic = "abort"
    is set on wasm, leading to aborts for proc_macro2::Span::unwrap()
    calls.

    All tests except trybuild seem to pass on stable, WIP #1587 next

  • further reduce namespace pollution

  • don't re-export all of hydroflow
    Reduces namespace pollution when wildcard-importing hydroflow_plus.

hydroflow_plus Commit Statistics

hydroflow_plus Commit Details

view details
Read more

Contributors

davidchuyaya
Assets 2
Loading

hydro_deploy v0.11.0

23 Dec 21:40
@hydro-project-bot hydro-project-bot
hydro_deploy-v0.11.0
9a7e486
Compare
Choose a tag to compare
hydro_deploy v0.11.0

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

New Features

  • use regular println when no tasks are active
    Significantly improves the appearance of Hydroflow+ logs when the
    terminal causes wrapping.

Commit Statistics

  • 3 commits contributed to the release.
  • 45 days passed between releases.
  • 3 commits were understood as conventional.
  • 2 unique issues were worked on: #1501, #1577

Commit Details

view details
  • #1501
  • #1577
    • Use regular println when no tasks are active (8d550b9)
  • Uncategorized
    • Update CHANGELOG.mds for big rename (e1a08e5)
Assets 2
Loading

hydro_cli v0.11.0

23 Dec 22:06
@hydro-project-bot hydro-project-bot
hydro_cli-v0.11.0
b58dccc
Compare
Choose a tag to compare
hydro_cli v0.11.0 Latest
Latest

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

New Features (BREAKING)

  • mark non-deterministic operators as unsafe and introduce timestamped streams
    Big PR.

    First big change is we introduce a Timestamped location. This is a bit
    of a hybrid between top-level locations and Tick locations. The idea
    is that you choose where timestamps are generated, and then have a
    guarantee that everything after that will be atomically computed (useful
    for making sure we add payloads to the log before ack-ing).

    The contract is that an operator or module that takes a Timestamped
    input must still be deterministic regardless of the stamps on messages
    (which are hidden unless you tick_batch). But unlike a top-level
    stream (which has the same constraints), you have the atomicity
    guarantee. Right now the guarantee is trivial since we have one global
    tick for everything. But in the future when we want to apply
    @davidchuyaya's optimizations this will be helpful to know when there
    are causal dependencies on when data can be sent to others.

    Second change is we mark every non-deterministic operator (modulo
    explicit annotations such as NoOrder) with Rust's unsafe keyword.
    This makes it super clear where non-determinism is taking place.

    I've used this to put unsafe blocks throughout our example code and
    add SAFETY annotations that argue why the non-determinism is safe (or
    point out that we've explicitly documented / expect non-determinism). I
    also added #![warn(unsafe_op_in_unsafe_fn)] to the examples and the
    template, since this forces good hygiene of annotating sources of
    non-determinism even inside a module that is intentionally
    non-deterministic.

    Paxos changes are mostly refactors, and I verified that the performance
    is the same as before.

Commit Statistics

  • 3 commits contributed to the release.
  • 45 days passed between releases.
  • 2 commits were understood as conventional.
  • 2 unique issues were worked on: #1501, #1584

Commit Details

view details
  • #1501
  • #1584
    • Mark non-deterministic operators as unsafe and introduce timestamped streams (9393899)
  • Uncategorized
    • Release dfir_lang v0.11.0, dfir_datalog_core v0.11.0, dfir_datalog v0.11.0, dfir_macro v0.11.0, hydroflow_deploy_integration v0.11.0, lattices_macro v0.5.8, variadics v0.0.8, variadics_macro v0.5.6, lattices v0.5.9, multiplatform_test v0.4.0, pusherator v0.0.10, dfir_rs v0.11.0, hydro_deploy v0.11.0, stageleft_macro v0.5.0, stageleft v0.6.0, stageleft_tool v0.5.0, hydro_lang v0.11.0, hydro_std v0.11.0, hydro_cli v0.11.0, safety bump 6 crates (9a7e486)

Contributors

davidchuyaya
Assets 2
Loading

dfir_rs v0.11.0

23 Dec 21:40
@hydro-project-bot hydro-project-bot
dfir_rs-v0.11.0
9a7e486
Compare
Choose a tag to compare
dfir_rs v0.11.0

Documentation

  • getLines support named section markers, fix #1320

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • fix some broken github tree/main links

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

  • cleanups for the rename, fixing links

New Features

  • initial flo implementation
    Basic first pass implementation, without changing the current scheduler,
    no breaking behavior. Adds the flo syntax (loop { ... }), basic graph
    structure checks, nested loop blocks, and two basic windowing operators
    (batch() and all_once()).

    Next steps:

    • Needs scheduler changes:
    • Implement un-windowing operators

  • Implement windowing operator repeat_n() -

Bug Fixes

  • avoid pulling in clap as a dependency
    We only need it for examples, so this reduces compilation burden for
    trybuild.

Other

  • fix stable tests
    Sets TRYBUILD=overwrite on stable, as the output messages will often
    not match pinned-nightly

    Adds env HYDROFLOW_EXPECT_WARNINGS=ignore to allow warning tests to
    pass

Chore

  • update links for renamed repo (excluding CHANGELOG.mds), fix #1571
  • bump versions manually for renamed crates, per RELEASING.md
  • cleanup snapshots
  • Rename Hydroflow -> DFIR
    Work In Progress:
    • hydroflow_macro
    • hydroflow_datalog_core
    • hydroflow_datalog
    • hydroflow_lang
    • hydroflow

Refactor

  • rename two_pc -> two_pc_hf for collision with hf+ two_pc example
  • use match_box macro to compile on stable
  • move rest of scheduler state into Context

New Features (BREAKING)

  • minimize dependencies pulled into trybuild builds
    We don't need hydroflow_lang in the runtime builds, do some feature
    flagging to avoid it.

  • remove import!, fix #1110
    in prep for rust stable #1587

    No good way to resolve the source file paths on stable

    No way to get good diagnostics on external files in general, at all

Refactor (BREAKING)

  • use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit
    Previous PR (#1587) website build did not work because panic = "abort"
    is set on wasm, leading to aborts for proc_macro2::Span::unwrap()
    calls.

    All tests except trybuild seem to pass on stable, WIP #1587 next

hydroflow Commit Statistics

hydroflow Commit Details

view details

Commit Statistics

Commit Details

view details
Assets 2
Loading

dfir_macro v0.11.0

23 Dec 21:40
@hydro-project-bot hydro-project-bot
dfir_macro-v0.11.0
9a7e486
Compare
Choose a tag to compare
dfir_macro v0.11.0

Chore

  • upgrade to Docusaurus v3
    Main breaking change is MDX parsing, which trips up on unescaped < in
    the generated docs, so we have to adjust the generator logic.

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • fix some broken github tree/main links

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

  • cleanups for the rename, fixing links

Chore

  • update links for renamed repo (excluding CHANGELOG.mds), fix #1571
  • bump versions manually for renamed crates, per RELEASING.md
  • Rename Hydroflow -> DFIR
    Work In Progress:
    • hydroflow_macro
    • hydroflow_datalog_core
    • hydroflow_datalog
    • hydroflow_lang
    • hydroflow

New Features (BREAKING)

  • remove import!, fix #1110
    in prep for rust stable #1587

    No good way to resolve the source file paths on stable

    No way to get good diagnostics on external files in general, at all

Refactor (BREAKING)

  • use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit
    Previous PR (#1587) website build did not work because panic = "abort"
    is set on wasm, leading to aborts for proc_macro2::Span::unwrap()
    calls.

    All tests except trybuild seem to pass on stable, WIP #1587 next

hydroflow_macro Commit Statistics

  • 3 commits contributed to the release.
  • 38 days passed between releases.
  • 3 commits were understood as conventional.
  • 3 unique issues were worked on: #1558, #1600, #1606

hydroflow_macro Commit Details

view details

Commit Statistics

Commit Details

view details
  • #1501
  • #1620
    • Rename Hydroflow -> DFIR (3291c07)
  • #1624
    • Cleanups for the rename, fixing links (6ab6252)
  • #1627
    • Bump versions manually for renamed crates, per RELEASING.md (03b3a34)
  • #1628
    • Update links for renamed repo (excluding CHANGELOG.mds), fix #1571 (5196f24)
  • Uncategorized
    • Fix some broken github tree/main links (c707659)
    • Update CHANGELOG.mds for big rename (e1a08e5)
Assets 2
Loading

dfir_lang v0.11.0

23 Dec 21:40
@hydro-project-bot hydro-project-bot
dfir_lang-v0.11.0
9a7e486
Compare
Choose a tag to compare
dfir_lang v0.11.0

New Features

  • initial flo implementation
    Basic first pass implementation, without changing the current scheduler,
    no breaking behavior. Adds the flo syntax (loop { ... }), basic graph
    structure checks, nested loop blocks, and two basic windowing operators
    (batch() and all_once()).

    Next steps:

    • Needs scheduler changes:
    • Implement un-windowing operators

  • Implement windowing operator repeat_n() -

Other

  • fix stable tests
    Sets TRYBUILD=overwrite on stable, as the output messages will often
    not match pinned-nightly

    Adds env HYDROFLOW_EXPECT_WARNINGS=ignore to allow warning tests to
    pass

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

  • cleanups for the rename, fixing links

Chore

  • update links for renamed repo (excluding CHANGELOG.mds), fix #1571
  • bump versions manually for renamed crates, per RELEASING.md
  • Rename Hydroflow -> DFIR
    Work In Progress:
    • hydroflow_macro
    • hydroflow_datalog_core
    • hydroflow_datalog
    • hydroflow_lang
    • hydroflow

New Features (BREAKING)

  • remove import!, fix #1110
    in prep for rust stable #1587

    No good way to resolve the source file paths on stable

    No way to get good diagnostics on external files in general, at all

Bug Fixes (BREAKING)

  • rename union to chain and restrict LHS to be bounded
    Returning a Stream from union on unbounded streams was unsound,
    since the order of outputs is not deterministic.

Refactor (BREAKING)

  • use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit
    Previous PR (#1587) website build did not work because panic = "abort"
    is set on wasm, leading to aborts for proc_macro2::Span::unwrap()
    calls.

    All tests except trybuild seem to pass on stable, WIP #1587 next

hydroflow_lang Commit Statistics

hydroflow_lang Commit Details

view details

Commit Statistics

Commit Details

view details
Assets 2
Loading

dfir_datalog_core v0.11.0

23 Dec 21:40
@hydro-project-bot hydro-project-bot
dfir_datalog_core-v0.11.0
9a7e486
Compare
Choose a tag to compare
dfir_datalog_core v0.11.0

New Features

  • initial flo implementation
    Basic first pass implementation, without changing the current scheduler,
    no breaking behavior. Adds the flo syntax (loop { ... }), basic graph
    structure checks, nested loop blocks, and two basic windowing operators
    (batch() and all_once()).

    Next steps:

    • Needs scheduler changes:
    • Implement un-windowing operators

  • Implement windowing operator repeat_n() -

Refactor (BREAKING)

  • use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit
    Previous PR (#1587) website build did not work because panic = "abort"
    is set on wasm, leading to aborts for proc_macro2::Span::unwrap()
    calls.

    All tests except trybuild seem to pass on stable, WIP #1587 next

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

Chore

  • bump versions manually for renamed crates, per RELEASING.md
  • cleanup snapshots
  • Rename Hydroflow -> DFIR
    Work In Progress:
    • hydroflow_macro
    • hydroflow_datalog_core
    • hydroflow_datalog
    • hydroflow_lang
    • hydroflow

hydroflow_datalog_core Commit Statistics

  • 2 commits contributed to the release.
  • 38 days passed between releases.
  • 2 commits were understood as conventional.
  • 2 unique issues were worked on: #1585, #1606

hydroflow_datalog_core Commit Details

view details
  • #1585
    • Initial flo implementation (b8acd84)
  • #1606
    • Use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit (251b103)

Commit Statistics

Commit Details

view details
Assets 2
Loading

dfir_datalog v0.11.0

23 Dec 21:40
@hydro-project-bot hydro-project-bot
dfir_datalog-v0.11.0
9a7e486
Compare
Choose a tag to compare
dfir_datalog v0.11.0

Refactor (BREAKING)

  • use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit
    Previous PR (#1587) website build did not work because panic = "abort"
    is set on wasm, leading to aborts for proc_macro2::Span::unwrap()
    calls.

    All tests except trybuild seem to pass on stable, WIP #1587 next

Documentation

  • add repository field to Cargo.tomls, fix #1452
    #1452

    Will trigger new releases of the following:
    unchanged = 'hydroflow_deploy_integration', 'variadics', 'variadics_macro', 'pusherator'

    (All other crates already have changes, so would be released anyway)

  • fix some broken github tree/main links

  • update CHANGELOG.mds for big rename
    Generated before rename per RELEASING.md instructions.

  • cleanups for the rename, fixing links

Chore

  • update links for renamed repo (excluding CHANGELOG.mds), fix #1571
  • bump versions manually for renamed crates, per RELEASING.md
  • Rename Hydroflow -> DFIR
    Work In Progress:
    • hydroflow_macro
    • hydroflow_datalog_core
    • hydroflow_datalog
    • hydroflow_lang
    • hydroflow

hydroflow_datalog Commit Statistics

  • 1 commit contributed to the release.
  • 38 days passed between releases.
  • 1 commit was understood as conventional.
  • 1 unique issue was worked on: #1606

hydroflow_datalog Commit Details

view details
  • #1606
    • Use cfg(nightly) instead of feature, remove -Z flag, use Diagnostic::try_emit (251b103)

Commit Statistics

Commit Details

view details
  • #1501
  • #1620
    • Rename Hydroflow -> DFIR (3291c07)
  • #1624
    • Cleanups for the rename, fixing links (6ab6252)
  • #1627
    • Bump versions manually for renamed crates, per RELEASING.md (03b3a34)
  • #1628
    • Update links for renamed repo (excluding CHANGELOG.mds), fix #1571 (5196f24)
  • Uncategorized
    • Fix some broken github tree/main links (c707659)
    • Update CHANGELOG.mds for big rename (e1a08e5)
Assets 2
Loading

variadics_macro v0.5.5

08 Nov 19:30
@hydro-project-bot hydro-project-bot
variadics_macro-v0.5.5
dcd48fc
Compare
Choose a tag to compare
variadics_macro v0.5.5

New Features

  • generalized hash trie indexes for relational tuples
    Generalized Hash Tries are part of the SIGMOD '23 FreeJoin
    paper by
    Wang/Willsey/Suciu. They provide a compressed ("factorized")
    representation of relations. By operating in the factorized domain, join
    algorithms can defer cross-products and achieve asymptotically optimal
    performance.

Commit Statistics

  • 1 commit contributed to the release.
  • 1 commit was understood as conventional.
  • 1 unique issue was worked on: #1503

Commit Details

view details
  • #1503
    • Generalized hash trie indexes for relational tuples (f7e740f)
Assets 2
Loading