-
Notifications
You must be signed in to change notification settings - Fork 37
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs(hydroflow_plus): polish quickstart and drop stale pages (#1576)
I've decided to leave some more challenging TODOs as comments for now, just so we can get the live site back into decent shape. Also changes the "Get Started" button on the landing to point to the HF+ quickstart.
- Loading branch information
Showing
14 changed files
with
86 additions
and
283 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
sidebar_position: 1 | ||
--- | ||
|
||
# Dataflow Programming | ||
Hydroflow+ uses a dataflow programming model, which will be familiar if you have used APIs like Rust iterators. Instead of using RPCs or async/await to describe distributed computation, Hydroflow+ instead uses **asynchronous streams**, which represent data arriving over time. Streams can represent a series of asynchronous events (e.g. inbound network requests) or a sequence of data items. | ||
|
||
Programs in Hydroflow+ describe how to **transform** entire collections of data using operators such as `map` (transforming elements one by one), `fold` (aggregating elements into a single value), or `join` (combining elements from multiple streams on matching keys). | ||
|
||
If you are familiar with Spark, Flink or Pandas, you will find Hydroflow+ syntax familiar. However, note well that the semantics for asynchronous streams in Hydroflow+ differ significantly from bulk analytics systems like those above. In particular, Hydroflow+ uses the type system to distinguish between bounded streams (originating from finite data) and unbounded streams (originated from asynchronous input). Moreover, Hydroflow+ is designed to handle asynchronous streams of small, independent events very efficiently. | ||
|
||
<!-- TODO(shadaj): link to collections section --> | ||
<!-- TODO(shadaj): show example of mermaid graph --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.