Skip to content

Commit

Permalink
Create gcloud-dataflow.md
Browse files Browse the repository at this point in the history
Added information on Dataflow
  • Loading branch information
KimtVak8143 authored Oct 9, 2022
1 parent 8f5ac40 commit af68549
Showing 1 changed file with 16 additions and 0 deletions.
16 changes: 16 additions & 0 deletions Data-streaming/gcloud-dataflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
## Google Cloud Dataflow

Google Cloud Dataflow is a fully managed service for executing Apache Beam pipelines within the Google Cloud Platform ecosystem.

Benefits:
- Streaming data analytics with speed: Dataflow enables fast, simplified streaming data pipeline development with lower data latency.
- Simplify operations and management: Allow teams to focus on programming instead of managing server clusters as Dataflow’s serverless approach removes operational overhead from data engineering workloads.
- Reduce total cost of ownership: Resource autoscaling paired with cost-optimized batch processing capabilities means Dataflow offers virtually limitless capacity to manage your seasonal and spiky workloads without overspending.

Features:
- Autoscaling of resources and dynamic work rebalancing: Minimize pipeline latency, maximize resource utilization, and reduce processing cost per data record with data-aware resource autoscaling. Data inputs are partitioned automatically and constantly rebalanced to even out worker resource utilization and reduce the effect of “hot keys” on pipeline performance.
- Flexible scheduling and pricing for batch processing: For processing with flexibility in job scheduling time, such as overnight jobs, flexible resource scheduling (FlexRS) offers a lower price for batch processing. These flexible jobs are placed into a queue with a guarantee that they will be retrieved for execution within a six-hour window.
- Ready-to-use real-time AI patterns: Enabled through ready-to-use patterns, Dataflow’s real-time AI capabilities allow for real-time reactions with near-human intelligence to large torrents of events. Customers can build intelligent solutions ranging from predictive analytics and anomaly detection to real-time personalization and other advanced analytics use cases.


For more info: [https://cloud.google.com/dataflow](https://cloud.google.com/dataflow)

0 comments on commit af68549

Please sign in to comment.