From 66a7e9de2aae0c599b62deecc804c897585bf1af Mon Sep 17 00:00:00 2001 From: Egor Kraev Date: Fri, 27 Sep 2024 13:00:12 +0200 Subject: [PATCH] Update Event-driven orchestration for AI systems.ipynb --- .../Event-driven orchestration for AI systems.ipynb | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/examples/Event-driven orchestration for AI systems.ipynb b/examples/Event-driven orchestration for AI systems.ipynb index 94d407b0..a6187b9d 100644 --- a/examples/Event-driven orchestration for AI systems.ipynb +++ b/examples/Event-driven orchestration for AI systems.ipynb @@ -4,15 +4,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Event-driven orchestration for AI systems. Is LlamaIndex Workflows basically Kafka Streams?\n", + "# Event-driven orchestration for AI systems: new ways and old\n", "\n", - "Data processing tools like Kafka, Spark, Flink etc. have long been used to build workflows. Today, as people create complex AI systems, they often find themselves (surprise!) needing orchestration frameworks. Is the field really so specific it needs new tools for solving the same problems? We argue that there is no difference: after all, LLMs are just APIs, and the fundamental patterns of workflow architecture remain the same.\n", + "Data processing tools like Kafka, Spark, Flink etc. have long been used to build workflows. Today, as people create complex AI systems, they unsurprisingly often find themselves needing orchestration frameworks. Is the field really so specific it needs new tools for solving the same problems? After all, LLMs are just APIs, and the fundamental patterns of workflow architecture remain the same.\n", "\n", - "A widespread data processing paradigm is event-driven architecture. The purpose of this example is to demonstrate how you can use tried-and-tested tools like Kafka Streams (ported to Python via Faust) to build workflows, side-stepping the need to re-invent the wheel for something as fundamental as event processing. This also comes with reliability and scalability of Kafka, easing production deployment.\n", + "A widespread data processing paradigm is event-driven architecture. The purpose of this example is to demonstrate how you can use tried-and-tested tools like Kafka and [Faust](https://github.com/faust-streaming/faust) to build workflows. This also comes with reliability and scalability of Kafka, easing production deployment.\n", "\n", - "We'll walk through the basic concepts of the newly released LlamaIndex Workflows library, and compare them to what you get with Faust.\n", - "\n", - "To get a basic understanding of event-driven workflows, we suggest you check out this guide to LlamaIndex Workflows: https://docs.llamaindex.ai/en/stable/module_guides/workflow/\n", + "We'll walk through the basic concepts of the newly released [LlamaIndex Workflows](https://docs.llamaindex.ai/en/stable/module_guides/workflow/) functionality, and compare them to what you get with Faust.\n", "\n", "Here we are using the well-maintained fork of Faust: https://github.com/faust-streaming/faust" ]