Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

使用Connector进行并流时,可能会出现数据未准备好的情况 #28

Open
cpf2021-gif opened this issue Jun 28, 2024 · 1 comment

Comments

@cpf2021-gif
Copy link

cpf2021-gif commented Jun 28, 2024

出现的问题

在示例8中,数据流图:
image

问题的描述

在程序中开启两个Goroutine分别执行Flow1和Flow2,如果Flow2执行到Function2,要执行Load操作时,Flow1并未到Function3并完成Save操作,就会出现下面的错误:

LoadScoreAvg12(): GetStuScoresByStuId err = avg_score not found for stuId: 101
context.Background
Function Called Error err = avg_score not found for stuId: 101

->Call Func SaveScoreScore12
panic: avg_score not found for stuId: 101

可能的方案

是否能在Connector中添加方法判断数据是否成功存储(Save),如果数据并未被加载完成,当前Function会阻塞等待,直到数据存储完成后再执行Load并进行后续操作

@aceld
Copy link
Owner

aceld commented Jul 4, 2024

是的,目前如果需要两个流的同步,只能有以下几种办法:
1、通过存储媒介做同步机制,比如阻塞、轮询等
2、直接抛弃当前包,交给上层消息队列MQ,如kafka等做重新消费处理

KisFlow目前还没有实现窗口windows的配置机制,实际上如果实现该机制,底层逻辑也是阻塞,或者交给delay 集合,稍后再重试等。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants