wait for s3 file to appear

[mlybarger]

hi, i have a longer running process that we're using citrus to test. the process is kicked off from an airflow job that we invoke from citrus. the job will launch some processing ec2's, then it will launch an EMR to convert the data to parquet, and load to our data lake.

when the emr is complete, it puts a done file in a known spot in an s3 bucket. current, we just have a <sleep seconds="1800" to give enough time for the process to finish.

what's a better way to say "wait for the done file to show up, or timeout after 30 minutes."

we're using xml dsl on an older version of citrus 2.7.9, , but are looking to update citrus soon.

[bbortt]

there are retry actions available, did you take a look at these?

https://citrusframework.org/citrus/reference/4.2.1/html/index.html#containers-repeat

[christophd]

there is also a repeat-on-error container that performs a nested action with retry mechanism. you can define the maximum number of retries and a delay between attempts.

https://citrusframework.org/citrus/reference/4.2.1/html/index.html#containers-repeat-on-error

[christophd]

BTW, looking forward to seeing you update to later Citrus versions. Please, let us know how it goes 🤞

[bbortt]

@mlybarger any news on this? it's been a while 😉