Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asynchronously run a pipeline multiple times. #61

Open
Shawn-Guo-CN opened this issue Oct 2, 2019 · 1 comment
Open

Asynchronously run a pipeline multiple times. #61

Shawn-Guo-CN opened this issue Oct 2, 2019 · 1 comment

Comments

@Shawn-Guo-CN
Copy link
Contributor

Shawn-Guo-CN commented Oct 2, 2019

Under the current version (0.5.2), users could run a same pipeline with same parameters multiple times. This would raise a potential problem, i.e. I/O conflicts of same tmp folders. Further, users wouldn't realise this problem, as the error wound only appear in stdout and never appears in our main window.

@Shawn-Guo-CN Shawn-Guo-CN changed the title Asynchronously run a pipeline multi-ple times. Asynchronously run a pipeline multiple times. Oct 2, 2019
@rogerhurwitz
Copy link
Contributor

rogerhurwitz commented Oct 2, 2019

I agree that this is a concern. @colehurwitz is also thinking about this.

It seems to me that on a per element basis there are three possibilities:

  1. Fully Concurrent: The element fully supports multi-instance concurrent execution. A bandpass filtering preprocessor, for example, should be able to run in multiple concurrent pipelines without any risk of failure.
  2. Parameter Concurrent: The element supports multi-instance concurrency, but requires a per-instance change to its parameters. A sorter, for example, that has a save path may be able to operate concurrently with another instance of the same sorter if and only if the save path is set per instance.
  3. Not Concurrent: The element has no support for multi-instance concurrency.

I'm tempted to solve this by adding a method to SpikeElement that would allow the app to query an element instance to determine its concurrency requirements, but I'm open to other suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants