Skip to content

Commit

Permalink
Remove BulkDownloader operator (#3161)
Browse files Browse the repository at this point in the history
This operator's design requires some more discussion. The current issues
are:
1. it is accessing the workflow context (i.e., workflowId and
executionId), which may not be permitted in the future.
2. It is directly interacting with the local file system.

For the short term, we will drop the support for such as downloader. We
can discuss how to support it in the future when needed.
  • Loading branch information
Yicong-Huang authored Dec 16, 2024
1 parent 8b92548 commit edaf799
Show file tree
Hide file tree
Showing 5 changed files with 0 additions and 213 deletions.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ import edu.uci.ics.amber.operator.cartesianProduct.CartesianProductOpDesc
import edu.uci.ics.amber.operator.dictionary.DictionaryMatcherOpDesc
import edu.uci.ics.amber.operator.difference.DifferenceOpDesc
import edu.uci.ics.amber.operator.distinct.DistinctOpDesc
import edu.uci.ics.amber.operator.download.BulkDownloaderOpDesc
import edu.uci.ics.amber.operator.dummy.DummyOpDesc
import edu.uci.ics.amber.operator.filter.SpecializedFilterOpDesc
import edu.uci.ics.amber.operator.hashJoin.HashJoinOpDesc
Expand Down Expand Up @@ -202,7 +201,6 @@ trait StateTransferFunc
new Type(value = classOf[RedditSearchSourceOpDesc], name = "RedditSearch"),
new Type(value = classOf[PythonLambdaFunctionOpDesc], name = "PythonLambdaFunction"),
new Type(value = classOf[PythonTableReducerOpDesc], name = "PythonTableReducer"),
new Type(value = classOf[BulkDownloaderOpDesc], name = "BulkDownloader"),
new Type(value = classOf[URLFetcherOpDesc], name = "URLFetcher"),
new Type(value = classOf[CartesianProductOpDesc], name = "CartesianProduct"),
new Type(value = classOf[FilledAreaPlotOpDesc], name = "FilledAreaPlot"),
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

0 comments on commit edaf799

Please sign in to comment.