-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* feat: submit job on tapis * add: tapis job submission handling (#26) * fix: issue with response * add: suscribe api * fix: using tapis unpublished package * fix: add fixtures webhook * fix: add fixtures * fix: add endpoint * update: execution status from webhook * fix: handle error * format and ununsed * update: import * fix: formatting * 24 tapis register outputs (#27) * fix: webpack nodaemon * fix: get output list * fix: support tapis execution * feat: download outputs into the ensemble (#28) * fix: add new paths * fix: refactor * fix: download files * fix: improve function name * Support tapis - Handle failed execution (#30) * fix: handle error if tapis auth fails * fix: migrate to mfosorio package * format * fix: docs
- Loading branch information
Showing
65 changed files
with
18,950 additions
and
103 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| { | ||
| "presets": ["@babel/preset-env", "@babel/preset-flow", "@babel/preset-typescript"] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| module.exports = { | ||
| "env": { | ||
| "browser": true, | ||
| "es2021": true | ||
| }, | ||
| "extends": [ | ||
| "eslint:recommended", | ||
| "plugin:@typescript-eslint/recommended" | ||
| ], | ||
| "overrides": [ | ||
| { | ||
| "env": { | ||
| "node": true | ||
| }, | ||
| "files": [ | ||
| ".eslintrc.{js,cjs}" | ||
| ], | ||
| "parserOptions": { | ||
| "sourceType": "script" | ||
| } | ||
| } | ||
| ], | ||
| "parser": "@typescript-eslint/parser", | ||
| "parserOptions": { | ||
| "ecmaVersion": "latest", | ||
| "sourceType": "module" | ||
| }, | ||
| "plugins": [ | ||
| "@typescript-eslint" | ||
| ], | ||
| "rules": { | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| ## Submission of Executions from UI to Ensemble Execution Queue | ||
|
|
||
| ```mermaid | ||
| sequenceDiagram | ||
| title Submission of Executions from UI to Ensemble Execution Queue | ||
| actor User as User | ||
| User->>+UI : Select models [:model1, :model2, ...] | ||
| UI->>+Task Catalog: Create Task ([:model1, :model2, ...]) | ||
| Task Catalog->>-UI: Return Task ID | ||
| User->>+UI: Select data and parameters | ||
| UI->>+Task Catalog: Update Task ([:model1, :model2, ...]) with data and parameters | ||
| Task Catalog->>-UI: Return Task ID | ||
| User->>+UI: Submit run ([:model1, :model2, ...] and :taskid) | ||
| UI->>+Ensemble Manager: Run models ([:model1, :model2, ...]) and task (:taskid) on ExecutionEngine X (TAPIS) | ||
| Ensemble Manager->>+Task Catalog: Get Task | ||
| Task Catalog->>-Ensemble Manager: Return Task | ||
| loop For each model in the task | ||
| Ensemble Manager ->>+Task Catalog: Get parameters and input selections | ||
| Task Catalog->>-Ensemble Manager: Return selections | ||
| Ensemble Manager ->>Ensemble Manager: Create executions using the selections (cross product) | ||
| loop Add each execution to the queue | ||
| Ensemble Manager ->>Ensemble Manager: Create Job Request | ||
| Ensemble Manager ->>+Tapis: Add execution on Tapis | ||
| Tapis ->>-Ensemble Manager: Added | ||
| end | ||
| end | ||
| Ensemble Manager->>-UI: Submitted | ||
| UI->>-User: Response status of the submission | ||
| ``` | ||
|
|
||
| ## Tracking the Progress of the Execution as Emsemble Manager | ||
|
|
||
| ```mermaid | ||
| sequenceDiagram | ||
| title Tracking the Progress of the Execution | ||
| Tapis ->>Ensemble Manager: Event: Execution Status Change | ||
| note over Ensemble Manager, Tapis: Tapis sends a notification to <br> Ensemble Manager when the <br> status of the execution changes | ||
| alt Execution is completed | ||
| Ensemble Manager ->>+Tapis: Get Job Output List | ||
| Tapis ->>-Ensemble Manager: Return Job Output List | ||
| loop For each output in the job | ||
| Ensemble Manager ->>+Tapis: Download Job Output | ||
| Tapis ->>-Ensemble Manager: Return Job Output | ||
| Ensemble Manager ->>Ensemble Manager: Modify URL to point to the output file | ||
| end | ||
| end | ||
| Ensemble Manager ->>+Task Catalog: Update Execution Status | ||
| Task Catalog ->>-Ensemble Manager: Return Execution Status | ||
| ``` | ||
|
|
||
| ## Tracking the Progress of the Execution as User | ||
|
|
||
| ```mermaid | ||
| sequenceDiagram | ||
| title Tracking the Progress of the Execution | ||
| User ->>+UI: Get Executions Status (taskid) | ||
| UI ->>+Task Catalog: Get Execution Status | ||
| Task Catalog ->>-UI: Return Execution Status | ||
| UI ->>-User: Return Execution Status | ||
| ``` | ||
|
|
||
| ``` | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,201 @@ | ||
| /** | ||
| * For a detailed explanation regarding each configuration property, visit: | ||
| * https://jestjs.io/docs/configuration | ||
| */ | ||
|
|
||
| /** @type {import('jest').Config} */ | ||
| const config = { | ||
| // All imported modules in your tests should be mocked automatically | ||
| // automock: false, | ||
|
|
||
| // Stop running tests after `n` failures | ||
| // bail: 0, | ||
|
|
||
| // The directory where Jest should store its cached dependency information | ||
| // cacheDirectory: "/private/var/folders/qn/xpsy3ssx5hbbb_ndr2sbt5w80000gn/T/jest_dx", | ||
|
|
||
| // Automatically clear mock calls, instances, contexts and results before every test | ||
| // clearMocks: false, | ||
|
|
||
| // Indicates whether the coverage information should be collected while executing the test | ||
| // collectCoverage: false, | ||
|
|
||
| // An array of glob patterns indicating a set of files for which coverage information should be collected | ||
| // collectCoverageFrom: undefined, | ||
|
|
||
| // The directory where Jest should output its coverage files | ||
| // coverageDirectory: undefined, | ||
|
|
||
| // An array of regexp pattern strings used to skip coverage collection | ||
| // coveragePathIgnorePatterns: [ | ||
| // "/node_modules/" | ||
| // ], | ||
|
|
||
| // Indicates which provider should be used to instrument code for coverage | ||
| coverageProvider: "v8", | ||
|
|
||
| // A list of reporter names that Jest uses when writing coverage reports | ||
| // coverageReporters: [ | ||
| // "json", | ||
| // "text", | ||
| // "lcov", | ||
| // "clover" | ||
| // ], | ||
|
|
||
| // An object that configures minimum threshold enforcement for coverage results | ||
| // coverageThreshold: undefined, | ||
|
|
||
| // A path to a custom dependency extractor | ||
| // dependencyExtractor: undefined, | ||
|
|
||
| // Make calling deprecated APIs throw helpful error messages | ||
| // errorOnDeprecated: false, | ||
|
|
||
| // The default configuration for fake timers | ||
| // fakeTimers: { | ||
| // "enableGlobally": false | ||
| // }, | ||
|
|
||
| // Force coverage collection from ignored files using an array of glob patterns | ||
| // forceCoverageMatch: [], | ||
|
|
||
| // A path to a module which exports an async function that is triggered once before all test suites | ||
| // globalSetup: undefined, | ||
|
|
||
| // A path to a module which exports an async function that is triggered once after all test suites | ||
| // globalTeardown: undefined, | ||
|
|
||
| // A set of global variables that need to be available in all test environments | ||
| // globals: {}, | ||
|
|
||
| // The maximum amount of workers used to run your tests. Can be specified as % or a number. E.g. maxWorkers: 10% will use 10% of your CPU amount + 1 as the maximum worker number. maxWorkers: 2 will use a maximum of 2 workers. | ||
| // maxWorkers: "50%", | ||
|
|
||
| // An array of directory names to be searched recursively up from the requiring module's location | ||
| // moduleDirectories: [ | ||
| // "node_modules" | ||
| // ], | ||
|
|
||
| // An array of file extensions your modules use | ||
| // moduleFileExtensions: [ | ||
| // "js", | ||
| // "mjs", | ||
| // "cjs", | ||
| // "jsx", | ||
| // "ts", | ||
| // "tsx", | ||
| // "json", | ||
| // "node" | ||
| // ], | ||
|
|
||
| // A map from regular expressions to module names or to arrays of module names that allow to stub out resources with a single module | ||
| // moduleNameMapper: {}, | ||
|
|
||
| // An array of regexp pattern strings, matched against all module paths before considered 'visible' to the module loader | ||
| // modulePathIgnorePatterns: [], | ||
|
|
||
| // Activates notifications for test results | ||
| // notify: false, | ||
|
|
||
| // An enum that specifies notification mode. Requires { notify: true } | ||
| // notifyMode: "failure-change", | ||
|
|
||
| // A preset that is used as a base for Jest's configuration | ||
| // preset: undefined, | ||
|
|
||
| // Run tests from one or more projects | ||
| // projects: undefined, | ||
|
|
||
| // Use this configuration option to add custom reporters to Jest | ||
| // reporters: undefined, | ||
|
|
||
| // Automatically reset mock state before every test | ||
| // resetMocks: false, | ||
|
|
||
| // Reset the module registry before running each individual test | ||
| // resetModules: false, | ||
|
|
||
| // A path to a custom resolver | ||
| // resolver: undefined, | ||
|
|
||
| // Automatically restore mock state and implementation before every test | ||
| // restoreMocks: false, | ||
|
|
||
| // The root directory that Jest should scan for tests and modules within | ||
| // rootDir: undefined, | ||
|
|
||
| // A list of paths to directories that Jest should use to search for files in | ||
| // roots: [ | ||
| // "<rootDir>" | ||
| // ], | ||
|
|
||
| // Allows you to use a custom runner instead of Jest's default test runner | ||
| // runner: "jest-runner", | ||
|
|
||
| // The paths to modules that run some code to configure or set up the testing environment before each test | ||
| // setupFiles: [], | ||
|
|
||
| // A list of paths to modules that run some code to configure or set up the testing framework before each test | ||
| // setupFilesAfterEnv: [], | ||
|
|
||
| // The number of seconds after which a test is considered as slow and reported as such in the results. | ||
| // slowTestThreshold: 5, | ||
|
|
||
| // A list of paths to snapshot serializer modules Jest should use for snapshot testing | ||
| // snapshotSerializers: [], | ||
|
|
||
| // The test environment that will be used for testing | ||
| // testEnvironment: "jest-environment-node", | ||
|
|
||
| // Options that will be passed to the testEnvironment | ||
| // testEnvironmentOptions: {}, | ||
|
|
||
| // Adds a location field to test results | ||
| // testLocationInResults: false, | ||
|
|
||
| // The glob patterns Jest uses to detect test files | ||
| // testMatch: [ | ||
| // "**/__tests__/**/*.[jt]s?(x)", | ||
| // "**/?(*.)+(spec|test).[tj]s?(x)" | ||
| // ], | ||
|
|
||
| // An array of regexp pattern strings that are matched against all test paths, matched tests are skipped | ||
| // testPathIgnorePatterns: [ | ||
| // "/node_modules/" | ||
| // ], | ||
|
|
||
| // The regexp pattern or array of patterns that Jest uses to detect test files | ||
| // testRegex: [], | ||
|
|
||
| // This option allows the use of a custom results processor | ||
| // testResultsProcessor: undefined, | ||
|
|
||
| // This option allows use of a custom test runner | ||
| // testRunner: "jest-circus/runner", | ||
|
|
||
| // A map from regular expressions to paths to transformers | ||
| transform: { | ||
| "\\.(gql|graphql)$": "@graphql-tools/jest-transform", | ||
| ".*": "babel-jest" | ||
| } | ||
|
|
||
| // An array of regexp pattern strings that are matched against all source file paths, matched files will skip transformation | ||
| // transformIgnorePatterns: [ | ||
| // "/node_modules/", | ||
| // "\\.pnp\\.[^\\/]+$" | ||
| // ], | ||
|
|
||
| // An array of regexp pattern strings that are matched against all modules before the module loader will automatically return a mock for them | ||
| // unmockedModulePathPatterns: undefined, | ||
|
|
||
| // Indicates whether each individual test should be reported during the run | ||
| // verbose: undefined, | ||
|
|
||
| // An array of regexp patterns that are matched against all source file paths before re-running tests in watch mode | ||
| // watchPathIgnorePatterns: [], | ||
|
|
||
| // Whether to use watchman for file crawling | ||
| // watchman: true, | ||
| }; | ||
|
|
||
| module.exports = config; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.