Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: prep for unified dataset schema #1414

Merged
merged 33 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
1798c0c
refactor naming of current dataset schema and dto. Added new unified …
nitrosx Sep 4, 2024
6ff6c6a
solved weird dependency in jobs
nitrosx Sep 5, 2024
74b6edd
still refactoring names, started working on tests
nitrosx Sep 6, 2024
cffb060
fix types on obsolete schemas
nitrosx Sep 11, 2024
fc81272
updated to use latest dataset schema in service
nitrosx Sep 13, 2024
41d304c
refactoring data types
nitrosx Sep 13, 2024
0c3b316
First two dataset test passing
nitrosx Sep 16, 2024
bc4532c
fixed tests in Datasets Simple
nitrosx Sep 17, 2024
5f84530
making progress with tests
nitrosx Sep 17, 2024
6bdbb43
refactor naming of current dataset schema and dto. Added new unified …
nitrosx Sep 4, 2024
2fdc8bf
solved weird dependency in jobs
nitrosx Sep 5, 2024
ac468a1
still refactoring names, started working on tests
nitrosx Sep 6, 2024
c6d59fb
fix types on obsolete schemas
nitrosx Sep 11, 2024
74a6c26
updated to use latest dataset schema in service
nitrosx Sep 13, 2024
15b2740
refactoring data types
nitrosx Sep 13, 2024
2244c18
First two dataset test passing
nitrosx Sep 16, 2024
ebc2666
fixed tests in Datasets Simple
nitrosx Sep 17, 2024
b9cf5b3
making progress with tests
nitrosx Sep 17, 2024
79b1ac6
all test passing
nitrosx Sep 18, 2024
5da5544
latest packages.json local version
nitrosx Sep 18, 2024
81e24e8
resolved conflicts
nitrosx Sep 18, 2024
cc77985
Renamed current dataset dtos to osbolete
nitrosx Sep 18, 2024
22abb54
fixed test dataset used in elasticsearch tests
nitrosx Sep 18, 2024
dc3ff8d
Updated datasets dto according to PR review
nitrosx Sep 19, 2024
e246213
Merge branch 'master' into swap-4073-single-dataset-schema
nitrosx Sep 19, 2024
2778acf
fix elastic search test fail
Junjiequan Sep 19, 2024
ef48549
added instrumentIds to ES datasetFieldMapping
Junjiequan Sep 19, 2024
ed115b3
Merge branch 'master' into swap-4073-single-dataset-schema
nitrosx Sep 19, 2024
4ef5b9f
implemented requested changes and fixed linting
nitrosx Sep 19, 2024
175be49
Merge branch 'master' into swap-4073-single-dataset-schema
nitrosx Sep 19, 2024
3472130
Merge branch 'master' into swap-4073-single-dataset-schema
nitrosx Sep 19, 2024
a3b02f9
added database migration
nitrosx Sep 20, 2024
c5e1040
Fixed mongo migrate script for dataset unified schema
nitrosx Sep 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions migrations/20240920111733-dataset-unified-schema.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
module.exports = {
async up(db, client) {
// TODO write your migration here.
// See https://github.com/seppevs/migrate-mongo/#creating-a-new-migration-script
// Example:
// await db.collection('albums').updateOne({artist: 'The Beatles'}, {$set: {blacklisted: true}});
await db.collection("Dataset").updateMany({}, [
{
$set: {
proposalIds: ["$proposalId"],
instrumentIds: ["$instrumentId"],
sampleIds: ["$sampleId"],
},
},
]);
await db.collection("Dataset").updateMany({ type: "derived" }, [
{
$set: {
principalInvestigator: "$investigator",
},
},
]);
},

async down(db, client) {
// TODO write the statements to rollback your migration (if possible)
// Example:
// await db.collection('albums').updateOne({artist: 'The Beatles'}, {$set: {blacklisted: false}});

await db.collection("Dataset").updateMany({}, [
{
$set: {
proposalId: "$proposalIds[0]",
instrumentId: "$instrumentId[0]",
sampleId: "$sampleId[0]",
},
},
]);
await db.collection("Dataset").updateMany({ type: "derived" }, [
{
$set: {
investigator: "$principalInvestigator",
},
},
]);
},
};
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
"test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
"test:api": "npm run test:api:jest --maxWorkers=50% && concurrently -k -s first \"wait-on http://localhost:3000/explorer/ && npm run test:api:mocha\" \"npm run start\"",
"test:api:jest": "jest --config ./test/config/jest-e2e.json --maxWorkers=50%",
"test:api:mocha": "mocha --config ./test/config/.mocharc.json -r chai/register-should.js ",
"test:api:mocha": "mocha --config ./test/config/.mocharc.json -r chai/register-should.js",
"prepare:local": "docker-compose -f CI/E2E/docker-compose-local.yaml --env-file CI/E2E/.env.elastic-search up -d && cp functionalAccounts.json.test functionalAccounts.json"
},
"dependencies": {
Expand Down
Loading
Loading