Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add new dataset v4 controller #1541

Open
wants to merge 47 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
e28d7b1
start with adding the v4 datasets controller
martin-trajanovski Nov 29, 2024
533789b
start the main cleanup on the dataset v4 controller
martin-trajanovski Dec 1, 2024
cdd9afa
cleanup and fix some linting errors
martin-trajanovski Dec 2, 2024
8089998
Merge branch 'migrate-principal-investigator' of https://github.com/S…
martin-trajanovski Dec 3, 2024
48118d2
use the versioning of datasets controller properly and finalize the i…
martin-trajanovski Dec 3, 2024
ab7136b
do some more cleanup in the new controller
martin-trajanovski Dec 3, 2024
6638d0f
use improved regex to extract the version from the URI
martin-trajanovski Dec 4, 2024
1a9575a
Merge branch 'master' of https://github.com/SciCatProject/scicat-back…
martin-trajanovski Dec 4, 2024
3054920
refactor how we use config module and make accessGroups easily access…
martin-trajanovski Dec 5, 2024
2056bd2
fix failing unit tests
martin-trajanovski Dec 5, 2024
7cbe9ea
improve dataset dto types for sdk generation
martin-trajanovski Dec 5, 2024
5b2c33e
add aggregation for findById and findOne on the datasets.v4 controller
martin-trajanovski Dec 6, 2024
5e6b6cd
add some notes and todos on whats left
martin-trajanovski Dec 6, 2024
1617446
do some more cleanup and improve include filters for aggregation
martin-trajanovski Dec 9, 2024
b528017
more cleanup and filter refactor and validation
martin-trajanovski Dec 10, 2024
8b7c4f2
resolve merge conflicts
martin-trajanovski Dec 10, 2024
500ed77
resolve merge conflicts
martin-trajanovski Dec 11, 2024
d6aaa63
try to revert some package upgrades
martin-trajanovski Dec 11, 2024
5f9b97e
Merge branch 'master' into SWAP-4319-add-new-dataset-v4-minimal-contr…
martin-trajanovski Dec 11, 2024
2f3d89a
try to revert some more package changes
martin-trajanovski Dec 11, 2024
8bb6982
test the new content property
martin-trajanovski Dec 12, 2024
8d9531d
content property final improvements
martin-trajanovski Dec 12, 2024
96ce2ea
use content variable instead of funciton
martin-trajanovski Dec 12, 2024
05eab47
cleanup
martin-trajanovski Dec 12, 2024
589bf1e
finalize all filters for find dataset endpoints
martin-trajanovski Dec 12, 2024
3b0ef0b
fix most of the PR review comments
martin-trajanovski Dec 13, 2024
8432254
fix v3 dataset controller access filters
martin-trajanovski Dec 13, 2024
e98d0b4
some small improvements and leads how to solve the nested relational …
martin-trajanovski Dec 13, 2024
848633f
try to revert some chages that are making tests fail
martin-trajanovski Dec 13, 2024
e6441a9
fix api tests as well
martin-trajanovski Dec 13, 2024
035ac08
fix: update GitHub Actions workflow to conditionally upload SDK artif…
Junjiequan Dec 16, 2024
f0a7830
add granular access checks for relational fields
martin-trajanovski Dec 16, 2024
9e4aade
Merge branch 'SWAP-4319-add-new-dataset-v4-minimal-controller' of htt…
martin-trajanovski Dec 16, 2024
8bad5cb
add new environment variable variable description in github actions
martin-trajanovski Dec 16, 2024
caf3d1a
feat: add public endpoints for better separation of concerns
martin-trajanovski Dec 16, 2024
768c6bd
add sort and skip in the find one complete
martin-trajanovski Dec 17, 2024
9e66825
resolve conflicts
martin-trajanovski Dec 17, 2024
a83609d
resolve conflicts
martin-trajanovski Dec 17, 2024
97683dd
move the public dataset v4 endpoints in own controller
martin-trajanovski Dec 17, 2024
c0b6f0f
review feedback improvements
martin-trajanovski Dec 17, 2024
e304181
pull latest parent branch
martin-trajanovski Dec 17, 2024
d5ccb0a
fix some review comments and add fullfacet to public endpoints as well
martin-trajanovski Dec 19, 2024
0b85180
fix default sort value
martin-trajanovski Dec 19, 2024
08f344e
Merge pull request #1576 from SciCatProject/SWAP-4365-add-public-data…
martin-trajanovski Dec 20, 2024
9322531
feat: add api tests for the new controllers (#1580)
martin-trajanovski Jan 14, 2025
99ecdef
resolve merge conflicts
martin-trajanovski Jan 14, 2025
f1dc1a7
revert some changes in package-lock
martin-trajanovski Jan 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/release-and-publish-sdk.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,9 @@ jobs:
env:
MONGODB_URI: "mongodb://localhost:27017/scicat"
JWT_SECRET: thisIsTheJwtSecret
# It disables the content property on some of the endpoints as the sdk generator complains about it.
# We want to keep it when the app is running to improve the swagger documentation and usage.
SDK_PACKAGE_SWAGGER_HELPERS_DISABLED: true
nitrosx marked this conversation as resolved.
Show resolved Hide resolved
run: |
npm install -g wait-on && npm install
npm run start & wait-on http://localhost:3000/api/v3/health --timeout 200000
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/upload-sdk-artifact.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ jobs:
env:
MONGODB_URI: "mongodb://localhost:27017/scicat"
JWT_SECRET: thisIsTheJwtSecret
# It disables the content property on some of the endpoints as the sdk generator complains about it.
# We want to keep it when the app is running to improve the swagger documentation and usage.
SDK_PACKAGE_SWAGGER_HELPERS_DISABLED: true
nitrosx marked this conversation as resolved.
Show resolved Hide resolved
run: |
npm install -g wait-on && npm install
npm run start & wait-on http://localhost:3000/api/v3/health --timeout 200000
Expand Down
31 changes: 0 additions & 31 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,6 @@
"chai-http": "^5.1.1",
"concurrently": "^9.0.0",
"eslint": "^9.0.0",
"eslint-config-loopback": "^13.1.0",
"eslint-config-prettier": "^9.0.0",
"eslint-plugin-prettier": "^5.0.0",
"globals": "^15.12.0",
Expand Down
2 changes: 0 additions & 2 deletions src/admin/admin.module.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
import { Module } from "@nestjs/common";
import { AdminService } from "./admin.service";
import { AdminController } from "./admin.controller";
import { ConfigModule } from "@nestjs/config";

@Module({
controllers: [AdminController],
imports: [ConfigModule],
providers: [AdminService],
exports: [AdminService],
})
Expand Down
12 changes: 6 additions & 6 deletions src/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,15 @@ import { LoggerModule } from "./loggers/logger.module";

@Module({
imports: [
AttachmentsModule,
AuthModule,
CaslModule,
CommonModule,
ConfigModule.forRoot({
load: [configuration],
isGlobal: true,
cache: true,
nitrosx marked this conversation as resolved.
Show resolved Hide resolved
}),
AuthModule,
CaslModule,
AttachmentsModule,
CommonModule,
LoggerModule,
DatablocksModule,
DatasetsModule,
Expand All @@ -51,7 +53,6 @@ import { LoggerModule } from "./loggers/logger.module";
LogbooksModule,
EventEmitterModule.forRoot(),
MailerModule.forRootAsync({
imports: [ConfigModule],
useFactory: async (configService: ConfigService) => {
const port = configService.get<string>("smtp.port");
return {
Expand Down Expand Up @@ -80,7 +81,6 @@ import { LoggerModule } from "./loggers/logger.module";
inject: [ConfigService],
}),
MongooseModule.forRootAsync({
imports: [ConfigModule],
useFactory: async (configService: ConfigService) => ({
uri: configService.get<string>("mongodbUri"),
}),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,10 @@ import { AccessGroupFromPayloadService } from "./access-group-from-payload.servi
import { HttpService } from "@nestjs/axios";
import { AccessGroupFromMultipleProvidersService } from "./access-group-from-multiple-providers.service";
import { Logger } from "@nestjs/common";
import { ConfigModule } from "@nestjs/config";
/*
* this is the default function which provides an empty array as groups
*/
export const accessGroupServiceFactory = {
imports: [ConfigModule],
provide: AccessGroupService,
useFactory: (configService: ConfigService) => {
Logger.debug("Service factory starting", "accessGroupServiceFactory");
Expand Down
4 changes: 1 addition & 3 deletions src/auth/auth.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { AuthController } from "./auth.controller";
import { JwtModule } from "@nestjs/jwt";
import { JwtStrategy } from "./strategies/jwt.strategy";
import { LdapStrategy } from "./strategies/ldap.strategy";
import { ConfigModule, ConfigService } from "@nestjs/config";
import { ConfigService } from "@nestjs/config";
nitrosx marked this conversation as resolved.
Show resolved Hide resolved
import { UsersService } from "src/users/users.service";
import { OidcConfig } from "src/config/configuration";
import { BuildOpenIdClient, OidcStrategy } from "./strategies/oidc.strategy";
Expand Down Expand Up @@ -42,9 +42,7 @@ const OidcStrategyFactory = {

@Module({
imports: [
ConfigModule,
JwtModule.registerAsync({
imports: [ConfigModule],
useFactory: async (configService: ConfigService) => ({
secret: configService.get<string>("jwt.secret"),
signOptions: { expiresIn: configService.get<number>("jwt.expiresIn") },
Expand Down
3 changes: 2 additions & 1 deletion src/casl/casl-ability.factory.spec.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { ConfigService } from "@nestjs/config";
import { CaslAbilityFactory } from "./casl-ability.factory";

describe("CaslAbilityFactory", () => {
it("should be defined", () => {
expect(new CaslAbilityFactory()).toBeDefined();
expect(new CaslAbilityFactory(new ConfigService())).toBeDefined();
});
});
Loading
Loading