Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] Aihub / JobInfo api (Notion) 구현 완료 #91

Merged
merged 24 commits into from
Sep 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e154174
feat: implement getAiHubModels api (need notion.yml)
shj1081 Aug 7, 2024
8d22f06
feat: change the field of model notion db
shj1081 Aug 9, 2024
ff1dd1b
chore: refine service code
shj1081 Aug 9, 2024
007f36f
feat: request and response with integer year (not string)
shj1081 Aug 10, 2024
78519c9
feat: implment getAiHubDatasets API and modify service code to reduce…
shj1081 Aug 10, 2024
9fe364f
chore: refine comments
shj1081 Aug 10, 2024
cdc4f50
refactor: relocate dto
shj1081 Aug 10, 2024
02edba8
feat: RestDocs, controller test code
shj1081 Aug 11, 2024
332034b
feat: implement JobInfo API
shj1081 Aug 11, 2024
0411bed
feat: implement JobInfoControllerTest code, RestDocs, change api URI
shj1081 Aug 12, 2024
a1af480
Merge pull request #50 from SystemConsultantGroup/feat/48-notion-api/1
shj1081 Aug 14, 2024
f63f998
Merge pull request #61 from SystemConsultantGroup/feat/48-notion-api/2
shj1081 Aug 23, 2024
8a62976
chore: change for handling merge conflict
shj1081 Aug 30, 2024
4be5630
chore: change for handling merge conflict (merge phase)
shj1081 Aug 30, 2024
a903b03
chore : modify CI.yml for notion config
shj1081 Aug 30, 2024
ee7f4ae
chore: add notion config in application.yml
shj1081 Aug 30, 2024
86e2f30
fix: fix the cascade option
shj1081 Sep 3, 2024
b7770a6
Merge pull request #86 from SystemConsultantGroup/feat/48-notion-api/3
shj1081 Sep 3, 2024
5a456c6
chore: add html files for restDocs
shj1081 Sep 8, 2024
842ce5b
Merge pull request #95 from SystemConsultantGroup/feat/48-notion-api/4
shj1081 Sep 12, 2024
6449912
chore: add html files for restDocs (ommited ones)
shj1081 Sep 12, 2024
32aa81a
chore: handle merge conflict
shj1081 Sep 12, 2024
ecb4ea8
chore: handle conflict for new develop commit
shj1081 Sep 15, 2024
609de19
[CHORE] add omitted html files for restDocs / handle merge conflict
shj1081 Sep 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:
run: |
echo "${{ secrets.OAUTH_YML }}" | base64 --decode > src/main/resources/application-oauth.yml
echo "${{ secrets.MINIO_YML }}" | base64 --decode > src/main/resources/application-minio.yml
echo "${{ secrets.NOTION_YML }}" | base64 --decode > src/main/resources/application-notion.yml
find src

- name: gradlew 권한 부여
Expand Down
16 changes: 16 additions & 0 deletions src/docs/asciidoc/aihub-controller-test.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
== AI HUB API
:source-highlighter: highlightjs

---

=== AI HUB 모델 리스트 조회 (POST /aihub/models)

====
operation::ai-hub-controller-test/get-ai-hub-models[snippets="query-parameters,request-fields,http-request,http-response,response-fields"]
====

=== AI HUB 데이터셋 리스트 조회 (POST /aihub/datasets)

====
operation::ai-hub-controller-test/get-ai-hub-datasets[snippets="query-parameters,request-fields,http-request,http-response,response-fields"]
====
4 changes: 3 additions & 1 deletion src/docs/asciidoc/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,6 @@ include::eventNotice.adoc[]
include::eventPeriod.adoc[]
include::gallery.adoc[]
include::application.adoc[]
include::project.adoc[]
include::project.adoc[]
include::aihub-controller-test.adoc[]
include::jobInfo-controller-test.adoc[]
10 changes: 10 additions & 0 deletions src/docs/asciidoc/jobInfo-controller-test.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
== JOB INFO API
:source-highlighter: highlightjs

---

=== JOB INFO 리스트 조회 (POST /jobInfos)

====
operation::job-info-controller-test/get-job-infos[snippets="query-parameters,request-fields,http-request,http-response,response-fields"]
====
43 changes: 43 additions & 0 deletions src/main/java/com/scg/stop/aihub/controller/AiHubController.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package com.scg.stop.aihub.controller;

import com.scg.stop.aihub.dto.request.AiHubDatasetRequest;
import com.scg.stop.aihub.dto.request.AiHubModelRequest;
import com.scg.stop.aihub.dto.response.AiHubDatasetResponse;
import com.scg.stop.aihub.dto.response.AiHubModelResponse;
import com.scg.stop.aihub.service.AiHubService;
import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.web.PageableDefault;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/aihub")
@RequiredArgsConstructor
public class AiHubController {

private final AiHubService aiHubService;

// Get the paginated list of models
@PostMapping("/models")
public ResponseEntity<Page<AiHubModelResponse>> getAiHubModels(
@RequestBody AiHubModelRequest request,
@PageableDefault(page = 0, size = 10) Pageable pageable) {
Page<AiHubModelResponse> models = aiHubService.getAiHubModels(request, pageable);
return ResponseEntity.status(HttpStatus.OK).body(models);
}

// Get the paginated list of datasets
@PostMapping("/datasets")
public ResponseEntity<Page<AiHubDatasetResponse>> getAiHubDatasets(
@RequestBody AiHubDatasetRequest request,
@PageableDefault(page = 0, size = 10) Pageable pageable) {
Page<AiHubDatasetResponse> datasets = aiHubService.getAiHubDatasets(request, pageable);
return ResponseEntity.status(HttpStatus.OK).body(datasets);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package com.scg.stop.aihub.dto.request;

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;

import java.util.List;

@Getter
@AllArgsConstructor
@NoArgsConstructor
public class AiHubDatasetRequest {

private String title; // 제목 : text
private List<String> dataTypes; // 데이터 유형 : multiselect
private List<String> topics; // 주제 분류 : multiselect
private List<Integer> developmentYears; // 구축 년도 : multiselect
private String professor; // 담당 교수 : text
private List<String> participants; // 참여 학생 : text (comma separated)
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package com.scg.stop.aihub.dto.request;

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;

import java.util.List;

@Getter
@AllArgsConstructor
@NoArgsConstructor
public class AiHubModelRequest {

private String title; // 제목 : text
private List<String> learningModels; // 학습 모델 : multiselect
private List<String> topics; // 주제 분류 : multiselect
private List<Integer> developmentYears; // 개발 년도 : multiselect
private String professor; // 담당 교수 : text
private List<String> participants; // 참여 학생 : text (comma separated)
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
package com.scg.stop.aihub.dto.response;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.JsonNode;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class AiHubDatasetResponse {
private String object;
private String id; // 페이지 id
private String cover; // 커버 이미지
private String title; // 제목
private List<String> dataTypes = new ArrayList<>(); // 데이터 유형
private List<String> topics = new ArrayList<>(); // 주제 분류
private List<Integer> developmentYears = new ArrayList<>(); // 구축 년도
private String professor; // 담당 교수
private List<String> participants = new ArrayList<>(); // 참여 학생
private String url; // redirect url

@JsonProperty("object")
public void setObject(String object) {
this.object = object;
}

@JsonProperty("id")
public void setId(String id) {
this.id = id;
}

@JsonProperty("cover")
public void setCover(JsonNode cover) {
if (cover != null) {
if (cover.has("external")) { // external cover image
this.cover = cover.get("external").get("url").asText();
} else if (cover.has("file")) { // file cover image
this.cover = cover.get("file").get("url").asText();
}
}
}

// Extract properties from the JSON response and set the fields of response dto
@JsonProperty("properties")
public void setProperties(JsonNode properties) {
if (properties != null) {
this.title = extractTextFromProperty(properties, "제목", "title");
this.dataTypes = extractMultiSelect(properties, "데이터 유형");
this.topics = extractMultiSelect(properties, "주제 분류");
this.developmentYears = extractIntegerMultiSelect(properties, "구축 년도");
this.professor = extractTextFromProperty(properties, "담당 교수", "rich_text");
this.participants = extractParticipants(properties);
}
}

@JsonProperty("url")
public void setUrl(String url) {
this.url = url;
}

// Helper methods
// Extract text from the text property
private String extractTextFromProperty(JsonNode properties, String fieldName, String textType) {
if (properties.has(fieldName)) {
JsonNode fieldNode = properties.get(fieldName).get(textType);
if (fieldNode.isArray() && fieldNode.size() > 0) {
return fieldNode.get(0).get("text").get("content").asText();
}
}
return null;
}

// Extract participants from the comma-separated text property
private List<String> extractParticipants(JsonNode properties) {
String participantsString = extractTextFromProperty(properties, "참여 학생", "rich_text");
if (participantsString != null && !participantsString.isEmpty()) {
return Arrays.asList(participantsString.split("\\s*,\\s*"));
}
return new ArrayList<>();
}

// Extract multi-select property
private List<String> extractMultiSelect(JsonNode properties, String fieldName) {
List<String> result = new ArrayList<>();
if (properties.has(fieldName)) {
JsonNode multiSelectNode = properties.get(fieldName).get("multi_select");
if (multiSelectNode.isArray()) {
multiSelectNode.forEach(node -> result.add(node.get("name").asText()));
}
}
return result;
}

// Extract multi-select property as integers
private List<Integer> extractIntegerMultiSelect(JsonNode properties, String fieldName) {
List<Integer> result = new ArrayList<>();
if (properties.has(fieldName)) {
JsonNode multiSelectNode = properties.get(fieldName).get("multi_select");
if (multiSelectNode.isArray()) {
multiSelectNode.forEach(node -> result.add(Integer.parseInt(node.get("name").asText())));
}
}
return result;
}
}
115 changes: 115 additions & 0 deletions src/main/java/com/scg/stop/aihub/dto/response/AiHubModelResponse.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
package com.scg.stop.aihub.dto.response;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.JsonNode;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class AiHubModelResponse {
private String object;
private String id; // 페이지 id
private String cover; // 커버 이미지
private String title; // 제목
private List<String> learningModels = new ArrayList<>(); // 학습 모델
private List<String> topics = new ArrayList<>(); // 주제 분류
private List<Integer> developmentYears = new ArrayList<>(); // 개발 년도 (now List<Integer>)
private String professor; // 담당 교수
private List<String> participants = new ArrayList<>(); // 참여 학생
private String url;

@JsonProperty("object")
public void setObject(String object) {
this.object = object;
}

@JsonProperty("id")
public void setId(String id) {
this.id = id;
}

@JsonProperty("cover")
public void setCover(JsonNode cover) {
if (cover != null) {
if (cover.has("external")) { // external cover image
this.cover = cover.get("external").get("url").asText();
} else if (cover.has("file")) { // file cover image
this.cover = cover.get("file").get("url").asText();
}
}
}

// Extract properties from the JSON response and set the fields of response dto
@JsonProperty("properties")
public void setProperties(JsonNode properties) {
if (properties != null) {
this.title = extractTextFromProperty(properties, "제목", "title");
this.learningModels = extractMultiSelect(properties, "학습 모델");
this.topics = extractMultiSelect(properties, "주제 분류");
this.developmentYears = extractIntegerMultiSelect(properties, "개발 년도");
this.professor = extractTextFromProperty(properties, "담당 교수", "rich_text");
this.participants = extractParticipants(properties);
}
}

@JsonProperty("url")
public void setUrl(String url) {
this.url = url;
}

// Helper methods
// Extract text from the text property
private String extractTextFromProperty(JsonNode properties, String fieldName, String textType) {
if (properties.has(fieldName)) {
JsonNode fieldNode = properties.get(fieldName).get(textType);
if (fieldNode.isArray() && fieldNode.size() > 0) {
return fieldNode.get(0).get("text").get("content").asText();
}
}
return null;
}

// Extract participants from the comma-separated text property
private List<String> extractParticipants(JsonNode properties) {
String participantsString = extractTextFromProperty(properties, "참여 학생", "rich_text");
if (participantsString != null && !participantsString.isEmpty()) {
return Arrays.asList(participantsString.split("\\s*,\\s*"));
}
return new ArrayList<>();
}

// Extract multi-select property
private List<String> extractMultiSelect(JsonNode properties, String fieldName) {
List<String> result = new ArrayList<>();
if (properties.has(fieldName)) {
JsonNode multiSelectNode = properties.get(fieldName).get("multi_select");
if (multiSelectNode.isArray()) {
multiSelectNode.forEach(node -> result.add(node.get("name").asText()));
}
}
return result;
}

// Extract multi-select property as Integer
private List<Integer> extractIntegerMultiSelect(JsonNode properties, String fieldName) {
List<Integer> result = new ArrayList<>();
if (properties.has(fieldName)) {
JsonNode multiSelectNode = properties.get(fieldName).get("multi_select");
if (multiSelectNode.isArray()) {
multiSelectNode.forEach(node -> result.add(Integer.parseInt(node.get("name").asText())));
}
}
return result;
}
}
Loading
Loading