diff --git a/content/projects/Ichiyo/index.mdx b/content/projects/Ichiyo/index.mdx index 27d8b8aa..ddfa03e4 100644 --- a/content/projects/Ichiyo/index.mdx +++ b/content/projects/Ichiyo/index.mdx @@ -29,10 +29,12 @@ SNSなどのコミュニケーションツールは我々の関係性を高速 本作品は、すれ違う人他者への想像を膨らませ、交流を行うことを目指す。二次元コードが印字された葉を介し、人々がデジタル空間上で自由にメッセージを残せるようデザインを行った。 +This work got Best Prototype Award at [TEI 2024](https://tei.acm.org/2024/) Student Design Competition This work got an Honorable Mention at [GOOD DESIGN NEW HOPE AWARD 2023](https://newhope.g-mark.org/award/2023/23NHA040011.html) This work was exhibited at: [東京大学制作展True 2022](https://iiiexhibition.com) More articles: [東京大学制作展 2022 Emulsion](https://www.iii.u-tokyo.ac.jp/research/221213iiiexhibition) +Slides: [https://drive.google.com/file/d/1XFnZUYT9VDWZyIfjNWg4R1Q_R8-BnjpK/view?usp=sharing](https://drive.google.com/file/d/1XFnZUYT9VDWZyIfjNWg4R1Q_R8-BnjpK/view?usp=sharing) DOI: [https://doi.org/10.1145/3623509.3635866](https://doi.org/10.1145/3623509.3635866) diff --git a/content/projects/Ichiyo/presentatino2.jpeg b/content/projects/Ichiyo/presentatino2.jpeg new file mode 100644 index 00000000..88c428f4 Binary files /dev/null and b/content/projects/Ichiyo/presentatino2.jpeg differ diff --git a/content/projects/Ichiyo/presentation.jpeg b/content/projects/Ichiyo/presentation.jpeg new file mode 100644 index 00000000..44d55901 Binary files /dev/null and b/content/projects/Ichiyo/presentation.jpeg differ diff --git a/content/projects/Ichiyo/presentation3.jpeg b/content/projects/Ichiyo/presentation3.jpeg new file mode 100644 index 00000000..d48c42ce Binary files /dev/null and b/content/projects/Ichiyo/presentation3.jpeg differ diff --git a/content/projects/MuZeroVAE/MuZeroVAE.jpg b/content/projects/MuZeroVAE/MuZeroVAE.jpg new file mode 100644 index 00000000..7f5d95fb Binary files /dev/null and b/content/projects/MuZeroVAE/MuZeroVAE.jpg differ diff --git a/content/projects/MuZeroVAE/figure1.jpg b/content/projects/MuZeroVAE/figure1.jpg new file mode 100644 index 00000000..a8f5612b Binary files /dev/null and b/content/projects/MuZeroVAE/figure1.jpg differ diff --git a/content/projects/MuZeroVAE/index.mdx b/content/projects/MuZeroVAE/index.mdx new file mode 100644 index 00000000..2295a956 --- /dev/null +++ b/content/projects/MuZeroVAE/index.mdx @@ -0,0 +1,21 @@ +--- +cover: "./MuZeroVAE.jpg" +date: "2024-02-27" +title: "MuZeroVAE" +areas: + - Research +--- + +人工知能の分野において，計画能力を備えたエージェントの開発は長年にわたる重要な課題である．完璧なシミュレーションが可能なチェスや囲碁のような分野では，木構造ベースの計画手法が顕著な成功を収めてきた． +しかし，実世界の問題に直面すると，環境を動かすダイナミクスはしばしば複雑で未知である． +この課題に対応するため，以前には木構造ベースの探索と学習モデルを組み合わせた，世界モデルを活用するボードゲームAIであるMuZeroが開発された． +このアルゴリズムは，基盤となるダイナミクスの詳細を知らなくても，挑戦的で視覚的に複雑な領域で超人的なパフォーマンスを実現する．特に，囲碁，チェス，将棋において，MuZeroはゲームのルールが提供されたAlphaZeroアルゴリズムに匹敵する性能を示した． +本研究では，MuZeroアルゴリズムを発展させ， VAE（Variational AutoEncoder）を使用してResNetの部分を変更した「MuZero VAE」という新しいアーキテクチャを提案する． +これによりオセロを学習する．MuzeroはRepresentationとDynamicsによる潜在空間を整合させる仕組みを持たないが，この2つの潜在空間を明示的に近づけることでPredictionの学習を安定化させる． +また，2つの潜在空間を近づけることで潜在空間の距離が近い位置で戦略的や評価値的に近い状況を学習することが期待され，VAEで盤面を再現することができる．これにより潜在空間内を探索し，戦略的に類似した盤面を再現することによりモデルの解釈性が増加するとともに，オセロというゲームの理解の深化にも大いに寄与する可能性が高い研究である． + +このプロジェクトは「東京大学松尾研主催世界モデルと知能2023」最終課題において優秀プロジェクトの一つに選ばれました． +This project was selected as one of the excellent works at the end-term project in "World Model and Intelligence 2023" Course held by Matsuo Institute. + +Paper Prescript: [https://drive.google.com/file/d/1ZxZ6zm2DwHUqnFqtQskJjIxW03lqxJpg/view?usp=sharing](https://drive.google.com/file/d/1ZxZ6zm2DwHUqnFqtQskJjIxW03lqxJpg/view?usp=sharing) +Poster: [https://drive.google.com/file/d/131KIxNktSpwvC5Ta4RMRBIVFFKvPB380/view?usp=sharing](https://drive.google.com/file/d/131KIxNktSpwvC5Ta4RMRBIVFFKvPB380/view?usp=sharing) \ No newline at end of file diff --git a/content/projects/SUMART/SUMART.jpg b/content/projects/SUMART/SUMART.jpg new file mode 100644 index 00000000..9ae26a5e Binary files /dev/null and b/content/projects/SUMART/SUMART.jpg differ diff --git a/content/projects/SUMART/figure1.jpg b/content/projects/SUMART/figure1.jpg new file mode 100644 index 00000000..8c94855e Binary files /dev/null and b/content/projects/SUMART/figure1.jpg differ diff --git a/content/projects/SUMART/index.mdx b/content/projects/SUMART/index.mdx new file mode 100644 index 00000000..fe3fec51 --- /dev/null +++ b/content/projects/SUMART/index.mdx @@ -0,0 +1,17 @@ +--- +cover: "./SUMART.jpg" +date: "2024-03-14" +title: "SUMART" +areas: + - Research +--- + +We propose SUMART, a method for summarizing and compressing the volume of verbose subtitle translations. +SUMART is designed for understanding translated captions (e.g., interlingual conversations via subtitle translation or when watching movies in foreign language audio and translated captions). +SUMART is intended for users who want a big-picture and fast understanding of the conversation, audio, video content, and speech in a foreign language. +During the training data collection, when a speaker makes a verbose statement, SUMART employs a large language model on-site to compress the volume of subtitles. This compressed data is then stored in a database for fine-tuning purposes. +Later, SUMART uses data pairs from those non-compressed ASR results and compressed translated results for fine-tuning the translation model to generate more concise translations for practical uses. +In practical applications, SUMART utilizes this trained model to produce concise translation results. +Furthermore, as a practical application, we developed an application that allows conversations using subtitle translation in augmented reality spaces. +As a pilot study, we conducted qualitative surveys using a SUMART prototype and a survey on the summarization model for SUMART. +We envision the most effective use case of this system is where users need to consume a lot of information quickly (e.g., Speech, lectures, podcasts, Q&A in conferences). \ No newline at end of file diff --git a/src/@lekoarts/gatsby-theme-emilia/texts/about-me.mdx b/src/@lekoarts/gatsby-theme-emilia/texts/about-me.mdx index 65fc596f..9218fba6 100644 --- a/src/@lekoarts/gatsby-theme-emilia/texts/about-me.mdx +++ b/src/@lekoarts/gatsby-theme-emilia/texts/about-me.mdx @@ -18,15 +18,19 @@ AIと人間のインタラクション（例えばAIを用いた学習支援や -------------------------- ## News -- 2024.2 One paper got accepted to [IEEE VR 2024 Workshop](https://sites.google.com/cluster.mu/1st-ws-seamless-reality/) -- 2024.1 I defended my master's thesis! -- 2024.1 I submitted one paper to [IEEE VR 2024 Workshop](https://sites.google.com/cluster.mu/1st-ws-seamless-reality/) +- 2024.02 My course work on reinforcement learning got excellent prize! Check our paper[paper](https://drive.google.com/file/d/1ZxZ6zm2DwHUqnFqtQskJjIxW03lqxJpg/view?usp=sharing) and [poster](https://drive.google.com/file/d/131KIxNktSpwvC5Ta4RMRBIVFFKvPB380/view?usp=sharing) +- 2024.02 My work got awarded at Telecommunication Interdisciplinary Research Award! +- 2024.02 Our work was selected as an Excellent Work at [World Model and Intelligence](https://deeplearning.jp/lectures/world-model-2023/) Final Course Project +- 2024.02 We got Best Prototype Award at [TEI 2024](https://tei.acm.org/2024/) Student Design Competition!
Read More -- 2024.1 I submitted my master's thesis! +- 2024.02 One paper got accepted to [IEEE VR 2024 Workshop](https://sites.google.com/cluster.mu/1st-ws-seamless-reality/) +- 2024.01 I defended my master's thesis! +- 2024.01 I submitted one paper to [IEEE VR 2024 Workshop](https://sites.google.com/cluster.mu/1st-ws-seamless-reality/) +- 2024.01 I submitted my master's thesis! - 2023.11 One paper got accepted to [TEI 2024](https://tei.acm.org/2024/) - 2023.10 I submitted one paper to [TEI 2024](https://tei.acm.org/2024/) - 2023.10 Our work 'Ichiyo' got an Honorable Mention at [GOOD DESIGN NEW HOPE AWARD 2023](https://newhope.g-mark.org/award/2023/23NHA040011.html) @@ -103,26 +107,31 @@ International Journal of Human-Computer Interaction (IJHCI), April. 2023. 14 pag In Proceedings of the Ninth ACM Conference on Learning @ Scale (L@S ’22), June. 2022. 6 pages. DOI: [10.1145/3491140.3528299](https://doi.org/10.1145/3491140.3528299). [[Paper](https://www.iplab.cs.tsukuba.ac.jp/paper/international/nishida_las2022.pdf)] 2. Koki Tamura, Nobushige Doi, Naoto Nishida, Junjie Chen, Hitomi Yanaka. -Japanese fluency filtering using surprisal, +Japanese Fluency Filtering Using Surprisal, 29th of the Association for Natural Language Processing (NLP2023), March. 2023. 6 pages. in Japanese. [[Paper](https://drive.google.com/file/d/1xYvQVpj7HwvHLk7hR9dvTfPNsWCzUJ8z/view?usp=sharing)][[HP](https://www.anlp.jp/proceedings/annual_meeting/2023/)] #### Workshops/Poster -1. Naoto Nishida*, Hirofumi Shibata*, Ayako Yogo*, Yu Shimada*, Toma Ishii*(*co-1st authors). +1. Naoto Nishida, Jun Rekimoto. +SUMART: SUMmARizing Translation from Wordy to Concise Expression. +31st the IEEE Conference on Virtual Reality and 3D User Interfaces (1st Workshop on Seamless Reality: AR Technologies for Seamless Perception and Cognition between Cyber and Physical Spaces), March 2024. 3 pages. DOI: tba + + +2. Naoto Nishida*, Hirofumi Shibata*, Ayako Yogo*, Yu Shimada*, Toma Ishii*(*co-1st authors). Ichiyo: Fragile and Transient Interaction in Neighborhood. -Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '24 Student Design Competition), February 2024. 3 pages. DOI: [10.1145/3623509.3635866](https://doi.org/10.1145/3623509.3635866). [[Paper](https://drive.google.com/file/d/1XJeN_yUejc5OVCOIMvGv5F9IdAIMGQ0i/view?usp=sharing)][[Supplemental Video](https://drive.google.com/file/d/1Rn16XSD0yEsv5f0ME2-vTtUw1qTBfTWK/view?usp=sharing)] +Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '24 Student Design Competition), February 2024. 3 pages. DOI: [10.1145/3623509.3635866](https://doi.org/10.1145/3623509.3635866). [[Paper](https://drive.google.com/file/d/1XJeN_yUejc5OVCOIMvGv5F9IdAIMGQ0i/view?usp=sharing)][[Supplemental Video](https://drive.google.com/file/d/1Rn16XSD0yEsv5f0ME2-vTtUw1qTBfTWK/view?usp=sharing)[Slide](https://drive.google.com/file/d/1XFnZUYT9VDWZyIfjNWg4R1Q_R8-BnjpK/view?usp=sharing)] -2. Naoto Nishida, Sakiho Horibe, Jun Rekimoto. +3. Naoto Nishida, Sakiho Horibe, Jun Rekimoto. Traverco AR: Transcription-assisted Verbal Communication via AR glasses. 30th Workshop on Interactive Systems and Software (WISS 2022 Demo/Poster). December 2022. 3 pages. in Japanese.[[Paper](https://www.wiss.org/WISS2022Proceedings/data/1-A02.pdf)][[Workshop Page](https://www.wiss.org/WISS2022Proceedings/)] -3. Naoto Nishida, Hinako Nozaki, Buntarou Shizuki. +4. Naoto Nishida, Hinako Nozaki, Buntarou Shizuki. Language Learning Assistance by Automatic Adjustment of Playback Speed Based on Facial Expression. 29th Workshop on Interactive Systems and Software (WISS 2021 Demo/Poster). December 2021. 3 pages. in Japanese. [[Paper](https://www.wiss.org/WISS2021Proceedings/data/1-A09.pdf)][[Poster1](https://drive.google.com/file/d/1kCp8FUoCV5tiuWHWerNOnFTmM6yz3i6i/view?usp=sharing)][[Poster2](https://drive.google.com/file/d/1sLXqn93z4wl_Ym30pDtErbGgvjnEJTmN/view?usp=sharing)][[Workshop Page](https://www.wiss.org/WISS2021Proceedings/)] -4. Yuga Tsukuda, Naoto Nishida, Jun Lu, and Yoichi Ochiai. +5. Yuga Tsukuda, Naoto Nishida, Jun Lu, and Yoichi Ochiai. **Insect-Computer Hybrid Speaker: Speaker using Chirp of the Cicada Controlled by Electrical Muscle Stimulation.** ACM CHI 2021 Workshop on Design and Creation of Inclusive User Interactions Through Immersive Media (ACM CHI IICW 2021). May 2021. 5 pages. [[Preprint](https://drive.google.com/file/d/1Gv5o8t94oQg8jvn8UkxMBDoYoXVzPQkq/view)][[Workshop Page](https://sites.google.com/view/acm-chi-iicw21/home)][[Demo Video](https://www.youtube.com/watch?v=wdhv4QByH6c)] @@ -140,6 +149,7 @@ Media (ACM CHI IICW 2021). May 2021. 5 pages. [[Preprint](https://drive.g - 2023.12 31st Workshop on Interactive Systems and Software(WISS2023) - 2021.12 29th Workshop on Interactive Systems and Software(WISS2021) - Internships + - 2024.04(Expected)-Present GENIAC Program at Matsuo Institute - 2022.03 OIST Science Challenge 2022 [[Certificate](https://drive.google.com/file/d/1yk5evdEUg2J872brfN5ucY3_iirmvh6r/view?usp=sharing)][[Presentation Slide](https://drive.google.com/file/d/1sdwNyr9eWYqfTge0oN01w2uCJ28088pW/view?usp=sharing)] ### Industory Services @@ -157,6 +167,9 @@ Media (ACM CHI IICW 2021). May 2021. 5 pages. [[Preprint](https://drive.g - 2019.07 IBM Hackathon [[Project Page](https://github.com/nawta/otomo)] etc... ### Award +- 2024.02 Excellent Project - World Model and Intelligence 2023 +- 2024.02 Telecommunication Interdisciplinary Research Award +- 2024.02 TEI 2024 Student Design Competition - Best Prototype Award [[Certificate](https://drive.google.com/file/d/1b5kn71FFk0SbktvuEMEXdPh6o_OcUeVP/view?usp=sharing)] - 2023.10 GOOD DESIGN NEW HOPE AWARD 2023 [[HP](https://newhope.g-mark.org/award/2023/23NHA040011.html)][[HP backup](https://drive.google.com/file/d/1V8C1y-mHujYVzm0flEFRshk5hD2Il9JK/view?usp=sharing)][[Certificate](https://drive.google.com/file/d/1nyr1hWeFxnmlV4HI50Psow37eVAcQh58/view?usp=sharing)] - 2022.03 Dean's Award from University of Tsukuba [[Certificate](https://drive.google.com/file/d/1VVcvHf4D0OGzFqIJX4q6-E5yZn-uRA1m/view?usp=sharing)] - 2022.03 Excellent Foreign Language Activity - 1st Prize [[Certificate](https://drive.google.com/file/d/1lFcv0DKXW69NMg9KNMNEghpnis63Wyvn/view?usp=sharing)] @@ -164,6 +177,8 @@ Media (ACM CHI IICW 2021). May 2021. 5 pages. [[Preprint](https://drive.g - 2019.09 FUJIFILM Hackathon - 1st Prize ### Grants + +- 2024.02 Telecommunication Interdisciplinary Research Award 300,000 JPY - 2023.10-2027.03 The Univerisity of Tokyo PhD Fellowship 180,000 JPY/month - 2023.09 Google East Asia Student Travel Grants 1,000 USD - 2023.04 Gary Marsden Travel Award 2,500 USD