Dialogflow V2: Unity Plugin Proposal

• Project Description
• Team Members
• Define the problem
• Address Greater Landscape
• Deliverables
• Timeline
• Documentation
• Documentation
• Accessibility
• Mentoring
• More about team
  • Ilana Pecis Bonder
  • Alice Sun
  • Lin Zhang
• References

Project Description

The Dialogflow V2 Unity Plugin is a continuation from Api.ai: Unity Plugin, which made it easy to integrate the Dialogflow(Api.ai) natural language processing API into your Unity project.

Team Members

• Ilana Pecis Bonder
• Alice Sun
• Lin Zhang

Define the problem

• Api.ai: Unity Plugin is no longer working in Dialogflow V2, due to the difference in the way the basic system works (Dialogflow API v2 versus v1).
• Dialogflow API V1 will be shut down in one year – on October 23rd, 2019.
• Our goal will be figuring out ways to build similar plugin that works in Dialogflow V2 system.

Address Greater Landscape

Opening new possibilities to involve voice-based conversational interaction in VR and AR environment.

Deliverables

• Feature: listening and sending voice data from Unity to the Dialogflow V2 service for recognition and processing
• Doc: show and explain how to use it (including STT process, if required)

Timeline

This project will be completed over 5 weeks (Nov 6 - Dec 11). Describe a timeline in detail below.

Week 1

• Defining project schedule and make proposal presentation
• Reaching out mentors
• Meeting with Nicole @1PM, Nov 8
• Dialogflow V2 Unity Plugin Hackathon Nov 11

Week 2

• Analyzing the authentication process in Dialogflow V2
  • Authentication via Google Cloud Service Account JSON key
  • It will give 401 error without adding the generated “client_email” in JSON file into the Dialogflow project authentication panel:

• Adding the “client_email” to "INVITE NEW PEOPLE" will give access to the Dialogflow project:

• JSON Web Token (JWT) for Google Clout Platform in Unity
• How can we automate this process?

Week 3

• Audio Request Support: Passing audio file directly via V2 API call

• Using Audio Source in Unity to capture audio from a microphone

Week 4

• Testing with a text call "hello", which received fulfillment response of "Hiiiii Unity~~~" from "Default Welcome Intent" in Dialogflow.

• Next step is to moving on with Audio Source instead of text query.

Week 5

• TBD

Documentation

• The documentation that is more related to detailed process or personal thoughts will be written in individual's blog posts, while information that can be used as instructions or to guide other people will exist inside this project repository for further users and contributors. Initial documentation including basic intro, proposal and schedule will be continuously updated in README.md.

Accessibility

All documentation will be done with considerations of the W3C Web Content Accessibility Guide.

Mentoring

• Nicole He: Dialogflow Mentor
• Igal Nassima: Unity Mentor

More about team

Ilana Pecis Bonder

• Creative Technologist and UI/UX Specialist
• Interactive Telecommunications Program (ITP)
• Portfolio

Alice Sun

• Graphic & interaction designer
• Interactive Telecommunications Program (ITP)
• Blog

Lin Zhang

• Front-end developer
• Interactive Telecommunications Program (ITP)
• Blog

References

• Api.ai: Unity Plugin
• Dialogflow API v2 versus v1