What is this project

This is a fully featured, free*, functional Multiplayer VR UNITY 6 project. Features:

• Netcode for GameObjects Host and Client option
• Ad-hoc join capability with Just In Time client/server protocol and a server repository to synchronize prefabs across all clients post app launch
• 2-Stage Client Sign-In process with freedom to build and use your own rigged avatars (FBX), no need to register and package the avatars in Network Manager due to the included Client-Server-Join protocol and ServerRepository object
• Hands, head and detailed finger tracking from your XR system to your avatars
• 4 preconfigured avatars (male/female, human/bot), easily replace with your own avatars
• Auto-IK component to automatically configure all IK constraints of your avatars hands, head and fingers
• Checkbox for using a local or a web-based prefab/avatar resource repository
• Tool to generate asset bundles to store on the web-based repository
• 2 different object grab implementations
• First Person and Third Person mode via the UI (FP3P)
• Vivox Voice Services with Spatial audio (need to add your own credentials)
• Debug Console (3rd party free component) for getting debug information on standalone VR/XR systems
• OpenXR based so in principle platform-agnostic
• NPC with speech interface: using STT+LLM+TTS cloud services
• NPC with Text to Mesh interface: using Sloyd.ai cloud services

(* => excluding paid cloud services for the NPC and assumes you don't exceed the Vivox Voice services complementary service tresholds)

What is new in this Branch

This branch is now running UNITY 6 and has DEEPSEEK-R1 and Ollama support!

Some additional new features in this version are:

• Generate 3D objects at runtime by talking to the NPC (create/generate model, edit/update/change model etc..)
• Uses GLTFast to spawn 3D objects in VR
• Added and updated support for multiple Speech Recognition, Speech Synthesis and LLM AI models
• 3D Spatial audio from the NPC for a more realistic experience
• Added Ollama component as an alternative for LLM_Groq service if you want to run your own LLM locally
• Support for the new DeepSeek-R1 LLM on GroqCloud and Ollama (download deepseek model for Ollama yourself)!
• Changed AI component dependency approach: introduction of a new AI_Orchestrator component that manages all AI component links and inter-component comms

Steps to get started

1. Pull the branch
2. Open Unity Editor, when prompted about Errors, select Ignore and do not start in safe mode
3. Go to File -> Open Scene -> BaMMain
4. In the VivoxVoiceManager, enter your Vivox service credentials (if not used (yet), leave them blank)
5. Import the uLipSync package: Top Menu -> Assets -> Import Package -> Custom Package -> Navigate to the uLipSync package in this repository and import it
6. You may need to re-import the XR Interaction Toolkit samples and the XR Hands samples
7. If you want the InGame Debug Console then go to the Unity Store and purchase it (free)
8. If you want to talk with the NPC, be sure to get API keys from the following cloud providers: HuggingFace (free), GroqCloud (free), Speechify (paid), RapidAPI (partially free) and Sloyd.ai (partially free).
9. In Assets/Prefabs, click on the DynamicPrefabStarter object, select its ClientServerJoinProtocol script and ensure that UseWebRepository is UNCHECKED
10. Check the Platform setting under Unity Menu->File->Build Settings, if you want to compile for standalone VR (eg. Meta Quest 2,3) select Android otherwise select Windows
11. Open the Network Manager and change IP address of the host to the IP address of the device/PC that will act as the host

If you want to use remote prefabs

1. In Assets/Prefabs, click on the DynamicPrefabStarter object, select its ClientServerJoinProtocol script and ensure that UseWebRepository is CHECKED
2. Enter the url of your own webserver where you want the AssetBundles to be stored
3. In the Assets menu in Unity, select Build Asset Bundles
4. When that is completed, open a File Explorer and find your Assets folder, there is an AssetBundle folder with a windows and an android subfolder
5. ZIP the AssetBundle folder and transfer it to your webserver, unpack under the the url you entered in step 2

Compatibility

• Currently this code is tested for Windows and Meta Quest 2 and 3 (Pico 4 not yet!)
• If you want to add Mac with a Web Repository then you must update the Editor/AssetBundlesBuild script to generate Mac AssetBundles ...
• AND open Assets/Scripts/Network/ClientServerJoinProtocol.cs and add code in the OnNetworkSpawn() method (pretty straightforward)
• If you have another VR brand headset, you will need to configure that headset in Project Settings -> XR Plugin Management