Introduction to ChatdollKit Building Conversational AI with 3D Models
ChatdollKit is an innovative SDK that empowers you to transform your 3D models into engaging, voice-enabled chatbots. Whether you're a developer, artist, or enthusiast, ChatdollKit provides the tools to create interactive virtual assistants with features such as voice recognition, AI-driven conversations, and dynamic facial expressions.
This platform supports a variety of LLMs, allowing for advanced AI capabilities within your virtual assistant. With ChatdollKit, create multi-platform applications (Windows, Mac, iOS, Android, WebGL) that can run on VR, AR, and web environments.
Key Features Explore the Capabilities of ChatdollKit
ChatdollKit offers a comprehensive set of features designed to make your virtual assistant project a success:
**Generative AI Native:** Seamlessly integrates with multiple Large Language Models (LLMs), including ChatGPT, Anthropic Claude, Google Gemini Pro, and Dify, providing function calling and multimodal capabilities.
**3D Model Expression:** Synchronizes speech and motion, controls facial expressions, and autonomously handles animations, including blinking and lip-sync.
**Dialog Control:** Integrates Speech-to-Text and Text-to-Speech (TTS) engines (OpenAI, Azure, Google, VOICEVOX, etc.), manages dialog state, extracts intents, and supports wake-word detection.
**Multi-Platform Support:** Compatible with a wide range of platforms, including Windows, Mac, Linux, iOS, Android, and other Unity-supported platforms such as VR, AR, and WebGL.
What's New Latest Updates in Version 0.8.15
Version 0.8.15 introduces significant enhancements across multiple areas:
**WebGL Enhancements:** Added Silero VAD support, camera switching with correct aspect ratio handling, file upload for images, optimized microphone data transfer, and lip-sync fixes.
**UI Control Improvements:** Streamlined UI controls that work out-of-the-box without configuration.
**Stronger Noise Resistance:** Enhanced speech recognition in noisy environments by combining multiple voice activity detection methods.
Explore previous updates (versions 0.8.14, 0.8.13, 0.8.11 and 0.8.12, etc.) to find features like echo cancelling support, improvements in conversational AI, multi-language support, and long-term memory features.
“ChatdollKit is a powerful solution for developers looking to create interactive and engaging virtual assistants.
The ChatdollKit Team
Explore Interactive Features
Dive Deeper into ChatdollKit's capabilities
AI Integration
Leverage the power of leading LLMs like ChatGPT and Gemini for intelligent conversations.
Speech Synthesis
Utilize text-to-speech capabilities for realistic and expressive voice output, including support for various voices and styles.
Multi-Platform Support
Create virtual assistants that run on Windows, Mac, iOS, Android, WebGL, and more, reaching a wider audience.
Get Started Setting Up Your First ChatdollKit Project
To get started with ChatdollKit, follow these steps:
**Dependencies:** Install the latest version of ChatdollKit.unitypackage and the required dependencies (UniTask, uLipSync, UniVRM).
**Project Setup:** Create a new Unity project or use an existing one. Ensure you are NOT using the SRP (Scriptable Render Pipeline) project template.
**Import Dependencies:** Import the necessary packages from the Unity Package Manager.
**Resource Preparation:** Add a 3D model to your scene and import animation clips for your model.
**Configure the AIAvatarVRM prefab**: Add the prefab and configure as detailed in the documentation to add the necessary components to the scene.