Why
Many of our projects rely on characters that listen and answer by voice. Greta is where we work on that interaction on its own: how a character holds a conversation, reacts while someone speaks, and feels present in a real room. She is our test bed for voice, expressive speech and avatar behaviour, on several devices at once.
Three ways to meet her
In your room
In passthrough mixed reality on Meta Quest 3 and 3S and PICO 4 Ultra. She stands on the real floor, and furniture can hide her.
In the browser
A web page with the same character and voice, with no headset needed.
On an open engine
A version built with the open-source Godot engine, for Quest headsets.
What she can do
- Hold a conversation as a character from a CareConverse scenario, for example to present the lab to visitors.
- Read a text aloud with emotion, cheerful, whispering and so on, following tags in the text.
- Behave like someone listening: she follows you with her eyes, turns towards you, blinks, moves her lips as she speaks, shows captions, and stops when you interrupt her.
Technology
- Mixed reality: Unity 6 with OpenXR and AR Foundation; also a port to Godot 4.
- Web: a static page with a 3D model, no installation.
- Voice: through CareConverse (Google Gemini Live and Gemini speech), or through a voice server that uses Mistral's speech and language models (cloud services).