Conversational AI demonstrations happen in quiet rooms. The questions are friendly, the network is good, and one person speaks at a time. Then the system goes into an airport concourse, a hospital reception or a hall holding several thousand people, and most of what made the demonstration work quietly disappears.
I build these installations. The AI is rarely the part that fails. The room is.
Hearing one person in a crowd
Speech recognition that is faultless on a headset degrades badly against background noise. The fix is physical rather than algorithmic: a microphone array that focuses on whoever is standing in front of the unit, directional audio so the reply reaches that person rather than the whole hall, and presence detection so the system knows someone has approached.
Noise also breaks something subtler. Before a system can answer, it has to work out that the question has ended. That judgement normally depends partly on silence, and in a loud room silence never arrives. Get this wrong in one direction and the avatar interrupts people mid-sentence. Get it wrong in the other and every answer arrives late.
The delay nobody forgives
People leave a gap of about two tenths of a second between conversational turns. No AI pipeline achieves that today. The working target across the industry is roughly 800 milliseconds from the end of a question to the first sound of a reply, and beyond about a second and a half people assume the system is broken and walk away.
Reaching that number is an architecture problem rather than a model problem. Recognition, language model, speech synthesis and facial animation each take time, and a system that runs them one after another will always feel slow. The ones that feel alive overlap the stages: partial transcripts stream to the model, the model streams its first words to synthesis, and speech begins before the sentence is finished.
Where the computer sits
Every stage you move onto hardware inside the unit removes a network round trip, and keeps the installation working when the venue connection falters. Every stage you leave in the cloud is cheaper and easier to update. Most real deployments settle in between, with a local fallback that can still answer the most common questions when the connection is gone.
This is why we treat these as hardware projects. Microphone design, on-device computing, display and enclosure are decided alongside the conversational system, not after someone has already chosen a standard kiosk. It is the same discipline as any other embedded hardware programme: the physical constraints are settled first because they are the ones you cannot change later.
The face is its own problem
A voice assistant can be a few milliseconds late and nobody notices. A face that moves out of step with its own voice is noticed instantly, including by people who could not explain what is wrong. Lip movement has to be driven by the audio as it arrives, and even accurate synchronisation can feel lifeless, because a real speaker’s face begins moving fractionally before the sound.
What to ask before you commission one
If you are earlier than that and still deciding whether an avatar suits your space at all, we set out the eight decisions worth making before commissioning one.
Designing for the bad day
Every installed system has one. The network fails, a group of teenagers decides to test its limits, a child asks something nobody anticipated, or the hall is louder than any test predicted. These are not edge cases to handle later; they are Tuesday.
So decide in advance what the system does when it hears silence, when it misunderstands three times in a row, when someone is abusive, and when the connection drops entirely. Staff need a simple way to pause or reset it without calling anyone. And visitors need to see something graceful when it is offline: a holding screen with directions to a human being is fine, a frozen face is not.
What the delay should look like while it thinks
Since some delay is irreducible, the honest approach is to design for it rather than hide it. Show that the system is listening the moment speech is detected. Let it acknowledge the question briefly while the full answer is still forming, which is what people do in conversation anyway. Keep small idle movement in the face so a pause reads as thought rather than failure. And let people interrupt: a system that talks over a visitor feels far worse than one that is slightly slow.
What does not work is scripted filler. A charming phrase deployed while the system thinks is pleasant the first time and irritating by the third visitor, because installations are used by the same staff and regulars every day.
Measuring it after installation, not just at acceptance
Response times drift. Content grows, conversations get longer, and providers change their infrastructure without telling you. A system that was quick in week one is not necessarily quick in month six, and nobody will report it as a technical fault; they will simply use it less.
Build the measurement in from the start: median and worst-case response time, how often the system failed to understand, and which questions it could not answer. That last list is the most useful thing the installation produces, because it tells the organisation what visitors actually want to know.
None of these are questions about artificial intelligence. They are questions about a machine that has to work in a building full of people, which is what an installation actually is.
