Simulate your LiveKit voice agents in an Environment, a voice-native Error Feed, and more

Simulate your LiveKit voice agents in an Environment

Connect a LiveKit voice agent you already run to an Environment, the same way you connect a Vapi or Retell agent. Enter the agent’s registered name, your LiveKit URL, API key and secret, and its system prompt. Preflight checks run before the environment builds, and in every simulation a simulated caller talks to your agent and puts it to the test.

What you get

• Bring your LiveKit agent as it is. Connect it under Build environment, Hosted platform, with the agent’s registered name, LiveKit URL, API key and secret, and its system prompt. No code to share and nothing to rewrite.
• Checked before you build. Preflight checks your LiveKit connection and that your agent is reachable before the environment builds.
• Test the agent your users talk to. In every simulation, a simulated caller talks to your hosted agent, so it’s tested with its real voice pipeline, tools and backend.
• Scenarios from your prompt. Scenarios are generated from the system prompt you paste, and your agent’s tool calls reach your real backend during every call.

How to turn it on

In Simulate, open Environments and choose Build environment. Under Hosted platform, pick the Voice agent type and LiveKit, fill in the agent’s name, LiveKit URL, API key, API secret and system prompt, and run preflight.

Why this matters

Until now, putting a hosted LiveKit agent through simulation meant connecting its source code. Now the deployed agent itself is tested, next to the Vapi, Retell and phone agents you already connect.

Who this is for

Teams that build and run voice agents on LiveKit and want to test them with realistic callers before every release.

A voice-native Error Feed

The Error Feed is now voice-native. It investigates a voice call from the audio of the call, along with the voice provider’s own warnings and errors, not just the transcript, so it can tell a silence or dropped connection caused by the provider apart from a fault in your agent, without burning a hole in your pocket. Grouping also scales to projects with tens of thousands of findings, and personal information in stored conversations is redacted.

Improvements

Simulate

Nine more call and voice metrics in run analytics. A simulation run’s analytics header now shows nine more call and voice metrics, including connected rate, agent latency, words per minute, talk ratio, and average duration and turns. The Detailed Analytics toggle controls the reliability section, and analytics stay visible even when a run has no reliability data.

Run results tables load up to 98% faster. On a 500-call run, the run results table now loads in about 0.3 seconds once it has loaded before, down from as long as 25 seconds, and a first load is about 3.5 times faster. The filter panel uses the same “Use case” and “Sub-goal” labels as the rest of the run, lets you exclude a use case, sub-goal or status with an “Is not” filter, and its status chips reflect your other active filters.

Filter a run’s calls by status. Click a status chip (Queued, In progress, Passed, Failed, Inconclusive or Error) to filter a run’s calls, with matching counts in every group, and see each sub-goal’s verdict in run details.

A cleaner Calls tab. On a run’s Calls tab, the toolbar wraps cleanly at narrow widths, calls you open from a Debug failures result appear right away with their groups expanded, and deleting a call updates the run’s totals immediately.

Simulated callers that behave more like real people. Simulated callers confirm a recap briefly instead of reading it back in full, ask for a repeat when the audio is unclear, and follow up on questions your agent left unanswered. Generated scenario suites also include a minimum share of adversarial cases and give every caller a unique name.

Simulated callers read codes out clearly. Simulated callers now read booking codes, order numbers and other letter-and-digit references one character at a time, the way people do on the phone, so your agent is tested on hearing the exact code.

More reliable simulated calls. Simulated calls run more reliably, so fewer calls end early because of a model provider issue.

Chat runs follow your scenario settings. When you change a scenario’s maximum number of turns, chat runs now stop at that number. The environment chat clears a whole scenario suite only when you explicitly ask it to, and hosted chat runs for agents that reply through a callback start reliably.

A resizable, collapsible builder chat. The environment builder’s chat panel can be collapsed and resized, remembers the width you choose, and wraps a long unbroken string such as a URL instead of scrolling sideways.

Observe

Set alerts for score evals. You can now set alerts for score evals. An alert tracks the eval’s average score against your threshold, so it fires as configured. The preview chart works before you pick a label, and threshold changes refresh it immediately.

Sharper dashboard charts. Widgets with no axis bounds fit the data tightly, the Out of Bounds toggle clips or widens as labeled, and a right-hand axis scales its series together. Stacked line charts show continuous lines and accurate tooltips, and minute-granularity widgets stay responsive.

Observe polish. A filter field or value that is cut off shows its full name in a tooltip. On a saved tracing view, editing a custom date range or a compare-mode filter shows the Save view control so you can update the view.

Evaluation

Edit and re-grade evaluations on a finished run. Edit a single evaluation from a run, pre-filled, and re-run just that evaluation on the run’s calls, or re-grade the evaluations you pick. The same edit is available from the environment’s Evaluations tab and from an evaluation column’s header on the run table, you can show or hide each evaluation’s column, and an evaluation added after the run finishes appears without a reload.

Feedback keeps few-shot examples in sync. Updating a dataset, experiment or eval-playground feedback submission replaces its full set of few-shot examples, and removing a submission removes its examples from retrieval immediately.

Stronger security for evaluation prompts. Security of prompt-template rendering in the JSON Validation evaluation has been strengthened.

Agents

Do more from your AI assistant through the MCP server. Connect your AI assistant to the Future AGI MCP server and it can now work through annotation queues, edit and clean up datasets, check fired alerts, review alert and guardrail rules, and inspect Agent Command Center sessions, right from the assistant you already use.

Smoother prompt editing. Select All inside a prompt message selects only that message’s text and treats an attached image, audio or PDF as one block. Copy and cut give plain text, undo and redo restore attachments, and attachments keep their filename and size after you save.

Gateway and API

Agent Command Center: more control over logs, configuration and alerts. Add a Request Logs column for any custom property your organization declares. Configurations are validated when you save them, so one that can’t apply is rejected with the exact field and reason while the previous version stays active. Set a model timeout per organization, override it per request with x-agentcc-timeout or x-agentcc-request-timeout, and get latency alerts on a window’s average latency, sent to the channels you choose.

Agent Command Center: broader route and provider coverage. Model overrides and conditional routes work on the Anthropic-native, Gemini-native and audio routes, and streamed speech is budget-checked and logged like every other request. tool_choice: "none" withholds tools from Anthropic models, transcription honors response_format (text, srt, vtt, verbose_json), Gemini text-to-speech on Vertex returns the format you request, models that need the Responses API are tagged in Edit Provider, and custom OpenAI-compatible or Anthropic-format providers use the correct /v1 path.

Clearer statuses in the simulation API. The run and call-execution API responses return queued or in_progress while a call is active and error when a call failed or was cancelled, and add a sub_goal_results field with each sub-goal’s verdict. The existing sub_goals field stays for compatibility.

Platform

Stronger SAML single sign-on. Security for SAML single sign-on has been strengthened. Sign-in now starts from the login page’s “Sign in with SSO” option, and admins who want an identity-provider app tile can point it at the Future AGI SSO login page. A SAML session can’t change two-factor, passkey, recovery-code or API-key settings, approve MCP OAuth apps or reset a password.