Join the event trusted by enterprise leaders for nearly two decades. VB Transform brings together the people building real enterprise AI strategy. Learn more
Editor’s note: Emilia will lead an editorial roundtable on this topic at VB Transform this month. Register today.
Orchestration frameworks for AI services serve multiple functions for enterprises. They not only set out how applications or agents flow together, but they should also let administrators manage workflows and agents and audit their systems.
As enterprises begin to scale their AI services and put these into production, building a manageable, traceable, auditable and robust pipeline ensures their agents run exactly as they’re supposed to. Without these controls, organizations may not be aware of what is happening in their AI systems and may only discover the issue too late, when something goes wrong or they fail to comply with regulations.
Kevin Kiley, president of enterprise orchestration company Airia, told VentureBeat in an interview that frameworks must include auditability and traceability.
“It’s critical to have that observability and be able to go back to the audit log and show what information was provided at what point again,” Kiley said. “You have to know if it was a bad actor, or an internal employee who wasn’t aware they were sharing information or if it was a hallucination. You need a record of that.”
Ideally, robustness and audit trails should be built into AI systems at a very early stage. Understanding the potential risks of a new AI application or agent and ensuring they continue to perform to standards before deployment would help ease concerns around putting AI into production.
However, organizations did not initially design their systems with traceability and auditability in mind. Many AI pilot programs began life as experiments started without an orchestration layer or an audit trail.
The big question enterprises now face is how to manage all the agents and applications, ensure their pipelines remain robust and, if something goes wrong, they know what went wrong and monitor AI performance.
Choosing the right method
Before building any AI application, however, experts said organi …