AI Agents Become New Internal Developer Platforms

AI agents are emerging as the next generation of internal developer platforms. At the European KubeCon & CloudNativeCon conference, Whitney Lee and Viktor Farcic discussed setting guardrails to control agent actions and using logs, metrics, and traces to understand agent behavior. Farcic argued that agents will revolutionize the platform space, offering developers tools that replace traditional solutions like Backstage. Agents receive developer input, combine it with system context, and send it to large language models, which either respond directly or invoke tools based on their capabilities. Semantic search enables agents to locate relevant information from sources such as Git repositories, PR discussions, Slack conversations, Jira tickets, and Zoom recordings of design meetings. Enterprises are building their own agents to enforce security guardrails, allowing or blocking specific actions automatically or with approval. Farcic noted that model inputs can be any user-provided content and outputs are actions the model deems appropriate, with no guarantee of constraints.

Lee highlighted tracing capabilities that make agent-platform interactions observable, showing which models were used, which tools were called, token costs, and execution paths. Farcic recommended capturing execution events through logs, metrics, and tracing, with OpenTelemetry providing a standard for generative AI spans that include model names, token counts, tool calls, and parameters. These spans can be exported to Jaeger, Grafana Tempo, Prometheus, or Loki, and are supported by commercial observability platforms such as Datadog, Honeycomb, Dynatrace, and Elastic. Farcic emphasized that real-world tracing data, including actual developer queries and agent execution paths, forms the most valuable evaluation dataset. InfoQ conducted post-conference interviews with Lee and Farcic, in which Farcic stated that the quality of semantic search depends on the data indexed and its partitioning, not on the embedding model or vector database. He warned against treating full documents as single vectors, advocating instead for granular chunks with preserved metadata such as repository, team, and timestamp.

He also stressed the need for continuous indexing and deletion synchronization to maintain timeliness, and advised against indexing mutable system states, which should instead be accessed via live tools. Metrics observation is essential to verify retrieved content and diagnose retrieval failures. Lee added that vector search abstracts source locations, allowing agents to retrieve information based on semantic similarity without requiring explicit keyword guesses. He noted that aggregated tracing data reveals developer intent and drives a feedback loop for platform improvement. Farcic concluded that tracing is indispensable for debugging agents, as repeated inputs can yield different execution paths, making trace data the sole evidence of actual event progression. He identified three critical aspects: agent decision-making, resource consumption, and auditability, arguing that trace data must capture tool selections, parameter usage, and spending patterns to ensure accountability.