Agentic AI, driven prominently by Generative AI (GenAI) technologies such as large language models (LLMs), has emerged as the focal point in many organisations’ AI transformation initiatives. However, despite the enthusiasm around GenAI, implementing it effectively poses significant challenges due to the complexity involved and the need for fundamental changes in organisational structures and processes. According to a recent post by software architect Shanmugam Sudalaimuthu, companies face mounting pressure to demonstrate tangible returns on GenAI investments quickly to maintain momentum, with a Deloitte survey revealing that 55% of senior leaders expect visible benefits within two years while 34% fear that failing to realise value could stall further adoption.
To balance the urgency for early results with the demand for long-term AI transformation, Sudalaimuthu highlights a tactical approach centred on a crucial feature in LLMs called Structured Outputs. This feature enables the generation of AI outputs that conform strictly to pre-defined data schemas, tackling one of the biggest integration hurdles for enterprises: aligning unstructured natural language outputs from GenAI with the structured data formats required by legacy IT systems.
Unstructured data constitutes over 80% of enterprise data, encompassing emails, social media, documents, logs, and more. Traditional natural language processing (NLP) tools have struggled to handle such data effectively, but LLMs—pretrained on vast amounts of internet text—can extract key information, convert it into structured formats using Structured Outputs, and feed it into existing databases and analytics tools. This capability not only utilises existing investments in business intelligence but also unlocks value by illuminating key performance indicators and improving risk and compliance management. Gartner anticipates that generative AI will accelerate delivery of value in data and analytics programmes by 40% by 2027.
Structured Outputs also integrate LLMs more deeply into data engineering pipelines. Platforms like MotherDuck have introduced SQL-based querying interfaces that use Structured Outputs to enable querying and transforming unstructured data within traditional data warehouse environments. This integration is pivotal for maintaining clean and reliable training data for AI models. Additionally, Structured Outputs support the generation of synthetic data to mitigate concerns about sensitive personal information in training datasets, as well as the enrichment of documents with structured metadata to enhance retrieval accuracy in GenAI applications.
In Retrieval Augmented Generation (RAG) — a method that supplements LLMs with relevant external documents to reduce hallucinations and improve fact-based responses — Structured Outputs prove instrumental. By transforming unstructured retrieved data into structured representations like tables or graphs, techniques such as StructRAG improve the reliability of reasoning-heavy AI tasks. Companies like Mistral have further utilised Structured Outputs for performance evaluation frameworks that judge RAG outputs, underlining their growing importance.
Beyond data and analytics, Structured Outputs enable real-time tracking of customer sentiment and operational indicators by extracting structured alerts from unstructured IT logs and other semi-structured sources. This improves system observability, troubleshooting, and threat detection, especially in legacy environments lacking structured logging. They also enhance automation workflows; for instance, ServiceNow employs them to automatically generate workflow definitions from natural language user requests, bridging gaps where traditional robotic process automation may falter on unstructured inputs like text and images.
Despite the promising applications, Structured Outputs do come with challenges. Their efficacy depends heavily on high-quality prompt and schema design; poorly designed prompts may yield structurally correct but semantically inaccurate outputs. Model performance also varies, and complex nested schemas can increase token usage, latency, and costs. Although McKinsey estimates GenAI model costs at only around 15% of total solution costs, these factors necessitate careful planning aligned with broader organisational AI strategies to maximise benefit.
The broader AI landscape corroborates the urgency and complexity in scaling GenAI initiatives. Gartner predicts that by the end of 2025, at least 30% of GenAI projects will be abandoned after proof of concept due to issues such as poor data quality, missing risk controls, unclear business value, and escalating costs. These warnings reinforce the importance of early tactical wins that demonstrate value quickly, such as those provided by Structured Outputs, to sustain momentum and justify substantive investments.
Moreover, ongoing shifts in data and analytics operating models underscore the role of senior leaders in orchestrating this transformation. A Gartner survey finds that 61% of organisations are evolving their data and analytics frameworks in response to AI technologies, with Chief Data & Analytics Officers (CDAOs) increasingly accountable for delivering tangible business outcomes from data initiatives. By 2026, over a quarter of Fortune 500 CDAOs are expected to directly oversee at least one top-earning product based on data and analytics, illustrating the growing strategic significance of these capabilities.
Women’s adoption of Generative AI technologies is also predicted to rise sharply, reaching parity or exceeding male adoption rates in the US by the end of 2025, according to Deloitte. Additionally, enterprises deploying AI agents—software systems that perform autonomous or semi-autonomous tasks—are projected to grow from 25% in 2025 to 50% by 2027, highlighting accelerating innovation fueled by GenAI.
In conclusion, while agentic AI and Generative AI hold transformative potential, their successful implementation demands a balanced approach that delivers early, measurable results without losing sight of the long-term strategic vision. Structured Outputs represent a pivotal innovation enabling LLMs to harness vast reservoirs of unstructured data into actionable, structured insights, thereby addressing integration challenges and unlocking new avenues for automation, analytics, and operational excellence. Yet, to truly achieve AI transformation, organisations must couple tactical application of such technologies with clear governance, high-quality data pipelines, and comprehensive strategies to scale agentic AI responsibly and effectively. As Sudalaimuthu aptly notes, structured data extracted from unstructured content—though seemingly mundane—may provide the critical foundation for unlocking the full value of AI across enterprise landscapes.
Source: Noah Wire Services
Subscribe to Industry Updates
