My experience in building LLM application and the necessity of Agent Framework

6/20/2026 • Atchayasri Rajkumar

My experience in building LLM application and the necessity of Agent Framework

I built an LLM based reporting tool during my career break. I am coming across posts and articles related to the usage Agentic Systems in LLM applications. From my experience, I can tell to people working on projects related to LLM applications is that you don't always need an Agentic System. If you have a clear workflow in mind and know the next steps, you probably don't need an Agent Framework. Before deciding on using CrewAI, LangGraph or Microsoft Agent, just think about the goal of your application. Lets assume your application connects to datasources, fetches metrics, calculates KPIs, detects anomalies, generates insights, creates report and sends to a client. This is a workflow and you can achieve it with 80% deterministic code with specific rule and 20% LLM to generate insights. You don't need an agent here. Instead, if your application decides on the data sources to use, the metrics and KPIs to investigate, then you need an agent. Basically if the application observes, reasons and chooses the next action then you need an agent. If your end goal is to show "What happened" then an automated LLM application is enough. But if you want to analyze "Why something happened", then you need an Agentic Framework. Automated LLM application says, "Conversions dropped 30% last week and Campaign B performed worst". LLM with Agentic Framework can answer, "Why the conversions dropped?" Was it because of traffic or CTR or CPC or landing page performance. Because an agent makes decisions while on the flow and reacts based on the result. To conclude, if the investigation path is known beforehand use deterministic code with LLM. If the investigation path depends on discoveries made during investigation, an agent becomes useful. Agree?