Levels of Autonomy in LLM applications

Code

• Code has zero autonomy and is 100% deterministic

• We all know that everything is hard-coded and it is not even really a cognitive architecture

  Disadvantage

  The problem? You'd need to write rules for every possible scenario - making it impossible to handle real-world complexity.

LLM call

• A single LLM call means your app basically does one main thing - you give it an input, it processes it, and gives you back an output.

• Think of chatbots that just take your message and respond, or apps that translate text.

• A single LLM call means your app basically does one main thing - you give it an input, it processes it, and gives you back an output.

• Think of chatbots that just take your message and respond, or apps that translate text.

• This was a huge leap from hard-coded rules, even though it's still pretty simple and is only in the 2nd stage of autonomy

  Disadvantage

  • Trying to get everything done in one shot often leads to confused or mixed, responses - just like how a single person can't be an expert at everything.

Chains

• Think of chains like having multiple specialists instead of one generalist. Instead of asking one AI to do everything, we break it down into steps where each AI is really good at one thing.
• Imagine a customer service chatbot: The first AI reads your complaint and figures out exactly what product you're talking about
• The second AI finds the right solution from the company's help docs, and the third AI turns that solution into a friendly response.
• Each step is simple, but together they create a much smarter system than a single LLM call could.

Chains (contd.)

• This is where we first started seeing AI apps that could handle more complex tasks - not just by being smarter, but by breaking big problems into smaller, manageable pieces.

  Disadvantage:

  The downside? These fixed sequences are like a rigid assembly line - they always follow the same steps defined by the human.

4. Router

• Now this is where it gets interesting - routers are like smart traffic cops for your AI. Instead of having a fixed path like in chains, the AI itself decides what steps to take next.

• Imagine a personal assistant bot: when you ask it something, it first figures out if you need help with scheduling, research, or calculations, then routes your request to the right tool or chain for the job.

  Disadvantage While it can choose different paths, it still can't remember previous conversations or learn from mistakes.

5. State Machine (Agent)

This is combining the previous level (router) but with loops. Agent ~= control flow controlled by an LLM This involves features like:

1. Ability to have human-in-loop, ask for approval before moving on
2. Multi-agent systems
3. Advanced memory management
4. Go back in history and explore better alternate paths
5. Adaptive Learning

This is were Langgraph, comes into the picture

Levels of autonomy in LLM applications

What is LangGraph ?

A framework for building controllable, persistent agent workflows with built-in support for human interaction, streaming, and state management. It uses the Graph Data Structure to achieve this

Key features of LangGraph

1. Looping and Branching Capabilities: Supports conditional statements and loop structures, allowing dynamic execution paths based on state.
2. State Persistence: Automatically saves and manages state, supporting pause and resume for long-running conversations.
3. Human-Machine Interaction Support: Allows inserting human review during execution, supporting state editing and modification with flexible interaction control mechanisms.
4. Streaming Processing: Supports streaming output and real-time feedback on execution status to enhance user experience.
5. Seamless Integration with LangChain: Reuses existing LangChain components, supports LCEL expressions, and offers rich tool and model support.