Routing: Building Step-by-Step AI Reasoning

In the world of large language models (LLMs), creating intelligent and helpful AI systems often goes far beyond prompting them with a single question. Modern AI products rely on structured reasoning workflows - where each step builds on the previous one, with clearly defined roles and goals. One of the most practical and powerful of these workflows is routing: the process of deciding what kind of query is being asked, and what kind of system or agent should handle it.

In this article, we'll walk through the concept of routing in LLM applications using a fun example - a magical wand support system inspired by the Harry Potter universe (aka Ollivander's AI Helpdesk). We'll look at:

  • What routing is in an LLM-driven app
  • How schema-guided prompting helps structure AI responses
  • A full code walkthrough (with fake wands and magical issues!)
  • Why modular routing improves reasoning and maintainability

Let's begin.

What is Routing in an LLM System?

Routing refers to the process of categorising or classifying a user's input in order to:

  • Send it to the correct sub-agent or subsystem (e.g., technical vs. product support)
  • Select the right prompt or reasoning template
  • Invoke specific tools or APIs (e.g., weather API, product database, etc.)
  • Think of routing as the AI equivalent of a customer support switchboard operator: its job is not to answer everything directly, but to intelligently direct each incoming request to the best destination.

This is especially important when building multi-functional assistants or agentic systems where multiple tools or skills are available.

Why Routing Matters

Imagine you're building an AI assistant for a support desk. Here's what happens without routing:

User: My wand won't stop casting water spells!
AI: That's interesting! Did you know the Elder Wand is made of elder wood?

Yikes.

This is a product info answer, but the user clearly needed technical help. Without a routing step, your assistant may pick the wrong reasoning strategy which would eventually lead to a confused or frustrated user.

Instead, if you first classify the input as either a product_info or technical_support query, then you can make sure that the LLM respond in a way that makes sense within that frame of reference.

Building a Magical Support Router (Code Walkthrough)

We'll walk through a magical wand support system with two departments:

  • product_info: Ask about wand types, features, materials
  • technical_support: Malfunctioning spells, reversed transformations, etc.

We use Google's @google/genai Node.js SDK to interact with Gemini, and JSON schema validation to enforce structure.

Let's dive into the key parts.

1. The Wand Product Catalogue

We simulate a fake in-memory database:

const wandCatalogue = [
  {
    name: 'Elder Wand',
    core: 'Thestral tail hair',
    wood: 'Elder',
    length: '15 inches',
  },
  {
    name: 'Holly Phoenix',
    core: 'Phoenix feather',
    wood: 'Holly',
    length: '11 inches',
  },
  // ...
];

This lets the AI answer specific product-related questions with actual reference data.

2. The Routing Schema

This defines what a valid routing decision looks like. We use @google/genai's Type.OBJECT to define the expected structure:

const routingDecisionSchema = {
  type: Type.OBJECT,
  properties: {
    category: {
      type: Type.STRING,
      enum: ['product_info', 'technical_support', 'unknown'],
    },
    reasoning: {
      type: Type.STRING,
    },
  },
  required: ['category', 'reasoning'],
};

By enforcing this schema, we get structured output instead of ambiguous text. This is key to reliable, parseable routing.

3. The Routing Prompt

Instead of asking the AI to do everything, we ask it to only determine the category:

const promptRouter = `
You are a magical support router at Ollivander's AI Helpdesk. Classify the query into one of:

- product_info: Wand types, cores, woods, sizes, general info
- technical_support: Wand not working, spells going wrong, magical anomalies
- unknown: If the query doesn't clearly fit

Explain your choice.

Query: ${userQuery}
`;

The AI is focused only on classification first, not solving the problem. This keeps reasoning simple and targeted.

4. Handling the Routed Query

Once the AI returns the category and reasoning (and remember, it does so by returning us a valid object as we have specified it via the schema), we dynamically craft the real prompt based on the routing result:

const routerRequest = {
  model,
  contents: promptRouter,
  config: {
    responseMimeType: 'application/json',
    responseSchema: routingDecisionSchema,
  },
};

const resultRouter = await ai.models.generateContent(routerRequest);
routingDecision = JSON.parse(resultRouter.text);
switch (routingDecision.category) {
  case 'product_info':
    prompt = `You are a wand specialist. The user asked: "${userQuery}". ...`;
    /*
    For this case you also want to include the available products, which you could do by the following:

    Here are the wand listings: ${wandCatalogue.map(w => `- ${w.name}: ${w.wood} wood, ${w.core} core, ${w.length}`).join('\n')}
    */
    break;

  case 'technical_support':
    prompt = `You are a magical technician. Help the user with: "${userQuery}" ...`;
    break;

  default:
    prompt = `The user's message was: "${userQuery}". Ask them to rephrase politely.`;
    break;
}

Now each type of query is answered in-context: with the correct tone, expertise, and reference data.

5. Sample Outputs

Let's test it with a few magical queries:

const queries = [
  'What\'s the difference between a phoenix feather core and unicorn hair?',
  'My wand randomly casts Aguamenti when I try Lumos!',
  'I dropped my wand into a cauldron and now it smells like basilisk.',
  'Where can I find Lord Voldermort?',
];

You'll get:

  • Expert wand lore answers for product questions
  • Detailed magical troubleshooting for technical issues
  • Friendly fallback messages for unclear input
Expand here for a full output sequence

🧙‍♀️ Processing: "What's the difference between a phoenix feather core and unicorn hair?"
🔮 Category: product_info
🪶 Reasoning: The query asks about the difference between two types of wand cores (phoenix feather and unicorn hair), which falls under product information related to wand features and materials.
🧙 Response:

Alright, let's talk about the difference between phoenix feather and unicorn hair cores! While my catalogue only features a few examples, we can still draw some inferences.

Here's what we can glean from the wands listed, and generally speaking, about these two core types:

Phoenix Feather:
• My catalogue shows the Holly Phoenix wand as Holly Wood with Phoenix feather core and 11 inches length.
• Generally: Phoenix feather wands are often said to be capable of the widest range of magic. They can be harder to tame and personalise, sometimes showing a strong independent will. They can be capable of great acts of both good and evil.

Unicorn Hair:
• The Willow Whisper wand is Willow Wood with Unicorn hair core and 14 inches length.
• Generally: Unicorn hair wands are known to produce the most consistent magic and are the least prone to blockages. They are deeply loyal and may retain attachment to their first owner. However, they are not the most powerful core type and are prone to melancholy if mishandled.

In summary: Phoenix feather wands are powerful and versatile, but can be temperamental. Unicorn hair wands are reliable, consistent, and deeply loyal, but perhaps lack the sheer power potential of other cores.

Hopefully, that helps clarify the differences! Remember, wand lore is complex, and the wood also plays a major role.

------------------------------------------------------------
🧙‍♀️ Processing: "My wand randomly casts Aguamenti when I try Lumos!"
🔮 Category: technical_support
🪶 Reasoning: The user is reporting an issue where their wand is casting the wrong spell (Aguamenti instead of Lumos), indicating a malfunction.
🧙 Response:

Ah, a classic case of the Wand Wishy-Washies! Casting Aguamenti when you're aiming for Lumos is certainly... dampening your chances of success, wouldn't you agree?

Here’s a diagnosis and remedy guide:

1. Wand Fatigue or Core Instability:
• Rest your wand for a few days
• Use "Essentia Pura" while gently cleansing it
• Meditate and recharge your magical intent

2. Magical Backwash or Residual Energy:
• Smudge with Muggle white sage
• Wipe gently with Dragon Skin Wand Polish and cast "Tergeo Magica"

3. Wand/Wizard Incompatibility:
• Re-establish your magical bond
• Practise simple spells
• If needed, consult a wandmaker

4. Focus and Intent:
• Take a deep breath before casting
• Visualise the desired effect clearly
• Be precise with your incantation

Important: If you've used dark magic recently, your wand may require additional cleansing. Treat it with respect — it's your magical partner.

------------------------------------------------------------
🧙‍♀️ Processing: "I dropped my wand into a cauldron and now it smells like basilisk."
🔮 Category: technical_support
🪶 Reasoning: The query describes a problem after an accident, indicating a potential malfunction.
🧙 Response:

No need to panic! A Basilisk-scented wand is unfortunate, but fixable. Handle with care.

I. Assess the Damage:
• Look for residue or irregularities
• Test from a distance with a non-combative spell

II. Cleansing Ritual:
• Mix: powdered Dittany, essence of Moly, rue, unicorn tears, powdered moonstone
• Soak the wand for exactly 6 minutes and 32 seconds
• Dry and polish with moonstone
• Chant: “Purge foul scent, dispel the dread…” (full incantation provided)

III. Aftercare:
• Store in a silken pouch for 24 hours
• Retest Lumos safely
• Consult a wandmaker if issues persist

Note: If actual Basilisk venom is suspected — contact the Ministry immediately!

------------------------------------------------------------
🧙‍♀️ Processing: "Where can I find Lord Voldermort?"
🔮 Category: unknown
🪶 Reasoning: This query is unrelated to wand support or functionality.
🧙 Response:

Oh dear! Are you sure that's what you're looking for? Perhaps you meant a book? Let's tread carefully with such names. Tell me more and I’ll help guide you — to knowledge, not danger!

Why Schema-Guided Routing Is Powerful

Schema-based routing provides multiple benefits:

  • Modularity: Cleanly separates classification from response generation
  • Interpretability: You get structured insight into why a query was classified
  • MaintainabilityL Easy to update prompt logic per category
  • Robustness: Can gracefully handle unknown queries

It's a simple pattern but incredibly powerful.

When to Use Routing

You should consider using routing when:

  • Your assistant has multiple skills, agents, or domains
  • You want to separate concerns (e.g., classify first, then respond)
  • You need detailed control over prompts, tools, or model strategies
  • You want to log and audit model decisions for debugging or analytics
  • It's a core design pattern for Agentic AI systems, and a lightweight alternative to full-blown orchestration frameworks.

Closing Thoughts

LLMs are incredibly versatile but their effectiveness depends heavily on how you guide them. By introducing a routing step, you give your AI the ability to reason more clearly and act more appropriately in multi-domain settings.

Whether you're building a whimsical wand assistant or a real-world customer support bot, routing is a foundational capability for intelligent, human-aligned behaviour.