My cousin runs Qwen 3.5 7B on her M1 Pro Mac (I was the person suggesting it for her basic writing tasks)
Yesterday, she called me being super frustrated.
She called Qwen SHIT.
So,
My young cousin, working on a college social science project examining gender distribution across professional sectors in India needed a clean, interactive heatmap.
Specifically, she wanted a state-wise visualization of registered medical doctors, categorized strictly by male vs. female healthcare practitioners.
Having spent a few months tuning local AI models, she figured this would take five minutes.
She fired up Jan.ai, pointed it to her local Ollama instance running Qwen 3.5 7B on an M1 Pro Mac, and entered a basic prompt:
“Build a single-file index.html application containing an SVG map of India that color-codes each state based on female doctor percentages, complete with hover tooltips displaying male and female doctor counts."What followed was a three-hour descent into local AI execution failures, token limits, and software architecture bottlenecks :)
1. The Context Collapse: The Hidden Token Tax of Reasoning Models
Modern open-source models like Qwen 3.5 are built with internal reasoning mechanisms (often wrapped inside <think> blocks).
While reasoning improves logical code generation, it creates a silent tax on local execution parameters.
[System Prompt] ➔ [Reasoning Phase (<think> ... 2,000+ Tokens)] ➔
[Code Output Generation]When Qwen attempted to generate a complex interactive SVG map, the sequence unfolded as follows:
- The model analyzed the geometry of Indian state SVG paths.
- It calculated state-by-state gender metrics.
- It formulated D3.js/vanilla JavaScript hover handlers.
By the time the model completed its internal thought process, it had consumed over 2,000 tokens inside the <think> tag alone.
Because local API servers (like Ollama or Llama.cpp) typically default to a context limit of 2,048 or 4,096 tokens, the generation hit a hard ceiling right as the HTML output began.
The UI returned an incomplete file ending abruptly at div id="map">, triggering context limit errors in the host client:
Model ran out of context size: This request needed 4,687 tokens, but the model's context window holds only 4,096.2. The Data Gap: AI Hallucinations vs. Ground-Truth Demographics
When forced to generate statistical data without external assistance, local LLMs default to pattern-matching approximations.
When building gender-disaggregated datasets, this introduces significant accuracy issues.
Global reports from the World Health Organization (WHO) estimate that women account for roughly 14% to 17% of allopathic medical doctors in India.
However, state-by-state breakdowns fluctuate dramatically based on regional social infrastructure:
- High Female Representation: States like Kerala, Delhi, and Himachal Pradesh consistently show higher ratios of female medical graduates.
- Low Female Representation: States like Uttar Pradesh, Bihar, and Jharkhand exhibit significantly lower percentages.
┌────────────────────────────────────────────────────────────────────────┐
│ National Average: ~14.2% - 17% │
├───────────────────────────────┬────────────────────────────────────────┤
│ High Female Doctor Ratios │ Kerala, Delhi, Himachal Pradesh │
├───────────────────────────────┼────────────────────────────────────────┤
│ Low Female Doctor Ratios │ Uttar Pradesh, Bihar, Jharkhand │
└───────────────────────────────┴────────────────────────────────────────┘When asked to generate this dataset on the fly, the local model made two distinct errors:
- Binary Reduction: It defaulted to 50/50 male-to-female distributions across every state to simplify its JS calculations.
- Mock Data Reliance: It substituted real WHO/NSSO health workforce metrics with generic placeholder values like
female_count: 100, male_count: 100
To build a statistically valid tool, the AI had to be explicitly constrained to work from grounded demographic estimates rather than generating random inline JSON.
3. Tool-Calling Loops and Search Pollution
Attempting to resolve the data accuracy problem by enabling Web Search in local interfaces like Jan created a new set of failures.
When Web Search is active on small local models (like 7B or 8B parameter variants), web queries often return unstructured, long-form text blocks. Rather than extracting specific numbers, the model gets trapped in recursive tool-calling loops:
- Search for “Ministry of Health doctor gender data by state”.
- Ingest thousands of tokens of raw web text.
- Exceed context memory limits before processing the HTML code layout.
- Fail to generate the final file output.
+------------------------------------------------------------------+
| Search Pollution Feedback Loop |
| |
| [Prompt] |
| │ |
| ▼ |
| [Web Search Call] ──► [Returns Massive Unstructured HTML] |
| │ |
| ▼ |
| [Context Limits Exceeded] ───┴──► [Model Halts / Fails Output] |
+------------------------------------------------------------------+4. Engineering Solutions for Local LLMs
To successfully generate a full application using a local LLM setup without crashing the inference pipeline, developers must implement specific system constraints.
A. Bypass the Reasoning Block
When context budgets are tight and the output requires large amounts of repetitive code (such as inline SVG vector coordinates), disable the model’s <think> output.
In Qwen-focused setups, passing explicit system directives or using flags like /no_think forces the model to skip reasoning tokens and write output directly:
/no_think
Write a complete single-file index.html with inline SVG map paths and
JavaScript...B. Expand Ollama System Parameters
Local host applications like Jan run on top of backend engines like Ollama. If the backend context parameter (num_ctx) is not explicitly increased, the model defaults to a standard 2048/4096 buffer regardless of available VRAM.
Increasing the allocation via environment configuration expands the available execution window:
# Expand context buffer before launching the local server
export OLLAMA_CONTEXT_LENGTH=16384
ollama serveC. Separate Data Ingestion from Code Generation
Small local models perform best when given a single, clearly defined task. Asking a 7B model to act as a database, demographic analyst, SVG geometry engine, and UI developer simultaneously almost always leads to failure.
A far more reliable workflow involves separating the tasks:
- Provide the structured JSON demographic data inside the prompt directly.
- Instruct the model strictly to handle the SVG rendering and event-handling code.
Local LLM architectures like Qwen 3.5 running on consumer hardware (such as Apple Silicon M-series chips) offer impressive privacy and zero-latency performance.
However, they demand structured orchestration.
Building even a simple data visualization tool requires managing context windows, suppressing unnecessary reasoning tokens, and providing structured baseline data.
Local models are powerful rendering engines, but keeping them on track requires clear constraints and active pipeline management.
A good lesson for her.
In case we are meeting for the first time, come over here, it’ll be worth the roller coaster of articles that are gonna come up in the next few weeks.
If you’re an established writer, here are the brands paying for sponsored articles.
I do not use AI in my writings and you shouldn’t either. So, How did I go from 0 to 1000 here ?