Got A Lot Of Products and Do Not Want To Give Your Data To Anyone For Spam Filtering? You’re welcome.
I have a bunch of products. If you maintain multiple community touch points like Telegram groups, support emails, discord servers, you know the pain.
Spam isn’t just annoying, it’s a constant evolving tax on your moderation team’s time.
We needed an intelligent filter across all our products.
The obvious route is routing everything through OpenAI or Anthropic, but passing every single inbound message to a paid API introduces latency, privacy concerns, and unpredictable costs.
So, this weekend I decided to bring the brain in-house.
The plan was to build a zero-cost, highly capable spam filtering engine using Qwen3.5 9B, hosted locally via Ollama, secured by Tailscale, and orchestrated by a custom Python gateway.
I’ll just write off the steps.
Firstly, Apple’s M1 pro is what I chose as the machine I’d like my whole setup in, as it’s an old machine, biting the dust for too long :’)
Instead of renting a massive GPU rig, you should always utilize existing hardware. Moreover, Apple’s unified memory architecture is a cheat code for local LLMs.
And by using Ollama, the heavy lifting of llama.cpp and Metal (MTL0) acceleration is handled out of the box.
Installing Ollama is super simple.
Next,
We wanted a model smart enough to understand the nuance between a frustrated user and a sophisticated crypto scammer, but small enough to run economically.
We did experiment with Gemma, Qwen and the in-house Hermes models initially.
Alibaba’s Qwen3.5 (9B parameter model) hit this sweet spot perfectly.
Looking at our backend, the Qwen3.5 9B model took up about 5.2GB of disk space.
Leaving plenty of overhead on our 16GB machine.
Running an LLM locally on a dev machine or a Mac Mini is great, but how do your production servers talk to it?
Exposing port 11434 to the public internet is a security nightmare waiting to happen.
Enter Tailscale.
By installing Tailscale on the host machine and all our production nodes, we created a secure, peer-to-peer mesh network.
The M1 Pro got a static internal IP. All we had to do was bind Ollama to this specific Tailscale interface and open up CORS for our internal services:
Now, any of our product backends, whether running on AWS, DigitalOcean, or Vercel can safely POST payloads to http://100.69.149.35:11434 as if the LLM was sitting on the same local server rack.
No complex reverse proxies, no exposed public ports.
Now, it was time for:
The Orchestrator or Building the Hermes Gateway
You can’t just plug raw platform webhooks directly into an LLM.
Why?
Real-world platforms are messy.
We built a centralized Python dispatcher we call the Hermes Gateway.
Instead of every product writing its own LLM integration, Hermes acts as the middleman. So, the middle man is the code where these messages and mails including spams are consumed, formatted and then sent to the model.
We started by hooking up our Telegram bots.
Hermes also handles the dirty work:
- Platform Polling: It maintains the connection to Telegram via long-polling.
- Resilience: If you look at our production logs, Telegram’s API drops constantly. We wrote Hermes to handle
httpx.ReadErrorandBad Gatewaytimeouts, automatically falling back to sticky IPs (149.154.166.110) and reconnecting without dropping inbound messages.
- Batching & Prompting: When a message comes in (e.g.,
user=Shashwat chat=149177630 msg='hey'), Hermes wraps it in our custom spam-detection system prompt and fires it across the Tailscale network to the Qwen model. - Action: Based on Qwen’s response, Hermes either lets the message through to the end user/support team, or instantly blackholes it.
By isolating the LLM on a Tailscale IP, I created a single brain that all our microservices can securely access.
When a user interacts with the Resend email setup, or chats with our Telegram bot, the payload is quietly fired to our M1 Pro, analyzed by Qwen3.5, and flagged or cleared in seconds.
Also, we aren’t paying per-token.
We aren’t sending our users’ raw chat data to a third-party cloud provider either.
This is still pretty raw and in-testing. We do need to have a robust, highly capable spam filter.
So that I can say this with pride:
Local AI isn’t just for tinkering anymore, it’s ready for production networking.
In case we are meeting for the first time, come over here, it’ll be worth the roller coaster of articles that are gonna come up in the next few weeks.