My studio network setup for AI workflows: 3 hosts, 0 cloud

by

In recent months, I have completely overhauled my entire AI infrastructure. No more cloud APIs when processing sensitive data. Why? Because I have no desire to leave my clients' IP addresses with OpenAI or Anthropic just to write text or generate images. In my opinion, that is disproportionately risky.

I am now building this locally, in our Frankfurt studio. Here is the technical reality check of my setup, how I use it for client projects, and why it is worth the money.

The hardware: Why I'm not skimping on 4090s

I didn't buy everything all at once. But I did prioritize quality. My setup consists of three dedicated hosts running over my studio network.

Host .87 – The workhorse

This is my main computer for AI. Windows 11, but with an NVIDIA RTX 4080 SUPER with 16GB VRAM. This is my workhorse GPU. It's not the most powerful one out there (the 4090 has 24GB), but it is the perfect value-for-money winner. I use it for FLUX-fast-FP8 models (for image generation) and complex text processing.

Host .94 – The Linux Server

Ubuntu is running here. Hardware: RTX 3060 Ti (8GB VRAM). Why a 3060 Ti? It is sufficient to run large LLMs (Large Language Models) in 4-bit quantization without relying on the CPU. I primarily use this host as an Ollama server for text models. The latency is completely negligible for clients on the network.

Host .31 – Der „Always-On“

Windows 11, slightly older hardware, but always online. Serves as an interface and backup.

The Software: Ollama and the Models

The software is basically simple: Ollama as middleware. It is open source, runs locally, and connects with any software that communicates via HTTP.

Hier sind die Modelle, die in meinem „Production-Stack“ sitzen:

  1. GLM-4.7-flash (for text): I decided on the Flash model here. It is extremely fast. I measured a text rate of approx. 38 tokens per second (t/s) on the 4080-SUPER. This is faster than most cloud APIs, which are hindered by network latency and queues. For quick responses in chatbots or content generation, this is worth its weight in gold.
  2. FLUX-Schnell-FP8 (for images): When I need an illustration for clients, FLUX is the gold standard. The FP8 variant is significantly faster on the 4080 than the standard version, without the image quality appearing artificially flattened.
  3. F5-TTS (for voices): For video workflows, I use F5-TTS. It allows you to clone voices or generate new ones that sound extremely natural. The hardware requirements are high, but the quality justifies the effort.

Warum lokal? Das „No-Cloud“-Prinzip

The greatest advantage is not the speed, but the data sovereignty.

Ein Kunde in der Finanzbranche hat mir gestern seinen Jahresbericht als PDF geschickt. Er wollte eine Zusammenfassung und eine Analyse. Er hat mich gefragt: „Kommt das in die Cloud?“

Ich habe gesagt: „Nein. Es läuft auf .87.“ Er hat das nicht nur akzeptiert, er hat sich erleichtert angehört. Das Vertrauen ist etwas, das man nicht kaufen kann, aber durch technische Transparenz leicht aufbauen kann.

Effort vs. Benefit:

Yes, the effort is higher. I have to download models (often 4GB to 20GB per file), I have to maintain hardware, and I have to maintain the code.

But: If I generate EUR 500,000 in revenue from AI projects per year and save €50 in cloud fees for every project (which happens quickly with API usage), my hardware will have paid for itself after two years. Additionally, I don't have to store credit card details in a thousand different services.

Conclusion:

For small to medium-sized enterprises, a local 4080 or a combination of 4080 and 3060 Ti is perfectly sufficient. The cloud is great for experiments, but for actual business operations involving sensitive data, a dedicated server in the studio is the better architecture.