In recent months, I have completely overhauled my entire AI infrastructure. No more cloud APIs when processing sensitive data. Why? Because I have no desire to leave my clients' IP addresses with OpenAI or Anthropic just to write text or generate images. In my opinion, that is disproportionately risky.<\/p>\n
I am now building this locally, in our Frankfurt studio. Here is the technical reality check of my setup, how I use it for client projects, and why it is worth the money.<\/p>\n
I didn't buy everything all at once. But I did prioritize quality. My setup consists of three dedicated hosts running over my studio network.<\/p>\n
Host .87 \u2013 The workhorse<\/strong><\/p>\n This is my main computer for AI. Windows 11, but with an NVIDIA RTX 4080 SUPER<\/strong> with 16GB VRAM. This is my workhorse GPU. It's not the most powerful one out there (the 4090 has 24GB), but it is the perfect value-for-money winner. I use it for FLUX-fast-FP8 models (for image generation) and complex text processing.<\/p>\n Host .94 \u2013 The Linux Server<\/strong><\/p>\n Ubuntu is running here. Hardware: RTX 3060 Ti (8GB VRAM)<\/strong>. Why a 3060 Ti? It is sufficient to run large LLMs (Large Language Models) in 4-bit quantization without relying on the CPU. I primarily use this host as an Ollama server for text models. The latency is completely negligible for clients on the network.<\/p>\n Host .31 \u2013 Der „Always-On“<\/strong><\/p>\n Windows 11, slightly older hardware, but always online. Serves as an interface and backup.<\/p>\n The software is basically simple: Ollama<\/strong> as middleware. It is open source, runs locally, and connects with any software that communicates via HTTP.<\/p>\n Hier sind die Modelle, die in meinem „Production-Stack“ sitzen:<\/p>\n The greatest advantage is not the speed, but the data sovereignty<\/strong>.<\/p>\n Ein Kunde in der Finanzbranche hat mir gestern seinen Jahresbericht als PDF geschickt. Er wollte eine Zusammenfassung und eine Analyse. Er hat mich gefragt: „Kommt das in die Cloud?“<\/p>\n Ich habe gesagt: „Nein. Es l\u00e4uft auf .87.“ Er hat das nicht nur akzeptiert, er hat sich erleichtert angeh\u00f6rt. Das Vertrauen ist etwas, das man nicht kaufen kann, aber durch technische Transparenz leicht aufbauen kann.<\/p>\nThe Software: Ollama and the Models<\/h2>\n
\n
Warum lokal? Das „No-Cloud“-Prinzip<\/h2>\n