I Tried to Build My Own AI. It Turns Out Data Centres Exist for a Reason.

Very empowering.

I admit, I fell for it too.

Except I didn’t exactly start small.

I bought this rather ridiculous machine with an AMD Ryzen AI Max+ 395, 128GB of LPDDR5X memory, a 2TB SSD and Radeon 8060S graphics.

All thanks to my UAE credit cards.
[WARNING : Doing this is injurious to your finances.]

Basically, one of those machines built for people who have decided that what their desk really needs is a small personal AI data centre.

On paper, it is absurdly impressive. The 128GB of unified memory means you can load models that would have been completely unrealistic on an ordinary consumer machine not very long ago.

So naturally I thought: excellent, I shall now build my own AI.

I installed Ollama and started experimenting.

At first it was genuinely exciting. You type ollama run, pull a model, and suddenly there is an LLM running entirely on your own machine. No API key. No cloud account. No mysterious server somewhere deciding whether it likes you today.

I tried smaller models first. They were quick, useful and surprisingly capable. For basic questions, summarising things, playing with documents and simple coding tasks, I could absolutely see the attraction.

Then, obviously, I started downloading bigger ones.

Because nobody buys 128GB of RAM to run the small model.

And this is where the education began.

The bigger models would run. That was never really the problem. Watching something enormous load into memory and actually answer questions locally is very impressive.

But then I started giving it the sort of work I actually give ChatGPT, Claude and other frontier models.

Long documents. Coding. Reasoning across several things at once. Large context. Going backwards and forwards over a problem. Asking it to rethink something. Asking it to analyse files and then produce something useful from them.

From “Can this model run?” I moved to, “Do I actually want to work like this every day?”

Those are two very different questions.

A model fitting into 128GB of memory sounds fantastic on a specification sheet. But capacity is only one part of the equation.

Once you go far enough down this rabbit hole, you stop talking about RAM and start talking about memory bandwidth.

Then GPU compute. Then quantisation. Then how much of the model is actually being accelerated. Then multiple GPUs. Then power. Then cooling. Then networking.

And somewhere around this point I realised that I was slowly designing a data centre because I was annoyed about paying companies that had already built data centres.

That was when the whole thing started looking slightly ridiculous.

It reminds me of heating.

Imagine deciding that communal heating is a bad idea because you want complete control over your heating infrastructure.

So every household builds its own little heating centre. Its own machinery. Its own spare capacity. Its own maintenance. Its own fuel supply.

Most of it sits idle for large parts of the day.

Technically, congratulations. You now have heating sovereignty.

Economically, you have recreated something the whole neighbourhood could have shared much more efficiently.

AI compute increasingly looks the same to me.

A data centre can buy enormously expensive hardware and spread that cost across thousands, perhaps millions, of users.

The GPUs do not need to sit idle because Amit has gone to teach Mathematics for six hours.

Someone else uses them.

When I go to sleep, someone on the other side of the world uses them.

The cooling is shared. The power infrastructure is shared. The hardware is constantly utilised. When better chips arrive, they replace them.

Meanwhile my beautiful 128GB AI workstation is sitting at home doing absolutely nothing.

And I paid for 100% of it.

This is why I have become slightly amused by the increasingly fashionable idea of: “Just build your own AI.”

What exactly are we building?

The model was built by somebody else. Ollama was built by somebody else. The libraries were built by somebody else. The processor was designed by AMD. The machine was built by GMKtec.

And after assembling all of this, I am still trying to reproduce, on one desk, a service whose entire economic advantage comes from sharing extremely expensive infrastructure between lots of people.

There are, of course, very good reasons to run AI locally.

Privacy is one. Offline access is another. Research, experimentation, sensitive data, specialised models and simply learning how all of this actually works are perfectly good reasons too.

And I will admit that there is something deeply satisfying about disconnecting the internet and watching a massive language model continue talking to you.

It feels slightly magical.

But for the kind of work I am increasingly doing with AI, I need considerably more compute than even this rather ridiculous little box can sensibly give me.

Could I spend more?

Of course.

A powerful NVIDIA GPU. Then another one. More memory. Better power supply. Bigger case. More cooling.

And then perhaps I could finally achieve my dream of spending CHF 10,000 to avoid a CHF 20-a-month subscription.

At some point you have to admire the logic.

I suspect this is why data centres will continue to win for heavy, general-purpose AI.

Not because local AI is bad. Local AI is actually becoming remarkably good.

It is because economies of scale are brutally difficult to beat.

Which brings me to the funniest part of my little experiment.

My brother-in-law has been talking about getting into AI development.

And suddenly the machine that makes questionable economic sense for me makes perfect sense for him.

For learning, experimenting with Ollama, running different models, understanding inference, coding and generally disappearing down exactly the rabbit hole I have just crawled out of, 128GB of unified memory is fantastic.

So after setting out to build my own independent AI infrastructure…

I appear to have bought my brother-in-law a very expensive computer.

Still cheaper than building a data centre in the tiny Swiss village that’s my home currently.