Hardware for local AI: which machine to choose

Local AI runs on hardware inside your company, so it pays to know what drives the choice of machine. Here is an overview without the technical fog.

Servers in a data centre — Photo: Christopher Bowns, CC BY-SA 2.0, Wikimedia Commons

What decides the machine

The size of the model. A larger model needs more memory, and memory is usually the first limit.
How many people use the model at the same time. One user needs less performance than a whole team.
The response speed you consider acceptable.
What data the model processes and where it may be stored.

A computer on the desk

For a small team a compact machine with a lot of shared memory is enough, for example an NVIDIA DGX Spark with 128 GB of memory or a computer with an AMD Ryzen AI Max+ processor. Large models run on the desk, with no server room and no noise.

A server with GPUs

For large models and many users, servers with graphics cards are used, from NVIDIA RTX PRO 6000 to HGX B200, or AMD Instinct MI350 cards with 288 GB of HBM3E memory. Plan for dedicated space, cooling and power.

Without buying: renting

A rented virtual server with a GPU (VPS), where the model runs on performance only for you.
A dedicated physical server with a GPU in a data centre: full control, steady performance, data on a machine only for you.

How we choose

From your data, team size and budget we determine the model and hardware, supply and set up the machine, connect it to your tools and verify on a real task that it all works. The price of the machine varies with the variant chosen, so we give it after a conversation. We verify each model's licence for your use case before deployment.