The M5 Ultra Mac Studio is insanely good. For how I work, it is the most compelling local AI machine I’ve used, and frankly it feels overpowered. I can have a capable model loaded, agents doing real work, Xcode open, and still use the computer normally. I keep trying to find the point where I have to choose between pushing the GPU and getting on with everything else, and so far I haven’t found it.
My configuration has 256GB of unified memory, but memory alone only gets a model onto the machine. The model needs to be fast enough to use, and the rest of the computer needs to keep up with it. M3 Ultra could run capable local models, but I could feel the desktop hitch when I pushed inference and tried to work at the same time. On M5 Ultra I have not had that problem in my use. The CPU is great, the GPU is great, and honestly it is hard to really push this thing. It's funny, a few days before the M5 Ultra was announced I was actually talking to a few friends and was telling them after nearly 18 months I was finally able to really hit the limits of M3 Ultra. Well, I have yet to hit that for M5 Ultra.
It's funny, if you've been following Creative Strategies for a bit, you may notice our website got a slight redesign recently. This was actually done using GPT 5.6 Sol and GPT 6 Astra recently, but both of these were running on the M3 Ultra Mac Studio I've been using for the past year or so. The reason I'm mentioning it is this was actually a full revamp of all the infrastructure hosting it, and part of that was creating a ton of simultaneous containers in OrbStack to test, convert data formats to retain articles when moving from WordPress to Ghost, testing and optimizing our self-hosting optimizations for Railway Docker containers, and help with design for it. Given each container needed its own database, worktree for real time updates for different UI changes, and needed to be available in real time for the agents, it was very memory and CPU intensive. I'm not sure any other computer I have would have been able to handle this as well! It did really push the 512GB memory M3 Ultra to its limits, believe it or not. This isn't super relevant to this report on the M5 Ultra, but it's an anecdote that I feel sorta explains where these ultra-level SoCs and workstations are incredibly useful regardless of where the inference happens.
You own the CPU too
There’s a lot of debate right now about owning a GPU for inference. Should you buy the hardware and run your own models? Should you rent the GPUs? Should you just use an API? Those are fair questions, and yes, owning the GPU can be great. What gets missed is that with a Mac Studio you also own the CPU that runs everything around the model. The agents, compilers, Xcode, the apps they are controlling, and the rest of your work have a place to run. For me, this is the best CPU and GPU combination I’ve had in one machine.
What makes the power of this machine so useful is that it is not locked into one job. I can push local inference, then move straight to development, or have the CPU and GPU working at the same time. As a consumer, that makes it my main computer rather than a dedicated AI appliance. For an enterprise, it could be a CPU hub, a GPU hub, or both. If models or inference software move in a different direction, the Mac is still an extremely capable computer. That is a lower-risk bet than committing the whole machine to one workload.
The GPU is obviously powerful too. Apple lists roughly 1.2TB/s of memory bandwidth for M5 Ultra, and its GPU includes Neural Accelerators for matrix operations. That memory bandwidth and compute are why a large local model can feel interactive instead of merely fitting in memory. Of course, the model and the software matter just as much. Apple’s specifications and its explanation of Neural Accelerators cover the hardware side.
Model architectures and inference frameworks change constantly. The machine stays the same. I can tune the software for a model I care about today, then use the CPU for development tomorrow, and I don’t need to predict exactly which model or framework will win three years from now to find the computer useful.
Ok, the local AI part
There are two things that decide whether I actually want to use a local model: can it do the work, and can it do that work quickly enough? Qwen 3.8 Flash Next has been very good for me in OpenCode 2. In my earlier agent testing I saw around 40 tokens per second on M3 Ultra, with waits of roughly 5–20 seconds before an agent turn got moving. On M5 Ultra it was around 75 tokens per second, with waits closer to 1–5 seconds. Over one turn that may not sound dramatic. Over an agent run with tool call after tool call, you feel every one of those waits. In practice this actually ends up being 5-8x faster on M5 Ultra.
Those are numbers from my agent workflow, not some universal speed claim for every Qwen setup. Context length, cache reuse, quantization, and the inference software all change the result. The 149-tokens-per-second decode figure I get into below is from later kernel tuning, so I wouldn’t put it on the same chart as the earlier 75-tokens-per-second agent observation without labeling the workloads.
I’ve also been able to keep multiple subagents doing real work at once. I haven’t done a controlled concurrency test yet, so I’m not going to give you a neat throughput number. What I can say is that the machine no longer feels like I have to choose between letting an agent work and getting on with my own work.
M3 Ultra could run these models. That wasn’t the problem. The problem was that I was spending too much time waiting for the next useful thing to happen. M5 Ultra changes that enough that leaving the model loaded in the background now makes sense for how I work.
The cache still matters. A lot.
Here’s the part that gets lost when people only compare tokens per second: an agent isn’t one continuous stream of text. It reads context, calls a tool, gets something back, and starts another turn. Whether it can reuse the work it already did on that context makes a massive difference.
Prefill is the work of processing the input before the model generates anything. If the server can reuse cached context, there is less to process on the next turn. If that cache breaks, the model has to chew through those input tokens again. It doesn’t matter that decode is fast if you’re sitting there waiting for prefill.
OpenCode 2 has given me better cache reuse in my setup, which is one of the reasons I’ve liked using it with Qwen 3.8 Flash Next. But I’ve still seen a cache break turn into a 30–40 second wait on a long context, even on M5 Ultra. You notice very quickly how much the software matters.
That’s also why I don’t think it makes sense to talk about the hardware as if the performance is fixed on launch day. The silicon gives you a ceiling, but the kernels and the rest of the inference stack decide how close you get to it. I’ve been working on that part directly.
Writing kernels for M5 Ultra
A model is a series of operations on numbers: matrix multiplications, attention, normalization, and so on. A GPU kernel is the code that actually carries out one of those operations, or a group of them, across the GPU. How you divide the work, reuse data, and move it through memory has a lot to do with the speed you end up seeing. MLX supports custom Metal kernels, so you can tune the operations that matter for the model you are running.
Take a matrix multiplication. A good kernel breaks it into tiles, reuses data in fast on-chip storage, and keeps the matrix hardware doing useful work. You can also combine compatible operations instead of writing an intermediate result to memory just to read it back. That’s fusion. Metal’s tensor operations can use the Neural Accelerators inside M5-class GPU cores, but you still have to feed that hardware well. The same chip can give very different results depending on the software.
This is one of my favorite parts of using the Mac Studio. I’ve had Opus 5 and GPT 6 Astra writing and optimizing Metal kernels for Qwen 3.8 Flash Next. Give the agents the Metal 4.1 programming guide, a token budget, and the actual M5 Ultra to test against, and they can write code, compile it, run the model, look at the result, and try again. The GPU runs the model, but the CPU is hosting the whole loop that makes the GPU faster.
In this round of tuning, I took prefill from around 2,400 to 4,000 tokens per second on the same machine. That’s a 66.7% increase without changing the hardware. Decode is around 149 tokens per second in this optimized setup. I’m still working on it, and I think there’s more to get out of the chip.
What does that mean for an actual task? At those prefill rates, the same input takes about 40% less time to process. If your agent spends a lot of time on long prompts or cache misses, that is a meaningful change. If it spends most of its time in tools, the overall improvement will be smaller. This is why an enterprise deploying a particular model has a reason to optimize for its own context lengths and workload, not just run one benchmark and call it done.
What I mean by owning the CPU
A good example of what I mean happened yesterday. I was on a plane, using the Mac Studio remotely while I had agents working on the Qwen model. The CPU was running the agent harness, compiling Metal code, managing the tools, and hosting the rest of the development environment. The GPU was where I was testing the optimizations. I could check in while traveling, but the interesting part was how much the machine itself was doing.
Then the optimization work finished and the machine moved on to my hobby projects in Xcode. It didn’t need a new purpose. It was already a macOS development machine with a great CPU, memory, storage, and all the apps I use. That’s the part I keep coming back to when people frame this as a question of whether to own a GPU. You’re also getting the host that can do real work before, during, and after inference.
On M3 Ultra I could sometimes feel the desktop hitch or lag when a model was running and I was trying to use the computer for something else. I haven’t noticed that on M5 Ultra in my use so far. I’m sure there are workloads that could push it there, but being able to leave a model loaded, call it when I need it, and keep working normally changes how useful the whole setup is.
For a consumer, this can just be your computer. For an enterprise, maybe it is a developer workstation, maybe it is a local inference node, maybe it is the CPU hub coordinating several jobs, or maybe it is all of those things. That flexibility matters because nobody knows exactly which models, inference frameworks, or deployment patterns will make the most sense in a few years. If one part of the stack changes, the other part of the machine doesn’t suddenly stop being useful.
And yes, the CPU is fast
My Geekbench 7.0.0 Pro result was 3,666 single-core and 51,448 multi-core, on macOS 27.0 build 26A428. The screenshot identifies the machine as Mac17,15. That is a strong CPU result, and one reason I don’t want to treat the CPU as a footnote in a discussion about inference.
A benchmark score is nice, but what I actually notice is the computer doing the work around the model. It is building code, running Xcode, hosting agents, and staying responsive while inference is happening. I’d need a comparable M3 Ultra Geekbench 7 run before using these scores to make a precise generation-over-generation claim.
One result that really stood out was building a Rust project from scratch. It took about six minutes on M5 Ultra versus 20 minutes on M3 Ultra. That’s a strange enough difference that I think something else may be going on, but it was striking to see in my own work.
I wouldn’t attribute all of that to the silicon without repeating it with the same project revision, Rust toolchain, dependencies, caches, and build settings. It is a workflow observation for now. Still, it’s another reminder that this CPU is doing real work for me, not just sitting there to launch GPU jobs.
It’s a whole computer
Another example: I used a local model through OpenCode to build a benchmark app for Apple’s Foundation Models framework in Xcode. The model ran locally, called tools in Xcode, wrote the app, built it, tested it, and kept going through that loop on the Mac Studio. Think about what is happening there. The model and the application it is building are on the same machine, and the CPU and GPU each have work to do.
I already have an M3 Ultra cluster that I use remotely through a personal inference proxy, so I’m not opposed to separating the model from my working computer. I can access that cluster from my M5 Max MacBook Pro or the Mac Studio. With M5 Ultra, though, I’ve found more value in having a capable model and the rest of my work on the same machine. Even when I’m away from it, I can reach the full computer remotely.
None of this means I think every agent should run on a local model. For many tasks, I still want a frontier cloud model doing the main reasoning, with a local model as a tool when that makes sense. But there are also jobs I want to run entirely on my own machine, and I don’t want to stitch together a bunch of smaller models just to make that possible. Having enough memory and speed to run one capable model when I choose to is a big deal.
The same goes for privacy. Local isn’t automatically the right answer for everyone. Privacy is about choosing where the data goes and who you trust with it. If a team needs an air-gapped system, or wants certain documents and tools to stay on hardware it controls, this Mac can host the model and the applications together. If the workflow can use a cloud model, it can do that too. The point is having the choice.
To be blunt, this thing is insanely good. Qwen is fast enough for real agent work, the CPU can build and host everything around the model, and I can keep using macOS as my main computer without feeling it slow down. I can use it for inference, make it a CPU development hub, or do both at once. There are certainly heavier workloads that could push it, and I want to try clustering for larger models, but in my work so far the M5 Ultra Mac Studio feels frankly overpowered.