JOEL’S LOCAL AI FIELD NOTES · OCTOBER 2026
You just bought aMac Studio.
Now what?
M5 Max · 128 GB · Local AI
An M5 Max. 128 GB of memory. A plan to run local AI. Here’s what happened when I stopped comparing models and started asking them to do real work.
Read the story Copy the staged setup prompt ↗I started with
a speed target.
Then I changed
the question.
If you’ve just bought a Mac Studio M5 Max with 128 GB to run local AI, the first question is obvious: what should you put on it? I leased mine and started in much the same place. I wanted to know which models would run and how fast they would be. By early October, I was asking for the smartest model I could run at a minimum of 200 tokens per second. About 19 minutes later, I asked what would change if I dropped that to 50.
That gap turned out to describe the whole project. I wanted a fast machine, but I was willing to wait for better work. The hard part was finding the combination of model, runtime and agent that actually delivered it.
I had already explored running Kimi on a Mac Mini, clustering machines, and why agents such as Hermes and OpenClaw struggled to keep doing meaningful work around the clock. The Studio turned those questions into experiments.
These were targets I asked about.
They weren’t benchmark results.
“Maximizing output,
business outcomes,
even if it’s slower.”
The goal became easier to state than to measure.
I wanted agents that could make useful files, work through coding tasks and keep going without losing the thread. Reliable tool use was mandatory. A convincing answer in a chat window was only the start.
“It fits” is only
the first question.
The model isn’t the only thing
living in memory.
My early questions were basic. Which version should I download? What does all that memory buy me? I also explored moving to 256 GB. That was an upgrade question, not an upgrade I had completed.
A model download is only part of the budget. Once it runs, it shares memory with the conversation, the operating system and the other work on the machine. A model that answers a short question can still struggle when an agent has been working for an hour.
That changed how I thought about 128 GB. The useful question was how much room remained after the model loaded. Try the memory illustration in the playbook.
MLX, the serving runtime, prompt formatting and the agent harness all affected the result. The harness manages the conversation, calls tools and decides when to compact an old session into a summary. I couldn’t change one piece and assume everything else was equivalent.
With agent assistance, I started building a local-model lab inside Agentic Workers. Now I could run tasks, save what happened and inspect the artifacts afterward.
Fast tokens.
Real files.
Mixed results.
The lab gave models jobs I actually cared about: produce an operating plan, build a spreadsheet, make a landing page, synthesize material and create an interactive scene.
I quickly found several different clocks. Prompt processing happens before the answer starts. Decode speed measures token generation. End-to-end time also includes processing and the work around the model. A task may involve several calls, tools and retries.
A single tokens-per-second number hides most of that. So I started opening the files.
Qwen3-Coder-Next
4-bit configurationA landing page passed critical checks in 52.79 seconds. The operating plan had wrong totals. Spreadsheet “formulas” were missing their equals signs. The overall run was rejected.
Qwen3.5-35B-A3B
4-bit configurationIt produced files, but geometry and camera-control checks failed. Generation speed didn’t make the scene usable.
Qwen3.5-27B
4-bit configurationThose three artifacts passed their critical checks. Synthesis and the NYC scene still failed. Useful evidence for particular tasks, with plenty left to fix.
Two agents. One shared GPU.
The synthesis task really did launch two analyst children, but their GPU calls were serialized. One child hit its iteration limit without saving its note. In another test, a parent exited while its background children were still running, killing the work.
Making the parent wait for its children was an orchestration repair. Crediting a different model would have been the wrong lesson.
A faster answer.
How much faster
to useful work?
Qwen3.5-27B gave me a baseline worth improving. One experiment kept the same weights and added DFlash: a helper proposes tokens that the main model checks before accepting them. That’s speculative decoding.
The paired probes showed a real improvement: 29.3 to 63.9 tokens per second during generation. The end-to-end rate went from 17.1 to 24.9. The first number more than doubled; the rate including overhead improved by about 46%. Compare the two clocks and exact test conditions.
The smaller end-to-end gain mattered just as much as the bigger decode number. The configuration was applied to serving, but I kept its qualification narrow.
The model list kept growing. The important distinction was whether I’d downloaded a candidate, tried it, used it as a judge or actually measured useful work.
A non-thinking DeepSeek CLI pilot produced an actual website in a 203.80-second attempt. Separate browser review confirmed desktop and mobile layout, solution tabs, FAQ and navigation.
It also hit its 24-step limit, encountered a nested terminal-sandbox problem and missed the required notes. The page was real. The task was incomplete. This figure does not describe high-thinking Hermes sessions.
What happens
after the
short test?
I work with large repositories. I wanted enough room for a session to keep making progress, not to paste an entire repository into the first prompt. My goal was a growing budget of half a million to a million tokens.
The earlier 65K setting felt too small. I worried about frequent compaction and what survived the summary. A trace review found ten summary generations: nine aborted and one committed, plus two lock deferrals. The successful compaction reduced the next prompt and resumed tool use. The failures needed their own explanation.
Eventually I configured 262,144 tokens with compaction at 80%. That was a larger setting, but I had only passed a short test. A reliable full-window coding session was still unproven. So was the 500K–1M goal. Explore what a configured session budget means.
These issues didn’t erase genuine arithmetic and task failures. They changed how I read the scoreboard. To choose a model fairly, I had to inspect the model, harness and evaluator together.
Then I asked it
to bake bread.
Local AI, inside Old School RuneScape.
GameBench made usefulness much more concrete. I built a local dashboard, terminal chat and a RuneLite overlay showing the chosen action, then used Tutorial Island to test the loop.
Pi was the player interface. A Qwen model planned. A quantized Clef model chose bounded actions from observed game state. A RuneLite bridge carried those actions into the game. Hermes was the separate development framework; during the latest tutorial work it was paused while Codex handled repairs.
Full-size Clef and Qwen couldn’t safely remain loaded together under that setup. A roughly 15.2 GiB, 4-bit Clef build let both stay loaded, taking turns for inference. Quantization bought room for a system, not just one chat model.
The catch was real.
The completion test was wrong.
A fishing run acquired Raw shrimps, but its test expected a later cooking milestone. Genuine fishing progress still appeared as an incomplete run. The original receipt stayed incomplete.
Milestones were confirmed through fresh game observations across multiple attempts. This illustration is not a live game or a recording. There were failed, incomplete, interrupted and takeover runs. Full Tutorial Island completion was not demonstrated.
At another point, the world produced “Bread dough” while the completion check expected “Dough.” Later observations confirmed cooked shrimp, entry to the chef’s area and baked bread at stage 170. Those observations didn’t turn every earlier attempt into a clean success.
A small model making a good constrained move doesn’t establish that it can modify a large repository. It can still tell me where a smaller local model belongs in a larger system.
What I’d do
first, today.
I’d start with a job small enough to check: a spreadsheet whose totals I know, a page with a few required interactions, or a code change with a test. I’d save the prompt, configuration and output. Then I’d run the same job again before changing the model.
That is the setup I can recommend building: a repeatable way to discover what works for your tasks. My own daily-driver decision is still open. I have useful business artifacts, measured serving improvements and observed game milestones, alongside unfinished runs and real failures.
The first milestone is simple: a local model completes one useful task, you can verify the result, and you know how to stop and restart it. Everything that follows has a baseline.
01What am I giving up for that speed?
02Does the agent finish the job?
03How much intervention does it need?
04Can it carry the work through a longer session?
Start with one useful result.
Build from there.
Make your
first session
count.
You don’t need to reproduce every experiment above. You need a working local stack, a task worth doing and a way to tell whether it worked. These are the decisions I’d make before downloading the biggest model I can find.
This is a setup guide, not a claim that one model is best for every 128 GB Mac. The instructions below deliberately ask your agent to verify today’s compatibility before installing anything.
- 01
Choose the job before the model.
Write down one input, one expected artifact and a pass/fail check. “Make a spreadsheet” is vague. “Create a sheet whose formulas reconcile to these known totals” gives you something to test.
- 02
Inventory the Mac you actually have.
Confirm the chip, memory, macOS, free storage and existing servers. Leave space for model downloads and the rest of your work. My machine is an M5 Max with 128 GB; your exact configuration still needs checking.
- 03
Start with one compatible stack.
Choose a supported model format and quantization, then a runtime and interface that support it. Get a short local reply before adding agent tools. A GUI can simplify the first session; a scripted setup can make repeated tests easier. Either route still needs verification.
- 04
Give the agent a small, safe workspace.
Test file reading and writing in a disposable folder. Check the endpoint really uses your local model. Keep the API on localhost and sensitive accounts out of the first test. A chat reply doesn’t establish reliable tool use.
- 05
Keep the receipt. Then improve one thing.
Save the model, quantization, runtime, thinking mode, context, prompt and resulting artifact. Record total time and intervention as well as decode speed. Repeat before increasing context or adding another model.
What does 128 GB actually buy you?
Unified memory gives the CPU and GPU a shared pool, but a model’s download size is not its total running cost. Weights, KV cache, temporary allocations, the OS and tools all need room. Quantization reduces weight precision and usually their size; the effect on quality must be tested.
Illustration only: a 128 GiB accounting budget, a fixed 16 GiB reserve and an assumed KV rate of 0.0625 GiB per 1,000 tokens. Actual cache use depends on architecture, precision, runtime and workload. This is not a measured fit test or a hardware prediction.
Model
The learned weights and architecture. A quantized copy uses a particular representation of those weights.
Runtime
The software that executes the model. A server exposes it through an API. Supported formats and flags vary.
Harness
The agent loop around the model: prompts, tool calls, permissions, retries and compaction.
Context
The token budget available to a call. Useful long-session capacity also depends on output allowance, cache, memory and retained task quality.
Which speed number should you compare?
Use total task time and artifact quality to choose a setup. Decode throughput helps diagnose one part of it. The paired experiment below held its stated configuration fixed; the other story measurements describe individual runs, not a model leaderboard.
weighted decode speed
Decode measures how quickly the model generates tokens. It leaves out initial prompt processing and other overhead.
Same Qwen3.5-27B weights; non-thinking; short, ~6K and ~49K prompts; 65K ceiling; 768-token outputs. Weighted decode: 29.3 → 63.9 tok/s. End-to-end rate: 17.1 → 24.9 tok/s. Small financial and real-tool checks passed. This was not a new run of the full business suite.
Qwen3.5-27B · 4-bit
Saved artifacts passed the critical checks for the operating plan, spreadsheet and landing page. Synthesis and the NYC scene still failed.
Later DFlash probes improved speed with the same weights. They did not requalify the full business suite.
The wrong thing
to blame.
An old rollback server was still holding a large copy of the weights while the selected server tried to load. Clearing the idle duplicate resolved the conflict. Changing the model would have been the wrong diagnosis.
~5 sec → ~2.9 sec
Two short high-thinking tests with 4K and 6K prefixes finished faster with a warm cache. Useful for repeated context under that profile; not a universal speed claim.
Settings aren’t
evidence.
Deeper MTP drafting passed parity and tool checks but didn’t beat both depth-one controls. TurboQuant settings were accepted, yet the actual cache type hadn’t changed.
How much context is enough?
Start with the session you can validate, then grow it. A model’s advertised maximum, the server’s configured limit and a dependable working session are three different measurements. My later 262,144-token configuration was a setting, not proof of a successful full-window coding session.
WHEN THE FIRST RUN GOES WRONG
Find the failed layer.
The model won’t load, or memory pressure spikes.
Check available memory, the exact format/runtime compatibility and whether another server already holds weights. Inspect before stopping anything. Try a smaller supported quantization or context with headroom. Don’t start by changing system memory limits.
Chat works, but the agent can’t use tools.
Check the exact model’s tool support, prompt template, server protocol and harness configuration. Then test one real read/write operation in a disposable folder. A natural-language promise to call a tool is not a tool call.
Tokens are fast, but the task takes forever.
Separate prompt processing, time to first token, generation, tool waiting and retries. Compare cold and warm runs fairly. More agents may queue behind one shared GPU rather than make the work faster.
The agent says “done,” but the output is wrong.
Open the artifact. Recalculate formulas, run the tests and try the interactions. Also inspect the evaluator: my records contain genuine model errors alongside broken checks, missing judges and orchestration failures.
A long session loses the thread or stops.
Inspect compaction, output allowance, tool errors, locks and memory. Confirm whether a summary actually committed and the agent resumed. Reduce the scope and reproduce the failure before deciding a larger model or window will fix it.
COPY / YOUR FIRST ADMIN-AGENT BRIEF
Give your new Mac a careful starting point.
Paste this into an agent that can inspect your Mac Studio. It needs authorized computer or terminal access, access to current documentation, enough disk space and your participation in installation approvals. An ordinary chat window with no device access can help plan, but cannot install the stack.
The prompt starts with an inventory and a plan. Copying it does not run commands or grant access. It is designed to work in stages; successful setup may take more than one exchange.
QUICK ANSWERS
Before you close the tab.
Is 128 GB enough to run local AI on a Mac Studio?
My experiments produced useful artifacts on a 128 GB Studio. That does not establish that every model or long context will fit. Choose the exact weights and runtime, reserve memory for the session and tools, and measure the actual workload.
Which local model should I install first?
Choose a supported candidate for one task you can check, with room to spare. Qwen3.5-27B in 4-bit gave me useful business-task artifacts, but it also failed other tasks. I have not established a universal winner or a final daily driver.
Can I keep everything private and offline?
Local inference can keep model execution on your Mac. Downloads, external tools, cloud fallbacks and remote APIs can still move data elsewhere. Verify each component, keep the server on loopback and test your intended offline workflow rather than assuming “local” covers everything.
Primary documentation to check before setup
Official references checked October 10, 2026. Use the current documentation for your installed version; my experiment configurations are not universal installation instructions.
- Apple: Mac Studio M5 Max / M5 Ultra specifications ↗
- Apple machine learning research: MLX and unified memory ↗
- MLX-LM: supported models, installation and context tools ↗
- LM Studio: local API server ↗
- LM Studio: listening address and server exposure ↗
These links document their respective products. They do not independently verify my private benchmark results or endorse every experimental runtime mentioned above.
These are results from my own setup and saved experiments, not a controlled cross-hardware leaderboard. Models, runtimes, thinking modes, output budgets and evaluators changed between runs. The paired DFlash comparison held its stated setup fixed; the other figures describe individual probes or tasks. Decode speed and total task time measure different parts of the experience.
The memory and context widgets are illustrations. The game scene depicts verified milestones, not a continuous successful run. Story and evidence reflect the project as of October 10, 2026.