DeskMind reads the screen, works out the next step and acts, all with a small model on your Mac. When a task could mean two things, it asks you instead of guessing.
A 0.8B model decides each step and hands the unsure ones to a 4B. Both run on your Mac: no cloud round-trip, no per-step bill.
Each step is a multiple-choice question: the model scores every option instead of writing text, so each one gets a probability. Unsure steps go to the 4B or to you, and any agent can call it through /v1/systemone.
Eyes, Brain, Hands and the Mac app are all open source, along with Bench, which grades them. Every result lists its sample size, so you can reproduce it.
One loop, three parts, packaged in one app. A separate bench measures the whole loop on a real desktop.
Recorded with the released model. Waits are shortened in the videos; nothing else is cut.
“Find Lisa Wong’s order in records.txt and add it to ledger.csv as Date, Customer, Order, Amount, then save.”
Guessing would write the wrong row. Asking costs one sentence.
“Open NetEase Cloud Music, search Billie Eilish’s BIRDS OF A FEATHER and play the live version.”
No API, only pixels: it reads the screen and picks the live track from look-alike results.
“Open parts.csv from the attached folder, append the table’s four rows sorted by Qty from high to low, then save and close it.”
Across two apps: read in Safari, open the file from a folder, write, save, close.
Measured on one Mac (M4 Pro, 48 GB) with the released 0.8B → 4B router. Small samples; we say where it still fails.
Graders check the final state of files and apps. Every attempt counts, environment failures are listed, and the bench is open.
All results and methods →Where these numbers come from, with caveats →The app itself only goes online to download the models. After that, every decision runs on your Mac and the model servers listen on 127.0.0.1 only. Apps it operates for you, such as Safari, use the network as usual.
Apple Silicon with macOS 15 or later. The decision models use about 7 GB of memory while running; Macs with less than 12 GB get a warning.
About 5.3 GB of models on first run, checked file by file. If Hugging Face is slow, the app switches to the ModelScope mirror by itself.
It shows each step, asks before sending, deleting, paying or publishing, stops on ⌘. and pauses when you touch the mouse or keyboard. It can still make mistakes, so start with tasks you can check.
Yes. The app, code and models are free. Code is Apache-2.0; model weights follow the terms on each model card.
得心，应手。 Download the app, or start with the models and the code.