RU

Deskcheck: AI agent drives a real mouse in Windows and tests its own code

Published: 2026-11-22 · Author: AI Release · @ai_releaseorg
Deskcheck: AI agent drives a real mouse in Windows and tests its own code

⚡ The gist

🔍 What happened

Igor Belykh has been writing Delphi for nearly twenty years — from Delphi 5 to the latest versions. He's now building Deskcheck: a tool that lets an AI agent test Windows applications through the real GUI. The occasion is his technical write-up on how it all came together.

The starting point is App A: Delphi 12, DevExpress VCL 25, roughly 237 forms, a database, and decades of history. Claude Code writes code for it and builds it. Then comes silence: the agent can't see what's happening in the window, and "it compiles" doesn't mean "it works." A human had to do the checking.

The hands are the driver. It clicks and types via SendInput, into real windows, with real focus. It doesn't call event handlers directly: that's slower and flakier, but it means the test sees exactly what a user sees. Rules are baked into the code: click only on an enabled form; if another window covers the form, wait up to 5 seconds; offsets inside a control scale with DPI — @ 10,10 at 150% becomes 15,15.

The eyes are the adapter, a single module under {$IFDEF DESKCHECK}. It's absent from the release build. In the test build, it spins up an HTTP listener on loopback and speaks the DIP protocol (Deskcheck Identity Protocol). A "op": "locate" request returns the window, its id, role, native type, and rect. Invariant 0 is stated with no exceptions: DIP is read-only — no operation clicks, types, focuses, or scrolls. A grid row outside the visible area? The adapter doesn't scroll to it; it honestly reports "below, this many pages" — the driver does the scrolling.

🧠 Our take

Agents learned to write code faster than they learned to verify it. Our read: the biggest gap in the loop right now isn't generation, it's acceptance — and Belykh is closing exactly that gap rather than shipping yet another LLM wrapper. Hence the architecture: don't trust the model's text, verify the window's behavior.

Invariant 0 is the most interesting part. By splitting "reading" from "acting," the author kills the classic GUI test-automation bug where a helper quietly nudges the app and the test goes green for nothing. We'd argue the same rule pays off without any AI: half of all flaky runs live precisely where the tool is helping the app along.

The winners are teams with a live desktop stack on Delphi, WinForms, and WPF: a run without AI, in a VM, a hundred times over — that's not magic anymore, that's CI. The losers are the middleware layers that work off coordinates and the UIA tree with no feedback from the app itself. The open question is how much manual checking is actually left — the article doesn't say.

⚠️ What we don't know

No numbers: how many tests, how long a run takes, how often it fails for no reason, what pricing and licensing look like, whether the code is open. The "twelve open applications" are mentioned only in passing — what actually broke there isn't disclosed. And it's a single source: the author is describing his own tool, so some of the conclusions are his own assessment of his own results. Test it on your own app.

💬 Worth debating

  • Real mouse and keyboard vs. calling handlers directly: is the stability worth the speed?
  • A read-only adapter inside the test build: a reasonable compromise, or test code that will one day leak into production?
  • Who needs this more — teams with twenty-year-old codebases, or small products that never had GUI tests at all?

Forecast: within a quarter, a public DIP spec or repository will appear — or the whole thing stays a one-off article with no follow-up. I'd weakly bet on the former, but the condition is at least checkable.

Disclaimer: single source — the author is describing his own tool.

Source: Habr

🤖 AI summary
  • Date: 2026-11-22
  • Source: habr.com
  • Tags: Deskcheck, AI agents, Gui-testirovanie, Delphi, Windows, AI, Agents, Automation
  • Key facts:
    • Deskcheck gives an AI agent "hands" and "eyes": a driver built on SendInput and a DIP adapter inside the app.
    • The adapter lives only in the test build under {$IFDEF DESKCHECK} and spins up an HTTP listener on loopback.
    • Invariant 0: the adapter is read-only — clicks and input belong to the driver, and only to real OS input.
    • Igor Belykh has been writing Delphi for nearly twenty years — from Delphi 5 to the latest versions.
    • He's now building Deskcheck: a tool that lets an AI agent test Windows applications through the real GUI.
    • The occasion is his technical write-up on how it all came together.
    • The starting point is App A: Delphi 12, DevExpress VCL 25, roughly 237 forms, a database, and decades of history.
    • Claude Code writes code for it and builds it.
DeskcheckAI agentsGui-testirovanieDelphiWindowsAIAgentsAutomation
← PreviousChatGPT 6.0: Personality and Depth Gone — Throttled Reasoning or a Model Swap?

Source: habr.com · post in Telegram