Skip to main content
Applications

07 · ACT

AI Desktop Operation

Desktop Operator

Letting AI read the screen and drive browser and desktop tools to carry out cross-application tasks — inside an explicit, auditable boundary.

01Overview

What this is

Many business systems have no API, so the work can only be done through the interface. This is a laboratory question: under what conditions is handing interface operation to an AI both safe and worthwhile? Boundaries, authorisation and the ability to stop are the substance of the investigation, not a footnote to it.

02Problem

The problem

Where there is no interface to call, automation must act through the UI — and UI actions come with no argument validation and no permission model. Without a boundary, a single misjudgement can be irreversible.

03Capability

Core capabilities

  • 01

    Explicit authorisation scope

    The applications and windows it may touch are declared in advance; anything outside is refused.

  • 02

    Human confirmation

    Irreversible actions — submit, pay, delete — must be confirmed by a person.

  • 03

    Full action log

    Every click and keystroke is recorded and can be audited afterwards.

  • 04

    Stop at any time

    A stop control is always available and halts execution immediately.

  • 05

    Restricted credential handling

    Passwords, payment details and identity document numbers sit on a never-type list; the moment such a field appears, control returns to the person.

04Flow

How it works

  1. 01

    Authorise

    The user declares the permitted scope and the forbidden actions.

  2. 02

    Read

    Identifies the current interface structure and its interactive elements.

  3. 03

    Plan

    Breaks the task into verifiable steps.

  4. 04

    Confirm

    Irreversible steps stop and wait for a person.

  5. 05

    Act

    Acts within scope, recording throughout.

05Position

Where it sits across the capability domains

InputOutcome
  1. 01 · SENSESense & InteractAlso touches
  2. 02 · UNDERSTANDKnowledge & JudgementNot directly involved
  3. 03 · CREATEContent & GenerationNot directly involved
  4. 04 · ACTAutomation & ExecutionPrimary domain
  5. 05 · ORCHESTRATEEnterprise OrchestrationNot directly involved
The primary domain defines what this application is responsible for; the domains it also touches show what it has to work with. The diagram describes position only — it is not a statement of delivery scope.
06Use cases

Use cases

  1. 01

    Data movement without APIs

    Transcribes information between legacy systems that expose no API.

  2. 02

    Repetitive UI work

    Runs structurally fixed, low-judgement interface tasks in volume.

  3. 03

    Process verification

    Tests in a controlled environment whether a process can be automated at all.

07Connections

Enterprise system connections

  • Browsers
  • Desktop office software
  • Legacy systems without APIs
  • Workflow engine
08Governance

Data, permission and deployment

This is not a system with unlimited authority. Scope must be declared in advance, irreversible actions require human confirmation, a stop control is always available, and all actions are logged. The system does not enter passwords, payment details or identity document numbers. It runs only in controlled environments.

09Availability

What is open today

An experimental project, not open for general use. Current work is focused on boundary definition, failure modes and audit mechanisms — whether this becomes a product depends on those answers.

Experimental

Apply for access

Want to know what this does in your own context?

Tell us the scenario and the systems already in place. We will start by judging whether it is worth doing at all, then talk about how.

A laboratory investigation into boundaries and risk. No productisation timeline is promised.