心地とインターフェース

Lately, I have been spending a little less time operating interfaces myself. I notice it especially when working in admin panels or booking travel and accommodation for the occasional business trip.

I open an admin panel or website I have not used in a while and search through the menus, wondering, “Where is that setting?” Before I can do what I came to do, I have to understand how that particular service has organized its UI. I used to accept this as part of the process, but lately, having to “first learn how to use it” has begun to feel a little tiresome.

At times like that, before trying to find my own way through a complicated UI, I turn to Computer Use. With Computer Use, I can describe what I want to do, and AI looks at the screens of websites and apps, clicking and typing on my behalf.

Where to click, and in what order. I have started leaving the work of figuring out a complicated UI to AI.

In fact, I also leave much of the work of preparing Feel & Interface articles for Substack to Computer Use. Previously, I would open Substack, enter the title and body, create and insert images, and configure the delivery settings, working through each step myself.

Now, when I ask, “Send this article out on Substack in Japanese and English too,” the agent uses Computer Use to open the admin panel and fill in the necessary information. I only look at the screen when something needs checking along the way. By now, the agent may know its way around Substack’s admin panel better than I do.

Until now, using a new app or web service meant first learning how to use its screens. I think an easy-to-use interface was one that let us work through those steps with as little confusion as possible.

When designing UIs myself, I have thought about whether people can find their way to where they want to go, and whether the result of an action comes across clearly. Of course, those things will still matter. After all, Computer Use is operating existing websites and apps built for people.

But there are now situations where AI is the one directly operating those screens. Substack’s admin panel has not changed, yet the way I use it has changed quite a lot. I can send out an article without remembering where the publish button is or how to set an image.

An agent now sits between me and the interface.

Recent research has begun to explore this change. In a study of Computer Use Agents published in 2026, Apple organizes the UX of delegating screen operations to AI around questions such as how users give instructions, how the AI communicates what it is doing, and how much users can intervene.

Alongside designing buttons and forms, we are beginning to design how people can make themselves understood when asking an agent to do something, how much of its current activity to show, and when they can stop it or take over themselves.

The study also suggests that there is no single right approach: the amount of explanation and control people need varies with the task and the user.

Meanwhile, Microsoft Research’s “AI at your Fingertips” explores whether people can delegate work to AI without looking at a screen at all. The team built a ring-shaped device that lets users ask AI to do something by touching it and speaking, then vibrates when the task is complete.

For simple tasks, participants found it convenient to delegate without looking at a screen. For more complex tasks, however, they reportedly felt uneasy without visual or audio feedback.

It seems that not having to operate something and not needing to see what is happening are two different things. This feels familiar from my own experience with Computer Use.

I review an article myself before it goes out, but I am comfortable leaving much of the work of creating a Substack draft to the agent. With a flight booking, though, I would like it to find options and fill in the necessary information, but I would feel a little nervous if it went ahead with the purchase before I had checked the price, date, and time.

On the other hand, if it asks, “May I go ahead?” at every step, I find myself thinking it would be faster to do it myself.

I want to leave it to the agent, but look in now and then.

Even without watching the screen all the time, I can find out what it is doing whenever I wonder. At the important moments, I can check things and decide for myself. When I can leave things to it in that way, I feel more at ease while doing something else.

Lately, there has also been a move beyond having AI operate UIs made for people: websites themselves are starting to be designed with AI use in mind. WebMCP, a specification currently under discussion, is one example.

Computer Use, when it works by looking at screens, uses websites much as we do. It finds search fields and buttons and interacts with them one by one. With WebMCP, a site can tell AI directly, “Here are the things you can do on this site.”

On a travel booking site, for example, people would still see search forms and booking buttons. AI, meanwhile, could be given direct access to functions such as “search for flights” or “start a booking.” It could use those functions without having to find the buttons on the screen.

If approaches like WebMCP become widespread, we will no longer have to assume that people and AI operate the same screens in the same way. We will be able to design clear screens for people and accessible ways in for AI, each suited to how it works.

When designing UIs, I have always thought carefully about how people interact with them. But with an agent between people and the UI, and websites offering ways in for AI, there may be a little more to think about. How much should we leave to AI? What should we show along the way, and when should we ask people to make a decision?

I feel that my own sense of what is “easy to use” has been changing a little too. Being able to use something without getting lost still matters, of course. But not having to operate it myself in the first place has also been helping me quite a lot.

How we make room for time away from operating a screen may become another question to consider when thinking about interfaces that feel good to use.