Microsoft's Copilot AI assistant can now optionally see your entire desktop—maybe it will be able to finally explain what Windows error code 0x8007002c means
Rejoice, for Microsoft has upgraded its Copilot Vision AI assistant with access to your entire desktop in Windows 11. Mercifully, this is an opt-in feature limited to preview builds of Windows 11, not something being rolled out compulsorily via Windows Update, for now. So our AI bot overlords haven't quite taken over the PC.
Microsoft announced the move on the official Windows blog. "Now you can choose to share your whole desktop with Copilot. When you share your desktop (or any specific browser or app window), Copilot can see what you see and talk to you about it in real time. It can help analyze content, provide insights, and answer your questions, coaching you through it aloud," Microsoft explains.
Previously, Copilot Vision could only be given access to individual apps and windows. Microsoft says you can use the new capability to, "get tips on making improvements to your creative project, help with improving your resume, or guidance while navigating a new game."
To get the whole process going, you "click the glasses icon in your composer, select the desktop that you’d like to share, and ask Copilot to help with whatever you’re working on. To stop sharing, press ‘Stop’ or ‘X’ in the composer."
In theory, you should be able to ask Copilot Vision almost anything related to what's being shown on the screen using natural language voice prompts. Maybe it could be helpful for, I dunno, working out how to do a macro in Excel. I'll tell you what could be really handy, having those inscrutable alphanumeric Windows errors codes explained. If it can do that, colour me impressed.
How this wil fit in with Microsoft's plan to enable customisable persona for Copilot isn't clear. Apparently, you'll even be able to have Copilot appear like ye olde Clippy, the animated paperclip assistant from Office 97.
So, if you want Clippy riffling through everything you do on the desktop, this update may be for you. As that implies, there are obvious privacy concerns.
Microsoft doesn't exactly have a impeccable track record in this area, what with it transpiring that the Recall feature in Windows 11 was originally storing desktop snapshots as unencrypted images. But, for now, this is very much an opt-in for hardcore Windows fans and devs on the WIndows Insider program.
Speaking of which, this update for Copilot is part of Windows 11 build 1.25071.125 and currently limited to the Windows Insider program. You'll also need to be in a market where Windows Vision is enabled, which right now means only the US.