Microsoft might be planning to let AI loose on your local video and audio files with Windows 11 'intelligent media search'
Microsoft's Recall feature was, I think it's fair to say, not entirely well received. The idea that Windows 11 would take screenshots of your desktop regularly to provide you with a searchable history might have seemed useful to some, but privacy and security concerns meant Microsoft had to backtrack pretty quickly.
Well, how about Windows AI taking a looksie at your local media files? Twitter user @XenoPanther has been digging around in the latest Windows Insider Preview Build and has apparently found reference to something called "intelligent media search" (via TweakTown). According to XenoPanther, the feature is planned to allow search by "spoken words in your indexed video or audio files".
"By clicking 'I agree,' you consent to scanning the media files on your device. If needed, the required model will be downloaded and installed in the background.
"Once the AI model is set up, it needs to transcribe your media files and index them before enabling content-based search. We'll inform you once the process is complete."
It's important to clarify that this looks like the only current source for this potential feature, and there's a lot that's left unclear. Taking what we know at face value here, —and assuming the source is correct—this would likely be a CoPilot+ integration, which means it would likely need an NPU for the AI processing.
These changes may already exist in other branches but in 27695..."Intelligent media search" references have been addedIntelligent media search availableSearch by spoken words in your indexed video or audio files. By clicking 'I agree,' you consent to scanning the media files…August 30, 2024
What's also unclear is whether this would be something you could point at a specific file or folder, or whether you'd simply hand over all the media files on your machine to the AI and let it have it. The latter seems pretty impractical, as processing a large number of media files at once with full transcriptions would likely be very hardware intensive—although the wording suggests that might be the current plan.
Then there's the privacy concerns. Even as an "opt-in" feature, letting an AI loose on your local media content for indexing and transcription wholesale seems like a privacy and security nightmare. Pointing it at one specific file or folder, however, may well have some practical uses. Recording a meeting or briefing, for example, and then specifically targeting it for transcription, is something that third-party cloud-based services like Otter.ai have been doing for some time.
That being said, after the Recall debacle, it seems unlikely that users would feel all that comfortable about letting AI loose on potentially sensitive content stored on their personal machines—so hopefully it'd be easy to ignore for those that'd rather their local media remained untouched by the tendrils of AI. Personally, if it's a machine-wide scraping of all my media files? I'll be opting out, thanks very much.
At the very least, for now it looks like it might be a feature merely in planning, rather than something that's ready for an imminent release.