Microsoft’s Snip Insights puts A.I. technology into a screenshot-taking tool

A team of Microsoft interns have thought up a new way to put A.I. technology to work – in a screenshot snipping tool. Microsoft today is launching their project, Snip Insights, a Windows desktop app that lets you retrieve intelligent insights – or even turn a scan of a textbook or report into an editable document – when you take a screenshot on your PC.

The team’s manager challenged the interns to think up a way to integrate A.I. into a widely used tool, used by millions.

They decided to try a screenshotting tool, like the Windows Snipping Tool or Snip, a previous project from Microsoft’s internal incubator, Microsoft Garage. The team went with the latter, because it would be easier to release as an independent app.

Their new tool leverages Cloud AI services in order to do more with screenshots – like convert images to translated text, automatically detect and tag image content, and more.

For example, you could screenshot a photo of a great pair of shoes you saw on a friend’s Facebook page, and the tool could search the web to help you find where to buy them. (This part of its functionality is similar to what’s already offered today by Pinterest). 

The tool can also take a scanned image of a document, and turn a screenshot of that into editable text.

And it can identify famous people, places or landmarks in the images you capture with a screenshot.

Although it’s a relatively narrow use case for A.I., the Snip Insights tool is an interesting example of how A.I. technology can be integrated into everyday productivity tools – and the potential that lies ahead as A.I. becomes a part of even simple pieces of software.

The tool is being released as Microsoft Garage project, but it’s open-sourced.

The Snip Insights GitHub repository will be maintained by the Cloud AI team going forward.