How Microsoft's Windows ML machine learning could help apps anticipate your needs Credit: Mark Hachman / IDG

How Microsoft's Windows ML machine learning could help apps anticipate your needs

Microsoft is using machine learning to make its Windows apps a lot smarter. At a Technology Sneak Peek on Monday at Microsoft's Build developer conference in Seattle, executives used Windows Ink to demonstrate the potential of Windows ML, a machine learning API that debuted in the April 2018 Update to Windows 10.

Because Windows ML is an API, regular users will never encounter it. But the API provides the framework for developers to tap the power of your CPU or GPU to anticipate and act upon your needs, in Windows and Windows apps.

The executives at the demo made it clear that the Ink features they were showing off could drastically change, or might never be released at all. Still, the future they showed moves a few steps forward from the way Windows works today.

Case in point: Today, OneNote’s app for Windows 10 is smart enough to translate an inked math problem into a solution that can be solved. But each time you want to perform that action, you have to “lasso” that equation with your pen, signalling to Windows what you want it to do.

The Windows Ink teams showed off how Windows ML could address that and other issues with the inking experience. For example, ink could be used to:

  • Ink the values you want on a slider bar, rather than making you move the slider back and forth until you find the right value.
  • Allow Windows to recognize when users are inking math, a flowchart, or a table.
  • Change behavior when ink is used in OneNote, as opposed to Windows’ Sticky Notes or WordPad apps.
  • Alter the texture of a paintbrush application by speed.
  • Use the recognizer to give Office Lens OCR capabilities.
Windows Ink Mark Hachman / IDG

Windows ML could adjust the depth and texture of an ink brush on the fly.

The Office Lens OCR example shows the potential of the recognizer platform created with Windows ML. Office Lens is a mobile app that’s superb at, say, aligning a photo of a whiteboard for easy reading. But it can’t yet “read” the text that’s been written on it. The ability to “see” the real world has been a Microsoft priority, and machine learning could move that priority forward for Office Lens and more.

As for whether Office Lens would actually gain this capability, Kyle Beck, a program manager at Microsoft said that his team just designs the inking platform—where it ends up isn’t his decision. But, he added, Office Lens would be the “natural fit” for it.

Windows Ink Mark Hachman / IDG

This is how Ink will (or. at least, could) recognize inked equations. 

IDG Insider


«How technology helped foil the rhino poachers


Pillars of Eternity II: Deadfire review: A strong ship in shallow waters»
IDG Connect

IDG Connect tackles the tech stories that matter to you

Our Case Studies

IDG Connect delivers full creative solutions to meet all your demand generatlon needs. These cover the full scope of options, from customized content and lead delivery through to fully integrated campaigns.


Our Marketing Research

Our in-house analyst and editorial team create a range of insights for the global marketing community. These look at IT buying preferences, the latest soclal media trends and other zeitgeist topics.



Is your organization fully GDPR compliant?