Microsoft says Windows 11 Edge Ink API improves ink latency on the web by up to 240%

Today Microsoft announced the preview availability of an enhanced inking experience in the latest Dev Channel builds of Microsoft Edge. Microsoft first introduced these enhancements during Build 2021, showcasing how this new web API, which has been implemented upstream in the Chromium open-source project, greatly reduces the latency between the tip of a physical stylus and ink as it’s drawn on the screen.

From their initial experiments on Windows 11 Insider Preview Builds, they have measured up to a 240% improvement in latency, which is highlighted in green in the animation below:
The new and improved inking experience on the web is now available by downloading the latest Windows 11 Insider Preview Builds and running the code sample presented on Microsoft’s blog post on the latest Microsoft Edge Dev Channel.

Technical Details

In Chromium-based browsers today, pen events are first sent to the browser process, which in turn forwards these events to a web application’s JavaScript event loop. The time delay between when the browser process receives these events and when they reach the application can sometimes be significant, depending on the rest of the main thread, resulting in the latency seen when inking.

To improve this, behind the scenes of the InkPresenter implementation on Windows 11, Microsoft Edge is using a new Windows API that will work directly with the operating system’s compositor to draw additional ink strokes outside of Microsoft Edge’s application loop. Thanks to this API, instead of waiting to deliver the event to the web application via JavaScript, we can take these points and provide them to the operating system compositor as soon as we receive them. The compositor can then connect the points with ink strokes and draw these strokes in the next frame that is to be presented to the screen, dramatically reducing latency.
For operating systems like Windows 10 and Linux that don’t have this API, a polyfill implemented directly in Microsoft Edge will take over and provide predictive rendering for ink strokes beyond the last known trusted PointerEvent. This implementation is intended to work similarly to the Windows 11 API – it leverages the points that the browser knows about, along with some predicted points to draw an extension to the application’s ink stroke at the last moment. While the effect isn’t as powerful as the Windows 11 API, it can still provide a much-improved experience for users!

Looking Ahead

As mentioned in the introduction, the work to implement this API was done as an upstream contribution to the Chromium project and is turned on by default for any Chromium-based browsers.

Going forward, Microsoft will be continuing to tune the prediction values provided by the Edge-implemented polyfill for non-Windows 11 installations. They hope to continue to see improvements in latency as they adjust the values and algorithm that the prediction system is using.

To read more about the Ink API and track its progress along the standards track, you can visit the draft specification page on GitHub: Ink API.

Developers can read more about implementing it at Microsoft here.