Stevie Bathiche Of Microsoft Surface Team Explains The Stylus Technology In Surface Pro 3 In Detail

During the Reddit AmA by Surface team from Microsoft, Stevie Bathiche from Surface team did a long reply on Reddit explaining the Stylus technologies available on the market and why Microsoft chose N-Trig over Wacom. Read his full response below.

Hi.. this is StevieB. I anticipated the pen digitizer question.. so I have been thinking about how to talk about it over the weekend. I hope you and other folks find it useful. I will try to answer the rest of your questions preceding. Below is about your Wacom and Pressure question.

There are 3 main types of pen digitizer technologies: Electromagnetic, passive capacitive (those conductive rubber tipped pens that trick the digitizer into looking like a finger), and active capacitive.

Electromagnetic works by putting a printed circuit board across the entire device, typically situated underneath the display and its backlight. This is a completely separate system from the touch digitizer that typically goes in front of the display. The circuit board has a bunch of planar coils that emit an electromagnetic field (like one side of a transformer). The Other side of the transformer is located in the pen itself. As the pen approaches these fields and coils it couples the EM signal and adds a load. This load is picked up across multiple coils and the position of the stylus is then interpolated. These field lines can emit 15mm or so above the display, and thus the mechanism for hover. Data is transmitted from the pen to the device (pressure and button data), by modifying the frequency content of the load. To get orientation there is typically a secondary coil/circuit.. and simple trig can extrapolate the orientation of the pen.. this is important for later to remove mechanical parallax. Among the three this technique has been around the longest.

Passive stylus, works by simply acting as an extension of your finger, by being a conductor to passively couple with the electrostatic signal from the transmit and receive rows and columns of the digitizer. This technique uses the same transparent conductive lines in front of the display to do both the finger and “fake” finger/stylus.

Finally, active capacitive solutions started coming into fruition in the early 2000s. Their mode of operation is to use the same transparent conductive lines as passive stylus does above, but rather the pen tip injects an electrostatic signal which is picked up by these touch capacitive lines. Think of the pen is a mini radio and the sensing lines in front of the display the device are little antennas. The junction(s) (where the rows and columns of the transparent conducting lines cross) which receives the strongest signal is correlated to the position of the pen. To do this, the pen typically needs a battery, but the pen can emit all kinds of signals from buttons, pressure, and other. We purchased a really fantastic active capacitive pen and touch technology a couple years ago: you may remember the company that did the CNN election boards.. Perceptive Pixel. This is no doubt the most fantastic pen technology around for large non-mobile screens. Active capacitive field is defiantly showing a lot of activity by a number of different touch makers.. cause it is a natural extension of touch technology and its integration with the touch solution.

Now let’s go into what makes a great stylus from a hardware perceptive :

1) Precision is king. The more accurate and consistent the pen tip is to the actual ink of the display the more natural and more accurate you as an artist and user are able to execute. a. The Precision problem is broken down into 3 categories:

i. Visual Parallax: from pen tip to ink on screen. This is where you think the pen tip is.

ii. Electronic parallax: from pen tip to digitizer and where the digester thinks the pen tip actually is.’

iii. Digitizer precision and linearity across the entire screen.

2) Feel and sound: should sound and feel like writing on paper.

3) Pen weight, feel, and ergonomics.

4) Consistent and accurate pressure sensitivity

5) Buttons for switching modes (erase, selection, and other commands)

6) Latency: the lag of ink behind the pen tip (highly app dependent)… good developers can keep this number to a minimum.

7) Palm detection so the computer can reject false touches.

8) Device Integration: how it fits, and the industrial design requirements.

Now that we have a bit of a background of the different popular pen digitizer options and the hardware characteristics of what makes a good pen. Let’s go into a brief pros and cons of each of these (I am gonna skip passive stylus as it is still offers an obvious compromise in experience for our current applications [for now]). But before we begin, please let me say first none of these 3 options are generally more superior than one another. It all depends on the application and even more importantly the implementation/execution of technology. The performance of any one of these technologies is highly dependent on how thoughtful, careful, the engineering put into by the device makers to integrate the digitizer system. I have seen some really poor implementations of all 3 of these solutions above by various device makers. Using the brand of technology does not guarantee performance in the least bit.

Electromagnetic Stylus:

1) Precision: This can be very good for electromagnetics but it is highly dependent on implementation. To get the corners working well, and avoid non-linearity, the digitizer sitting behind the display must extend slightly beyond the display. Also metal objects or magnetics objects near the digitizer or in front of even in the device greatly effect noise and performance. This highly constraints the device maker from having a certain boarder size, and also the types of materials they use on the device and in the pen. Because magnetic fields shift with the environment, you will see drift and offsets… The device maker must do a really good job of calibrating the device.. and if the user puts something infront of the device (say a case that has metal in it), then they must be able to do the same level of calibration. Outside of these constraints, EM pen can give very good results.

2) Visual Parallax: this just depends on thickness the cover glass is… and none of the technologies really have an advantage of disadvantage here.

3) Electronic parallax: because the EM digitizer is buried behind the display and the coils are not located the tip of the pen, the digitizer must calculate the orientation of the pen and translate position from that… this is really never perfect and will often be dependent on where you are on the display.. so it is not a single mathematical transform for all points on the display.. it can be highly complex.. most of the time, the simple route is what is taken.

4) Digitizer precision and linearity across the entire screen: best way to test this.. is take ruler and draw straight diagonal lines across the display. Note how the lines are never really straight… this is very hard to do.

5) Feel and sound: Typically today we work with various materials to change the coefficient of static and dynamic friction of the tip on glass.. but there are other techniques we are working on to make this even better no matter what pen technology is used.

6) Pen weight, feel, and ergonomics. Because it is magnetic based the pen cannot be made out of metal. EM stylus has all kinds of shapes and sizes… from really thin and uncomfortable (but can be docked) to ones that feel like a pen. The pro here is that the pen does not need batteries.

7) Consistent and accurate pressure sensitivity: generally known to do a great job. This is much more about how the pressure curve looks like than the number of bits… I will explain below.

8) Buttons for switching modes (erase, selection, and other commands): because the pen and modify the signals actively (powered by the coils).. it can communicate buttons and pressure information.

9) Latency: the lag of ink behind the pen tip (highly app dependent)… good developers can keep this number to a minimum.

10) Palm detection so the computer can reject false touches: does not really have advantage of active capacitive.. but over passive stylus it does.

11) Device Integration: how it fits, and the industrial design requirements: because the pen digitizer is a separate digitizer from the touch, this solution will add anywhere between 0.4-1mm in thickness, a few mm around the bezel of the device, and a few 10’s of grams of weight. It is a bit harder ti integrate into the device cause of the constraints around materials and mechanics.

Active Capacitive Stylus:

1) Precision: in the past I have seen some not so good implementations here, but I am so pumped to see our current results in Pro3. We really did move the mark here. The Pen is really much more precise, linear, and linear across the entire device. The first comment I hear from artists when they use the device, is how precise the pen is.

2) Visual Parallax: this just depends on thickness the cover glass is… and none of the technologies really have an advantage of disadvantage here. In Pro 3 we have dropped the optical parallax to .75mm.. this one of the lowest parallax I have seen anywhere for inking tablets. This means as you move your head around your pen tip, the pen tip stays closer to the ink.

3) Electronic parallax: because the antenna lines are just behind the cover glass (for us that is .55mm thick!) the electronic parallax is further reduced.. and this is one of the reasons our pen feels more accurate.

4) Digitizer precision and linearity across the entire screen: do the ruler test!

5) Feel and sound: We are using new materials to change the dynamic and static friction of the pen tip. The result is a more paper like feel. As an industry we can do better.. but it is going to have to go to a different mechanism.. more on that later

6) Pen weight, feel, and ergonomics. Because the signal emits from the tip of the pen, metal objects in the body do not effect performance.. this is why we were able to do a beautiful anodized aluminum pen, that feels like a high quality pen in hand. Now we do need a battery, but the advantage of a battery is that one can emit a more powerful signal for other functions.. like click note: click the top of the pen once and OneNote automatically gets pulled open (even over the lock screen [secured]).. and double click and you get the acetate layer for clipping out portions of the screen into OneNote… pretty neat!.. and you can hold the pen about 3-5 feet away to do that… and you cannot do such an experience if you did not have a battery.

7) Consistent and accurate pressure sensitivity: just as good as our previous implementations.. as good as they come in my opinion. More on that below.

8) Buttons for switching modes (erase, selection, and other commands): because the pen is powered it can emit all kinds of commands via its emitted signals (buttons, pressure information, click note).

9) Latency: the lag of ink behind the pen tip (highly app dependent)… good developers can keep this number to a minimum. We have one caveat during hover.. while our latency is still best in class when you are inking, you may notice a bit of a lag during hover.. but only during hover mode..

10) Palm detection so the computer can reject false touches: about the same as EM.

11) Device Integration: Active capacitive digitizer are integrated into the touch controller, and use the same touch sensing lines. This is a fantastic form of integration, which makes for a thinner and lighter device. Also there are less restriction on materials.. for example our Type Keyboard clicks into the bottom portion of the device bezels via magnetics.. this would really be a bad thing for an EM digitizer.

Pressure I have heard a few folks worry about the 256 pressure levels vs. 1024… One can claim absurd amounts of resolution 10,12,14,16 bits.. whatever.. but in the end eventhough system is spitting out a 10 or 16 bit number does not mean there is a 10 or 16 bits worth of useful information there.. just like an overspeced digital camera.. the sensor is 20 megapixels.. does not mean the resultant image is 20 megapixels worth of information. You can do this experiment yourself.. I did it over the weekend to try to prove a point to my friends: I took the best known a EM based device I know of and compared it to Pro 3. I started by first downloading and installing a Microsoft PowerTool software called “digiInfo”.. this allows you to record and see Windows messages… I set the software to record pressure on both device. And then a built a small rig to hold the stylus above the digitizer with a about 50 grams downward pressure. Recorded the data of the static pressure.. imported to excel and did some statistics.. here is what I saw: the 1024 pressure tip static-pressure-number had a standard deviation 3 times greater than that of the 256 pressure tip. In end, the performance was the same.. eventhough one had 2 bits less reported info. This makes a lot of sense.. let me put it another way.

The Pro 3 pen measures from 10grams-400 grams of pressure and maps 256 levels to that… the mapping is nonlinear.. cause the human hand force activation is non-linear… but one can approximate about 1-1.8 grams per level. The 10 bit pen.. goes from 10-500 grams.. and supposedly does about ~0.4 grams. Think about both those numbers and that is both super super sensitive.. the best weight scale I have can do .1 gram increments…. The only reason it works is cause it averages the heck out of the numbers which adds a considerable amount of lag.. this lag one cannot do on a stylus.. so you are stuck with a nosier signal comparatively in a stylus. With any new stylus there is a difference in the force curve that you have to get used to… and that is likely what people will notice.. not the difference in bit resolution. We are going to make that easier for you by later giving you a piece of software that allows you to map your own force curve! I encourage you to get one of those scales and try to control it to the .1 grams.. will shed some light on the topic. The feedback from artists I have heard.. is that they do not see a difference.. and that is cause really the resultant info is not different.

WinTab: yes we have wintab driver support. See the link below to download and install it for pro3. In the future I hope apps start using the more modern APIs.. Wintab is old and outdated.. adds latency, and inserts itself in the pen path.. http://www.ntrig.com/Content.aspx?Page=Downloads_Drivers select the windows 8.1 option