The ‘Quantified Self’ revolution has led to millions of us wearing fitness bands of some kind, but there has been little evidence that this has led to enduring weight loss or cause any change in the growing obesity epidemic.
Now a new study by the Stanford University School of Medicine may explain in part why – the fitness trackers are just not very good.
They tested the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2 against gold standard medical equipment with 60 volunteers and found their accuracy left a lot to be desired, especially when it came to the important element of the calorie count.
None of the seven devices measured energy expenditure accurately, with even the most accurate device off by an average of 27 percent and the least accurate by 93 percent.The Fitbit Surge was the best of a poor lot, with an error rate of roughly 27%, while the Microsoft Band showed an error rate of around 33%, beating out the Apple Watch, with an error rate near 40%. The PulseOn performed the worst with a shockingly high error rate of 92.6%.
“People are basing life decisions on the data provided by these devices,” said Euan Ashley, DPhil, FRCP, professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford. “The magnitude of just how bad they were surprised me.”
The devices did much better when it came to heart rate, which is being directly measured by optical pulse oximetry. In contrast, calorie expenditure is being estimated by proprietary algorithms in each device.
“My take on this is that it’s very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height and weight, etc,” said graduate student Anna Shcherbina.
The study included sixty volunteers, including 31 women and 29 men, who wore the seven devices while walking or running on treadmills or using stationary bicycles. Each volunteer’s heart was measured with a medical-grade electrocardiograph. Metabolic rate was estimated with an instrument for measuring the oxygen and carbon dioxide in breath — a good proxy for metabolism and energy expenditure. Results from the wearable devices were then compared to the measurements from the two “gold standard” instruments and the results published online May 24 in the Journal of Personalized Medicine.
The take home message would likely be that while regular exercise is important to health, basing your portion size on regular weighing rather than your wristband would likely yield better weight management results.