Is Your Fitbit Crap?

Let me cut to the chase: No.

Full disclosure: I own a Fitbit Flex™. I haven’t worn it in months, but I do like the thing. It’s a long, boring, irrelevant story why I’m not wearing it, so I won’t get into that. Just know that I am sincere when I say that I like my Fitbit.

I don’t think my Fitbit is telling me exactly how far I’ve walked or how many calories I’ve burned. I do think that it gives me a ballpark figure that can be used to compare one day to other days. I don’t think that it’s telling me exactly when I am sleeping, either, but the information about how restless I am some nights compared to others is interesting and even sometimes useful.

I also like that it buzzes on my wrist to wake me up in the morning.

Recording some basic information about my daily activity, giving me a reason to record what I eat, waking me up in the morning, and reminding me to get off my ass. These are not big goals and perhaps I don’t need a $100 gadget to accomplish them, but is the device bullshit?

Mother Jones sure seems to think so. Last week the site posted a piece titled “Science Says Your Fitbit is a Joke“. When I searched for more reports on the topic I found a Jezebel piece titled “Your Fitbit is Bullshit, Says Science“, and one on RYOT titled “Science Says Your Fit Bit’s Full of It“.

Since I don’t particularly trust any of these sites as a source of accurately-interpreted science news, especially those which appear to be trying a little too hard to find clever, edgy headlines, I was skeptical.

The Jezebel piece is easily dismissed as simply lazy blogging. The author links to Mother Jones, paraphrasing one of the main points of the piece (bold mine):

Mother Jones points to a new study showing that your iPhone or device of choice does just a good a job[sic], if not a better one, at doing things like tracking calories and measuring activity.

Except that’s not what the Mother Jones piece said. It reads:

But, according to a new study published Tuesday in the Journal of the American Medical Association (JAMA), the apps on your smartphone do the job just as well, or even better—at least in terms of measuring your steps and your calories.

So Jezebel incorrectly paraphrased Mother Jones. And they aren’t the only ones. RYOT wrote this:

A new study came out showing that iPhone apps and other (much cheaper) devices do a better job than the bands at tracking your activity, steps, and even calories.

Even assuming Mother Jones is correct, these paraphrases are not. Walking is, of course, an activity, but it’s only one type of activity. Mother Jones was correct in noting that smartphones can do the work of pedometers, but Jezebel and RYOT are overstepping (pun intended). Wearables such as the Fitbit Flex™ and Jawbone Up™ can track your activity while you are swimming, bowling, or doing jumping jacks. Can your smart phone do that? Well, I suppose if you’re holding it in your hand, it can, but who does that? Also, if you don’t sleep with your smart phone strapped to your body, it cannot track your activity during sleep.

But is Mother Jones right? Might smartphones do a better job? Well, let’s find out.

Although Jezebel linked to the original source, a press release by University of Pennsylvania School of Medicine, it does not appear that the author read the press release, nor did the author of the RYOT piece, who linked only to the press release (failing to even give Mother Jones credit for the thoughts expressed). There is no reference to smart phones doing a better job at anything, much less a list of activities. The only study which is discussed is the one described in the Mother Jones piece, which examined the accuracy of several devices at counting steps and found all to be similar.

But let’s get to the bottom line. Is it a fair assessment that science says these devices are bullshit? A Joke? Full of it?

First, the study discussed involved counting steps. That’s it. There is absolutely no comparison of different devices in regard to tracking calories and no examination of other activity or reports such as distance walked. The study was fairly well-designed–something that does not come across in the reporting. Although there were only 14 participants, each wore every device at the same time and each walked both 500- and 1500-step trials twice.

Here are the results from the 500-step trials:



The results of the 1500-step trials are nearly identical. Now, it appears that the Fitbit One™ and the Fitbit Zip™ are spot-on, and the fact that the error bars are practically non-existent is pretty impressive. Unfortunately, the authors failed to do any sort of statistical analysis at all. Instead, they stated these results:

We found that many smartphone applications and wearable devices were accurate for tracking step counts. Data from smartphones were only slightly different than observed step counts, but could be higher or lower.Wearable devices differed more and 1 device reported step counts more than 20% lower than observed.

Given the error bars for the rest of the devices, including the Fitbit Flex™, I’d believe that most did not differ from one another significantly, with the possible exception of the Nike Fuelband™, which apparently sucks. However, there are two very clear winners, with the Digi-Walker™ close behind. A simple statistical analysis would have confirmed this.

Regardless, it is very clear that smartphones do not outperform most of the wearables. What is really in question is whether people use the devices and use them to their advantage. This may be questionable, but it has hardly been decided by science and it has certainly not been decided in favor of tossing one’s Fitbit™. The JAMA article concludes:

Increased physical activity facilitated by these devices could lead to clinical benefits not realized by low adoption of pedometers. Our findings may help reinforce individuals’ trust in using smartphone applications and wearable devices to track health behaviors, which could have important implications for strategies to improve population health.

So there’s hope that devices, whether they are wearables or just smart phones, will translate to improved health for at least some users.

To be fair to Mother Jones, their piece is much more fleshed out than the other two, covering more than this one study. In my opinion, the Jezebel and RYOT pieces simply plagiarized Mother Jones. However, I saw nothing in the rest of the piece to warrant such harsh criticism of wearables.

Now after writing most of this I came across this wonderful piece on Mashable that was posted yesterday. I could have simply linked to it, saying “read this” because it’s pretty much what I have said, right down to a comparison to the game of “telephone” that I just edited out of this post.

In the end, I have to agree with its author, Chris Taylor, who opens with:

There are poorly designed scientific studies, and then there’s poor reporting on scientific studies by journalists who should know better.

Print Friendly, PDF & Email

1 comment to Is Your Fitbit Crap?