JUMP TO:

Whoop 3.0 Band & Platform In-Depth Review

DSC_5648

There’s arguably no product I’ve *ever* reviewed in over a decade doing this that has me as conflicted as the Whoop strap. It’s both fascinating and perplexing at the same time. It’s got enormous potential, and after four months of 24×7 usage, I’ve got a pretty darn good handle on what works exceptionally well, and what needs an exceptional amount of work.

At its core, the Whoop band/platform is an optical HR strap that does 24×7 monitoring and, in conjunction with the app, measures two core things: Recovery (via your sleep), and load (via your workouts/daily activity). It doesn’t show the time, it doesn’t track your steps, and it doesn’t make you a coffee. But it can track a tremendous number of other things via its daily manual entry journal. Everything from whether you spent the night on a redeye flight, to whether you took an ice bath – or even whether you did the horizontal shuffle (or a solo variant of that).

The overarching purpose of the Whoop platform is to tell athletes whether they need to take it easy (to improve recovery) or go harder (to increase strain). That’s it. It’s singular in its purpose. And like all single-tasker things, the bar for success is higher. When you only do one thing you have to do it really damn well. Else, nothing else matters. And that…that’s where the confliction with my usage of this device comes in.

Let’s get started. Oh – and yes, I bought this myself last summer. Since then I have been paying the usual required $30/month subscription since then – albeit I only started using it back earlier this year. Note, you *have* to commit to 6 months. So in effect, the device is $180 as a baseline, or $360 per year if you’re not quite sure you want to commit to it (if you do commit to a full year upfront, then it’s $288). Euro pricing is 25EUR/month and extrapolated with the same same 12-month/18-month savings.

image

Got all that? Good, let’s get rollin’.

What’s in the box:

DSC_5649

Once you’ve got the box unzipped, you’ll find the band with Whoop pod attached, the charging pod (plus micro-USB charging cable), and a carrying bag you’ll never use

Here’s all the items out on a table, with the included bag/pouch in the upper left corner.

Seriously, I don’t understand why they included a bag for a device meant to be worn 24×7. Whoop went to the trouble of designing the battery charging pod such that you *never* have to take the Whoop off your wrist to charge it (sans-cables). Why on earth would I need a travel bag for it?

Here’s a closer look at the charging pod and very short micro-USB charging cable. Every once in a while I see one of these types of cables that are flat on the end so that you think you can insert it either direction. You can’t. It’s just annoying. Good news: Just swap it out for any standard micro-USB cable. You have 300 of them around your house.

The Whoop pod slides into the battery pack/charger, which we’ll dive into in the basics section.

Next, here’s the pod:

 

Now, in addition to the base Whoop band, you can also buy (separately) the bicep band. The idea here being to potentially increase accuracy by moving the pod to your bicep for workouts. After seeing accuracy issues early on I went and bought this added band:

And while it might help for people with bigger biceps, what they don’t tell you is that it’s simply a longer regular strap. That’s notable because what I eventually realized is that it was far less work to simply use the existing strap and loosen it just enough to fit around my bicep. Given I’m more of a runner/cyclist, my biceps aren’t too big – so I can get away with this (in case you’re wondering, I’ve tried it in numerous spots, inside of arm, outside of arm, on upper forearm, on both sides, etc…):

I point that out because if the original strap is big enough to fit around your arm, it’s a ton easier to use than the super finicky process of removing the strap and re-doing it every time you work-out.

Ok, with that, onto the basics.

The Basics:

Before we start talking numbers like Matt Damon in Good Will Hunting, we should talk hardware. And that part is relatively easy. The Whoop strap is simply a band with an optical heart rate (HR) sensor on the bottom of it. There’s no display, no GPS, no dedicated buttons (though, it can recognize being tapped for pairing purposes). Think of it as a piece of jewelry more than a normal watch/wearable.

It should be worn about two fingers from your wrist, or basically, away (and not atop) your wrist bone. Normal stuff for optical HR sensors. Whoop also recommends you wear it on your non-dominant wrist, but honestly, that’s an unrealistic request. Most people are still wearing a watch, and most people tend to wear a watch on their non-dominant wrist (in my case, left wrist). Plus, no other optical HR sensor watch/band/wearable maker has this requirement.

DSC_5630

The base kit you buy includes the fabric strap which is pretty easily adjustable. The clasp can lift up to effectively loosen the strap for quick removal. Though, the entire system is designed to never remove it.

DSC_5613

The main portion of the pod is made of matte plastic, while the clasp is metal. From a build quality standpoint there’s nothing special here – they aren’t using any premium materials (within the wearable world anyway). But, it does look nice, and sorta blends well with most things.

DSC_5614

On the underside is Whoop’s optical heart rate sensor, which features two LED’s (green light in this case). Companies vary on the number of LED’s and the specific light colors used (green is most common, but yellows/oranges/reds can also help with different skin tones). The quantity varies as well, with 2-3 LED’s being among the most common, but some companies go upwards of 10 LED’s (Polar).

DSC_5617

The entire pod is water-resistant spec to IP68, and Whoop says it’s specifically fine for swimming. I’ve worn it swimming, countless showers, and everything else water-related without any issues.

Next, the charging occurs via the small charging dock. Except this dock has a party trick: It’s got a battery inside of it!

DSC_5623 DSC_5624

So if you remember to charge the dock, then you can simply attach it to the Whoop pod without ever removing the Whoop from your wrist – thus allowing for true 24×7 charging! It clicks in using a set of 5 different contact points, and is pretty rock-solid there. I’ve slept with it charging a few times, as well as also ridden home (cycling) with it too, with the charging dock on there.

DSC_5633

The only challenge here is I find the charging dock is super finicky when it comes to which devices it’ll recharge from. I’ve tried charging it a few times with different Mac & Windows computers, and it doesn’t seem to take the charge (yet these computers can easily charge every other device on the planet). And god help you if you forget to charge the charging dock and just want to insert the Whoop strap into the charger and charge the strap. I’ve only found a couple of USB ports around that actually cooperate there. I can even come back hours later and nothing has happened. I don’t know what’s up there, but I’ve since made a habit of as soon as I complete charging the Whoop band, I plug the dock in to recharge so I have it ready for next time.

DSC_5627

To be clear – I LOVE the concept of what Whoop is doing here. The idea of mid-flight refueling of your band is brilliant. No really, it is. And when it works, it’s a dream. But it just doesn’t always work. Said differently: Just don’t wait till your band is at 4% battery like me. Also, oddly, I found that it’ll pause sync/catch-up of data while it’s charging.

Which is as good a time as any to talk about the app. Given the Whoop band has no screen on it (it does have four tiny LED’s that are primarily used for pairing or checking the battery), almost all your interactions are on the app.

Once you’ve got it paired up we’d dive into the app and see it showing as connected along the bottom – ideally with a time that’s fairly recent (that shows data sync). In general it usually stays connected well. But a few times a week it falls behind in data sync by some number of hours. If it falls behind by perhaps 3 hours, no biggie, it’ll catch-up easily. But if you let it fall behind by say, a day or so, then it’ll take many hours to re-sync the data.

2020-05-25 00.26.24 2020-05-26 15.49.01

Ultimately, the Whoop band is really all about tracking two things: Sleep (aka recovery) and Workouts (aka Strain). Technically speaking, strain can be non-workout induced, but by and large, it’s about workouts. I note that because the Whoop band doesn’t really track anything else. It’s not going to show you steps, nor will it show you miles run, or sets completed in the gym. Again, all it cares about from a technology standpoint is your sleep and your workouts. Everything else in between is mostly just noise to it.

However, there are non-technological elements in the app that lets you track all assortment of factors, like whether you were sick, or if you drank alcohol or caffeine, or whether you shared your bed (and if you did something creative while sharing the bed). The band won’t track those factors by itself, but you tell it about each day, and then at the end of the month it provides insights based on what happened after you did those things. It’s gloriously fascinating.

And we’ll get to that.

But first, you’ll need to spend some time decoding the user interface. Cause frankly, it’s confusing initially. But once you get the hang of it, it’s pretty straightforward. As I’ll explain probably numerous times in this review – Whoop’s advantage (compared to a Garmin/Polar/etc device) is how it enumerates data and insights. But I wouldn’t say the actual user interface is doing it any favors. There’s swiping up, and down, and left, and right like unlocking a Rubik’s cube.

But every time you crack open the app you’ll start on this ‘Today’ page (at left), and if you want to see your last 30 days overview, you can swipe left to see this page (at right):

2020-05-28 16.38.27 2020-05-28 16.38.22 2020-05-28 16.41.49

The main page (above left) is where you’ll see the impact of sleep from a recovery standpoint (as well as the actual time), plus your HRV, calories burned, and day strain. Also, it’s where you’ll see activities. You can then swipe down (above right) to see those two core items (Strain & Recovery) overlaid. In an ideal world the recovery would stay in green each day, and the Day Strain would be up on hard workout days (and lower on recovery days).

If you swipe to the right you’ll see an overview of strain for the last 7 days, along with three options (Day Strain, Avg HR, Calories), along the bottom. Each of those options simply changes the graph above.

2020-05-28 16.42.33 2020-05-28 16.42.35 2020-05-28 16.42.38

Swipe again to the right and you’ve got recovery. This is showing you the Recovery (in %), then HRV (Heart Rate Variability), then RHR (Resting Heart Rate). In general, you want Recovery % to be as high as possible, you want your HRV values to be higher, and you want your RHR values to be lower. Both HRV & RHR are common values used by countless wearables and devices for many years. The Recovery %, however, that Whoop gives you is unique to Whoop based on their own algorithms (though, other companies have similar concepts). Note that the bottom numbers are 7-day rolling averages, where the graph up top is individual days. Also, Whoop is *only* capturing HRV data while you sleep, not during the rest of the day.

2020-05-28 16.42.42 2020-05-28 16.42.45 2020-05-28 16.42.47

Swipe one last time and you’ve got sleep data. This is the end of the road for data on the mobile app. This shows your sleep performance (%), your respiratory rate, and your hours needed (vs actual). It determines each day what you need to recovery from the previous day’s effort and then gives you always-unrealistic recommendations of ‘sleep need’. Like before, note that the bottom numbers are 7-day rolling averages, where the graph up top is individual days.

2020-05-28 16.42.53 2020-05-28 16.42.50 2020-05-28 16.42.51

Don’t worry, we’ll get into the sleep bits in just a second. Actually, let’s talk workouts first, then we’ll talk recovery/sleep from there.

Oh – and it should be noted that Whoop does have a website, which in general allows you to dig in just a slight bit deeper on some stats. Maybe deeper is the wrong word, more like ‘slice it different ways’. Same data, just shown in different formats/tables and more flexibility in how you compare/chart it.

image

I mainly used the app, but there are aspects of the site that can be useful if you want to dig into a specific issue in your training or recovery from a trending standpoint, but I wouldn’t say it’s a big difference from the app.

Workout Tracking:

image

The Whoop band is always monitoring what you’re doing, and thus its workout tracking function is mostly there to bookend the timestamp definition of a workout, rather than to actually have any impact on your daily strain. Meaning, whether or not Whoop sees a workout has no bearing on your daily strain numbers.

See, Whoop has two ways it can define a workout: Automatically, or manually.

The vast majority of the time, I use automatic workout tracking – and the vast majority of the time it works just fine (more on what happens when it doesn’t, later). To use automatic workout tracking you do nothing extra. You just leave the band on your wrist like normal, and then go off and do your workout. There’s no tapping, rubbing, or giving extra interaction to the strap at all. Just treat it like a piece of jewelry and do your workout. At some point after the workout, it’ll give you a notification that it recognized a workout, and then show it on the home screen. Here’s a couple of variations of that on different days. Note that the icon changes based on what type of activity it detects.

2020-05-28 13.27.01 2020-05-26 00.07.35

From there you can open up that session to get more details about that specific automatically recognized workout. You’ll see your heart rate graph, calories, average heart rate, max heart rate, and duration. And of course, up top you’ll see the strain gathered from that workout.

You can also tweak/edit the start/end times in case they’re off a bit, as well as change the exact sport type if you need to:

2020-05-28 13.27.21 2020-05-27 22.14.26

But let’s go back to the ‘Strain’. Everything in Whoop is ultimately about two numbers: Strain and Recovery. Strain is generally generated by workouts, but it can also be generated by other day to day activities and life. Specifically strain in Whoop is measured by how much time you spend in a given heart rate zone (that’s it). Your strain number will slowly increase throughout the day as you go about doing normal human things if those things cause your HR to rise. Whereas just sitting at your desk generally won’t increase it much.

Each morning after it processes your sleep (and thus Recovery number),  it’ll generate a target strain value for the day. You can adjust this target value slightly, based on what you want to achieve from a training standpoint. As with any training program, you have to apply a stimulus (load) in order to get stronger. Then you need to recover for the actual gains to occur. The challenge here is that Whoop doesn’t know what you’re training for. Unlike a real coach (or an automatic program), Whoop has no knowledge of the finish line or your goals. It simply looks at today, and past data, and tells you what condition you’re in based on your HRV values at night.

It’s up to you to figure out how to use that data to decide whether today is a hard day or not. To me, this is a super critical point to understand about Whoop – because despite costing $30/month, it can’t tell you how to train for something, because it doesn’t know what that something is. It doesn’t know you have a game or race coming up this weekend, or that you’re trying to build for a competition in three months. It doesn’t know that this is the last big day in this training block. It just focuses on today, and your history up until this point.

Below is what the Strain coach is telling me I should do for today (the entire day), before I go out and do it. It has no idea that I’ve got a 10-mile run planned here in a few mins, or that I have a 90-minute interval ride scheduled for tomorrow. Nor does it really give me guidance on how much that workout should be to achieve the specified Strain goal for the day. In other words, it doesn’t say ‘Go out and do a hard 45 minute run’. It just says “Build 13.0 Day Strain” (screenshot at left below).

2020-05-27 10.44.18 2020-05-27 10.43.13

For example, before I head out for this run, it’s telling me I’ve accumulated ‘4.1 Day Strain’ thus far. The strap is about 2 hours behind on syncing still, but that 2 hour gap is just time I’ve been sitting here at my desk typing this, so it’s unlikely to matter much in the context of this conversation.

However, what it’s not telling me for that “Build 13.0 Activity Strain” (screenshot at right above) is how much of that should be in a workout. It doesn’t know that because it doesn’t know my planned intensity. But it should have more than enough data to know the inverse:

How much daily strain I normally accumulate from now till the end of the day excluding a workout? And thus, it should be able to tell me how much strain to accumulate in a workout?

Except, it doesn’t. At least not correctly. See, when I tap on that ‘Strain Coach’, it then shows me to do an “optimal” workout, that I should be “13.0 Activity Strain” to reach my Day Strain. But wait a minute, it just also said (look above at the two screenshots carefully) my Day Strain goal in total was 13.0? Which was is it? As Whoop notes, they aren’t the same.

The scales of these are based on different time parameters. Day strain is logarithmic and counted from when you went to sleep till when you go to bed. Whereas activity strain is based on time spent in a given heart rate zone.

And it’s these sort of inconsistencies that are really hard to reconcile from a practicals standpoint. And ultimately, it’s not telling me anything that assists in training against a schedule or for a specific performance. In any case, out for a run I go. I’m using automatic recognition this time, because I don’t want to bring my phone along for a 10-mile run.

Here, I’m back, and now you see my run clocked a nice 18.3 in the Activity Strain department, and now my Day Strain is 18.6.

2020-05-27 14.20.07 2020-05-27 14.28.54

To which, you’re probably asking yourself how is it that my day strain pre-run was ~5.0, and then I added a workout strain of 18.3, and yet now my Day strain is ‘only’ 18.6. Well, that gets into the underbelly of Whoop’s proprietary algorithm (of which, they have little documentation of). However, if you dig through enough Reddit threads you’ll find out that the Day Strain is instead operating on that logarithmic algorithm I mentioned above. As such, the strain of the activity is relative to the duration of the activity (in my case, about 1hr 20mins). Whereas the day strain is since I went to sleep (not since I woke up). And the max Day Strain you as a human can apparently attain is 21. That’s the ceiling of the Whoop platform from a math standpoint.

Sure, that sounds nice written out in a sentence or two, but it’s frankly the most confounding thing I’ve ever written. Why? Because you can’t use that logic to figure out how exactly a given workout will contribute to your day strain. To which some might say “Well, use the Strain Coach app”, except, that’s not practical in many sports, and it *still* doesn’t answer that question. I can’t take the Strain Coach app swimming, and I don’t put my phone on my handlebars riding. Nor do I want to take it running to a track workout.

But wait, this rabbit hole gets deeper. Since my workout was at noon, I could as a triathlete easily go out and do another 90-minute (or longer) workout. An equally intense one. Remember before that run my day strain was ~4.1, and then immediately post run jumped to ~18.6. So, if I did a nice hard bike interval that would bring my Day Strain from 18.6 to something less than 21? How on earth is that intuitive? Or usable to plan by?

Oh no, we’re not done yet. As random luck would have it, I decided to weed my garden last night.

Seriously.

I sat on the ground and quietly pulled small weeds for 35 minutes. Also, I picked up some dog poop. But Whoop thought I did a two-part interval workout. I’m not kidding – I couldn’t make up this if I tried.

So in this case, it for some bizarre reason thought my heart rate topped out at 156bpm (it wasn’t – that’s my long-run heart rate pace). But nonetheless, it automatically created an activity which it asked me to label for 9.2 activity strain. And thus, my day strain is now 19.5.

2020-05-27 22.15.07 2020-05-27 22.15.24

Now Whoop thinks I’m a MoFo of a Weed Puller (or that my weeds are giant sequoia trees). That’s fine, but what’s not fine is I can’t get rid of that. That has now impacted my stats (incorrectly) for the day somewhat substantially. It has downstream impacts on what it thinks my recovery rate will be, how much sleep it recommends, and how it trends the rest of my profile going forward.

In any case, let’s move on and talk about Strain Coach.

That’s the feature that ultimately allows you to do a workout and record it manually. But, the main thing of doing that manual recording is seeing the strain accumulate in real-time. That could be useful in a gym setting perhaps, or some other static environment where you can mount/see a phone and watch it slowly grow. That said, it does support GPS activities too.

To launch the Strain Coach you’ll hit the menu option, then choose Strain Coach. When you do that you’ll want to check that the ‘Strain Coach’ toggle is enabled at the bottom:

2020-05-25 11.34.59 2020-05-25 11.35.03

As you do so, it’ll offer an optimal strain target, which is based on how much strain you’ve accumulated within your day, and how much it thinks you can sustain. Once you’re ready, hit the ‘Start Activity’ button and get rollin’. As you go through the workout you’ll see your current heart rate (and zone), as well as activity strain, calories, average heart rate, max heart rate, and the map:

2020-05-25 12.25.06 2020-05-25 12.25.03

Also notable is that you can create little video snippets with overlays during your workout (or anytime actually), such as this:

2020-05-28 12.17.50

Once you’re done, you’ll hit end and save, and then you can manually enter in your rate of perceived effort, performance level, and whether you were able to complete the workout you intended to.

2020-05-25 12.53.29 2020-05-25 12.53.48

At that point, it’s no different than any other automatically recognized activity in terms of how things appear in Whoop dashboard, except that when you open that activity you also get the map information and total distance/duration/speed too:

2020-05-26 00.07.17 2020-05-26 00.07.20 2020-05-26 00.07.24

So, up till now I’ve talked about when automatic workout recognition works properly (most of the time), and when you want to do a manual workout (in my case, sparingly). But what happens when things go wrong and it doesn’t see your workout automatically?

Yesterday morning is a great example of exactly that. I did a RAMP test, which is a roughly 20-25 minute long cycling test to determine one’s FTP, in this case done indoors on a stationary bike.  After that, I did about 30 mins of core work on mats. Now, normally Whoop does a good job at tracking my indoor cycling workouts. However, I find it struggles with tracking shorter indoor cycling workouts – as if it’s unsure about itself. So, if we look at left, you’ll see that it doesn’t show any workout yet for the day, yet my day strain is of moderate levels for theoretically not doing a workout. So instead, I’ll manually add in one. You’ll tap the little icon of a person with a plus next to it:

2020-05-26 15.23.14 2020-05-26 15.26.00

It’s here that you’ll see my ability to manually enter a start and stop time, as well as choose from a long list of sports. I’ve always found the sport list in Whoop a bit peculiar. Sure, it’s got the common stuff like running/cycling/etc.. And then the ‘ok, I guess’ things like ‘Caddying’, ‘Gaming’, or even ‘Ice Bath’ or ‘Motor Racing’, but oddly not yardwork or manual labor type category. There’s been a bunch of recent weekends where I’ve worked my ass off shoveling dirt for hours building credit (or weed pulling apparently), but no proper categorization aside from using some other random gym category. A minor nit, I know.

In any case, after choosing the times and sport you’ll press save and be given the usual option to rate your perceived exertion and performance level, as well as the always incredibly confusing question of “I was unable to complete the workout I intended to”. Each time I see that I ask myself, why not just word it as “Were you able to complete the workout as intended?” with a Yes/No toggle?

2020-05-26 15.29.01 2020-05-26 15.29.08 2020-05-26 15.31.39

I can then open up the workout and clearly see my heart rate for that, along with some minor analytics. But ultimately, that’s it. You’ll note that this didn’t change my Day Strain value from the 13.3 it was before I specified this time to indoor cycling. Rather, it simply allocated each chunk properly within it. That’s good and bad. It’s good because it means its focusing purely on heart rate data – not on nuances such as the actual sport itself.

But what’s problematic is the other side of the equation – when it automatically detects something that doesn’t exist. For example, the weeding I did earlier. Or, another example, this activity below on Sunday. This cycling workout with an activity strain of 11.3 factually didn’t exist. I had simply pedaled 5 minutes to the hardware store using an e-cargo bike to pick up a single shower curtain. In other words, I’d expend more effort washing two coffee cups than this. Yet, it thought I did one hell of a 36-minute long intense workout. The problem? Just like with the weed pulling, you can’t delete it.

2020-05-26 15.48.27 2020-05-26 15.48.34

Ahh, sure – you can delete that ‘workout’ item, but as it reminds you immediately upon pressing that button: There’s no way to delete the underlying heart rate data. And as such, it doesn’t decrease your daily strain. So in effect, that fake workout was greater than most of my workouts, due to the effect of having such a high HR value for a relatively long time.

2020-05-26 15.48.53 2020-05-26 15.48.46

And this is an example of where the cracks start to add up on Whoop. Sure, the number of times it’s overestimated my HR by creating fake activities is perhaps 3-5 times a month. Inversely though, underestimating HR is a far bigger problem in my testing. Vastly bigger. And for that, we need to dive into the actual data in the HR accuracy section.

Ultimately, the crux of the sport aspects is that you can’t delete any data (or zero-out/normalize it), and you can’t pair to any other strap for more accurate data when you want it.

Sleep Tracking:

DSC_5622

I’ve decided that in its former life, Whoop must have been an Instagram model. Someone who apparently has the time and lifestyle to blissfully sleep 10 hours a day with no actual commitments. Which doesn’t mean its optimistic look at how much I should sleep is wrong, it’s just not realistic. The good news is that while it tells me what it thinks I need, it does have a three-level toggle for deciding if I want to ‘Get By’, ‘Perform’, or ‘Peak’.

Actually, on second thought – it pretty much is like an Instagram model telling me my outfit isn’t good enough: “Yes, those jeans and shirt are fine if you just want to get by.”

In any case, there’s no interaction with the Whoop band when it comes to going to sleep or waking up. It just does it all for you behind the scenes, pretty much like any other wearable. At some point, after you wake up (usually around 15-45 mins in my experience), your phone will give you a notification that it’s finished processing your sleep, and you can take a look at it. You’ll also see it on your dashboard as well. Once opened, you’ll see the heart rate for the sleep session, as well as four tabs to get more detailed information about it.

2020-05-27 16.58.52 2020-05-27 16.59.11 2020-05-27 16.59.16 2020-05-27 16.59.19

Each tab will display a different metric or insight. For example, within the ‘Time in Bed’ tab, I can drill down to the different sleep stages (Awake/Light/REM/Deep). I don’t have a viable way of validating these, however there is a study from a few months ago that does appear to largely validate the majority of Whoop’s sleep-related metrics (though, not all of them).

In general, I find Whoop does a good job at tracking sleep, though, I do find it tends to often over-estimate ‘Awake’ time. For example, last night the baby woke up for some odd reason at just after 5AM. I gave her a bottle and was back asleep in about 20 mins. However, the Whoop thought that took nearly 43 mins. That’s incorrect.

2020-05-27 16.59.25

Another way more visible example is last Monday. The kids (and me too) were up around 7:00AM, to which my wife and I then did the entire three-ringed circus act of feeding two toddlers and a baby, getting them all dressed, and then eventually out the door and walked the 5-minute or so walk to their school. After which, we walked back home. All of which put us home around 9AM.

This was the 2nd week of school after it being closed, so we were finally able to take a few hours break. Which, in our case, meant going back to sleep for about two hours. With me so far? Well, the Whoop wasn’t. It just counted it as a single night, and then stuck a 2-hour ‘Awake’ time in there.

2020-05-27 17.05.51 2020-05-27 17.06.15

To me, this would have been Sleep + Nap. Anytime I walk down the block, across a multi-lane road with two sets of tram tracks, then down another block, then yet another block to school, talk with teachers, etc… and do the whole thing in reverse – I wouldn’t really count it as one night. Just my take.

So, what about naps you ask? Well, unlike a lot of wearables, the Whoop actually can track naps. And in my experience it’s pretty good about figuring out when a nap takes place and marks it properly in your log:

IMG_6042 IMG_6043

Except for one problem seen above: It thinks that a nap will reduce your next night’s sleep need by the exact same amount as the nap (1:09). A concept which is pretty darn questionable. Typically speaking, most of us nap because we’re tired from the previous night (late night out, on a flight, other disturbances, etc…). And atop that, there’s almost never a 1:1 connection between a 45-minute nap reducing your sleep need the next night.

For the vast majority of people/times, if I’m taking nap, it’s likely because I took a transatlantic redeye flight the night before and landed back home exhausted. I’ve flown a few million miles to know that a quick 60-minute late morning or midday nap lets me get through the day at the office being semi-productive. Same goes if I was up for large portions of the night with a sick child. To say there’s a 1:1 replacement of the following night’s sleep is mind-boggling.

In any case, I appreciate that Whoop tracks naps (unlike others), but the application of that recovery period just doesn’t align with reality.

So, let’s dig a little bit deeper into the sleep coach now. As you remember before, it’ll give you guidance for what sleep you’ll need to achieve your goals for the following day. And as you near bedtime, it’ll trigger a smartphone notification as a reminder. I appreciate this, even if I mostly ignore it.

2020-05-27 22.57.18

And as noted earlier, you can adjust the ‘I want to ‘Get By’/’Perform’/’Peak’ drop-down at the top. Again, while I think these recommendations live in fantasy-land, I absolutely love the concept of it, and the structure of it. It’s super easy to understand. Note the changes that occur as I toggle through the three different recommendations for last night, following that run (and then yard work).

2020-05-27 22.57.552020-05-27 22.57.51 2020-05-27 22.57.30

However, where it gets really fun is if you then tap on that green ‘Sleep Needed’ text box. Here’s where (at bottom) you can see what would have been your sleep if you didn’t do any workouts or take any naps. After that, it then will show you the impact of that workout. In this case, that workout added an extra 1hr and 8mins of strain. Then, it shows how much sleep debt I had from previous nights/months/years. And finally, it subtracts any naps.

2020-05-27 22.58.03 2020-05-27 22.58.17 2020-05-27 22.58.20

Again – the presentation of this data here is fantastic. Seriously, it’s probably one of the best screens in the entire app. And with that, I went to sleep, and the cycle began again.

By the way, if you connect your account to TrainingPeaks, it will transmit this daily metrics data over (but not workouts):

Now, once you wake up there is one last thing to do – which is to enter your journal entries for the day before. This is where Whoop goes through a set of questions (which you can customize, more on that in a second). So it’ll ask you things like whether you spent the night on a plane or drank alcohol or caffeine, or shared a bed. It uses this as part of your larger monthly reports that you can access.

2020-05-28 15.44.29 2020-05-28 15.44.54 2020-05-28 15.45.06 2020-05-28 15.45.09

Some of the answers have more detail – like time of last drink. It only takes a couple of seconds and I usually do it over breakfast (or, coffee, as it may be).

You can customize these questions too. As you can see from below, the current list is huge, and seems to grow every few months

The idea here is that you can then look at the data in aggregate as part of your Monthly Performance Assessment, which is accessed from the menu:

This then opens up a PDF that’s generated each month (usually on the 1st or 2nd day of the month) with insights based on all your journal activity. I’d argue, this is probably the coolest part of the platform. Though, it’s disappointing this is only generated once per month. After all, there’s no reason why (as I sit here on May 28th), I shouldn’t be able to see my entire month’s worth of data for May.

The first part of the report essentially summarizes the strain, recovery, and sleep aspects:

image

You can then see the impact for each of our journal questions on data like caffeine or using a device in bed.

image

Or, if you decided to track sex (one of the options in the journal), you could literally see the impact of that on your training and recovery (or, what Whoop thinks is the impact anyway).

Which, nearly four months down the road I’ve just now realized I’ve clearly misunderstood the value of Whoop all along. My mistake was thinking the HRV driven tracking of workouts and recovery would offer insights into how I can get faster as an athlete. In reality, it’d have been far more useful to simply have a PDF ‘Monthly Performance Assessment” report at the end of each month that says (to my also athletic wife): “Look, this huge AI-driven platform clearly states that we should have more sex to increase our athletic performance and recovery.”

And thus, nothing else in my review matters to a bunch of you now.

Heart Rate Accuracy:

DSC_5639

As you’ve no doubt figured out by now, the heart rate data that Whoop collects is *EVERYTHING* to Whoop. It’s like a jetliner and fuel. Without it (or with bad fuel), it’ll crash. Unlike most other wearables, the Whoop can’t connect to 3rd party chest HR straps or sensors that may be more accurate. And it can’t import any other data into the platform. So everything has to come from that little sensor.

In previous iterations of the Whoop band, you couldn’t meaningfully export that data out, or compare it. But with the Whoop 3.0 band you can now pair up the strap to any 3rd party app/device and broadcast your heart rate to that device just like any Bluetooth Smart heart rate strap. That means I can pair my Garmin, Polar, or Zwift app to the Whoop 3.0 band and use it (Whoop) as a heart rate strap. That’s handy from a practical standpoint if you don’t have another strap, but it’s even more valuable from a validation standpoint because now I can easily (and precisely) compare the heart rate data produced by the Whoop strap to other known good devices.

And so, that’s what I’ve been doing for four months. Every single workout I’ve recorded – so far some 74 different recorded workouts across a variety of sports from cycling to running to skiing to gym work and more. For each of those workouts, I’ve then compared it to 3-4 other heart rate straps and sensors. Chest straps from Polar, Garmin, and Wahoo Fitness. Dedicated optical HR sensors from Mio, Polar, and Wahoo. And watches from Suunto, Garmin, Apple, Polar, Timex, and more. Wearing it with the included strap, then the bicep strap. Then the included strap on my bicep. Then on the other wrist, the other arm. I’ve separated all the sensors so they are not near each other as to not impact accuracy. No sensors share the same arm portion (above or below elbow) – standard industry practices for properly comparing devices.

Seriously, I have so much data it’s almost overwhelming. Almost. Rather than spew out nearly a hundred data sets, I’ve basically consolidated things into the most representative bunch of them (good and bad). There are trends in how well the sensor does, and so my goal here is to show those trends.

And ultimately, at the end of the day, I’ll save you more rambling: It’s the least accurate heart rate sensor I’ve tested in a long whilerivaling only the Suunto 7’s less than stellar optical HR accuracy.

And in a device that is 100% based on the accuracy of that sensor, it’s a huge problem.

But don’t take my word for it, let the data tell the story.

We’re going to start with something simple here first, a relatively tame 10KM run with no major changes in heart rate. Don’t worry, we’ll get to the intervals in just a second. Here’s the first data set, compared against a Garmin HRM-DUAL chest strap and a Fitbit Charge 4.

image

Digging into the first part a bit you can see that both the Fitbit Charge 4 and the Whoop strap are initially inaccurate during the build. They ramp-up too quickly, and are likely picking up cadence there. The slow-roll of the chest strap is more accurate to what I was actually feeling as I began my run down a very slight decline.

image

Still, by the two-minute marker all three were in agreement. Now don’t let this zoomed-in view below confuse you, the changes in HR are mostly minor here, it’s just the scale has changed as well. Most of this is only changes of a couple of BPM. The Whoop strap was consistently underperforming here though. But the Fitbit Charge 4 and chest strap were nearly identical. Even in some minor surges I did, the Fitbit Charge 4 stayed the course.

image

The problem here though is that, in general, it was about 5-7bpm low the entire run. This is somewhat rare in my testing, usually it’s either similar, or entirely off the map. Rarely is it just slightly offset.

Next, it’s interval time (this was a bicep strap run). This first bump you see is more of a kick up in pace than a true interval. Basically a build. But after that, those are legit intervals at approximately 6:10-6:20/mile (3:45-4:00/km). Nice cruisin’ pace. The Whoop is again in blue:

image

The Whoop strap in blue did less than awesome here. The Timex R300 totally lost the boat on the first build interval but was perfectly fine after that. While the Fitbit Charge 4 and chest strap were pretty close most of the time except the shorter intervals.

But really, the issue is those intervals. Again, it literally missed the entire point of my interval workout. Based on this, I should have just done a simple easy run:

image

Next, we’ll look at an interval workout on the track, with the Whoop data in teal. Here’s that data set:

image

The problem here is better seen once I highlight the Whoop strap. Sure, it’s good for the warm-up and the first interval. But then it loses the plot entirely during the intervals themselves. This is a huge problem because this is where your ‘Activity Strain’ will accumulate. So in effect, I’m not getting credit for three of my five intervals at the correct high intensity. It sees these as lower intensity intervals.

image

Next, we’ll look at a Peloton bike ride – nothing crazy intense here actually. Just a mostly casual ride. The Whoop strap is in green here, and you can see it struggles tremendously throughout this workout from an accuracy standpoint. Sure, it might look roughly like the others, but if we compare it to all the other sensors I have on that ride, it’s arguably one of the worst (save the TICKR strap which wasn’t wetted enough for the opening 8 minutes). Here’s the data:

image

And if we zoom in a bit on some sections, we’ll see how often it’s off – usually by 10-12bpm. For reference, that’s basically an entirely different zone.

image

If we look at an outdoor ride – most of mine were actually just like this – reasonably good. But in almost every case I rode outside I wore the Whoop strap on my bicep – and that helped substantially there (it didn’t seem to appreciably help in running however):

image

Next, we’ll switch to a longer interval workout indoors on a trainer. This is actually a reasonably good showing for the Whoop strap, with only some minor errors (it’s in green here). And here’s the data

image

The Whoop errors were early on in the warm-up, but recovered quickly. Some later errors were during the recovery phases, but overall this is one of the better showings by Whoop. The fact that I’m on a bike inside is what makes this easier for Whoop (as with any optical HR sensor), because it reduces the pounding of running, or the strain on my wrist of some gym-type workouts.

And here’s another indoor workout that’s very similar for Whoop to the other sensors. All of these units would have largely produced the same Activity Strain within the Whoop model. Here’s that data set:

image

Of course, you’re likely realizing what the pattern here is by now: Relatively stable workouts Whoop tracks well. But anything with any significant intensity, and Whoop often will lose the plot.

And that’s the real issue here: Once you add short high-intensity, the accuracy of Whoop tends to go down substantially. Longer intervals tend to be better. This is a pattern that’s not at all uncommon. Many watches’ optical HR sensors share the same issues, though, I feel in general Whoop’s sensor is more prone to high-intensity errors than others.

But that ignores the biggest difference: For most other sport watch/wearables, you can pair to another heart rate sensor (like a chest strap) and get more accurate HR data if you need it. You can’t with Whoop. So effectively it becomes bad data in, bad analysis out.

(Note: All of the charts in these accuracy sections were created using the DCR Analyzer tool.  It allows you to compare power meters/trainers, heart rate, cadence, speed/pace, GPS tracks and plenty more. You can use it as well for your own gadget comparisons, more details here.)

Wrap-Up:

DSC_5618

Here’s the thing: On the surface, the Whoop platform and promise is incredibly impressive. The user interface, once you warm up to it, is astounding in how it shows the cause and effect of what you do from an athletic and recovery standpoint. It’s really, really, really good at that. In addition, the automatic exercise recognition is usually quite good when it comes to recognizing that an activity has taken place, and the start/end times. And finally, the industrial design of the Whoop band is pretty nice, and I love the concept behind the snap-on charger for mid-flight refueling. All that is awesome.

The problem though? All of that is undercut by the accuracy of the optical heart rate sensor. And based on the overwhelming amount of feedback you all have given me as I’ve been testing this over the course of this year, many of you that have tried Whoop, agree. That sensor simply isn’t all that accurate when you sit down and compare it second by second to other products on the market. And given it’s the *only* input into Whoop, thus the *only* input into the Whoop platform is the data coming off that sensor, it becomes a real issue. Atop that, you can’t manually override failures when they do occur. When my 5-minute pedal to the store looks like the most intense 35-minute effort of the week, it screws up all the numbers. And there’s literally nothing I can do about that. I can’t just zero that out.

It didn’t take long to realize this, and I suspect this is the reason that up until Whoop 3.0 the company has heavily pushed back on opening data access or allowing any form of meaningful data exports. The Whoop 3.0 band finally allows broadcasting as a Bluetooth HR strap, and it’s that feature that really allowed me to understand where things were going so wrong. Of course, that’s still semi-limited. I can’t spot-check point in time HRV samples from the rest of the day (it only does HRV sampling at night while sleeping), to see if those are correct (which would validate or invalidate sleep metrics). I can only see an average for the night.

Don’t get me wrong – if we set aside the sensor (but we can’t, because that’s the product), what Whoop is doing from an analytics standpoint, a journaling standpoint, a data collecting standpoint, and a user interface standpoint – it’s all very very good. The app in particular is something that every other company in this space could take notes on. In fact, I’d say Garmin, Polar, and Apple should force 20 of their engineering folks to wear Whoop straps for 3 months to really understand it (the good and the bad). But to primarily understand the app and data flow and how to surface data to athletes in a mostly cohesive manner.

So what can Whoop do to address these issues? Honestly, not likely very much. Based on everything out there, this underlying sensor is the same one they’ve had for quite some time. And most companies tend to top-out on accuracy tweaks to their optical sensors after about a year or so. I think they realistically need two things: A) A new sensor, and B) To allow you to pair another chest/etc heart rate strap for workouts when you want. If they allowed that, it’d likely resolve 95% of my issues. Or, if they pulled in data from their already existing TrainingPeaks partnership and automatically matched up workout data in TrainingPeaks to Whoop’s workouts and then overwrote the heart rate data.

So, unfortunately, at this point, despite really liking the promise of the platform and the app itself, I just can’t justify spending $30/month anymore for a service whose foundations are based on inaccurate data, has no knowledge of my training goals, and doesn’t provide me with guidance other than telling me to sleep 11 hours a night…every night.

With that – thanks for reading!

FOUND THIS POST USEFUL? SUPPORT THE SITE!

Hopefully, you found this post useful. The website is really a labor of love, so please consider becoming a DC RAINMAKER Supporter. This gets you an ad-free experience, and access to our (mostly) bi-monthly behind-the-scenes video series of “Shed Talkin’”.

Support DCRainMaker - Shop on Amazon

Otherwise, perhaps consider using the below link if shopping on Amazon. As an Amazon Associate, I earn from qualifying purchases. It doesn’t cost you anything extra, but your purchases help support this website a lot. It could simply be buying toilet paper, or this pizza oven we use and love.

Post a Comment

Your email address will not be published. Required fields are marked.
If you would like a profile picture, simply register at Gravatar, which works here on DCR and across the web.

Click here to Subscribe without commenting

Add a picture

*