JUMP TO:

A First Night with Apple’s New Sleep Tracking Feature

DSC_6221

While I had other reviews to finish writing this week, I got distracted spending half my afternoon getting an older phone reset and loaded up with the first iOS14 developer beta, to then allow my Apple Watch Series 5 to get updated with the latest watchOS 7 developer beta. All in the pursuit of tracking sleep. Now, this update won’t land on your watch till likely September. And it’s also likely we’ll see numerous tweaks before then. But still, the general framework of what Apple is going for here related to sleep is ready to start poking at.

Of course, not settled with merely having a single watch track my sleep last night, I threw down all the sleep things. In total, the following devices were being Big Brother:

A) Apple Watch Series 5 (with watch OS 7 beta installed)
B) Garmin Fenix 6 Pro (with new sleep beta installed)
C) Withings Aura Sleep under-mattress sensor
D) Whoop 3.0 band
(I also had a Beddit 3.5 sensor, but that failed to gather data last night)

If you wanted sleep data – I’m gonna give it to ya! Note that all devices were positioned/configured as such not to interfere with each other. For the watches there were one per wrist, and for the Whoop band it was using the bicep band instead.

Got all that? Good, let’s go.

Oh, wait, here’s this whole thing condensed into a video too:

Ok, now we’re ready for more text.

[Oh, some nitpicker’s corner items – I specifically used a clean and completely reset iPhone with a clean and completely reset Apple Watch on what is actually a clean/new Apple ID. There were zero apps installed on the iOS14 iPhone aside from factory defaults, thus no ancillary data went into that pot except for the Apple Watch with watchOS 7.]

Getting The Watch Ready to Sleep:

Now, as it stands today, actually getting the Apple Watch to track sleep is a rather long affair. Unlike Fitbit, Garmin, Samsung, and well…every other wearable ever – the Apple Watch won’t actually track sleep by simply wearing it and heading to sleep. Instead, you need to fully detail your night’s intentions to the parents, as if you were a teenage boy about to take a girl out to the prom.

So, assuming you got everything all updated on both your phone and watch, then you need to open the ‘Sleep’ app on your watch. This is what tells the watch that you want to do sleep things on both the watch and the phone. It does seem you can do some of this via the Watch configuration app, but you can’t do all of it. More on that in a second.

On the Apple Watch, you’ll open the Sleep app, and the first thing it’ll ask is for your sleep goal. This is basically telling it how many hours you want to sleep each night:

2020-06-23 21.19.04

[Note from the future: I think you can technically skip the sleep schedule/goals bit, but the Apple Watch Sleep UI flow very strongly pushes you through these steps, at least until you’ve set it up once – then you can go back and delete it. More on that in a second.]

It’s on this same page you can configure an alarm or not:

2020-06-23 21.19.11

Next, is the toggle to actually enable sleep tracking. This is the part that basically keeps track of how much you sleep each night:

2020-06-23 21.20.19

After that, you’ll toggle the Sleep Mode on the watch. This piece turns off the watch display unless tapped. So the previous setting enables data tracking, and this setting turns off the display as well as automatically enabling Do Not Disturb based on your planned sleep time.

2020-06-23 21.20.36

Then it’ll prompt for ‘Wind Down’. This is the piece that’ll turn on Do Not Disturb ahead of what would normally be your Do Not Disturb time. So, like a pre-DND game time. Most of the Wind Down aspects actually happen on your phone. In this setting, you’re configuring the exact time duration, effectively a count-down.

2020-06-23 21.20.50 2020-06-23 21.21.00

After that you can toggle whether to enable Charge Reminders or not, which, will remind you whether or not your watch has enough juice to match your planned sleep:

2020-06-23 21.21.08

Finally, four years later, you’ll get a summary of everything you just did. Assuming all parties agree to this legal document, you’re ready to go to sleep. Between taking these photos and 15 minutes later I’d ultimately end up changing the bedtime to 11:45PM instead.

2020-06-23 21.21.26 2020-06-23 21.21.17

Honestly, I’m exhausted from all this configuration. My simple man’s view here would be that for all of the machine learning sleep magic that Apple talked about in their keynote, this should just be on by default and tracking by default. After all, if it’s doing its job properly it shouldn’t cost any added battery life (just like it doesn’t for any other wearable on the market for the last decade). Most wearables do ask for a ‘Do not disturb’ window, but Apple already knows that anyway from your phone. So it should carry that over.

Note interestingly, if you go into your phone’s Watch app, and then to apps, and then to Sleep, you’ll find these settings pages. However, this is *AFTER* I had already configured it via the watch. Also, there’s no obvious method I can find to configure the ‘Wind Down’ period from the phone. Though I suspect it’s just a matter of me finding that treasure somewhere. Point being, it’s plausible you could be able to skip all the shenanigans above and then just quickly swipe to enable these settings here.

2020-06-23 21.28.55 2020-06-23 21.28.58

But again – that’s the point of this 3-month beta period – to hopefully account for this feedback and figure out how to tweak the experience. And Apple does have a good history of doing exactly that. We often see substantial changes in the first 2-3 weeks especially of the developer betas after WWDC. Anyways, let’s go to sleep.

Now You Sleep:

Well actually…you almost sleep. Since we’re here, we’ll talk about what happens as you approach bedtime. In my case, I had configured a DND (Do Not Disturb) and sleep time of 11:45PM. So, starting at 11:14PM (yes, 11:14PM), my iPhone notified me that it was about to start the bedtime Wind Down period:

2020-06-23 23.14.12

Then, at 11:15PM, my phone went into the Wind Down lock screen, first showing ‘Good Evening’, and setting DND (at left below). I didn’t have an alarm configured. Notably, this isn’t actually your normal lock screen. This is atop the lock screen. If you press ‘Dismiss’, then you see your normal lock screen, which is also darkened and notes that it’s in the Wind Down Period (at right below):

2020-06-23 23.15.35 2020-06-23 23.16.10

This is an interesting design choice. It’s basically Apple’s way of telling you ‘For the love of god, stop playing with your phone’, by having a two-part lock screen. There’s plenty of data out there that shows that screen-time as you approach bed time doesn’t help you sleep. So by having the dual-gates, they’re hoping you’ll pause an extra second.

Next, I puttered around cleaning up a few final things, brushing teeth, etc… and making it to the bed. At 11:45PM (my designed sleep time), the Wind Down screen overlay changed to ‘Sleep Well’. Again, double-lock screen here still in effect (at right), notice how now it says ‘While Sleeping’ as opposed to ‘During Wind Down’. It’ll continue this way until morning.

2020-06-23 23.45.58 2020-06-23 23.46.03

Meanwhile, over on my watch, nothing special happened during the Wind Down period. Do not disturb was enabled automatically 30 mins out, but it didn’t have any unique watch-face or such. Instead, at the 11:45PM marker the screen shut off. If I tapped a button, then the screen turned back on again to a dim overlay with just the time:

vlcsnap-2020-06-24-15h34m42s501

Then, it too stayed this way until I woke up.

I took a screenshot at 12:10AM as I turned off my phone, and fell asleep about 5 mins later.

[Sleeping time]

Fast forward a number of hours and it’s 7:29AM to the sound of a hungry baby. I got the baby and went out to the kitchen with her to make a bottle. At this point the Apple Watch gave me a ‘Good Morning’ message, showing my battery stats and weather for the day. It did NOT show sleep at this point.

vlcsnap-2020-06-24-15h34m32s551

I then went about finishing her bottle prep, feeding her, and getting her settled to play. Still though, even as 8:00AM passed, the Apple Watch didn’t show any sleep data on the watch itself. It was just empty:

vlcsnap-2020-06-24-15h35m08s278

It did however show ‘Time in Bed’ data on the iPhone in Apple Health. But that data is exactly as it says “Time in Bed”, not “Time Asleep”. It was a bit early on the ‘Time in Bed’. While I did initially get to bed at 11:17PM, I also got up a few times including to take the watch photo (in the kitchen) at 11:46PM. Either way, I’ll assume the spirit of what it says was correct:

2020-06-24 07.55.29 2020-06-24 07.55.25

Geek side note though – this data actually comes from the iPhone. If you dig deep into the Apple Health records, you’ll see this entry is *not* from the Apple Watch, but from your phone, basically looking at when you put it down. That’s listed under ‘Source’ = Ray’s iPhone’.

2020-06-24 08.05.27 2020-06-24 08.05.20

Finally, at 8:12AM – some 45 mins after I got up, I was able to see my sleep data on the Apple Watch itself:

vlcsnap-2020-06-24-15h35m17s574

There’s nothing to tap there on that section to get extra data. However, if I had slept for numerous days, then I’d see sleep trending data too. Interestingly though during that 41 minute period, Apple appears to be doing the processing. I had poked through Apple Health data earlier in that timeframe, and the only data showing was the Time in Bed data, not any added sleep data. But, now after 8:10AM, I then got added actual ‘Time Asleep’ data in Apple Health.

And in fact, it then corrects it to my actual time I woke up – a very close 7:27AM. In fact, I’d even agree that 7:27AM is probably slightly more correct than 7:29AM. I no doubt pondered for a minute or so whether baby was in fact going to be upset if I waited longer (the answer, yes). And as for the time that I fell asleep – that too was super close. I had guessed about 12:14-12:15AM for when I actually feel asleep – it says 12:17AM. I’m good with that.

2020-06-24 09.41.12 2020-06-24 09.41.38

Above you can see some coloring of the sleep data. This indicates the sleep blocks (not phases). And in fact, if you backtrack a bit in Apple Health and then dive into the specific sleep records recorded by the Apple Watch, you can see these chunks:

2020-06-24 09.48.14 2020-06-24 09.45.16

While I don’t have any recollections of waking up at 1:10AM, 1:51AM, or 2:26AM – I did wake up at 5:18AM to go to the bathroom. And then fell asleep about 2-3 mins later. It’s also notable that you see the ‘Date added to Health’ shows at 8:08AM. Meaning, it took the watch about 40 mins before it posted the night’s sleep results.

That in and of itself isn’t super unusual. For example, Whoop typically takes 10-20 mins (sometimes longer) before it shows sleep. Garmin is about the same with the newer firmware. Inversely, Polar is much much faster, and usually knows as soon as you start interacting with the watch that you’re up.

Other added data bits, it records your heart rate (BPM) all night long roughly every 5 mins. And then it recorded two HRV samples at 12:58AM and 4:42AM, as seen below:

2020-06-24 09.49.48

It also recorded environmental sound levels every 30 mins. Note that I suspect things like rubbing the watch against sheets is incredibly loud to the mic, so I’d take that with a grain of salt. Also, these are specifically from the Apple Watch, not my phone.

2020-06-24 09.51.23 2020-06-24 09.51.34

There’s no other sleep-related data or body metrics from the Apple Watch at this time. I know there have been hopes or rumors about other metrics, but as of today and this specific build, that’s what’s there.

Comparison of data:

So how did Apple do compared to everyone else? Well, let’s keep in mind this is a mere single night. As anyone with a wearable can tell you – you can easily have bad nights on otherwise good devices. Just as you can have the random good night on bad devices.

Here’s a comparison of the different sensors I had. Notables listed below:

Apple Watch Sleep Comparison

Data MetricMy RealityApple Watch Series 5Garmin Fenix 6 ProWhoop 3.0Withings Aura Sleep
Time to Bed (start time)11:17 PM11:17 PM (via iPhone)N/A12:12 AM11:24 PM
Feel Asleep (start time)~12:15 AM12:17 AM12:16 AM12:12 AM12:16 AM
Interruptions/Awake Time3-5 mins27 mins*54 mins44 mins50 mins
Morning Wake Up Time7:27-7:29 AM7:27 AM7:27 AM7:29 AM7:23 AM
Average Sleep Respiration-N/A13 brpm14.5 brpmNot captured
Nightly HRV Values-2 Values takenNot exposed1 Value takenNot captured
Average Sleep HR-Not calculatedNot calculatedNot calculated51
Sleep ScoreDamn goodNone78 - 'Fair'65% - 'Falling Behind'TBD
Sleep Phase/Stages-NoYesYesYes
Total Sleep Time7hrs 13 mins6hr 43 mins6hr 17 mins6hr 51 mins7hr 3mins

A couple of random notes:

– The Apple Watch doesn’t list ‘Awake Time’ directly like most wearables/sleep systems, so you have to do some quick math. Doing the math backwards, it would mean 27 mins of awake time based on specifying 6hr 43mins sleep time between 12:17AM and 7:27AM

– All the other devices specify an actual ‘Awake’ time, so I used that. I agree, the math doesn’t always work in those cases. I just wrote what it showed.

– For HRV values, Whoop and Apple Watch do record them. Whoop shows a single value, the Apple Watch recorded two values. Garmin does track HRV values at night (it’s how they get Body Battery), but doesn’t explicitly list HRV values in the app anywhere.

– For Sleep Scores, each company does their own magic. Note that for Whoop I used the ‘Sleep Performance’ score, and not the recovery score. For this night the recovery score in Whoop showed 49 , which is basically a body recovery score upon HRV. Garmin does the same thing with HRV. Withings Sleep score comes from mixing sleep stages. However, for last night the Withings for some odd reason didn’t calculate a sleep score yet. So, maybe later?

– For average sleep heart rate, all companies track heart rate constantly throughout the night. And all of them showed the values. And some of them showed max/min. But Withings had a metric that showed the average for sleep explicitly.

– For respiration rates, I used them as-is when shown.

Phew, got all that? Good.

I had hoped to show you data from both the Beddit 3.5 sensor and the Emfit QS sleep system sensor. But both were upset about different things. The Beddit sensor appears to be having some sort of Bluetooth or power problem. And the Emfit QS sensor looks like it needs to be reset. I’ll have to troubleshoot those in time for later this summer with a more final Apple Watch sleep tracking version.

Wrap-Up:

DSC_6219

Like every other sleep tracking device I used last night, the Apple Watch easily nailed the time I fell asleep (+/- 2 mins) and the time I woke up (+/- 2 mins also). Every device varied on how much they thought I was awake at various points night, and the Apple Watch doesn’t explicitly list that metric, so my math-based 27 minutes for the Apple Watch isn’t really comparing…err…apples to Apples.

For heart rate data points, the values look believable and in line with all the other units. Nobody was out in left field here for any point during the night. The two HRV values are also believable, but hard to compare against the Whoop which was taken at an unknown time during the night.

Ultimately though, what Apple is doing here is super basic. On one hand, every wearable company big and small is doing more and doing better with sleep data than Apple. Heck, even Apple’s own Beddit gives more information. On the flip side, Apple appears to be taking a ‘just the facts’ approach. Which is mostly how they’ve done things around health and fitness monitoring to date. Apple’s activity app (now called ‘Fitness’) has never been the pinnacle of athletic or fitness details. The rings interface and associated logic is very good at getting people to be active (which is great), but that doesn’t imply that how people can look at all that data later is anywhere near the levels of Fitbit or others.

We’ll have to see where Apple takes things. While Apple seems to be trying to nudge people towards the ‘Health’ app, I’m not really sure that massive repository is the best place to show sleep trends (just like it’s not for daily activity, and why Apple has the Fitness app now). I’d even argue that the sleep data from your Apple Watch specifically should go into the Fitness app, since that’s where all your other daily activity metrics are.

Either way, it’ll be interesting to see where Apple takes this. Both between now and September, and then beyond that.

With that – thanks for reading!

Post a Comment

Your email address will not be published. Required fields are marked.
If you would like a profile picture, simply register at Gravatar, which works here on DCR and across the web.

You can click here to Subscribe without commenting

Add a picture

*

31 Comments

  1. Kiara

    I use AutoSleep right now and based on your review, I will most definitely not be changing to the native sleep tracker. I refuse to tell technology when I intend to sleep. AutoSleep does a good job of tracking whenever I fall asleep, which changes due to my work shifts.

  2. Mike

    It would be interesting to see how the native app compares to AutoSleep which works very well.

  3. David

    Sorry if I missed it Ray… what if after I’ve configured the sleep app for the first night I forget the second night. I just climb in bed with my watch on and just… sleep. Does it track anything if I don’t “turn on the sleep mode” each night? I understand why Apple wants us to initially configure this and there benefits to activating the mode each night to take advantage of wind down and configuring the screen to be off/dim but if they don’t have the watch track sleep even if you do nothing at all its all a big fail… thanks again!

  4. Donna

    I’m too engrossed in the Garmin ecosystem to bother with Apple’s stats. I find it too cumbersome (the same with their running stats). Just my opinion. Anxiously awaiting the new sleep data to roll out to my Fenix 6S.

  5. inSyt

    Great article! Please try to include some of this to your already very in depth reviews.

  6. Frankwin Aerden

    Hi Ray,

    How is it in comparisson with polar sleep tracking. Totaly different or almost the same.

    Thnx for the great review.

    • I’d say that Polar is probably one of the deepest sleep tracking wearables out there now actually, at least in terms of metrics. So in that sense, this is super basic comparably.

    • My girlfriend has an Apple watch, while I have a Polar Vantage M. When discuss our sleep in the morning, there is a huge difference in metrics. 😉 The granularity of the Polar Vantage M sleep tracking is better … and super happy with the data and insights it has provided to me over the past year.

  7. Josh

    Random, but is the watch face showing the time of 11:47 and the small date a new face in the beta OS?

  8. Alex

    Hi Ray,
    you are by far the best. Nobody else is diving so deep.
    Thank’s a Lot, Alex

  9. Nathan M.

    Hey Ray,

    I noticed in the sleep chart you have the Fenix 6 listed as not having a calculated average sleep HR. I found today that it will calculate an average HR while sleeping but it’s buried under the “pulse ox while sleeping” tab in the sleep widget on Garmin Connect (I’m still on 9.0 software).

  10. seb

    As many posted I’ll probably stick to autosleep. Honestly I can’t believe how autosleep is good compared to my fenix 5. Garmin is terrible at tracking sleep for bad sleeper like me. I wake up most night for an hour and garmin always stop the night there even if I go back to sleep before my usual wake up time. I really wish garmin would improve that a lot on that.

  11. Nick K

    Ray!

    A little nerd nitpick if I may…

    WHOOP records HR and HRV throughout the sleep. The single value you see reported is the average HRV in the last deep sleep (SWS, slow wave sleep) episode that was 5 minutes or longer. If you try other devices that capture nightly HRV such as Night Charge enabled Polar devices or Ōura ring, you can actually cross reference your deep sleep periods from WHOOP to night long HRV chart, and WHOOP’s values will be right on the money.

    Now, why are they doing this? Two reasons. First, they have a white paper that suggests an average HRV for the night doesn’t really correlate to recovery the following day. Second, there’s a research that one should measure HRV only during SWS to avoid any waking time distractions. Yes, the consequence of this approach for WHOOP is that depending on your wake up time your recovery numbers might be drastically different since HR and HRV for the last deep sleep period are weighted heaviest.

    Oh… And biceps isn’t a very good for WHOOP to capture sleep data 🤓 The stage analysis will be off.

    • Thanks for the detail. The point I was making though was that only 1 value is taken (which is matched by what you noted), which also is shown here (link to whoop.com.). They may or may not record beyond that (some of their podcasts indicate they don’t, some do), but from a data presentation standpoint only once.

      The challenge to this approach though (only being in SWS) is that assumes ones actually gets their sleep phases correct. And there’s no evidence of that to date (else, my guess is that the study they often cite would have discussed it). In fact, in that study they skirt around the issue and state it doesn’t actually measure sleep phases, but only REM or non-REM time.

      Point being, they’ve put themselves into a pickle of double-assumptions. I’m not saying anyone else is better, but I have a funny feeling looking at more HRV data points is better than an assumption on one.

      As for bicep – indeed, it’s not ideal. And not where I wear it for sleep (t’s honestly uncomfortable there), but in terms of sleep times, it was spot-on. And I didn’t want it on the same wrist as other wearables, as that’d most definitely interfer.

      Cheers!

  12. JJS

    Does it make sense to track sleep only using sensors that look on movements? In my understanding only when HRV data come in you will get variable informations. The solution Apple offers at the moment could also be made with an iPhone App and putting the phone under my pillow! So why wearing my watch at night, prevent it from fueling the battery and carry that brick on my wrist during sleep when I get such more or less usefull (or useless) metrics?

  13. Stefan Kasteren van

    Hey Ray, does this Apple watch not give details like your REM sleep time etc?

    I tested the Philips watch five years ago and that watch gave all details of every face in your sleep. If this watch only gives you the time you was in bed then it is useless i.m.o.
    I want to know how wel my sleep was. Right?

  14. V Paul

    No blood oxygen / sleep apnea monitoring is very disappointing. I’m hoping the Watch Series 6 has special hardware that allows for more advanced sleep tracking or something, otherwise this seems like a big let down.

  15. V Paul

    Also Apple currently takes HRV values at night, I haven’t found the values useful or seemingly accurate.

  16. Chris W

    Looking forward to the upcoming sister website ‘dcsleepmaker’ 😉

    In all seriousness, thanks for another deep dive article.

    Have you used/considered the sleep cycle app, in conjunction with these wearables? I’m now using a Garmin 245 and it’s associated metrics, but used sleep cycle for quite a while prior to that. The 245 (and g connect) do a poor job of nailing my sleep and wake time, compared to sleep cycle. And like your suggestion for 20 Garmin engineers to use whoop and learn what a good user experience is, so too with sleep cycle.

  17. Devin

    Hey thanks DC Rainmaker! Big fan of your reviews.

    I’ve been using the beta sleep tracking since 6/22 as well and have had 2 nights out of these last couple weeks where it just didn’t track the sleep at all. The watch app says “no recent sleep data” under the the analysis section. The health app displays only time in bed. I’ve forced refreshes, restarts, etc. with no luck. There wasn’t anything unusual about my sleep those 2 nights, either. Beats me what’s up.

    I guess it works most nights! Have you had any nights where it just didn’t record anything?