Visual Lottery Analyser
Norine Wiltshire
I have a Garmin Fenix 5s Sapphire with metal band. It is lousy at monitoring sleep. I love it for everything else (except maybe reporting calories burned). I wish to update to a new watch that monitors and analyses sleep much better. My research shows that the Withings ScanWatch is by far the best at sleep analysis, but it has the only one button and a tiny display. The Samsung Galaxy Watch 6 seems to be the next best, but has limited options in look and feel. I love the look of my Fenix 5s and I can get the Epix Pro (Gen 2) in exactly the same style.
Sleep tracking is based on movement and optical HR - optical HR works differently for everyone and its not guarantee as to how well it will do with you. I would personally not suggest buying any one watch purely for sleeping. If you want the most accurate device purely for tracking sleeping do yourself a favour and by the withings sleep mat: -analyzer Depending on your age group a series of french hosiptals that did a fairly large trial on them found that for most age groups it was about 75% accurate (note that for some age groups it was less than 60% - if I remember correctly it started to fall off if you were over 55).
b) Stress is great at showing changes in lifestyle/habits/illness/injury - all factors that can cause issues around sleep and adversley impact it (case point - have an alcoholic beverage within 8 hrs of sleep and see the impact on your stress score / conversley don't (not the don't blame me if you stop drinking anything in the evening) - I would focus on this main factor and try and improve (of course not possible with illness/injury - but at least you know reason for poor sleep) . Even with poor duration as long as stress is good you will still be around 60% or fair stress & fair duration be +75%.
a) duration - yes it can sometimes get it wrong - they all can - if you are really inactive and your HR drops sufficiently then yes you watch may consider that you are asleep. Conversely if you don't actively move around upon awaking - again it might consider that you are still asleep - they all have this issue. When getting up, always check sleep score widget on watch - if watch not correctly recognized wake time - you get the option to stop sleeping - so can finalize wake time. I generally find that I am within 15 mins of start & wake time of when went to bed and got up - and maybe once every two months or so I have to manually stop sleep (usually when have to get up way before scheduled wake time - so suspect this is more a case of watch thinks you might roll over and go back to sleep.
c) deep - match against results of CPAP - when I have a bad CPAP record - I have bad deep sleep. I can't remember a time I have had a good CPAP result and a poor deep sleep result. So duration and when it occurs wrong (but who cares) but sleep score result seems accurate
f) awake/restlessness - doesn't always get this right but would say that around 80% of the time it does, and if using multiple alarms it usually shows a wake point for those alarm times - about 80% of the time (if it doesn't register one its usually the first alarm).
Re QS reviews - considering all the other main reviewers all say the same thing about why they don't analyse sleep tracking (they usually give a small synopsis of what the watch offers) - the tech isn't here yet - so no point. I give him kiudos for his attempt and his belief in himself - but considering he fails to state his gold standard device isn't gold standard as only up to 80% accurate. His lack of decent size data polls to analyse (he does 3-7 data polls - if that many) and justifies his decision on that (Ray, Des, Chase the Summit, Fit Gear Hunter etc - are all using these watches for 1-3 months before putting our reviews and are basing theirs on at least 7 times more data polls). While everyone has inherit bias QS seems pretty blatent - but not sure if just language barrier (how he phrases stuff) or just is biased - and I do wish that we would make a much better clarification'/justification as to how he rates each device - as he will rate one as good and give it 3 stars but rate another one as good and give it 4 stars - generally with no justification between his different goods as to how a good can be 3 yet also 4 stars. I have also noticed that he is generally completely out of lock step with other reviewers. Most of them tend to find pretty simliar results - however he seldom does - does make me wonder a bit - but possibly purely due to this testing methods (several of which are flawed) and size of data polls - as so few one bad result can make a massive difference - compared to the others with far more they are able to consider whether its a trend or a one off.
Let me first clarify that I currently own 3 new generation Garmin watches plus some Edge cycle computers. Also I am a software engineer / data scientist. In my opinion, Quantified Scientist does a VERY reasonable job of collecting data and presenting valid comparisons between devices. Clearly a lot of Garmin fans here (and on YouTube) do not like QS's (unfavorable to Garmin) results, but that does not make him wrong. Likewise downvoting me does not make me wrong.
Having said that, QS is a single human tester and it may be that his unique body shape/composition lead to exceptionally bad performance with Garmin devices. So although I believe his methodology is valid, his results (N=1) do NOT reliably extrapolate to an entire population of humans (as he readily admits).
I also disagree with the claims in this thread that other reviewers (DCR, Des, Chase the Summit, Fit Gear Hunter) have substantially different results from QS. The general consensus that I see pretty consistently from ALL legit reviews (including QS) is that:
Given the feedback and all the research I have done, I have determined that knowing whether any watch is going to work for me in regard to monitoring sleep is going to be a lottery. If I really want to monitor my sleep properly, I should forget about the possibility of using a watch and source a dedicated device specific to that purpose that is consistent, regardless of body type, build or whatever (if one exists).
I am more than happy with my Garmin 5s in all other ways. I am not into all the new fandangled stuff about answering my phone and reading texts through my watch, etc. So I am sure my 5s will last me for years moving forward.
1) his test poll size of 3-7 events - surely you can't consider that a reasonable test pool size. Technically the other reviewers basing theirs on 20-60 (based on 1-3 months of daily use) is barely sufficient but overall will give a much better idea of accuracny levels as are more able to distinguish between an ongoing trend, occasional hiccup or one off occurance. You cnanot do that with 3-7 events. So no his data collection is woeful and can potentially give very flawed results.
2) valid comparison - he only ever tests with a baseline i.e. chest strap & device. He never now includes additional devices - so can never be sure that a chest strap isn't having a wobble (it happens- dry contacts etc) or whether the wobble that does occur might have occured with other watch devices - note this is someting that all other reviewers do (at least he has stopped wearing 4 watches on his wrist like some of his really old reviews where it could potentially have impared results due to constricting blood flow). And I will say that at least he seems to be trying to improve his methods i.e. adding charts so that can see lag and TBH the correlation charts I am not sure they are that helpful, as if hitting peaks and troughs but just a lag you are still getting all the training beneit -its just a lag in HR.
3) sleep stages - garmin normally pretty much nails deep sleep in his reviews - its more light/rem - the reality that even his so called gold standard EEG device (which is only up to 80% accurate at its best and is often less than that- so far from gold standard - and why doesn't he reflect the reality is that it isn't gold standard and is at its max only 80% accurate - or is he concerned that it will only validate how inaccurate watches are) will struggle to accurately determine light/rem - in fact even sleep clinics struggle to differentiate and they are using higher quality, more sensors and delving into visuals clues in much more depth. So never overly worried about difference between light/rem. And why doesn't he do any clarification measurent' analysis around HRV which is far more accurate and effective than measurement than light/rem sleep especialy as its very effective in highlighting changes in habits/lifestyle/illness/injury - or is due to non fitness watches not having it and having to use a 3rd party app on apple watch and then still have to heavily massage the data to get decent readings......The reality is the most accurate consumer sleep tracking is a withings sleep mat - and that about 75% accurate for most age groups between 25-50 (and still struggles with light/rem where its far less accurate) and pretty woeful outside that group - which is why I suspect French hospitals (they commisioned and trialed the device across a 300-500 poll group over an extended period) decided not to invest installing them.
Yes QS has "fewer events" but each of HIS events measure the watch against an imperfect-but-credible reference device Dreem2 EEG headband. I personally find that to be a more valid test than other reviewer wearing just the watch for 3 months (and no other measurement to compare with).
1) Dreem is only for sleep tracking - nothing to compare it to as no one else does it. Credible - if you consider something that is 80% accurate as credible - either way it should be clarified in his reviews - which he does not.
2) His latest reviews on the Elevate 5 sensor definitely do not agree with what Ray and Des and the others have stated. if you think it does - then it suggests that a correlation co-efficient of 0.85-0.99 is neligible and of no material difference (which depending on test subjects is generally a reasonably accurate conclusion - but is not how he states it).
64591212e2