Yesterday the New York Times published in print and online a rather bizarre article on GPS running watches, one that essentially concluded that they were an “unreliable running partner”. Now my goal isn’t meant to defend GPS watches, but rather clarify a lot of oddities and inconsistencies in the piece, and why I believe the New York Times missed the boat entirely.
Not all watches are created equal
The article starts off describing the basic features of most GPS watches in the market, such as displaying pace, distance and time – all of which represent the primary reasons that most people pickup a GPS enabled watch. From there the author goes into a singular test case where she and a friend met up on a run this past Sunday in the following scenario as a basis for much of the remainder of the article:
The challenge here is that I’d ask first – what were the watches in question? What brands, devices and software versions? For example, was the one that was off by .42 miles (97.3% accurate) an older model from 5-6 years ago, or was it more recent? Who made it? What firmware version?
In the world of GPS watches, the reality is that not all devices are created equal. As I’ve shown before in four posts of accuracy tests, some units do simply perform better than others. Sometimes that is correlated to price, and other times it’s tied to the GPS chipset used and/or the firmware. To base the entire article (and all GPS watches in general) on what appears to be a single watch on a single run being off seems a bit of a stretch. For example, when the Timex Global Trainer first came out, there were indeed accuracy issues with it. On average, it was 2.5% off (short) – was her watch a Global Trainer? Or perhaps, it was an original Garmin FR610 – which also had issues early on with some routes showing about 2% short. Yet, both have been fixed by their respective companies (June for the FR610, August for the Global Trainer).
I found it strange that the author didn’t note the brand, nor contact them for an official reason, explanation, or PR response. Isn’t that the most basic journalistic thing to do?
In my mind, this is no different than saying “cars are unreliable”, because your particular car is in the mechanics shop. As in fact the author noted, her friends route was just about spot on, within .08 miles after 19 miles – or 99.58% accurate.
And finally – there is of course the very real possibility that something might have simply been wrong with the unit, either temporarily or permanently. For example, on some older Garmin units, sometimes the unit will get stuck on a particular satellite set that will cause significant distance inaccuracies (i.e. a mile or more off). These are easily fixed through a soft reset, which clears the satellite cache database and forces the unit reacquire the satellite information upon starting up the next time. No electronics device is perfect, and while this isn’t common, it’s also easily fixed in two button presses and about 10 seconds of time.
Misunderstandings about race distances
The article then segues into a discussion on running race participants and having incorrect distances shown. It talks with one of the Race Directors for the Rock n’ Roll Marathons, regarding complaints that users often have about courses. This ultimately ended up with this snippet on accuracy:
This is actually a really interesting quote, but not for the reasons you’d think – and this is where the author shows a lack of research.
See, in a race scenario, it’s nearly impossible for a GPS unit to be short. The most common scenario is for a GPS distance to be long.
Why you ask?
Well, because GPS measures distance in a series of points, usually about 1-second apart. This means that with the exception of trail runs where you have a slew of switchbacks, it’s virtually unheard for GPS to be short…unless the course itself is short.
And in general when you have multiple people saying the course is short – it usually really is actually short.
See, most long distance road races with large crowds aren’t going to have switchbacks in them, and very rarely do they have a significant number of 180* turns (due to overcrowding) – turns that even if they missed a data point here or there, wouldn’t account for a third of a mile.
As I’ve gone into in (probably painful) detail in the past, when you’re running a big race with lots of folks, you usually end up running quite a few corners wide. And those corners add up. Remember that races are measured according to USATF standards and certified non-GPS devices, which require that the measuring person take the absolute shortest possible route during the measurement, right up to the edge of the curb. That’s not how the vast majority of folks run their races though. Instead, most folks are forced into much wider paths, often with swerving around other runners. Every time you swerve around a runner – you’ve probably added 5-10 feet to your path. But let’s look at a corner scenario:
During that post I noted that for the National Marathon course that year, there were over 64 roadway turns – that’s 64 opportunities to run wide. With each one likely being an extra 40+ feet of running! Even in the simplest example of the street above, with just two lanes, that’s half a mile of extra running!
One fun game to play is to visit Garmin Connect after a race, and search on the race title and date – then look at the results, sorted by distance (there are so many folks with Garmin devices, it’s virtually impossible to find a race without lots of uploaded data). Based on that, you’ll find the interesting pattern that typically the fastest runners at the front of the pack run the fastest lines (with less congestion), and thus have much closer accuracy rates – distances that are closer to the actual distance of the event.
Now, that’s not to say that there aren’t ways that the GPS device can malfunction and display a distance short – but most of these are edge cases. For example, even with tunnels – most tunnels are straight, thus the GPS unit will simply connect the entry points and exit points on the tunnel and measure the distance in a straight line between them.
And while tall buildings have tended to cause issues for older GPS units, newer units tend to have fewer issues here. One could certainly argue that products of years past should have been more accurate, but the reality is that technology does advance, and thus publishing an article about GPS technology without actually noting whose GPS technology is no different than saying broadly that cars don’t work, without saying which make and model you have.
Online Maps, Phones, and Tracks
Last but not least, the article uses Google Maps as a basis for comparison. At a high level, there’s nothing wrong with this, but at a more concrete and scientific level – mapping out your route on Google Maps just isn’t valid when comparing to a precise distance your unit recorded.
When you map out a route on Google Maps (or any other mapping provider) you’re usually following the street that you ran down, and then when you make turns, you follow the next street (typically on the white/yellow portion of the map that represents the street). But the reality is that most people don’t run to the middle of the street’s intersection to turn – they run on sidewalks, and each time you make a turn, you’re cutting that corner in comparison to the map you outlined. This is really then no different than the previous section on running corners long during races. Online maps also don’t typically account for the normal left and right ebb and flow of most running/biking paths, slight twists that over time really do add up.
That’s not to say services like MapMyRun aren’t useful – because they are, they’re incredibly useful. I used one Sunday for mapping out my long run, and again for Thursday’s long run. But not because I expected to get the measurement down to the hundredth of a mile, but because I got a rough approximate of my route within perhaps half a mile. Or enough to ensure that my planned 18 mile run wasn’t a 21 mile run (as my original route would have taken me on).
Next up is phones. While the article didn’t talk to it specifically, I do want to call it out just for fun. Running with phones has become incredibly popular, and for good reason. For many, a $1 or free app can do what a $200+ GPS running watch can do. But you really have to be careful in choosing your applications. Some of these apps do a tremendous amount of development around GPS data smoothing, to make your routes and workout distances more accurate. Data coming from GPS receivers in cell phones is notoriously unreliable, thus the significant work that folks like RunKeeper have to do to make it consistently usable. That’s one of the reasons why I generally recommend sticking to the major and most popular running apps when it comes to GPS recording – because those are usually the same apps that have the resources to invest in development fine tuning, which often goes on forever. If you look at the updates to the extremely popular RunKeeper application, virtually every update notes “GPS Accuracy Improvements”.
Finally, running tracks. The article focuses on tracks, based in part on two different sources, talking about inaccuracies on the track, this section highlights it best (the quote was from the Rock n’ Roll Race Director):
And, in fact, what they are saying at a high level is indeed largely true. GPS devices today do struggle to get down-to-the-meter accuracy levels on a track. The technology in consumer GPS devices just isn’t there today. Though, it is there in this system I tested back in September.
But it’s also important to note that depending on your recording settings (again, which vary by model/device and even different firmware versions on the same device), you can get some pretty darn curvy lookin’ ovals, such as a recent track run below:
The little edges that you see around the 8 and 7 markers are simply where I was grabbing a water bottle every few intervals during a rest portion while walking around. Otherwise, it seems rather oval-like to me.
That said, I still do believe that consumer GPS running devices can improve in the area of tracks. One idea I talked about during my ANT+ Symposium Keynote speech was the idea of a track mode, which would enable the GPS unit to understand what a track is and automatically have it ‘lock’ to the track, thus producing correct and exacting distances each lap.
Summary
The point here isn’t to say that GPS watches are perfect, because they aren’t (trust me, I’ve seen just about everything that can go wrong). However, they are generally quite reliable, for both training and racing.
When looking at GPS accuracy like the New York Times article did, it’s incredibly important to speak to exactly which units were used, and the circumstances around them. Sure, that may sound detailed for a newspaper article – but in the vast majority of other pieces I read from the New York Times (which I subscribe to) – I do get that level of detail. Whether it’s a piece about technology or politics in Eastern Kenya – that’s ultimately why people read the New York Times over an article on CNN.com; For comprehensive and thorough analysis. And that’s what disappointed me the most here, the lack of understanding of the subject, or the detail behind it.
As always, thanks for reading!
FOUND THIS POST USEFUL? SUPPORT THE SITE!
Hopefully, you found this post useful. The website is really a labor of love, so please consider becoming a DC RAINMAKER Supporter. This gets you an ad-free experience, and access to our (mostly) bi-monthly behind-the-scenes video series of “Shed Talkin’”.
Support DCRainMaker - Shop on Amazon
Otherwise, perhaps consider using the below link if shopping on Amazon. As an Amazon Associate, I earn from qualifying purchases. It doesn’t cost you anything extra, but your purchases help support this website a lot. It could simply be buying toilet paper, or this pizza oven we use and love.
amen to that Ray!
Well. From and engineers viewpoint ray is just as incorrect as the times. It’s really not that simple. First an understanding of how GPS works is key. It measures time not position and converts that into positions in a coordinate system. Next you need to understand how accurately and how often it recalculated this. Then it records the data points into a flat file. You need to look at the data points and see how man are logged, the rate, and the number of decimal places in the position data. You are likely going to find about +\- 19 meters on a decent unit. And maybe a 1 second recording time on a good unit. Lots of direction and elevation changes are going to create errors. Straight lines like road running will be much better. Additionally, some gps units use a really simple algorithm to calc distance. Sum the distance between logged points. A more accurate unit will use some advanced algorithms such as circular interpolation. Finally loading these over the top of a map or sat image lends itself to another discussion with much more detail around the accuracy and interpolation of a 2d map image representing a 3d contour. When the satellite takes a picture the center is much closer to the lens than the edges of the photo …………..
Which part exactly was I incorrect on?
GPS, I’d argue that you are starting up the right path, but you fall short. But upfront- I think the article here is spot on with regards to the related errors. From the engineers’ perspective and coming from a professional in space operations and GPS support operations, you could discuss many errors, but you really touched on the least important. If time was the significant Sigma (multiple sigmas result in a predicted GPS Error) we’d see pinpoint accuracy due to time standards being maintained to billionths of a second. You do have errors from scintillation in the atmosphere, the geometry of utilized satellites (resulting in predicted dilution of precision-PDOP, HDOP, VDOP etc), multi-pathing and many other sources of inaccuracies impacting GPS receivers. The point I’m attempting to make is that GPS error is by design a function of three dimensions (4 including time) and, barring an extreme path being followed, will average itself out better and better as the duration becomes longer. Even for relatively short courses of 5 km, the errors resulting in the authors articles will be more drastic that the mean error resulting from the GPS system.
Now, with regards to the 3D v 2D maps; you are very close to correct. When terrain and elevation data (normally DTED is used) is not applied to a 2D map you will see slight skewing. This is fixed through a process caused orthorectification. A good friend of mine works for a commercial imagery company, and they generally rectify all imagery prior to posting – otherwise straight road on hilly terrain would not look straight. Maps like Open Source Map data bases have inherent errors, but on both counts the most significant source of error will again be based on the authors comments.
I’m generally not smart on many things, but I can probably address any follow on questions relating to GPS etc –
Hi,
You seem to understand about GPS. Can you please help me out with 1 questions. I recently bought my first Garmin watch. Prior to this I used phones which I found very reliable for calculating distance.
I have been having the weirdest issues with the Garmin Fenix 5. I tells me I ran 1km in under a minute or 12km when it should have been 7km. When I look at the map, it looks like I have been teleported all around the place. I’m running in an area that has tall buildings. Which type of watch would you recommend. Thanks.
The first thing that comes to mind is a multi-path problem. If you notice bouncing around when you are near tall buildings or extreme terrain it can be from your receiver accepting signals that appear to have traveled much further (after bouncing). There are some other things the receiver may assume like multiple signals etc but that’s more rare. To rule out any hardware problems I’d take it for a test in a more open environment. Haven’t used the Fenix-5, but with the 2 it always tracked well. Rainmaker may have some advice on settings for how often take the GPS positioning – I’m not familiar with those settings.
Many thanks for your quick response Dave. Much appreciated.
I ended up replacing the watch and then still had the same issues. So I requested a refund. The refund period is only a week so I had to act fast. I will do more research before getting a GPS watch. I’m hearing that since the Fenix 5 has a metal casing it will impact the signal. So maybe go for a plastic model like the new Forerunner 935. This might improve a bit but will not completely eliminate the problem.
I guess my other other option would be not to use GPS at all in that area and just rely on a foot pod for example.
Once again thanks!
For your information, below is a link to the GPS issues that I’m facing:
link to connect.garmin.com
Very dense high-rise buildings are tough…and honestly always have been. It doesn’t matter if I go back to my 10 or so year old Garmin FR305 or today’s latest.
That said, that track looks particularly bad. Just to clarify, are you waiting until you have full green signal? Also, are you using GLONASS or not?
One trick is after you’ve got green (with GLONASS), wait another 30-45 seconds. That just ensures it truly has solid signal.
Thanks for jumping in Dcrainmaker!
Yes it’s a bad track and I have more of these.
Yes, I wait for the green light first and then wait a minute or two before pressing start.
I’m also using Glonasss and 1 second reading.
I reckon it’s a very bad area. It even throws the phone off especially the first 1 or 2 km. I didn’t realize it until now. I always wondered why my first kilometers always appeared to be faster by one or two minutes, when I knew I was just trying to warm up and start really slow. Hahaha
Oh well, I’m just looking for an option to limit the damage. Either completely turn off the GPS and / or use a foot pod. Or avoid these “Bermuda triangle” spots.
Buying a sports watch opened me up to a new world of metrics and complexities that I never knew existed. But now that I have gotten a taste of what a Garmin watch can do. It’s hard to stop!
I have returned the watch now! That was a replacement watch too. I will think hard about the forerunner!
If you were to run in Manhattan , what would you use to track your distance? Would you rely on GPS?
DC Rainmaker: You do an awesome job with your blog. Thanks for doing it!
Great post, Ray. By the way, if I use 1 second recording on an all weather track, the GPS comes extremely close to recording 1600M for 4 laps every time. You are very correct about what you said.
I stopped reading Gina Kolata’s articles in the Times a long time ago. Her stories are always full of anecdotes presented as training or racing facts. This article is pretty typical of her style. It’s a pity the Times can’t do better for their fitness writer!
You just experienced the problem that most people encounter once in a while: When journalists write about subjects you know about, it turns out the journalist has very little knowledge about the subject and never ask any decent expert. When the journalists on the other hand write about subjects you don’t know about, you think they are experts and tell you every single detail worth knowing…
I expect that a lot of the high level issues with accuracy will go away once the GPS Block-III satellites are finished and launched.
Not that there’s anything wrong with your explanation, Ray, but this article posted on the Hampton Rockfest page is so good I reposted it on our club’s website with permission.
link to hamptonrockfest.com
It’s a great reference anytime I hear people complaining about course measurement.
Brilliant piece. You should offer it to the Times as an op-ed, maybe with a paragraph suggesting that the initial piece was very obviously below the (theoretical) standards of the Times. (It’s likely they will reject it simply as being pre-published and/or too minor a topic to run, but why not try? It woud be cool to have a Times byline!). At a minimum you should send a short version in as a letter. Great work!
Excellent post Ray. I know this isn’t what you were going for, but this is really indicative of the problems with “traditional” media. I’m sure it took you quite a while to write this post… but you also probably did it in a fraction of the time it took Ms. Kolata to write her article (or maybe not… maybe that’s the problem)., and yours is much more thoughtful, informative, and even-handed.
I had a similar experience when “This American Life” (which I normally love) did a show on the patent system. I’m a patent attorney, and I was aghast at how one-sided and just plain sloppy the TAM story was. It makes me wonder if they do such a half-assed on every story.
Glad you posted a response to this! Hopefully the author will see it. All I could think when I was reading that article the other day was that they really needed to read some of your posts!
Very glad you weighed in on this – I was shaking my head as I read it, it was a singularly ill-informed article! I agree with Josh that GK is not doing a good job with these fitness-related pieces!
Oh my god, someone who actually understands how a GPS watch works and has done some testing on the subject? So glad I found your blog, I’m tired of seeing that article shared everywhere since the author is so misinformed about the subject. Haven’t finished reading your post, but I HAD to say thank you for writing about this.
I’ve used my Garmin in my training for years… and it’s helped me run on pace and set PRs in a number of races, including a 2:54 marathon. This journalist telling people they’re basically worthless is… well, worthless.
I felt the same way about that NY Times article. I run with a Garmin 205 or 305, and even though they are older watches, they always seem to be spot on for me. Excellent rebuttal.
Let’s take the author’s one bit of anecdotal evidence – the *one* run that she performed and reported on.
The route (according to Google Maps) measured out to 15.96 miles. For purposes of this comment, I’m going to accept that, down to the hundredth of a mile, as fact – the route was in fact 15.96 miles.
Her watch said it was 15.54 miles.
That’s a distance of less than one half of a mile.
When you’re talking about going out and running 16 miles, does it **really** make that much of a difference if it was off by less than a half of a mile?
Really?
Shoddy, sensationalistic reporting.
Excellent post, did you send this to the NYT editors?
Great article. I did the Annapolis half this year and people with GPS watches were all complaining that the race was short. The race featured a hairpin turn on a jogging path in the woods. I tried to explain to several of them that GPS watches aren’t operating the way they think their are operating. They should think of it as a route that “connects the dots” when it has a good signal from the Satellites. Rounding that turn in those woods would have easily caused most watches to miss a few dots and cut off a good portion of that hairpin turn. If the course had continued in a straight line the watch would have just connected the dots further along the path and you would never be the wiser.
Exactly how did you do the search in Garmin Connect to see the race data you showed in this ‘review’
somebody put something uneducated in a news paper article as “twoof”. imagine that…
I agreed with everything except the notion that GPS watches error on the side of length. Unless Gmaps (which I use to compare to my Garmin distance after each measurable run) has a variable error rate between 2.5%-6%, my FR210 counts short most of the time.
How do we know Google Maps is perfectly accurate? Perhaps some of the error lies with using that as the reference measurement.
Excellent post! Thanks for the analysis (missing from the NYT article).
Thanks for posting your rebuttal to the NY Times article. I have used a Garmin 305 for years and I don’t have any issues with accuracy.
“I found it strange that the author didn’t note the brand, nor contact them for an official reason, explanation, or PR response. Isn’t that the most basic journalistic thing to do?”
I don’t necessarily agree. Generally my biggest problem with blogs like this, run by enthusiasts, is that they are *too* connected with the manufacturers. While it is sometimes cool to know the latest news releases from Garmin, as a user all I care about is that the device works. If it doesn’t work, then I don’t want it. And if it is something that can be fixed by a simple software reset, then I want it even less—because that is strong evidence of very poor design.
As a running enthusiast, I don’t care about Garmin’s excuses. All I want is something that works. GPS nowadays is pretty low tech anyway. Slap a $20 GPS chip in a $20 watch and you have a $200 running watch.
Great job, right on dude! I always enjoy reading your evaluations. Happy Holidays,
I have Garmin 405 and it reads short occasionally — most often that will on on a trail, but also just in my neighborhood. Wouldn’t that be the case if it loses the signal for some period of time?
Even on my neighborhood runs I don’t get the 1km beeps at exactly the same spot. close but not exact.
you need to write for the nytimes – agree they completely missed the boat for triathletes and runners – must be after a different type of audience… makes me question thier approach on other topics. ah but nobody is pefect least of all gps watches and the nytimes
‘Anonymous’ that agreed with the NYT article probably would never buy another car if they had bought a lemon. If you have one poor manufacturer, do you really say the whole product category is worthless?
I 2nd the comment that articles written by the general media about topics you are familiar with come through with a lot of inaccuracies and seem comical, but sound authoritative when you don’t know the subject matter.
So now how about writing an article about politics, showing all the falicies in most media reports?
The fact that the NYT author compared distances between GPS and Goggle Maps clued me in right away that she have no idea what she was talking about. People need to understand that there are inherent errors when measuring distance on Google Maps/Google Earth in the first place. Goggle maps are merely aerial images captured and stitched together to form a seamless map. The issue lies in the fact we are trying to represent earth (a spherical object) on a 2D mapping system. Its call map projections errors and the errors are inherent with any 2D mapping system, depending on the map projection used. The following properties can be preserved (AREA, SHAPE, DIRECTION, BEARING, DISTANCE, SCALE), but understand that not all properties can be preserved at the same time. In simple terms, all aerial images have to be geo-rectified and referenced to known ground control points, hence, the images will have distortion. Anyway, without me rambling more about it, there are entire GIS/RS (Geographic Information System / Remote Sensing) industries that deal with these issues on a daily basics. To think that there are no errors associated with distance measured by clicking a path using a mouse on Goggle Map on some computer screen is a joke!!!
“there are entire GIS/RS (Geographic Information System / Remote Sensing) industries that deal with these issues on a daily basics. To think that there are no errors associated with distance measured by clicking a path using a mouse on Goggle Map on some computer screen is a joke!!!”
If this is significant enough to make a difference (which depends on the scale, of course), then I would guess that Google makes the necessary adjustments. Google isn’t stupid.
“‘Anonymous’ that agreed with the NYT article probably would never buy another car if they had bought a lemon. If you have one poor manufacturer, do you really say the whole product category is worthless?”
A watch isn’t a car. Many years ago, before iPods, I bought an iRiver brand MP3 player. It was horrible, to use it you needed to memorize all sorts of random combinations of buttons. Eventually iPods came out and later I bought one, but yes that first experience soured me on the product category for a few years.
I think this makes sense, too. Especially in a category where the products are fairly similar (which seems to be the case with GPS running watches today). After having a bad personal experience, it is hard to trust reviews because reviewers tune their standards to the status quo instead of to the ideal situation. For example, Rainmaker thinks that having to reset the watch occasionally isn’t a big deal. That’s the status quo today. But ideally and for normal people who expect it to “just work,” having to reset your *watch* seems crazy. It’s a watch!
Even heard some rumours on this article here on the other side of the Atlantic 🙂 Are you sending your piece to the editor?
A few thing to keep in mind.
1) Google measure the right down the middle of the road. So if you run to on side or the other it can increase or decrease the distance of the run!
2) Keep the unit up to date and clearing the history to reduce the chance for errors.
3) The level off accuracy of the satellite at the start of the race.
4) How was the course certified? Wheel,bike,car, mapped of google. You can not run the intangibles of a course! How many people did you pass?
5) Nothing is perfect! When your racing or training Give it all you got & leave nothing to chance! Play HARD or GO HOME!!!!!!!!!
I think there is a big piece you are missing here Ray. Consumers have different expectations of technology than people like you in the industry. We talk about “expectation management” – but the reality is that consumers don’t care about things like percentage difference – they want an absolute level of accuracy on total distance, say within 50 feet. And some would argue – this is a valid complaint and we need to rise to the challenge.
Get her Ray. You should definately send in what you have written. You don’t mess with the Maker! Someone in that position should be upheld to higher standards of research. Please call her out, I would love to see if the Times does or says anything in response.
“If this is significant enough to make a difference (which depends on the scale, of course), then I would guess that Google makes the necessary adjustments. Google isn’t stupid. “
The point I was trying to make is that EVERY mapping applications have inherent errors in the distance measurement, just because Goggle said it was 15.54 miles, doesn’t mean it is 15.54 miles exactly, there is always a +/- (Goggle just doesn’t like to publish that information). In a normal GIS system, road errors can be anywhere 1-5 meters (sometimes even more so depending on the dataset). How do you think digital roads on Google Maps get drawn in the first place? Usually 2 methods: 1) Hand digitalize by using aerial imagery 2) Driven using GPS. Or in Google’s case, they purchase third party data. People need to understand that mapping system isn’t exact, but rather it’s a fuzzy logic, with acceptable tolerance.
Comparing GPS distance to Google distance is no different than comparing distance from two GPS watch, they will never agree 100%, and neither is absolute right, the true distance is somewhere in the middle. That was the point I was trying to get across. Goggle Map is just a representation of the real world, captured at a specific moment in time. Just like any time you go out for a run with your GPS watch, it captures a representation of the real world at that given moment in time.
Dumb question – how do you get Garmin Connect to compare the races like in your screen shot? I can only get them to display individually.
Thanks for the article. As an anecdotal experience, I did ten miles nonstop at my local high school track at a 7:00/mile pace and my forerunner 610 gave a distance of 10.01 miles. Interestingly, it was off by .01 after the first 400m and stayed that way throughout.
Forerunner 610, latest firmware. I don’t get many track runs in, but this experience convinced me that the 610 remains accurate even after six months of use.
After reading all the comments I wanted to address “Anonymous” and those others looking for absolute accuracy. Please check out the following websites:
link to www8.garmin.com
link to google.com
Both of the web pages, openly admit that they are not 100% accurate.
GPS watches are tools that help us in our training and racing. Just like all other tools, a GPS watch is only as good as the person using it and their understanding of the tool’s purpose and limitations.
Bravo, well said, thank you very much for responding to that poorly done article. What infuriated me the most is that you can’t comment on the articles, so hopefully enough people find their way to your response.
I’m sure you’ve seen this by now, but Runner’s World had a post and a counterpoint post, referencing you!
link to trainingdaily.runnersworld.com
-Chad
I think it should be, “How they missed the boat” not “Why”. Great read. If I were a sports tech. comp. I would consult you cuz you know your shit.
Well done, Ray. I read your piece on GPS v ‘official’ course measures. When I ran a half in October, I paid attention. Ran straight. Hugged the corners. My Garmin 305 was spot on for the 1st couple of miles relative to the course markings…then got a bit long…and a bit longer. Exactly as you suggest – imperfect running path compared to the ‘official’ accumulating error the farther you run. How much error? My 305 sez 13.11 miles. NYT = misinformed and poorly executed.
Awesome post. Thanks for parsing out all the details and doing the good kind of reporting that the Times ought to be doing.
My previous watch – Garmin Forerunner 110 – was regularly measuring 5-8% under. I remember running a trail running event with many runners around me wearing GPS watches. It was easy to see the problem when all the watches around me started beeping at the end of the first mile and mine was still at 0.95 mile.
I returned Forerunner 110 and bought Forerunner 610 instead. The GPS chip is supposed to be the same in both watches but I’ve never had the same issues with the 610 model. Also I’ve never seen again random spikes on the altitude charts – something that bothered me with the 110 model.
So yes, my personal experience is that not all GPS watches are the same in terms of precision.
By the way, Ray, you mentioned GPS watches are supposed to interpolate distance in tunnels. My own experience was different. When I ran Seattle Half last month, my Forerunner 610 watch lost signal in I-90 tunnel, and when it recovered at the end of the tunnel, the distance was off by 0.5 miles. It seemed like the watch paused distance mearement without pausing the time. The discrepancy continued towards the end of the race.
This comment has been removed by the author.
Did you get any reaction from the New York Times or the original Author? I am quite curious….
Nice work, Ray. Totally agree on track mode. Since moving to 1 second mode and away from smart recording, I get a pretty accurate track no matter what I’m doing. Thanks for digging in on this.
Excellent post, thank you! Links to Garmin 305 and Google Maps versions of this morning’s 10-miler are at link to dailymile.com . The Garmin route came out about 2.5% shorter.
You would think that someone with a Masters in Applied Mathematics (according to her NYT Bio) would have a much better grasp is the statistical principles involved in GPS tracking and mapping. This was neither good science nor good journalism. Sadly, it may turn many amateur endurance athletes away from a very useful training tool.
Most importantly realize that a USAT certified course by their own calibration method is measured long by inclusion of a small calibration factor to ensure that a road race course is never short. A runner is always assumed to run the shortest path. Unfortunately many races are not certified.
You say the august firmware fixed the accuracy of the Timex watch but looking at the release notes it seems like only time to satellite lock is faster with no improved accuracy so I don’t see where you are getting that from. link to timexironman.trainingpeaks.com
Hi Eli-
There were actually two Aug updates, one in 2011 and one in 2010. The 2010 one largely fixed the majority of issues, while the 2011 added addition improvements. Additionally there was a Nov update that helped quite a bit.
Given the Timex Global Trainer wasn’t an end to end Timex product, they learned a lot of lessons that aren’t being repeated with the Run Trainer.
Anybody know what sort of accuracy rate you get when using Google Maps? +/- ??%
Gmap-pedometer.com is usually 1-3% longer than a GPS watch. Two reasons…1) the GPS watches often underestimate (somehow they are losing distance and time), and 2) Gmap-pedometer.com measures based on the center of the road/trail, taking turns at the center. We runners usually cut the corners…and it adds up.
I have no means of verification, but I would say about 2/3 the discrepancy is due to the watch, with 1/3 due to Google’s overestimation.
As an aside, I have used the FR60…and when calibrated properly (1 mile on a track), I found it accurate to within 0.5% of Gmap-pedometer.com (half a percent!) when running at a similar pace to calibration.
I have Garmin 305 and Forerunner 110. I have tested them by wearing one on each arm. They consistently disagree with themselves in the following sense. I will run until one of them chirps that 3 miles has been done then I turn around and run back along the same course which is generally very straight ( some very slow turning ). On the way out the chirping garmin will say 3,0 ( of course ), but after running back on the same course that same garmin will give a reading of 3.1 to 3.2 miles, and it does this consistently. THe other garmin behaves similarly. However, the 305 seems to be a bit more accurate. But both consistently measure 3 miles to be less than three miles ( on the way out that is ). The distance has been measured with a wheel by the way.
I find that my GPS watchES are often short. Two days ago, I ran 12.21 miles. Today I ran the same route, except I ADDED 3 blocks. However, today it said I ran 12.01 miles. At the point where my watch ‘beeped’ the other day saying I was at 3 miles, today I was only at 2.88. At the 5 mile beep point, I was at 4.81. Somehow the watch was ‘losing’ distance. And, if anything, today I was running fewer straight lines. Odd.
Btw, this was a Timex Run Trainer, the first version, with the latest Firmware update. I have noticed a few less issues with my Garmin Forerunner, it, generally, produces similar distances on routes.
My worst experience was with the atrocious TomTom GPS Runner. My first run was a 9.5 mile route (measured by multiple sources)…the TomTom said it was less than 8.5. The next day I stood in the same parking lot for over 30 MINUTES waiting for a signal….and it never found the satellite. I am sad to admit a bit of profanity ensued on my part. Needless to say, this GPS watch was returned.
Apologies, the Garmin Forerunner mentioned above is the Forerunner 10, with the latest Firmware updates. If the watch had a presentation field for lap average pace and lap distance, I would use it every day!
Can you explain why strava and garmin connect have different upload distances and times from same upload? Is strava not accurate enough to cope?
Not easily. 😉
Actually, it’s a fun idea for a post. But, it’s not just limited to Strava, or Garmin. Rather, every sports application on the planet likes to massage data and they believe they are right. Each company of course thinks it’s smarter than another, and thus you end up with differences.
How about them hairpins? A lot of people complained at a local marathon that the course was short – most of them recorded 42.1-42.2 km – could the fact that the course was 2 loops, 6 hairpin turns per loop be the reason here? (I assume most of the time the GPS would just drop the extreme points around the hairpin – even if it loses 10 m per one turn, that would be 120 m (I think a bigger loss is more likely).
I have sometimes had the same experience with the Garmin Forerunner 110 as Anonymous on 12/23/2011 re: the mileage being short on a run that goes through a tunnel. I would have thought the watch would be smart enough to understand that I ran through the tunnel and wanted that mileage included. Can you comment?
All Garmin watches do simply take the tunnel start/end point and in the track file add the distance. I’m surprised your seeing something different. I’d also validate that what you see on the device is the same as what you see on Garmin Connect.
Wouldn’t autopause triggered by signal loss cause the distance to be shorter?
It could, though, autopause is disabled by default on Garmin devices.
The Garmin ForeRunner 110, as you mentioned, doesn’t have autopause. It didn’t used to omit the tunnel mileage as often as it does now, but lately every run that goes through a tunnel is short of the Google Maps distance by a significant amount, more than can be explained by curves and statistical variances. Your statement that ALL Garmin watches include the tunnel distance is obviously inaccurate because mine doesn’t, and obviously I am not the only one, as I referenced in my comment. I was hoping you could provide a bit more insight than saying that you’re surprised.
I’m not really sure what to say. Every Garmin device I’ve ever used (including my FR110) simply takes the entry/exit points from a loss of GPS signal situation and bridges the gap. I’m not quite clear why yours isn’t. If you look at the Garmin forums, you can see other peoples are working (including an example of a FR110).
Have you contacted Garmin Support?
Thanks. I will check out the Garmin forum, but I really don’t like contacting technical support for any company. They will probably keep me on hold for two hours and then deny that my watch is doing what it does. Thanks again.
I have definitely seen situations where Garmin underestimates mileage when signal is lost. For example, in Chicago last year, I did a 15K race in which a small part of the course was on underground roads. The GPS signal was lost when I went underground. I started at point A, traveled north to point B, then traveled west to point C where signal was regained. When Garmin tried to fill in what it assumed the path to be, it assumed a straight line going northwest from point A to point C. Of course, that path was not possible to take because that would mean running through solid walls. Because of things like this on the course, my Garmin told me that the 15K was only 9.05 miles long.
I understand the NYT. GPS running watches can be useful if you travel and often run in unfamiliar surroundings, but for local running courses I find them unnecessary and potentially very distracting. To know your pace you can just measure each of your local courses in gmaps (manually) and just keep track of the time if you need to. That’s all you need really. Learn to feel your cadence and pace, checking your times and counting your steps on occasion. I like technology, but only when it adds, e.g. bike power meters, but for running or swimming a regular watch is more than enough.
The track idea is great, except one detail. Not each lane in a track is created equal like a pool lane. Each lane has a different total length, and unless the GPS can lock down which lane you’re running in, this calibration tool will only be so helpful. (unless you’re able to manually input which lane you’re generally adhering to.)
Hi! What’s the most accurate watch in terms of GPS? Thanks!