GPS Accuracy Experiment

DaveLeeNC · December 25, 2008

I did an experiment with an iGolf Caddie (original model) a couple of years ago. I wondered if anyone had ever done anything similar.

What I did was to walk around my neighborhood and store 4 positions that were 100-200 yards from a specific point in my front yard (very few trees) and these points were basically North, South, East, and West of the fixed point in my front yard.

I then repeatedly measured the distance to these four points. Each measurement (actually four measurements) was separated by at least 2 hours. I ultimately did this 70 times over a week or so. This was 280 total measurements. Note that what I was measuring was NOT absolute accuracy, but I was measuring the repeatability of GPS distance measurements - accuracy certainly cannot be any better than this (but could be worse).

What I found was that 91% of the time the measurements were within +- 3 years.

97% of the time they were within +- 4 yards

99% of the time they were within +- 5 yards.

This seemed to match my experience on the course (once I learned how to properly use the device). Interestingly, one of the worst ways to use the iGolf device is to go somewhere and just stand there for 30 seconds or more. The device works best if you just walk up to a point, stand there a couple seconds for it to stabilize, and that's the number.

In my experience these devices will on rare occasions generate a really bad number. When I was using a GPS rangefinder I always had kind of a sense of where I was so a reading that was (for example) 20 yards off is pretty obvious. I excluded those in my experiment as I typically did on the course. They happen but not often.

I ended up being a laser rangefinder guy primarily because the thing that I care about the most is good accuracy TO THE PIN from inside 100 yards - something I'll never get from GPS. Actually from 175 yards I have a slight preference for numbers to the front, middle, and back of the green vs. rangefinder info.

dave

draver · December 28, 2008

The current concensus is that GPS equipment has improved through several generations since your experiment. As a result of this improvement in technology, your results are probably quite outdated and do not represent the current accuracy of modern units.

My understanding is that 1-3 yards accuracy is common, and expected from the latest golf handhelds. I own a Skycaddie SG-5 and have owned a Upro and the readouts measured within 1-2 yards at virtually every course marker I checked on courses I play regularly. I also have a Leica CRF 1200 laser rangefinder and it agrees with my SG-5 to the 1-2 yard variance.

Progress, fortunately.

phil75070 · December 28, 2008

Progress, fortunately.

Much progress! I am also not sure that doing a golf GPS accuracy experiment with an iGolf unit is very meaningful. I have always had the impression that iGolf targets the lower end of the range when it comes to these devices. Not saying there is anything wrong with that. :)

DaveLeeNC · December 29, 2008

The current concensus is that GPS equipment has improved through several generations since your experiment. As a result of this improvement in technology, your results are probably quite outdated and do not represent the current accuracy of modern units.

My understanding is that 1-3 yards accuracy is common, and expected from the latest golf handhelds. I own a Skycaddie SG-5 and have owned a Upro and the readouts measured within 1-2 yards at virtually every course marker I checked on courses I play regularly. I also have a Leica CRF 1200 laser rangefinder and it agrees with my SG-5 to the 1-2 yard variance.

Progress, fortunately.

Progress, fortunately.

Much progress! I am also not sure that doing a golf GPS accuracy experiment with an iGolf unit is very meaningful. I have always had the impression that iGolf targets the lower end of the range when it comes to these devices. Not saying there is anything wrong with that. :)

Just out of idle curiosity I was hoping that this might motivate folks with other brands of GPS to do the same experiment. It is pretty easy to do.

dave

draver · December 29, 2008

Isn't that a little like the T shirt " I'm with the Dept. of Redundancy Dept." ?

The course has provided markers, Sky Golf Has sent their GPS guy out to map it, and my GPS is agreeing with both of those measurements and my $700 laser rangefinder that is guaranteed accurate to 1 yard in 400.

What am I going to learn?

ragu421 · December 29, 2008

I havent owned a GPS largely because you have to download certain courses from your computer {and Im not very good on the computer} but I have owned a couple of laser rangefinders.I,first,bought the entry level Bushnell for about $199. at my local golf shop and would get some "crazy" readings.Like,I knew I was between 125 and 135 yards {as I was inside the 150 yard marker on the course} and would come up with 250 yards,270 yards,etc.What was happening,I deduced, was I was picking up trees and such that were located behind the green.I got my American express catalog {Id accumulated a lot of points} and went for the really high end Bushnell "pinseeker 1500 slope edition" and have found it to be VERY accurate,even when there is no wind and the flag is laying limp.This thing has literally improved my score by an average,Id say,of 3-4 strokes per round.I can measure to bunkers,etc.What I like about a lot of the GPS systems Ive seen advertised on TGC is that it shows an overhead view of the green.You can actually see how much room you have in front,behind,side to side,etc of the flag.What are the opinions of you that have owned both GPS and laser?On my home course,I pretty much know the distances from memory but when on vacation and playing a new course,I could def. see the advantages of GPS.What is a decent,mid level,GPS?Any recomendations?There are just so many out there to choose from.Id like to take both {on a slow day} to my home course and really see how much {if at all} it would improve my score.Im right on the brink of being a "scratch" golfer on my home course.Do any of the GPS's help with the slope of the green?My bushnell is the "slope" edition but Im not so sure exactly how accurate it really is.Any thoughts?

Wayneo · December 29, 2008

Phil,

while the unit might be cheap, the internals aren't. From Wikipedia on the chip found in the Neo. "Receiver sensitivity of -159 dBm while tracking; GPS receivers based on this chipset have routinely performed better than receivers based on other chipsets.[3] Reviewers have praised the SiRFstarIII chipset for its superior sensitivity and tracking capabilities.[4]"

http://en.wikipedia.org/wiki/SiRFstar_III

DaveLeeNC · December 30, 2008

Isn't that a little like the T shirt " I'm with the Dept. of Redundancy Dept." ?

The course has provided markers, Sky Golf Has sent their GPS guy out to map it, and my GPS is agreeing with both of those measurements and my $700 laser rangefinder that is guaranteed accurate to 1 yard in 400.

What am I going to learn?

How accurate (more specifically repeatable) your GPS readings are.

dave

phil75070 · December 30, 2008

Phil,
while the unit might be cheap, the internals aren't. From Wikipedia on the chip found in the Neo. "Receiver sensitivity of -159 dBm while tracking; GPS receivers based on this chipset have routinely performed better than receivers based on other chipsets.[3] Reviewers have praised the SiRFstarIII chipset for its superior sensitivity and tracking capabilities.[4]"

http://en.wikipedia.org/wiki/SiRFstar_III

Oops. I was wrong then. Thanks for the clarification. I did not know what chip set they were using. Do you know if the iGolf devices utilize WAAS/DGPS?

draver · December 30, 2008

They are within 1-2 yards each and every time I use it at a point that has been precisely measured by at least two different agencies. That being the course itself, and Sky Golf. My SG-5 also correlates to within 1-2 yards of my Leica rangefinder.

Conclusion: My golf GPS is accurate to within 1-2 yards virtually every time I take a reading, after taking hundreds, if not thousands of readings.

Your experiment seems to have proven your unit seems to be less accurate.

I also wonder if you haven't compounded the error you unit has by using it to check a point you measured with it in the first place? Say, it was reading 3 yards off on the first measurement, then 2 yards off but in a different direction the next few times. You'd show a 5 yard error, which was really only 2-3 yards from the true origination point wouldn't you?

There have been a few threads in the past few months regarding accuracy, someone was trying to compare a Marine GPS unit to golf units. If you can find that thread you may see what has already been discussed, and why accuracy is pretty much a dead issue.

Wayneo · December 30, 2008

Do you know if the iGolf devices utilize WAAS/DGPS? - Yes, it uses WAAS that can be turned on/off by the user. I've used it both ways.

More about the SG5 .........

"At the heart of the SkyCaddie SG5’s is a custom-engineered GPS receiver from Swiss GPS receiver IP manufacturer u-blox. A cooperative engineering effort between ublox and SkyGolf resulted in the advanced positioning technology TruePoint. In the SkyCaddie SG5, TruePoint provides positioning accurate to under a meter and an ultra-fast time to first fix in a small, easy to use, hand-held device with extremely low power requirements. SkyGolf selected Sarantel’s GeoHelix GPS antenna for the SkyCaddie Pro Series."

Taken from http://www.gps-practice-and-fun.com/golf-gps-update.html

DaveLeeNC · December 30, 2008

They are within 1-2 yards each and every time I use it at a point that has been precisely measured by at least two different agencies. That being the course itself, and Sky Golf. My SG-5 also correlates to within 1-2 yards of my Leica rangefinder.

Conclusion: My golf GPS is accurate to within 1-2 yards virtually every time I take a reading, after taking hundreds, if not thousands of readings.

Your experiment seems to have proven your unit seems to be less accurate.

I also wonder if you haven't compounded the error you unit has by using it to check a point you measured with it in the first place? Say, it was reading 3 yards off on the first measurement, then 2 yards off but in a different direction the next few times. You'd show a 5 yard error, which was really only 2-3 yards from the true origination point wouldn't you?
There have been a few threads in the past few months regarding accuracy, someone was trying to compare a Marine GPS unit to golf units. If you can find that thread you may see what has already been discussed, and why accuracy is pretty much a dead issue.

I never knew or cared about what the distance to my reference points actually was. As I stated in the experiment description, all I was measuring was the repeatability of a measurement. I stored four points that were "somewhere" and just measured the distance from a different fixed point to those four stored fixed points that didn't move (they were recorded as fixed coordinates in my GPS device), just determining how those readings varied over a week or so. Accuracy can't be any better than that, but could be worse (probably not worse if your mapped points are accurate in the first place).

FWIW the Garmin site as of 12/30/08 states that (to a 95% confidence level) you get "better than 3 meter accuracy with WAAS" ( http://www8.garmin.com/aboutGPS/waas.html ). I would interpret this as somewhere between 2 and 3 meters (95% of the time).

The FAA does regular WAAS accuracy evaluations and typically finds an absolute max error between 3 and 4 meters. I would assume (but am not an expert in this by any means) that they are using the best receiving equipment and calculations (probably done off line rather than real time), so this would represent the best that can be achieved with the WAAS system as implemented right now. See page 16 at http://www.nstb.tc.faa.gov/reports/waaspan26.pdf (the 10/08 report). I would guess that how far you are from a WAAS receiver is a factor here (400 miles for me IIRC) and in your specific case, 2 meter accuracy might well be real.

dave

ps. I ultimately became a laser guy simply because the answer that I cared about the most (accurate yardages TO THE PIN from inside 100 yards) just isn't doable with GPS. The accuracy even with my older system was OK w/me for usage well outside 100 yards. I am guessing that "combo systems" (laser and GPS) will be available in the next 3-5 years. I would rather have GPS data (yardage to front, middle, back) from 180 yards rather than laser (yardage to the pin). But yardage to the target from inside 100 yards (both practice and on the course) is my primary requirement.

draver · December 30, 2008

Dave,

My point on the compound error is, you stored a point with a device then remeasured to that point. If the first point is not precise, then all your measurements begin with that amount of error, plus whatever is introduced by the new measurement. As an example, your device stores N33.3333, W152.2222, but the true location is: N33.3331, W152.2219. From that point forward you believe you are measuring to the first set of numbers, you're really measuring to the second set and reading the difference as a distance. This cannot produce the type of measurement you're using for your accuracy evaluation. I also suspect there is some additional rounding of the numbers on the internal calculations that may contribute to this compound error.

This may not be as clear as I'd hope, but I would suggest using 3-4 permanent markers at a local course to determine how repeatable your unit really is, not your own units' positions.

DaveLeeNC · December 30, 2008

Dave,

My point on the compound error is, you stored a point with a device then remeasured to that point. If the first point is not precise, then all your measurements begin with that amount of error, plus whatever is introduced by the new measurement. As an example, your device stores N33.3333, W152.2222, but the true location is: N33.3331, W152.2219. From that point forward you believe you are measuring to the first set of numbers, you're really measuring to the second set and reading the difference as a distance. This cannot produce the type of measurement you're using for your accuracy evaluation. I also suspect there is some additional rounding of the numbers on the internal calculations that may contribute to this compound error.

This may not be as clear as I'd hope, but I would suggest using 3-4 permanent markers at a local course to determine how repeatable your unit really is, not your own units' positions.

We'll probably end up having to agree to disagree on this one, but there was no "intended point". I actually started out to do the experiment that you suggested, but it didn't take any time at all to realize that is was an impossible experiment. I had no way to store a "known point". All I could do was to mark a known point using the technology that I was trying to assess in the first place. This marking process had (probably) the same error as what I was trying to figure out. So I decided to take the problem apart and went after repeatability first (which, from what I could tell in reading about GPS technology, was probably most of the error assuming a 100% accurate marked point).

All I needed was a fixed point somewhere in the vicinity of where I intended (I wanted points in four different directions and somewhere between 100 and 200 yards out). Take the example of a digital scale that you are trying to assess. You could just go find a crowbar in the garage and measure it with the scale a bunch of times across the range of conditions that cause uncertainty in the scale.

Assume that you got the following

20 readings of 9.1 pounds

20 of 9.2 pounds

20 of 9.3 pounds

20 of 9.4 pounds

20 of 9.5 pounds

Unless there is something really weird going on about reading the weight of a crowbar, there is no way that this particular scale (for purposes of weighing things that weigh about as much as a crowbar) is more accurate than +- 0.2 pounds. Note that we have no idea what the crowbar really weighs, so the scales might well be off by more than +-0.2 pounds. And it might well be biased one way or the other, but this particular scale gives answers that vary randomly (from what we know) across a window that is 0.4 pounds wide (for purposes of weighing things of roughly the weight of a crowbar).

That is all that I did. I cared about this because my intention here was to build kind of a "poor man's Skycaddie" and mark all four sides of each green on the courses that I play to help me in assessing yardages to the pin from inside 100 yards (where 5 yards off does matter to me). Ultimately it was easier to buy a laser rangefinder - sold my GPS to a friend who wanted it. He is an older guy who doesn't get around really well any more and he wanted to avoid hunting up sprinkler heads - it was perfect for him.

dave