Why are course ratings and slope not based on score data?

[highbombs77]

My understanding of course rating (in the US)  is that a person from the governing body walks the course and measures the length of holes, size of the fairway, size of hazards, etc and then calculates a course rating and slope from each tee box.  Once this course rating is calculated, it remains the same until the course is re-rated in about 10 years or so.   Please correct me if this is wrong

 

I don't understand why the course rating wouldn't adjust over time based on the scores actually shot on that course.  It seems to me that over the long run, basing the course rating on actual scores would be much more accurate.   For example imagine a private club that has 2 courses.  Both are rated 70.0/113 by the rater.  Both are played frequently by the same group of members many times a year. If one course has an average net score of +2, and another course has an average net score of +6, it seems like it would make sense to change the ratings to reduce this disparity.  Make the rating of the hard course harder and the easy course easier to even things out.  I say average to be simple but you could use mean, median, 25th percentile, or whatever stat you want. 

 

Thoughts?

 

 

[North Butte]

It's a chicken or egg problem. A scratch golfer (or a bogey golfer or a golfer with a 15.7 index) is defined by the scores he posts on courses that have been rated on an absolute scale involving, as you say, measurements and so forth. Every golfer who holds a handicap is rated by his index relative to those absolute course ratings.

 

If you wanted to base the course ratings on scores, whose scores would you use? You'd just be starting with "golfers" having no handicap index to rate their ability (because you can only give them indices once the course ratings have been established).

 

With neither course/slope ratings or golfer handicap indices fixed and absolute the two would be circularly defined. There would be infinitely many pairs of course/slope ratings and golfer handicaps that were all equivalently valid.

 

The historical origins of that rating process to which you refer was to construct two theoretical standard golfers. One was called "scratch" and one was called "bogey". Buried in the handicap manual somewhere definitions about how far each them drive the bal and so forth. But the rating process is a thought experiment by the rating panel (not an individual but a small group actually) who go around the course taking measurements and making semi-objective evaluations of various aspects with the goal of coding each element of the course on standardized rating forms. Then those elements are combined and an estimate is made of how difficulty (resistant to scoring) the course might be for the standard "scratch" golfer and the standard "bogey" golfer. Those two estimates form the basis of the course rating (difficulty for "scratch") and slope rating (relative difficulty for "bogey" compared to "scratch"). 

 

At some point in the process, some parameters in the equations have to be fixed to identify a unique solution. You can define handicap versus course/slope ratings and course/slope ratings versus handicaps in a circular manner with no fixed anchor points. 

[highbombs77]

That is a good point about needing to fix a value to calculate other relative values.

 

I still feel that the handicaps and course ratings could be initially calculated as they are now.  Then if any course is consistently high or low in some measure of average net score over a long enough period of time where there are 1000's of rounds, you should adjust the rating. 

 

I wonder if this data exists anywhere?  If an area has 10 courses is there data that course A has an average net score of +3.1, course B has an average net score of +3.6, course C +2.1 etc.  I would assume GHIN must be able to calculate this data since you could know the course handicap on the day of play and the final score for each player. 

[North Butte]

If the owners of the course feel the ratings are leading to wrong handicaps they can request the course to be re-rated. That actually happens quite often, not unusual at all. A private club I belonged to years ago felt that way and the revised ratings actually changed by quite a bit. I think that was mostly because a lot of trees had grown up to tighten the course and it played slightly more difficult because of it. 

 

P.S. There is a per-round adjustment called PCC built into the handicap system. Basically, on any day when a certain minimum number of rounds are posted (fairly large number) if the average differentials across all those rounds are unsually large or small then all handicaps for that day will be adjusted by -2, -1, +1 or +2 strokes to compensate. I'd think if there were a serious rating problem with a course it would easy to notice that PCC kicks in all the time instead of just a couple times a year at random due to weather or course conditions. 

Edited November 12, 2022 by North Butte

[highbombs77]

Good point about PCC.  If the course rating is way off then it should kick in quite frequently.  That hasn't happened at my courses.  

 

This came up because the facility I play at the most has 2 courses.  The easier one is rated 69.6/127, the harder is rated 70.5/135.  My handicap is usually around 3-4, and I find that I average about 5 shots better on the easy course than the hard one.  And I feel that several others in my league are the same way. 

 

My handicap fluctuated a lot this year (low of 2.4, high of 5.9 that would have been 6.4 without reduced upward movement) and it seemed that the biggest factor was how many rounds I played on the easy course.  I had lots of differentials in the 0 to 4 range on the easy course, almost none on the hard course.  Maybe it is just an issue of comfort, confidence, or my game fitting the easier course better. 

[jvincent]

IMHO with all the shot data and course mapping data available today it would be very easy to create a statistical model for scoring that would take any bias out of the rating system.

[Newby]

12 hours ago, highbombs77 said:

Maybe it is just an issue of comfort, confidence, or my game fitting the easier course better. 

Very likely to be the case and the latter could be the most significant. Your playing style (strengths & weaknesses} may not exactly match the "standard" players mentioned by North Butte above.

[North Butte]

4 minutes ago, jvincent said:

IMHO with all the shot data and course mapping data available today it would be very easy to create a statistical model for scoring that would take any bias out of the rating system.

But that assumes you think there is some sort of “bias” in the rating system to start with. I see no evidence of it myself.

[Newby]

14 hours ago, highbombs77 said:

My understanding of course rating (in the US)  is that a person from the governing body walks the course and measures the length of holes, size of the fairway, size of hazards, etc and then calculates a course rating and slope from each tee box.  Once this course rating is calculated, it remains the same until the course is re-rated in about 10 years or so.   Please correct me if this is wrong

 

This data is available from the WHS. (I'm not sure about GHIN input in the US implementation). But clubs and other course management should (or could) be aware of scoring patterns and request regrading if they think it necessary.

 

Incidentally, 10 years is the minimum rating interval.

[North Butte]

When you boil an entire golf course down to two numbers which describe it from each set of tees there is no conceivable system that will perfectly reflect the difficulty for every one of the millions of golfers in the world.
 

If someone really thinks they can game the system into giving them a stroke or two of advantage by oh so clever selection of a certain set of tees on a certain course they are welcome to go for it. Most of us just want a fair handicap so we can play the darned game.

[jvincent]

4 minutes ago, North Butte said:

But that assumes you think there is some sort of “bias” in the rating system to start with. I see no evidence of it myself.

 

The ratings are done by humans, so even with guidelines there's always a chance of bias. Or even just differences between ratings teams.

[North Butte]

1 minute ago, Newby said:

This data is available from the WHS. (I'm not sure about GHIN input in the US implementation). But clubs and other course management should (or could) be aware of scoring patterns and request regrading if they think it necessary.

 

Incidentally, 10 years is the minimum rating interval.

At least in USA I have known courses to request and receive re-rating after just a year or two when it is obvious something isn’t quite right.

 

And in at least one case there were course renovations then a rating but once the course grew in and matured for a year it really played a lot differently.

[North Butte]

1 minute ago, jvincent said:

 

The ratings are done by humans, so even with guidelines there's always a chance of bias. Or even just differences between ratings teams.

Any algorithm you might want to come up with using big data is designed by humans as well, I would remind you. Not to mention all the utter made-up crap scores that are entered into GHIN every day. 
 

I’d much rather have my course rated by an experienced local rating committee than have it based on a bunch of vanity cap numbers typed ito an app by doofuses who have no intention of ever playing a round under the rules, ever. 

[jvincent]

22 minutes ago, North Butte said:

Any algorithm you might want to come up with using big data is designed by humans as well, I would remind you. Not to mention all the utter made-up crap scores that are entered into GHIN every day. 
 

I’d much rather have my course rated by an experienced local rating committee than have it based on a bunch of vanity cap numbers typed ito an app by doofuses who have no intention of ever playing a round under the rules, ever. 

 

Shotlink, Arccos, and Shotscope are all based on shot data, not scores. The sample set is so large that any anomalies in the data are inconsequential.

 

I know with a small team of programmers I could have a statistical handicap system built in under a year.

 

The only challenge would be for courses that don't have mapping data.

[North Butte]

8 minutes ago, jvincent said:

 

Shotlink, Arccos, and Shotscope are all based on shot data, not scores. The sample set is so large that any anomalies in the data are inconsequential.

 

I know with a small team of programmers I could have a statistical handicap system built in under a year.

 

The only challenge would be for courses that don't have mapping data.

Not one Arccos or Shotscope user in ten bothers to fully check and edit their shots after rounds. And again, relatively few Arccos and Shotscope users are playing strictly by the rules of golf. 

 

That data as a whole is approximately the same quality as GHIN postings. 

 

An algorithm that tries to tease "unbiased" (or even realistic) information about the real world from convenience-sampled self-reported data is doomed. There's no statistical or programming principle that says if you gather lots and lots and lots of garbage data, you'll eventually get something other than garbage out. 

 

P.S. And I hate to be so snarky but I've got to say it's funny to hear someone toss around the word "bias" so freely and then then turns around and suggest that USGA handicapping should be based on the 1% of rounds played every day using some shot tracker or another. That's just wild. 

 

Edited November 12, 2022 by North Butte

[jvincent]

10 minutes ago, North Butte said:

Not one Arccos or Shotscope user in ten bothers to fully check and edit their shots after rounds. And again, relatively few Arccos and Shotscope users are playing strictly by the rules of golf. 

 

That data as a whole is approximately the same quality as GHIN postings. 

 

An algorithm that tries to tease "unbiased" (or even realistic) information about the real world from convenience-sampled self-reported data is doomed. There's no statistical or programming principle that says if you gather lots and lots and lots of garbage data, you'll eventually get something other than garbage out. 

 

P.S. And I hate to be so snarky but I've got to say it's funny to hear someone toss around the word "bias" so freely and then then turns around and suggest that USGA handicapping should be based on the 1% of rounds played every day using some shot tracker or another. That's just wild. 

 

 

You clearly don't understand what I'm getting at.

 

With shot data the score does not matter. What matters is that for a shot of X yardage what is the statistical distribution of where that shot ends up. The rules of golf have no bearing on that data. Nor does score. 

 

So, with distribution data for shots of a given yardage you can statistically "play" a million rounds and generate an average score. And since the shot distribution data is different for scratch / bogey golfers you end up with two different average scores.

 

I would argue that the validity of the shot data available today is as good or better than the judgements of any rating team.

[North Butte]

2 minutes ago, jvincent said:

 

You clearly don't understand what I'm getting at.

 

With shot data the score does not matter. What matters is that for a shot of X yardage what is the statistical distribution of where that shot ends up. The rules of golf have no bearing on that data. Nor does score. 

 

So, with distribution data for shots of a given yardage you can statistically "play" a million rounds and generate an average score. And since the shot distribution data is different for scratch / bogey golfers you end up with two different average scores.

 

I would argue that the validity of the shot data available today is as good or better than the judgements of any rating team.

Only if the starting and ending points are accurately recorded. Which they may or may not be. 

 

And only if the 1% of shots being recorded are representative of the other 99% of shots hit by people who do not use trackers.  

 

You'll get the statistical distribution of GPS coordinates (subject to error) hit by people who happen to be using a GPS shot tracker. There is no way to get from that distribution to the distribution of shots hit by all golfers without making enormous, unjustifiable assumptions about how shot-tracker-recorded shots differ from shots hit by the other 99% of golfers. 

 

All of these tracking systems compute their own "handicap" and report it to the user. They are doing exactly what you describe, within the subset of golfer who use that system and for the subset of rounds that they record. So you don't need to create such a system, it exists.

 

In my experience, those Arccos or Game Golf or V1 Game computed handicaps do not agree well at all with my actual USGA handicap index or course handicap. They even seemed (for my game) to have systematic differences between the various trackers. So that's fine for what it is, a number produced for the entertainment and self-evaluation of golfers using a shot tracker. 

 

But there's no way to get from there to something usable by the millions of golfers who are never going to subscribe to Arccos or whatever. Those are the golfers served by actual handicap systems. And there's no need for those "handicaps" to agree anyway. The pseudo-handicaps generated by shot tracking apps are for people who want to self-evaluate. Real handicaps are for people looking to have a fair game with others of varying abilities.

 

[ChipNRun]

14 hours ago, highbombs77 said:

My understanding of course rating (in the US)  is that a person from the governing body walks the course and measures the length of holes, size of the fairway, size of hazards, etc and then calculates a course rating and slope from each tee box.  Once this course rating is calculated, it remains the same until the course is re-rated in about 10 years or so.   Please correct me if this is wrong

 

For starters, in 2015 I went out with a USGA course rating team fielded by the Metropolitan Amateur Golf Association of the St. Louis region. And, new courses get rerated after five years.

 

Here is the link to the updated 54-page USGA Course Rating Manual. This details the different factors rated on each hole to come up with the hole ratings. Each hole has its own worksheet, and the data for each hole is entered into a computer spreadsheet.

 

Factors taken into account include:

Quote

The Course Rating and Slope Rating is the evaluation of the playing difficulty of the course for the scratch player and the bogey player under normal playing conditions. The effective playing length is determined from the measurement of each hole, adjusted for the impact of roll, wind, elevation changes, altitude, dog-legs and forced lay ups. In addition to the effective playing length, there are 10 obstacle factors evaluated on each hole for both the scratch player and the bogey player. These are: topography; fairway; green target; recoverability and rough; bunkers; crossing obstacles; lateral obstacles; trees; green surface and psychology. The Course Rating System uses table values, adjustments and formulas to calculate ratings.

For more on this + overview of Rating/Slope interplay, see USGA Handicapping rules, Appendix G. 

 

14 hours ago, highbombs77 said:

I don't understand why the course rating wouldn't adjust over time based on the scores actually shot on that course. 

 

Reason? The course rating system uses a rating algorithm that is uniform in structure. And as NB pointed out,

14 hours ago, North Butte said:

If you wanted to base the course ratings on scores, whose scores would you use?

 

How golfers score on a given course involves in part the skill of the golfer.

15 hours ago, highbombs77 said:

For example imagine a private club that has 2 courses.  Both are rated 70.0/113 by the rater. ...

 

Being that each hole is rated on some 18 different factors, and the combined ratings of the 18 different holes create the course rating and slope, the chance of this occurring are very small. If this did occur, this what statisticians and economists call a unicorn, a freak occurrence. No way of telling, as this is a hypothetical.

 

If this did occur to a statistically significant degree, the course Greens Committee could request a re-rating.

[jvincent]

45 minutes ago, North Butte said:

Only if the starting and ending points are accurately recorded. Which they may or may not be. 

 

And only if the 1% of shots being recorded are representative of the other 99% of shots hit by people who do not use trackers.  

 

You'll get the statistical distribution of GPS coordinates (subject to error) hit by people who happen to be using a GPS shot tracker. There is no way to get from that distribution to the distribution of shots hit by all golfers without making enormous, unjustifiable assumptions about how shot-tracker-recorded shots differ from shots hit by the other 99% of golfers. 

 

Statistical analysis will tell you that the number of shots recorded on the various systems is well above the threshold required to be valid. If you're using validity as an argument then the hypothetical scratch and bogey golfers used by the raters are even less valid. 

 

The current rating methodology was devised in an era where we didn't have the tools and data we have now.

 

We've got better tools now so we should use them.

[North Butte]

You’re just throwing out words you don’t even understand. Validity is not established by sample size. The question is the representativeness of a tiny subset of self-selected shot tracker users for golfers in general. The 1% of golfers who choose to use a subscription shot tracking system are entirely different than people who are not interested in such things. No amount of non-representative data can magically replace all the missing data from people who would never use Arcos in a million years.

 

what you’re trying to design is a system that will in great detail reflect the nature of shot tracker users and in your opinion somehow do a better job of handicapping for them than a more general system aimed at all golfers. and to repeat myself that is already built in to the shot tracking apps.

 

Tell me again what alleged problem you are trying to solve by all this? You seem to be imagining that the handicap system is some sort of onanistic tool for golfers to judge themselves against a mass of the big data. that’s not a handicap system, that’s a shot tracking system, which you already have.

 

Why do you feel there is a need to replace the handicap system with some tweaked version of Arcos or Shotoink or whatever?

[jvincent]

You clearly don't understand how statistical sampling works.

 

You also seem to be brushing aside the fact that the current course rating system doesn't factor in how any actual golfers play. It's an arbitrary assignment of "effective" playing length based on an arbitrary assessment of obstacles for mythical scratch / bogey golfers.

 

Once again, typing slowly, the shot tracking system is just the source of the shot data. Based on shot data, for which there are billions (yes, billions) of data points, it is possible to calculate an average score with a high confidence factor that is based on actual data, not arbitrary factors.

 

For reference, are you aware what the actual course rating calculation is? To save you looking it up, here it is:

 

Scratch Rating = Scratch_effective_playing_length / 220 + 40.9

Bogey Rating = Bogey_effective_playing_length /  160 + 50.7

 

Use those numbers on your home course to figure out how much of a contribution the "obstacle factor" contributes to the course rating at your home course. At mine, it's a whopping 1.36 strokes for the scratch golfer. 

 

Where did the mystery constants in those equations come from? Are they based on some data?

 

Like I said earlier, we've got better tools, we should use them.

[North Butte]

You are implying random sampling when what you are proposing is using a convenience sample. Those are as different as chalk and cheese.

[StudentGolfer4]

Interesting discussion here and I’ll admit that I don’t fully understand everything that goes into course ratings. 
 

One thought I can’t shake, using scoring data to rate a course would require scores to be recorded accurately. I think we all know the percentage of golfers that play by the rules 100% of the time is slim to none. 

[Newby]

1 hour ago, jvincent said:

For reference, are you aware what the actual course rating calculation is? To save you looking it up, here it is:

 

Scratch Rating = Scratch_effective_playing_length / 220 + 40.9

Bogey Rating = Bogey_effective_playing_length /  160 + 50.7

 

Interesting. Where did you get those formulae?

 

The formula we work to for an initial or temporary SR is:

Scratch Rating = (Yardage / 220) + 42.0

and of the 42, 36 is taken care of by the number of expected putts.
 

Edited November 12, 2022 by Newby

[jvincent]

48 minutes ago, StudentGolfer4 said:

One thought I can’t shake, using scoring data to rate a course would require scores to be recorded accurately. I think we all know the percentage of golfers that play by the rules 100% of the time is slim to none. 

 

Scoring data is not what you want to use. Too many variables and inaccuracies in the data.

[jvincent]

46 minutes ago, Newby said:

Interesting. Where did you get those formulae?

 

The formula we work to for an initial or temporary SR is:

Scratch Rating = (Yardage / 220) + 42.0

and of the 42, 36 is taken care of by the number of expected putts.
 

 

I'd have to search for it again but it was from a rating manual.

 

I suspect that the difference in the 40.9 vs 42.0 is that for a temporary rating there's no adjustment data for obstacles so that formula uses actual length vs effective length.

[Pepperturbo]

Establishing course rating off scoring is a recipe for excessive labor, and who knows what for human error and input.  As I see it, two pieces of data make the process relatively simple, scratch for rating, and bogie golfer for slope. 

 

""I don't understand why the course rating wouldn't adjust over time based on the scores actually shot on that course.  It seems to me that over the long run, basing the course rating on actual scores would be much more accurate.""

 

Based on above, rating would fluctuate based on who plays the course, and how many post erroneous scores.   Say a group of (25) buddies play the course; none of them have indexes above 4, ten are scratch.  After they post their scores, the course rating would be changed.  Two weeks later 30 bogie buddies play the course, most post typical scores, but 10 or so post lower than actual; those would affect slope.  The course rating and slope would be worthless.

Edited November 12, 2022 by Pepperturbo

[ChaosTheory]

JV is saying, instead of breaking down a hole with theoretical distances of scratch and bogey golfers, and adjusting for obstacles, instead use actual distances and dispersion (which relates to obstacles) to run a lot of simulated plays of the hole.  Which would much more accurately rate the difficulty of a hole.

 

Makes a lot of sense to me.

 

Correct me if I’ve misstated you, @jvincent

[PNWGuy]

3 hours ago, Pepperturbo said:

Establishing course rating off scoring is a recipe for excessive labor, and who knows what for human error and input.  As I see it, two pieces of data make the process relatively simple, scratch for rating, and bogie golfer for slope. 

 

""I don't understand why the course rating wouldn't adjust over time based on the scores actually shot on that course.  It seems to me that over the long run, basing the course rating on actual scores would be much more accurate.""

 

Based on above, rating would fluctuate based on who plays the course, and how many post erroneous scores.   Say a group of (25) buddies play the course; none of them have indexes above 4, ten are scratch.  After they post their scores, the course rating would be changed.  Two weeks later 30 bogie buddies play the course, most post typical scores, but 10 or so post lower than actual; those would affect slope.  The course rating and slope would be worthless.

Actually, the statisics are simple and tons of data are available in electronic form in the WHS and GHIN databases.  Everything can be calculated automatically in real time.

 

The problem is if there is a "bias" (sand bagging?) in the posted scores as North Butte implies, the statistics may not be accurate.  I happen to play with a group that posts honestly, I can't speak for the integrity of others.  Also, there are simple analytical statistics that compare tournament scores to non-tournament scores to indicate potential sandbaggers.

[jvincent]

16 hours ago, Snowman9000 said:

JV is saying, instead of breaking down a hole with theoretical distances of scratch and bogey golfers, and adjusting for obstacles, instead use actual distances and dispersion (which relates to obstacles) to run a lot of simulated plays of the hole.  Which would much more accurately rate the difficulty of a hole.

 

Makes a lot of sense to me.

 

Correct me if I’ve misstated you, @jvincent

 

You've got it.