Why are course ratings and slope not based on score data?

[ChaosTheory]

This seems like a great project or dissertation for a Phd statistics program.

You're welcome.  🙂

[DaveLeeNC]

20 hours ago, Pepperturbo said:

Establishing course rating off scoring is a recipe for excessive labor, and who knows what for human error and input.  As I see it, two pieces of data make the process relatively simple, scratch for rating, and bogie golfer for slope. 

 

""I don't understand why the course rating wouldn't adjust over time based on the scores actually shot on that course.  It seems to me that over the long run, basing the course rating on actual scores would be much more accurate.""

 

Based on above, rating would fluctuate based on who plays the course, and how many post erroneous scores.   Say a group of (25) buddies play the course; none of them have indexes above 4, ten are scratch.  After they post their scores, the course rating would be changed.  Two weeks later 30 bogie buddies play the course, most post typical scores, but 10 or so post lower than actual; those would affect slope.  The course rating and slope would be worthless.

If you are going to do that you would have adjust for the index of the players posting the score. Then you still have the issue that not all bogey golfers are the same (short hitting 80 year old vs. long hitting once a month 20 year old).

 

dave

[North Butte]

7 minutes ago, DaveLeeNC said:

Then you still have the issue that not all bogey golfers are the same (short hitting 80 year old vs. long hitting once a month 20 year old).

And a 35-year-old bogey golfer paying $155 to subscribe to Arccos because he's aspirational is not the same as a retired bogey golfer playing in Mon-Fri morning men's group at the local country club. 

[Pepperturbo]

16 hours ago, PNWGuy said:

Actually, the statisics are simple and tons of data are available in electronic form in the WHS and GHIN databases.  Everything can be calculated automatically in real time.

 

The problem is if there is a "bias" (sand bagging?) in the posted scores as North Butte implies, the statistics may not be accurate.  I happen to play with a group that posts honestly, I can't speak for the integrity of others.  Also, there are simple analytical statistics that compare tournament scores to non-tournament scores to indicate potential sandbaggers.

Always entertained hearing all the data is online, easy peasy... Yet, nobody with actual successful business experience values online data.  Me thinks it has something to do with "just because it's online doesn't make it true or useable in the real-world."  LOL

 

As we both have known, sandbagging is a problem - no matter what formula is used.  To my thinking, it's easier to use a less labor intensive data vs info online that may or may not be accurate.

Edited November 13, 2022 by Pepperturbo

[klebs01]

This process of using the shot data seems very circular. How do you know if the player is good or not only using the data? Hard course or bad player? At the end of the day, I’m not sure what problem is being solved. 
 

also, the shot data is very questionable. I’ve never seen a decent golfer using one.  I don’t trust 5he accuracy, and I’m not sure that many people scrub the data properly. In the end, garbage in garbage out. 
 

 

[North Butte]

5 minutes ago, Pepperturbo said:

Always entertained hearing all the data is online, easy peasy... Yet, nobody with actual successful business experience values online data.  Me thinks it has something to do with "just because it's online doesn't make it true or useable in the real-world."  LOL

There are certainly ways to apply ML and what have you to data gathered online, in terms of figuring out how to target advertising and so forth. But nobody in science or industry is going to be around long if they think you can just download a boatload of "free" data and take it at face value. 

[jvincent]

3 hours ago, klebs01 said:

This process of using the shot data seems very circular. How do you know if the player is good or not only using the data? Hard course or bad player? At the end of the day, I’m not sure what problem is being solved. 
 

also, the shot data is very questionable. I’ve never seen a decent golfer using one.  I don’t trust 5he accuracy, and I’m not sure that many people scrub the data properly. In the end, garbage in garbage out. 
 

 

 

I know several very good golfers who use Arccos.

 

And if you use the PGA Tour ShotLink data then the data is very accurate.

 

Where it gets a little bit dodgey is how do you separate "scratch" golfer data from "bogey" golfer data? The obvious way is to group the data based on handicap index. So now you have datasets based on self identification of ability.

 

The WHS approach to this is based on driver distance, the ability to hit two shots a total of 470 yards, and some words about ability play to zero course handicap which is presumably a round about way of describing the overall quality of their shots/putts.

 

A data based approach would be to define distance / dispersion metrics for the scratch and bogey golfers for various distances as the sort criteria.

[North Butte]

10 minutes ago, jvincent said:

A data based approach would be to define distance / dispersion metrics for the scratch and bogey golfers for various distances as the sort criteria.

Everything you've said in this thread seems targeted at building detailed statistical models of shot dispersion on a certain golf course. That is simply a non-sequitur when talking about the basis relative to which handicap indices are defined.

 

Even if it were practical to build a model that somehow predicts the shot dispersion on every hole of the course for every 70-year-old women, 15-year-old boy, scratch golfer, 36 handicap beginner and everyone in between, it adds absolutely nothing to the goal of providing handicaps allowing fair competition between golfers of different abilties.

 

You don't need to know that a 14-hcp, short-hitting, 60-year-old senior male averages 27 yards from the center of the 17th green when approach from 140 yards. That information can't possibly be used in a handicap calculation. That's self-improvement Arccos/ShotScope navel gazing stuff.

 

So I'll ask one more time and promise not to repeat it again. What shortcoming of the WHS handicap system do you think needs fixing by your propose Big Data algorithm? 

 

And do not say just say "Bias" which has so far been your only description of the purported problem you're trying to solve. Or do say it, I don't care at this point.  

Edited November 13, 2022 by North Butte

[jvincent]

3 minutes ago, North Butte said:

Everything you've said in this thread seems targeted at building detailed statistical models of shot dispersion on a certain golf course. That is simply a non-sequitur when talking about the basis relative to which handicap indices are defined.

 

Even if it were practical to build a model that somehow predicts the shot dispersion on every hole of the course for every 70-year-old women, 15-year-old boy, scratch golfer, 36 handicap beginner and everyone in between, it adds absolutely nothing to the goal of providing handicaps allowing fair competition between golfers of different abilties.

 

You don't need to know that a 14-hcp, short-hitting, 60-year-old senior male averages 27 yards from the center of the 17th green when approach from 140 yards. That information can't possibly be used in a handicap calculation. That's self-improvement Arccos/ShotScope navel gazing stuff.

 

So I'll ask one more time and promise not to repeat it again. What shortcoming of the WHS handicap system do you think needs fixing by your propose Big Data algorithm? 

 

And do not say just say "Bias" which has so far been your only description of the purported problem you're trying to solve. Or do say it, I don't care at this point.  

 

No, it is not about building a model per golf course.

 

It is based on building two models. A shot distribution pattern for a "scratch" golfer and a shot distribution pattern for a "bogey" golfer. You then apply the models to every golf golf course out there and generate average scores for both models on every golf course.

 

The scratch scoring average becomes the rating and the bogey average is used to calculate slope. Doing it this way would actually allow you to create different slopes for different index ranges, but that's probably more complicated than it needs to be.

 

The shortcoming with the current rating system is that it is entirely dependent on the course raters assessment and a couple of mystery adjustment factors. Yes, there was probably some analysis done to come up with those factors, but it was done in an era where the amount data available was not nearly what we have today. If you can explain to me where the 40.9 constant in the course rating equation comes from I'm all ears.

 

As I've said repeatedly, we there is better data available today. We should use it.

[klebs01]

50 minutes ago, jvincent said:

 

I know several very good golfers who use Arccos.

 

And if you use the PGA Tour ShotLink data then the data is very accurate.

 

Where it gets a little bit dodgey is how do you separate "scratch" golfer data from "bogey" golfer data? The obvious way is to group the data based on handicap index. So now you have datasets based on self identification of ability.

 

The WHS approach to this is based on driver distance, the ability to hit two shots a total of 470 yards, and some words about ability play to zero course handicap which is presumably a round about way of describing the overall quality of their shots/putts.

 

A data based approach would be to define distance / dispersion metrics for the scratch and bogey golfers for various distances as the sort criteria.

which courses have shotlink data for scratch golfers? I’ve never seen any shot link equipment outside a pga stop. 

[PNWGuy]

4 hours ago, Pepperturbo said:

Always entertained hearing all the data is online, easy peasy... Yet, nobody with actual successful business experience values online data.  Me thinks it has something to do with "just because it's online doesn't make it true or useable in the real-world."  LOL

 

As we both have known, sandbagging is a problem - no matter what formula is used.  To my thinking, it's easier to use a less labor intensive data vs info online that may or may not be accurate.

I think the people at Alphabet (Google) and Meta (Facebook and Instagram) generate huge revenues by selling online data.  Advertisers analyze that data and use very effectively it to identify customers for their products.  That's why we are flooded with individualized ads.

I know you mentioned that you are the CEO of a company.  I don't know how big your company is or what industry you operate in.  I assure you large corporations with actual successful business experience value and make mind boggling profits by harvesting and using online data.

[jvincent]

1 minute ago, klebs01 said:

which courses have shotlink data for scratch golfers? I’ve never seen any shot link equipment outside a pga stop. 

 

Arccos and others have shot data for non PGA tour golfers. I was saying that if the model wanted to use PGA Tour data, the data exists.

 

And remember, shot data is NOT course specific. If you have 150 yards to the target it doesn't matter where that is.

[North Butte]

31 minutes ago, jvincent said:

 

No, it is not about building a model per golf course.

 

It is based on building two models. A shot distribution pattern for a "scratch" golfer and a shot distribution pattern for a "bogey" golfer. You then apply the models to every golf golf course out there and generate average scores for both models on every golf course.

 

The scratch scoring average becomes the rating and the bogey average is used to calculate slope. Doing it this way would actually allow you to create different slopes for different index ranges, but that's probably more complicated than it needs to be.

 

The shortcoming with the current rating system is that it is entirely dependent on the course raters assessment and a couple of mystery adjustment factors. Yes, there was probably some analysis done to come up with those factors, but it was done in an era where the amount data available was not nearly what we have today. If you can explain to me where the 40.9 constant in the course rating equation comes from I'm all ears.

 

As I've said repeatedly, we there is better data available today. We should use it.

So your suggestion boils down to taking the model of scoring (for scratch and bogey) that USGA is using based on their decades of rating courses with a model of scoring (for scratch and bogey) based on data accumulated by Arccos users over the past several year. 

 

In that case, much ado about nothing. In the end your model and the existing model both examine the physical attributes of a golf course and assign two numbers, scratch rating and bogey rating. One uses heuristics whose parameters have evolved by successive refinement of the USGA system over many years. The other would use predictive models whose parameters are fine tuned by a team of programmers based on Arccos (or similar) data. 

 

There's not going to be a hill of beans difference in the scratch and bogey ratings of most courses between those systems. And when there is a difference, which set of rating parameters do you think will appear more transparent and believable to the majority of golfers. The ones derived from a proprietary database of shot tracking from the tiny subset of golfers who subscribe to a tracking app? Or the ones derived from a team of raters going out and evaluating the course on the ground? 

 

The entire problem you propose to address is you, personally, don't trust that the USGA rating teams know what they're doing and you do trust Arccos (or whatever tracker data) and a bunch of algorithms to do a better job. I'd reckon you are in an extreme minority in that view. 

 

There is no demonstrable problem with the current rating system. Just your gut feeling that surely a bunch of guys out there measuring stuff and running it through a formula can't be as good as Arccos measuring stuff which you run through a formula.  

[klebs01]

11 minutes ago, jvincent said:

 

Arccos and others have shot data for non PGA tour golfers. I was saying that if the model wanted to use PGA Tour data, the data exists.

 

And remember, shot data is NOT course specific. If you have 150 yards to the target it doesn't matter where that is.

So we arrive right back where we started with two statistical golfers and measurements from point to point on a golf course. 

[jvincent]

Just now, klebs01 said:

So we arrive right back where we started with two statistical golfers and measurements from point to point on a golf course. 

 

I would argue that the current definitions are not statistical. The WHS formulas have some baked in numbers which presumably have been adjusted to get numbers that "feel right". The obstacle factors based on the judgement (yes, subject to guidelines) of the rating team.

 

The truly statistical model I am proposing is different. Specify the shot distribution patterns and let the computer do its thing. Some shots in the model will go OB or into a hazard and result in higher scores. Most won't. Have the computer "play" a hole 1M times and get an average score. Add the scores and you get the ratings. 

[North Butte]

5 minutes ago, klebs01 said:

So we arrive right back where we started with two statistical golfers and measurements from point to point on a golf course. 

Yep.

 

I see now I totally misunderstood what he was proposing. He just thinks his team of crack programmers and a few gigabytes of Arccos data can examine a satellite view of my home course and assign values to those two parameters that are better than what the rating team came up with when they visited to play and measure the course. 

[jvincent]

19 minutes ago, North Butte said:

The entire problem you propose to address is you, personally, don't trust that the USGA rating teams know what they're doing and you do trust Arccos (or whatever tracker data) and a bunch of algorithms to do a better job. I'd reckon you are in an extreme minority in that view. 

 

There is no demonstrable problem with the current rating system. Just your gut feeling that surely a bunch of guys out there measuring stuff and running it through a formula can't be as good as Arccos measuring stuff which you run through a formula.  

 

Again, I'm saying there is a better, more data driven way to generate the ratings that removes any subjectivity from the process.

 

Edited November 13, 2022 by jvincent

[North Butte]

2 minutes ago, jvincent said:

 

Again, I'm saying there is a better, more data driving way to generate the ratings that removes any subjectivity from the process.

 

No it removes the subjectivity on the part of course raters with subjectivity of the guys running those algorithms that convert satellite images to course ratings.

 

Surely you realize, someone goes through each satellite image of each hole and guides the software through mapping fairways/green/bunker/rough whatever when Arccos maps a course. And in your hypothetical system, someone will be doing that to apply you derived model parameters to each course in the handicap database.

 

At some point it's all got subjectivity, measurement error and the sort of "mystery factors" you find so worrisome in the USGA rating procedure. Somebody, somewhere tweaks the whole thing until it looks about right. You simply trust a bunch of algorithm designers and satellite mapping jockeys more than you trust USGA committees and volunteer rating teams.

[klebs01]

5 minutes ago, jvincent said:

 

Again, I'm saying there is a better, more data driving way to generate the ratings that removes any subjectivity from the process.

 

There isn’t. Both scenarios is are just attempts to define a “good” golfer and a “less good” golfer. Then use those definitions to measure obstacles and run the calculation predicting score. These are exactly the same thing.  You just see it as better because it’s new and different the in fact they are the exact same thing. 

[DaveLeeNC]

11 minutes ago, jvincent said:

 

Again, I'm saying there is a better, more data driven way to generate the ratings that removes any subjectivity from the process.

 

I don't have a dog in this  hunt,  but 150 yards into the 9th green of Pinehurst No. 2 is not the same as 150 yards into the 8th green of Pinehurst No. 1.  Subjectivity can sometimes be a big help.  dave 

Edited November 13, 2022 by DaveLeeNC

[jvincent]

1 minute ago, North Butte said:

No it removes the subjectivity on the part of course raters with subjectivity of the guys running those algorithms that convert satellite images to course ratings.

 

Surely you realize, someone goes through each satellite image of each hole and guides the software through mapping fairways/green/bunker/rough whatever when Arccos maps a course. And in your hypothetical system, someone will be doing that to apply you derived model parameters to each course in the handicap database.

 

At some point it's all got subjectivity, measurement error and the sort of "mystery factors" you find so worrisome in the USGA rating procedure. Somebody, somewhere tweaks the whole thing until it looks about right. You simply trust a bunch of algorithm designers and satellite mapping jockeys more than you trust USGA committees and volunteer rating teams.

 

There's nothing subjective about whether an area of a golf course is fairway/rough/bunker. It's either mapped correctly or it isn't. 

 

Measurement errors, assuming you are talking about shot data, probably get smoothed out with the size of the data set.

 

[jvincent]

4 minutes ago, klebs01 said:

There isn’t. Both scenarios is are just attempts to define a “good” golfer and a “less good” golfer. Then use those definitions to measure obstacles and run the calculation predicting score. These are exactly the same thing.  You just see it as better because it’s new and different the in fact they are the exact same thing. 

 

Clearly they aren't the same thing. Same goal, but differently methodologies which probably come up with different answers.

[North Butte]

4 minutes ago, jvincent said:

 

There's nothing subjective about whether an area of a golf course is fairway/rough/bunker. It's either mapped correctly or it isn't. 

 

Measurement errors, assuming you are talking about shot data, probably get smoothed out with the size of the data set.

 

There's no future in this. You're not talking about the real world. 

I'm out. 

[jvincent]

4 minutes ago, DaveLeeNC said:

I don't have a dog in this  hunt,  but 150 yards into the 9th green of Pinehurst No. 2 is not the same as 150 yards into the 8th green of Pinehurst No. 1.  Subjectivity can sometimes be a big help.  dave 

 

I don't remember those holes specifically, but over the long term if you are trying to hit a ball 150 yards there no statistical difference in the dispersions.

 

I'm ignoring elevation effect here assuming there are any.

[jvincent]

Just now, North Butte said:

There's no future in this. You're not talking about the real world. 

I'm out. 

 

Agree to disagree.

[DaveLeeNC]

3 minutes ago, jvincent said:

 

I don't remember those holes specifically, but over the long term if you are trying to hit a ball 150 yards there no statistical difference in the dispersions.

 

I'm ignoring elevation effect here assuming there are any.

 

No offense, but that is nonsense. Pinehurst (and many golf courses) have holes where you miss the green by a few yards and you are 30+ yards away when the ball stops (maybe on 2 sides, maybe all 4 sides, ....). 

 

This will make a HUGE difference in dispersions. I landed a ball on the 12th green of Pinehurst No. 6 the other day. It landed on the front edge (middle pin - poorly struck iron). My next shot was around 50 yards from the pin (short of the green). 

 

dave

[jvincent]

9 minutes ago, DaveLeeNC said:

 

No offense, but that is nonsense. Pinehurst (and many golf courses) have holes where you miss the green by a few yards and you are 30+ yards away when the ball stops (maybe on 2 sides, maybe all 4 sides, ....). 

 

This will make a HUGE difference in dispersions. I landed a ball on the 12th green of Pinehurst No. 6 the other day. It landed on the front edge (middle pin - poorly struck iron). My next shot was around 50 yards from the pin (short of the green). 

 

dave

 

Ah, I see what you are saying now.

 

My gut tells me that the effect of this is smoothed out by a large enough data set.

[DaveLeeNC]

14 minutes ago, jvincent said:

 

Ah, I see what you are saying now.

 

My gut tells me that the effect of this is smoothed out by a large enough data set.

 

The larger sample size will more reliably reveal the difference - whatever that might be and my example was extreme. But balls falling off edges 15'ish yards more than 'normal' is pretty common around here (as is balls staying on edges - depends on the green). 

 

dave

[Pepperturbo]

21 hours ago, PNWGuy said:

the people at Alphabet (Google) and Meta (Facebook and Instagram) generate huge revenues by selling online data. 

Being a business owner, I've seen internet data from both those companies, just don't rely on it to make decisions.  Why?  Your two examples are similar to Twitter, too much vaporware...  When Musk told Twitter he wanted an accurate measure of account data to merit value, they freaked.  If he had taken their online data at online face value, he would have paid too much. 

 

Online data presented to businesses such as mine, is too often discolored by bots of a human kind; look what's happened to Wikipedia since it opening it up to editors, a serous lack of accuracy.

[betarhoalphadelta]

On 11/11/2022 at 5:26 PM, highbombs77 said:

That is a good point about needing to fix a value to calculate other relative values.

 

I still feel that the handicaps and course ratings could be initially calculated as they are now.  Then if any course is consistently high or low in some measure of average net score over a long enough period of time where there are 1000's of rounds, you should adjust the rating. 

 

I wonder if this data exists anywhere?  If an area has 10 courses is there data that course A has an average net score of +3.1, course B has an average net score of +3.6, course C +2.1 etc.  I would assume GHIN must be able to calculate this data since you could know the course handicap on the day of play and the final score for each player. 

 

I think you'd run into a lot of problems trying to do this, even if you assumed that everything in GHIN is perfectly accurate and played to the official RoG, which we know isn't the case...

 

• Private courses where there is very little overlap between players on that course and nearby public (or other private) courses, so the private course effectively becomes a "closed system" where you can't get enough data on the quality of the players to accurately rate the course based on scores.
• Courses which effectively select certain strata of customer. Better golfers are probably more invested in the game, and play more rounds at harder (often more expensive) courses. Worse golfers are playing a lot of public/muni courses which are cheap and designed to be easy so you don't lose a ball every time you have an errant shot. If you don't have as much overlap, you can have data which is of reduced value because of selection bias. 
• Golfers who play certain courses much more often than others will get to know those courses, i.e. the basis for whether your index "travels". If I play 40 rounds a year at 2 area courses, and play 6 rounds a year at various other area courses, I'm liable to score worse for my cap on those 6 rounds because I don't know the course, the acceptable miss areas, the green complexes and break for putts, etc. 
• Most recreational US golfers (as far as my limited anecdotal data suggests) don't even carry a handicap. Many who do (myself included) do so mostly to track our own improvement. I.e. I don't compete in tournaments nor do I compete for money outside of our little low-stakes skins game where a Grint "estimated" cap is sufficient for us to assign strokes. So the handicap system is ALREADY biased towards players who take the game more seriously, which is the same issue as self-selected Arccos users. You may be introducing selection bias simply by relying on GHIN. 

 

It may be easy to think that with a sufficient enough sample size, all of these problems will disappear, but I'm not sure that points like the above doesn't introduce enough bias into the system that you could never tease it out. And given that self-reporting in GHIN may or may not be rounds played to the RoG, you have an unreliable data set to begin with. You would have to assume that rates of inaccuracy would ALSO even out from one course to another, but I'm guessing that rounds played at a lot of low-end public/muni courses have higher rates of scores that don't even approximate the RoG than courses that attract better players. 

 

Is the USGA system perfect? Probably not. Is anything about it being imperfect something that I'd really call it broken? Also, probably not.