Ratings Intro Concerning The Set Up Of Formulas Options

[ethone]

Let me start with two issues I felt were the most important to me. With time I'll add new sections on specific ratings.
I'd like this to develop into a discussion about formulas and the theoretic approach to them. I don't care if you build your own or use one of the custom formulas probably to come. And especially this is not a critique of Snyderart's formula. If I refer to them, it is because they're the only publicly available formula and he has done a truckload of research on the topic.
I want this to be about information, deliberation and discussion.

Separating Ratings:
The major issue in calculating the ratings from the real life statistics is that the ratings in N2003 are seperated differently from the real life statistics. There is no real life statistic reflecting finishing. There isn't even one reflecting qualifying! In the latter case the car/team performance would play a vital role. As will the points situation, as non-Top35 teams tend to go kamikaze in qualifying and qualify well above their "objective" ability.
I put "objective" in quotation marks because apart from the seperating issue, objectivity in real life statistics is marred by another issue:
Incidents like pit stop screwups,
penalties and crashes taint statistics like AvgFinish. Jeff Gordon in 2008 has worse statistics than would with his crashes and DNFs sorted out. So while reliability is a seperate rating in N2003, in reality it figures prominently in AvgFinish too. This is a major issue and combating it seems largely futile.
Even if this might seem like fighting
a rash with a sunburn, I suggest not relying on AvgFinish alone. One way would be to include the DNF statistic whenever you use AvgFinish to counterbalance the cases when a bad AvgFinish comes along with a high number of DNFs. However this will give an advantage to those drivers who themselves (or their cars) are naturally bad. David Reutimann, as much as I like him, would profit from this approach as it stands to suspect that he would not have finished in the Top 10 every time he had a DNF.
This is a tradeoff for which one would need to know what one values most.
I'd rather see Jeff Gordon up front than in the middle all season because he had so many DNFs. On the other hand putting drivers up front who don't belong there isn't really a nice sight for most of you either I guess. When I played around with a ratings formula the other day I ended up with Kyle Petty winning at Daytona. That may be due to the huge random factor at Daytona and me using fixed Car ratings so he had a chance in the first place, but it was a sight I'd definitely want to avoid.

Range:
I'm not a fan of the ranges Snyderart used until now. For Consistency I would avoid a high number of 100 like the plague. It will make cars impossible to overtake for the AI, for example at Daytona with the lower line being key. No AI driver would be reasonably able to pass a Consistency-100 car on the bottom.
Also his findings with the rating points pool influencing who gets picked for the race and who doesn't means we need some control over the range to exert some influence on that. If ever I want ratings to be "capped", I want it to be the upper border rating so only the better drivers are affected. What I mean is that if I calculate a rating with a given range (e.g. 20), I make sure the lower rating always stays below 100. The higher rating may get capped at 100 if the lower rating is above 80 in this case.

[Snyderart]

The problem that I'm having with the formulas is that in order to get ALL of the drivers to participate in a season, their ratings have to be close. However, what this causes is drivers who made all 36 races to get bumped by another driver who is pretty much a "field filler". Plus, the ratings are so close to achieve this that the entire field rolls around the track in a big knot. I'm still looking at the "ratings pool points" as a possibility as that does seem to control who races when.

Another thought is instead of rating the cars using absolute values, maybe rate the drivers on a bell curve? Take each driver that was best in each area and that is the best rating possible for that rating and then rate the other drivers based on that?

The great thing about this program is that it is so easy to change ratings that a race-by-race rating system seems the best way to go now. That's just a matter of checking a box in the race list and two seconds later, you're done!

This post has been edited by Snyderart: Jun 7 2008, 09:21 AM

[ethone]

Getting the drivers to participate...
Yeah this is difficult. Your rating points pool seems sensible though. You would need to make sure that the lowest a Top35 driver can be (assuming he has a bad draw and gets the lower value on every rating) would be higher than the highest a non-Top35 driver can be (assuming he maxes out every rating). That would avoid a non-Top35 driver bumping a Top35 driver from a race.
For this one would need to keep the ranges predictable and constant (a from-100-down range would be difficult to implement).

About the Bell curve, I'm not sure I understand what you're getting at.
If you take the best driver in a category, he would automatically get the highest rating anyway.
Taking that driver and basing the difference in rating between him and the second best one on the difference in results is what we have now. Giving the second best driver a say 5 Point worse rating would not reflect how much worse a driver is in a category. For example if Ryan Newman is the best qualifier with an average starting position of 4.5, Jeff Gordon is the second best with 5.0 and Kyle Busch third best at 8.0, giving Ryan Newman a rating of say 100, Jeff Gordon one of 95 and Kyle Busch one of 90 would not reflect the real difference in skill.

Taking a Bell curve to determine the spread in the rating would be one way to accurately reflect the difference between how a driver qualified in his best race and his worst race. This would conflict with the fixed-range I think is necessary to predictably determine who gets into the race and who doesn't.

[PbFoot]

Hey guys, I've been working on creating a ratings XML file that would re-create realistic qualifying and finishing results on a single race basis.
The problem I'd been struggling with was getting a set of formulas that would do both- since the qualifying ratings are also influenced by other ratings, I'd either end up with trucks qualifying as they finished a race if I used finishing results for things like aero/chassis/engine ratings etc. or race results more like the qualifying results if I used more of the qualifying results to calculate the ratings. (IMG:style_emoticons/default/banghead.gif)

Then I thought what I needed were separate XML files- one for qualifying and one for racing! (IMG:style_emoticons/default/yahoo.gif)

Here's what I've been doing:

1. Load up a single race in NRatings and apply my "qualifying results" forumula.
2. Save the changes and create a roster if needed and close NRatings.
3. Start NR2003 and run through qualifying for the chosen race. Once qualifying is over, save the results and exit the game.
4. Start up NRatings again and load the roster you're working with.
5. Now import the results from that race again, but this time apply my "race results" forumla. Save and exit NRatings.
6. Fire up NR2003 again, go back to the same race you saved the qualifying results from and select yes to resume from qualifying and start racing!

All the testing I've been doing has been on big tracks (Daytona and California mainly) so I'm not sure what kind of results you'd get on the intermediate/short tracks. I also edit the track.ini file for a given track to get the AI to run realistic qualifying and lap times- stuff like drafting distance, drag, inverse slip curve...
Here's a link to the XML files I'm using.
Qualifying/Finishing XML files

Disregard the above link- I made a slight change to the finishing formulas but easy-share isn't letting me upload the new file for some reason so I've attached the XML files here.

Also attached a test race at California. Pretty comparable to the real race results IMO- except for Busch wrecking, or getting wrecked. (IMG:style_emoticons/default/drive2.gif)

This post has been edited by PbFoot: May 1 2008, 02:20 PM

Attached File(s)

 Finishing_ratings.xml ( 2.31K ) Number of downloads: 34
 Qualifying_ratings.xml ( 2.34K ) Number of downloads: 15
 cali_5_1.html ( 28.11K ) Number of downloads: 25

 

[msouphy]

Hail to da man for the first public release of custom written formulas (IMG:style_emoticons/default/headbang.gif)

I'll be trying them out tomorrow.

Keep up the good work, and may others follow his footsteps (IMG:style_emoticons/default/smile.gif)

[ethone]

The qualifying rating itself only has a very minor effect on the qualifying performance of a driver. However there's a way by modifying the papy_ai.ini to change that, which would be my preferred way of creating more realistic differences between qualifying and race performances.
I'll write it up as soon as I get the chance.

[ethone]

Ok here's what you can deduce from the papy_ai.ini.
The actual driving skill of an AI car is determined by other variables than the ratings might imply. For example the qualifying rating is just a modifier to existing ratings.

Driver Skill:
Traction circle (acceleration and braking efficiency):
- AI Strength (1.0 per 1 setting)
- AI Strength (8.0 spread per 1 Setting)
- Q: Qualifying rating (0.04 per 1 Setting)
- RC: Road Course rating (0.03 per 1 Setting)
- ST: Short Track rating (0.03 per 1 Setting)
- SW: Speedway rating (0.03 per 1 Setting)
- SS: SuperSpeedway rating (0.03 per 1 Setting)
- Chassis rating (0.05 per 1 Setting)
What it does:
Maximum deceleration is calculated by braking efficiency coefficient * traction circle (* mechanical issues)
Maximum acceleration uses it too.

Aero Drag (mostly top speed)
- AI Strength (0.3 per 1 Setting)
- AI Strength (4.0 spread per 1 Setting)
- Q: qualifying rating (0.06 per 1 Setting)
- Aero Rating (0.13 per 1 Setting)

Engine Power (mostly acceleration but also top speed)
- AI Strength (0.2 per 1 Setting)
- AI Strength (2.0 Spread per 1 Setting)
- Q: qualifying rating (0.03 per 1 setting)
- Engine rating (0.07 per 1 Setting)

Driver Consistency (ability to perfectly follow the .lp line?)
- base value: 0.95 [I guess it's overwritten with the actual rating]
- Finishing (10.0 per Setting) !!!
What it does:
. modifies driver_commitment by 0.04 per Setting. The higher the consistency, the higher the commitment.
. modifies the dlong_lookahead by 0.04 per Setting. The higher the consistency, the better a driver can look ahead (and identify crashes, possibly turns)
. modifies dlat_lookahead by 1.5 per Setting. The higher the consistency, the better a driver can look "beside" him and identify cars/grip levels
. modifies left_offset by 0.08 per Setting. I'm guessing it keeps the car more centered and in less danger of spinning due to hitting the apron

Pitcrew Speed: modified by pitcrew speed rating by 0.1 per Setting
Pitcrew Consistency: modified by pitcrew consistency by 0.9 per Setting

What you can see:
- Finishing has an effect (huge one actually) on the consistency variable NR2003 uses. I'm not 100% positive it is the same as the consistency rating though.
- Track Type rating modifies the braking and acceleration efficiency (which explains why a SS Track Type rating might subjectively not have much of an impact as on the restrictor plate tracks you hardly ever accelerate or decelerate).
- Qualifying rating modifies various variables by a tiny little bit

PbFoot, I'd rather modify the papy_ai.ini to make the qualifying rating more important than use two different rating formulas for each race. After all if the game would modify the engine and aero rating according to the qualifying rating, that would be the same as you're doing with your seperate formulas.

[PbFoot]

Hey I'm all for simple... if the papy_ai can be modified to save us all a step, all the better! (IMG:style_emoticons/default/cool.gif)

[ethone]

I'm currently testing these lines in the papy_ai:
driver_qualifying_to_traction_circle = 0.08
driver_qualifying_to_aero_drag = 0.08
driver_qualifying_to_engine_power = 0.060
Keep in mind that you'd need a good qualifying rating formula then. The higher the values, the more the qualifying rating alone will determine the qualifying results.

I attached my current formulas, which finally seems decent to me.
I had an issue with only the same drivers qualifying up front but that seems solved. At least Elliott Sadler got the pole in my last test at Daytona. (IMG:style_emoticons/default/wink.gif)
Qualifying ability is reflected in other ratings as well, like track type rating (a ratio of 4 (avgFinish) to 1 (avgStart)) and aero/engine. This is to counter good drivers getting bad ratings because they DNF'ed in a race and thus have a lower avgFinish than they would have had if they had finished normally.

This post has been edited by ethone: May 3 2008, 04:27 AM

Attached File(s)

 ethone_2008_v10.xml ( 7.27K ) Number of downloads: 42

 

[PbFoot]

Hey bud, I tried your formula at Daytona w/ the 07 CTS mod and the qualifying results were pretty scrambled. I'm guessing this is mainly due to the spread- for example Darnell sat on the pole in the race, but NRatings gave him a 79/100 qualifying range. This combined with his lower aero/engine/chassis ratings in the 75/85 range would keep him off the pole for sure, right?
I changed my papy_ai to what you posted also.

Are your formulas geared towards single events or a "rolling rating" system as the season goes on?

I see others have downloaded our formulas- any feedback from the community? (IMG:style_emoticons/default/thmbup.gif) or (IMG:style_emoticons/default/thmbdn.gif) ?

[nascarfan15]

i did a daytona cts 08 race with one of these rating sets and brent raymer won (IMG:style_emoticons/default/unsure.gif)

This post has been edited by nascarfan15: May 8 2008, 01:35 PM

[PbFoot]

Who's ratings did you use?

[Snyderart]

Well, here's my two cents. First, let me throw out the safety net and say that this ratings formula is a work in progress. The 100-down stuff is out the window as the cars were just too perfect. Looking at the results I'm getting, I'd say that I'm starting to zero-in on something here. I haven't tested this on a full season yet. I'm pretty much giving up trying to create a ratings formula that will try to include ALL of the drivers that drive in a season because it's almost impossible to get them to run right once they are in a race. I can get them all in at least one of the 36 races, but the results are really screwy.

With Nratings (did I say this was a COOL program?), it's so easy to create rosters for each race that there really isn't a need to have full season ratings any longer. However, what is cool about full year ratings is it mixes it up a little. The one thing that you have to make sure of is that you have all the drivers that ran that year or you'll come up with real life drivers not linking to car files. The attached ratings work for both, but after testing with single races, seemed to work pretty good. Not totally done with them, but getting close.

For those of you who haven't tried to create a single race roster and formulas for that roster, here's the steps.

To create a single race roster.

1. Go into Nratings, click on the series.
2. Once in the series, go into the roster you want to use (I use a roster with ALL of the drivers for a season in it for this).
3. Then import ratings from real life.
4. Choose the series and season (the season should match the roster you used above in step 2) you want to use.
5. Check the box "Do not calculate ratings..." (Were just creating a roster this time around).
6. Check the race that you want to create the roster for.
7. Click "OK".
8. Back at the main area, click on "Create roster from selected cars." Make sure that there are no drivers that did not map to car file, if so, get out of Nratings and create a car file with that driver's name and number correct (also find some paint for it), then go back through the steps until there are no unmapped carfiles.
9. If everything checks out OK, name your roster. Name it something you can recognized later like "Nextel 2007-04 Atlanta" for the 4th race of the Nextel series.

To rate the cars that are in the roster you just created.

1. Go into Nratings, click on the series.
2. Once in the series, go into the roster you want to use (I use a roster with ALL of the drivers in it for this).
3. Then import ratings from real life.
4. Choose the series and season you want to use.
5. Check the race that you want the rating formulas to use (should be the same as the above race in step 6).
6. Choose the custom rating formulas you want to use.
7. Click "OK".
8. Back at the main area, click on "Save modified cars."

You're done! Just do these steps before each race in your season. The beauty of this is a driver may have done great at one race and the following week ran like garbage, the ratings will now reflect that as you "update" the car ratings before each race.

msouphy, again thanks for a GREAT program!!! (IMG:style_emoticons/default/winner.gif)

This post has been edited by Snyderart: May 9 2008, 12:34 PM

Attached File(s)

 Snyderart_v1.xml ( 5.67K ) Number of downloads: 103

 

[msouphy]

Very cool stuff you came up with there SnyderArt !

Question:

to rate my 2008 COT cars, my tendency is to use all the available races up until now (so that is up to and including Richmond) to rate the cars. I do that because I like my AI cars to be "evened out", I mean if I were to rate single races then I'd be pretty sure Ryan Newman would win Daytona, etc ... now, after each real race weekend, I just select all races to rate the cars, so I even them out over the whole season.

However, I always create a carroster like for single races, so I simulate the real life participants into each race.

Will that methodology work allright with your new formula, or is it mainly focused on single race ratings ?

thx and have a good weekend all !

Mathieu

[rupe]

What do you think of these ratings. I started with snyderart's ratings and did extensive editing. download here.
Click on misc. button.

rupe

This post has been edited by rupe: May 22 2008, 10:28 PM

[msouphy]

What do you think of these ratings. I started with snyderart's ratings and did extensive editing. download here

rupe

Hi Rupe, thank you for the file, I'll be checking it out. One remark though, I've opened the xml you've made and seen you've renamed the tag <Author> into <Authors>

It's not a good idea to rename tags. Although the <Author> tag is currently not used in the program and only serves as information in the xml, it might be used in the future, but if you rename it then the program won't recognise it.

It would be good if you could update your file.

Also, you have to me a strange statement in <DriverQualifyingMin> --> if (@avgStart == +30)

In fact that is exactly the same as if (@avgStart == 30). So the @avgStart variable must be exactly 30 for your condition to be true. Sure you don't mean if (@avgStart >= 30) ?

[ethone]

To reply to the earlier question: I use ratings the way msouphy applies them.
I take all available races for a season and apply the ratings formula to it

My aim is having the drivers rated in a way that race winners are capable of winning and that drivers with an average finish of >30 will only show up in front if they're very lucky. I don't aim to recreate exact race results and in fact I put some time into making midfield drivers (e.g. Elliott Sadler) able to break into the top 10/5.
I'll rework the whole concept of my ratings with the new variables msouphy put into the new version, but they'll focus on a full season again.

[msouphy]

Thinking of something, maybe it would be a good idea to have a common space on the forum or even on the website to group all downloads of custom made formulas ?

OmegaSeven, can you propose something ?

[OmegaSeven]

I can propose a lot of things, most of them unhelpful (IMG:style_emoticons/default/tongue.gif)

It actually crossed my mind within the last couple of days. Getting a spot on the website right now probably isn't going to happen. Not because it's not worth the space or effort, but the site is in the midst of a system upgrade. The other issue I can see arising from it is updating any download on the site takes admin intervention. Since I can see formulas going through multiple evolutions, that's a major pain in the ass for the admins, which we try to avoid at all costs (they get cranky).

What we can try, if it sounds good, is to create a sticky topic in the forum for only formula downloads. The original NRatings thread is getting really long, and expecting a person to wade through 100+ posts to find the formulas is kind of inconvenient. I can create another thread for only the formulas. As a person creates a formula, or updates it, they post a reply to that thread with the download link. Once I see the link, I can update the original post with the download link and delete the reply. This way, all of the downloads should be right there for all to see within one post.

[msouphy]

I can propose a lot of things, most of them unhelpful (IMG:style_emoticons/default/tongue.gif)

It actually crossed my mind within the last couple of days. Getting a spot on the website right now probably isn't going to happen. Not because it's not worth the space or effort, but the site is in the midst of a system upgrade. The other issue I can see arising from it is updating any download on the site takes admin intervention. Since I can see formulas going through multiple evolutions, that's a major pain in the ass for the admins, which we try to avoid at all costs (they get cranky).

What we can try, if it sounds good, is to create a sticky topic in the forum for only formula downloads. The original NRatings thread is getting really long, and expecting a person to wade through 100+ posts to find the formulas is kind of inconvenient. I can create another thread for only the formulas. As a person creates a formula, or updates it, they post a reply to that thread with the download link. Once I see the link, I can update the original post with the download link and delete the reply. This way, all of the downloads should be right there for all to see within one post.

Sounds good to me (IMG:style_emoticons/default/headbang.gif)