Power Ranking Systemgreenspun.com : LUSENET : MAME Action Replay : One Thread
I've been following this site for several months now (it's one of my favorites!) but I've noticed some problems with the leaderboard. I've thought about this for while and come up with an alternate ranking system to address some of the current deficiencies. Without getting into too much detail right now, the things that are happening that I don't like (and most of you would probably agree) are as follows:
- there is no solid score verification process currently in place - there are bad and bogus INPs floating around - there are some 1st place recordings that have bogus hi-scores - there are too many clones available which allow point leeching - there are clones where people have submitted the same recording!! - some players are after leaderboard points rather than actual game points - the leaderboard doesn't quite answer: how does a player rate when compared to others? - hi scores outside the top 3 don't count for beans
So, in the interest in fairness, I've come up with a Power Rating System
- only verified and authenticated scores count regardless of ranking - no scores from any clone games are allowed (only originals count) - actual game scores with ratios are used. 1st place score=100% - ranking and scoring high relative to the top score is rewarded - total score domination within a particular game is also rewarded - players are rewarded for playing well within the games they submit - the Power Rating scale is from 0 to 10 where the average player is set to a 1
Here are the top twenty players as of 3/5/99: Rank Player--Power Rating 1 BBH 9.19 2 JSW 9.04 3 JoustGod 8.68 4 Angry 8.34 5 German Krol 7.27 6 Zwaxy 5.73 7 BeeJay 5.46 8 Steve Krogman 5.27 9 Krool 5.06 10 TAB 4.83 11 Dith 4.08 12 Oldie 3.33 13 Crash 3.32 14 SportsDude 2.72 15 Neil Chapman 2.68 16 Frode Gjerde 2.34 17 Donut 2.33 18 Renzo Vignola 1.93 19 JGustavo 1.93 20 Barry Rodewald 1.88
Total Players: 124 Average Player: 1.00 Possible Games: 1178 Original Games: 631
It's interesting the note the differences. If anything this would force players to submit better recordings. Let me know what you guys think!!
-- Patrick Laffaye (firstname.lastname@example.org), March 05, 1999
I understand where is coming from. About a week ago I was thinking about how unbalanced the point system is when comparing scores amongst the players. My philosophy for a scoring system that will give a player a percentage based on there score compared to the highest score recorded for that game (also varified by Twin Galaxies associates). Below is an example of 5 scores with the players percentage rating based on the score they got. Pat has the same idea I have with giving the highest score a 100% rating and then all the other scores get rated accordingly. This was a rating system Twin Galaxies has been using for almost 20 years now. It has been proved time and time again to be successful and accurated when trying to rate scores amongst players on a certain game. I also agree that there is alot of clones floating around which can benefit a player in lots of ways. I mean, if your the best on donkey kong, I think everyone on marp knows that and have no problem with awarding the player his 10 pts, but to play 4 clones and get another 40 pts just sounds unfair to me. The player diserves the 10 pts, but it can get carried away. Anyway, let me explain what I'm talking about with the scoring rating system for the one's that haven't seen it before. Then I'll wait for some responses to it (I'm sure the 1st one will be Zwaxy! :)
Note: The game, score and players are an example.
*** Pang ***
1)Steve Krogman: 2,958,350 100% 2)BBH : 2,495,080 84.34% 3)Angry : 2,104,890 71.15% 4)Krool : 1,790,450 60.52% 5)Zwaxy : 700,000 23.66%
Please let me know if you understand the rating system. Otherwise I can break it down and explain how I came up with the percentages.
Regards, Steve Krogman
-- stephen krogman (email@example.com), March 05, 1999.
I think these are all good ideas, particularly the % scoring system - that makes a lot of sense. I also like the idea of ignoring clone games, something we considered a long time ago. I know some clones are slightly different in gameplay but essentially if you are good at one you have an advantage if there are many clones. Ban the clones. As for confirming scores, if there are no clones, there will be less INPs and more competition which will both lead to more time and reason to watch and confirm other peoples INPs.
-- Dith (firstname.lastname@example.org), March 06, 1999.
Oh yes, please get rid of the clones. I mean, I may never have the skills at the "standards" (ie Galaga, Pac Man) to go for #1 on the leader board, but I'm not looking forward to 3 versions of 10 Yard Fight or 5+ chip revisions of each Street Fighter/Mortal Kombat game.
The only problem I see going to a percentage scoring system is that 3 scores per game won't be adequate. 10 scores would make sense, but that could eat more server space than Zwaxy has. Maybe after a certain amount of minimum display time the recordings for weaker scores some could be moved offline. For instance, with the 1,000,000 point barrier shattered for Galaga (by however much Steve Krogman felt like), I don't think there will be much demand for recordings below 500,000.
-- aquatarkus (email@example.com), March 06, 1999.
I certainly like some of the ideas proposed regarding the Power Rating System. However, the issue of using only verified and authenticated scores may cause a problem in the area of scores recorded with some of the earlier versions of MAME that have not yet been confirmed.
This is due to the massive changes that have been made to MAME's ROM management system (i.e., merging, renaming). MAME has corrected many problems with bad ROM dumps, but this leaves most of us with ROM collections that can't play some games on MAME versions prior to MAME v34.
I agree that we should dump clone games. It just creates too many leeching opportunities given (see my Pengo & Amidar scores) =) But there are some instances when clones are completely different games, a good example of this is the Trog Prototype game. And would dumping clone games also include dumping multi-game variations, like the 4-in- one bowling games?
In response to Aquatarkus, MARP actually stores every players' top score that has been submitted into the database, and is not limited to just the Top 3 scores. The leaderboard simply uses the top 3 scores to calculate the point values (a long, long time ago, it just tallied up the number of 1st place scores each player had achieved).
-- Angry (firstname.lastname@example.org), March 06, 1999.
A quick note. Dith made a good point about all those extra inps to varify from cloned games. I think at least 20% of the games available to mame are clones. This can contibute to a couple of hundred extra inps that sooner or later will need to be varified by myself, or anyone of the other Twin Galaxies judges. It's also possible that all clones will not make the book for the simple fact that only the original versions will be used. This may be an issue that can only be determined by Chris as far as weather or not the scores should be either removed, or moved to another list or page, not to be confused with the original versions. If it creates to much of a problem, then they can be left alone and just these games won't be varified for the book.
Sheesh! so much for a quick note.... ^ _ ^
-- steve krogman (email@example.com), March 06, 1999.
I should just turn off the news. I'd feel better and would have had proper faith in the coolness of the database :)
-- Aquatarkus (firstname.lastname@example.org), March 07, 1999.
Hmm. For the most part, I think Angry's said it best on this issue... making only confirmed scores eligible for this Power Ranking system isn't really fair to everyone, especially those who play obscure games that alomst no one else plays :P and of course, the ROM changes that often occur between versions. While there are bound to be some bogus scores uploaded (although I can't think of anyone who does this intentionally), make sure the version is correct, try turning sound off/on, etc. If it still doesn't work, let Zwaxy know about it.
It seems unanimous that clones should go, and I agree. Without clones you get more competition over the games that ARE there, and I think that's good. But! Wait! As Angry said, some "clones" are actually pretty unique, like the infamous Trog prototype. And what happens if you were to kill off the clones of, let's say, Choplifter? The original Choplifter romset does not currently work, but there are two bootleg versions. The two versions are slightly different in terms of enemy placement, and one set awards 20,000 for a Perfect bonus on a level and the other awards 50,000. Which one do we use?
Oh yeah, regarding games where an identical recording is used for its clones... didn't "the mob" already rule against this? :)
-- BBH (email@example.com), March 07, 1999.
:o) Great, everyone seems up for making MARP more challenging by disallowing clone games. However, it's not going to be a simple change is it?
We need to build a list of all games which are going to be allowed. This will initially be generated by just removing all the games listed as clones, easy! But MAME is not very reliable with it's definition of what is a clones, there will be exceptions and some games only have a clone version of a game working. Also, the list will have to be updated with every beta release.
Still wanna go ahead with it? I'm still interested so to get things started I suggest we work on three lists. Replacement clones, legal clones and banned originals. I've started the lists below but it's far from definitive. If you disagree with anything or have any suggestions or additions then please post them here. Once we get a near complete list, I'll create a web page with all the suggested changes and an MWR file of the proposed list. I've based the list on MAME 0.35 beta 5 (released today).
List 1: Replacement clones This is where a clone game is used in place of the original, usually because the original doesn't work. These are the games I suggest we replace with the bootleg version.
chplft - chplftbl ddragon - ddragonb firetrap - firetpbl ninjakd2 - ninjak2a (no sound in original) popeye - popeyebl rainbow - jumping renegade - kuniokub slapfigh - slapbtuk superqix - sqixbl tigerh - tigerhb1 tokio - tokiob
List 2: Legal clones There are many clone games with differences in gameplay. A few examples that come to mind are:
Circus Charlie (set 2) - no level selector Trog (Prototype) - completely different gameplay Arkanoid 2 (US) - different screen layouts Donkey Kong (Jap) - different level order & "top of ladder" trick Donkey Kong Jr. (Jap) - different level order Crazy Kong (Alca) - mhorter beams on first level Galaxian Turbo - faster, harder gameplay FHMC Q*Bert - faster, harder gameplay Super Zaxxon - faster Mr.Do (Yukidaruma) - Older version with less features Space Demon - Rapid firing, different attack waves and different shield Star Jacker (Stern) - Only a dip switch setting , but changes gameplay. 600 - Different gameplay and enemies speed up Puck-man - Different maze layout Piranha - This is not a Pac-man clone really is it? Miss Pac-man Plus - Different mazes Hangly-man - Different mazes, invisible mazes
Many of these differences are not significant enough to warrant it legal. If you are champ at the original then it is highly likely that you will also be a master at the clone too. The games I would consider different game on the same hardare and vote as legal clones are:
trogp puckman piranha hangly mspacatk
List 3: Banned originals This list will include games which are essentially clones but listed as originals. It will also list games which don't work and games which have no scoring system. We can probably generate most of this list from the games on MARP which have no score but I'd also like to add:
hyperoly - Hyper Olympic is the European version of Track & Field csprint - Championship Sprint is a two player version of Super Sprint
Finally, a couple of questions to vote on.
Do we allow speed up cheat versions for the pac-man games to be included in leaderboard scores? I vote NO.
Do we still split some games (such as Bowl-O-Rama, 3 Wonders and Invinco/Deep Scan) into several sub-games? I vote YES (there's no other way to do it).
Have I missed anything?
Looking forward to your comments, Dith
-- Dith (firstname.lastname@example.org), March 08, 1999.
Well....banning clones seems really a good idea to me, and the detailed (even if preliminar) analysis Dith made is a good starting point....
I simply suggest to update the list of allowed games avery *final* version, not to get crazy with the dozens of betas....and maybe (why not), sooner or later, accept only .inps recorded with final versions only.
BTW...how is the points leeching banned games list ?!
-- Cicca (email@example.com), March 08, 1999.
There seem to be three distinct classes: sequels, revisions, and modifications. Sequels are new games, even if they aren't always any good. Revisions are minor, and basically have no play difference from the original. Modifications are major changes that make a real difference to the gameplay.
Sequels, like Street Fighter 2: Champion Edition or Super Zaxxon, have different gameplay and graphics that make them a new game. Some of the pirate or prototype boards, like Piranha or Trog, aren't really the same game and need sequel status.
I get the most annoyed with the minor revisions that don't affect gameplay counting as seperate games. Examples of this I can confirm are Mat Mania/Exciting Hour, 10 Yard Fight, and Discs Of Tron.
The most blatent revisions are like Mat Mania/Exciting Hour where you can play back a recording made with Mat Mania on Exciting Hour and vice versa. I think in cases where recordings are compatible, we should pick an "official" version and treat any recordings that play back on that version as acceptable. I'm not sure how common it is because I haven't tried it much.
Modifications are probably going to be the most fought over, because some of what the game designers considered fixing us players considered breaking. The best example of this is Asteroids, where the saucer hunting was made much harder. I could never really do the trick well, but to me Asteroids just isn't the same without it.
I wouldn't count the versions of Donkey Kong with different level orders as an important enough difference to overall scoring, but I'd expect the ladder bug would cause a big difference in scores.
So much for easy :(
-- Aquatarkus (firstname.lastname@example.org), March 08, 1999.
2 notes that Dith made I'd like to give my input on. Any clone that has any difference in gameplay, in my mind, is considered another ver. of that game. These 2 ver. of Ms Pac Man (original, and turbo) are unique and different. Therefore I vote to have scores for each one of them. Now, compare these 2 games, (galaga midway and galaga namco, or galaga (m or n) and galaga bootleg) There is absolutely no difference in gameplay between these games. With Ms Pacman turbo, it's different. May be easier to some players, but it's still different. I watched Billy Mitchell 2 weeks ago at an arcade we went to, and he tried to perform the "grouping method" he uses on the original and he said "man this IS different, and pretty hard to pull off". Basically, you make your own pattern and most of the time your trying to stay alive for 132 screens. Besides, if it was THAT easier, how come now one can get to the kill screen on marp? (I should speak for myself) :) I will work on a 1st kill screen on marp, but with better controls, I have done it many times in the arcade. Gallag is another clone that can be removed, it is identical with exceptions to 2 things: 1) Instead of Galaga, it's spelled Gallag and 2) It only shows 5 men max on the bottom of the screen instead of 8. Gameplay is 100%
Just my $.02 :)
Regards, Steve Krogman
-- stephen krogman (email@example.com), March 08, 1999.
Here's another slant on this thread.
Rather than "banning" the clones why not just have them linked.
ie: Record the score for whatever version of Galaga you want, and have the upload score it as a galaga Power Ranking recording regardless of whether or not it was gallag, galaga, galaganm, galagamw etc (or whatever they're called these days). Likewise for all other clones that are considered so similar to their parent game as to be insignificantly different in gameplay.
That way we keep the existing high scores for games that are "clones" without having to re-record them. Put me in for counting all versions of wonderboy as the same. Admittedly some of them are significantly harder energy wise but who cares. People will just record whichever version is the "easiest" and it will be "fair" as it will become obvious which version(s) are the easiest for any particular game.
I do agree that there are differences in some clones that significantly alter the gameplay. Perhaps an approach would be to group them for the purposes of Power Ranking leaderboard points, but keep them as separate valid entities for uploading of recordings. Keep the best of both worlds. Perhaps even have the old style leaderboard kept on some obscure link and the new style Power Ranking System as the "default leaderboard".
I think this would be better than outright banning any given version, of course it may mean a little more work for Chris when a new release of Mame comes out.
What say you fellow Marp'ers to this concept?
-- BeeJay (firstname.lastname@example.org), March 08, 1999.
Just a pointer to a separate thread where you can vote on these issues. Just cut and paste my message and change votes where req'd.
-- PatL (email@example.com), March 08, 1999.
I really like Beejay's idea of simply linking all clones to their originals. The changes we were talking about were starting to get very complicated and would need constantly updating with new MAME releases. Why don't we allow recordings for any clone to count as an entry for the original game in the Power Ranking system. We can build a simple list of clones which are obviously very different, such as trogp and piranha. These games can have there own entry.
This system will eliminate lots of problems, allowing us to sort this quickly and then get on with the competition.
-- Dith (firstname.lastname@example.org), March 09, 1999.
I just made a change to the leaderboard script which awards points using the 'percentage' method that has been discussed here. It's not the default - you have to tick a box at the end of the board screen to get the new numbers - but you might like to check it out.
I've not removed clones from the scoring, either, mainly because I don't know how to tell what's a clone and what isn't very easily, and also because I don't have time to mess with the script any more right now.
I wrote about the new routine on the MARP news page. Let me know what you think...
-- Zwaxy (email@example.com), March 12, 1999.
I have very simple propositions to make to the ranking problem About the bogus or unconfirmed inps, there is no question about it, they should be ignored. It may be tough for a player who didn't cheat, but you can't give any points until it is confirmed. You don't give Ben Johnson the WR if he is on drugs, do you? Instead there should be a penalty for someone that uses cheats, like ban him from the game.
If you like the percentage system, that is fine. But you can't have too many positions, or too less. I think the top 10 would be best. With that system you award someone that has made a super performance in 1st place compared to the second. e.g 1st place 100 points, second 70 points.
But when you compete in hundreds of games, the points in individual games don't make much sense. For an overall ranking you need the TOP 3 positions only! No need to mess with points at all! The first position gets the Gold Metal, the second the Silver and the Third the bronze. So, the player with the most gold’s, silvers and bronzes wins. First priorities are the gold’s of course. If you have the most gold’s, then you get TOP 1 EVEN if you have less silver or bronze. That makes sense because if you have more gold’s, the others will have the silvers.!!!
As for removing the clones, I disagree. Some clones are different than the main game, and what if I have the hi score and my main set becomes a clone in the next version of mame? What I propose is simple; A player can not be awarded with points more that 1 time in a game, either it is a parent of clone set.
I hope these make a lot of sense to you Grendal74
-- grendal74 (firstname.lastname@example.org), June 05, 2001.