Challenge Motiongreenspun.com : LUSENET : MARP Editors : One Thread
As far as I'm concerned - Q.T. Quazar has posted the motion to permit a challenge system.
I'll second the motion for debate.
I think it really needs to happen especially for really suspicious recordings. There can only be one challenge administered by one person at one time. If nothing comes out in one month's time - then that recording is history. We get to put in the recording's description of "failing to show-up for challenge"
There needs to be an unbiased judge. In other words, the challenger/s is/are not the judge/s.
Any other debate? Thanks :) GB9
-- Anonymous, June 24, 2000
This sounds reasonably ok, but i have a few questions:
(1) why have a only one challege per one person? If we can keep track of 5 people challenging 5 recordings we can keep track of one person challenging 5 recordings... we would not allow one person to challenge ALL recordings, but they shouldn't be limited to one.
(2) if a game is challenged, does the challengie have to make the recording in m35tg3 or m36tg4? if they do and if the game in question can be recorded in m35tg3, it's obvious no one should be making recordings using non tg mames... if they can make they're challenge with a non tg mame version, they will use the same cheating methods that they used in their origonal submission.
(3) double jeparday: can you challenge a m35tg3 or m36tg4 recording? if so some people would have to continue to make challenge submissions just to satisfy the curious.
To me introducing challenges makes marp takes another step towards mccarthy-ism if the challenges are public knowledge. Yes there are some people that deserve the crucible, but obviously many peopele are taking steps to cheat less and this could hurt those people...
I've said it before and i'll say it again, have a tg leader board and a regular leaderboard. Where if people question recordings on the main leader board they can ask the player personally or publically to submit a tgmame recording, where the threat of having their recordings removed is not so overbearing. If there is a tg leaderboard, people can choose to have their recordings scrutinized to the fullest exent when they submit a tg mame recording.
-- Anonymous, June 25, 2000
I think the answer for now is to allow challenges, but in a constrained manner. Limiting people to one active challenge is actually a good idea to prevent an initial flood. Another idea is to privatize the concept where unlimited formal challenges could be presented in writing with valid complaints/evidence explaining the issues. A judge gets assigned and oversees the process from start to finish. A final decision is made by him in 3-4 weeks time.
In general, I would say any 0.35 or 0.36 recording could be challenged since these are the easiest to cheat, but there must be a VERY SOLID reason. A small difference between 1st & 2nd place scores can't be challenged, for now. Likewise pre-0.34 recordings MADE BEFORE 1999 could be grandfathered and are exempt. TG MAME scores could be challenged, but only only the grounds of speed or autofire concerns. In all cases, the challengee would have to repeat the performance using the latest TG MAME. If 90-95% of the original score is achieved the original score stands. If it's exceeded, this of course would be the new top score.
-- Anonymous, June 26, 2000
Pat, I agree with most of what you say. Just a couple of amendments:
1) Autofire isn't really a concern any more. The combined efforts of Chad and Ben Jos with AnalInp (and all the other tools now being incorporated) has virtually eliminated the problem. Slowdown is still the primary cheating method, and even it seems to be on the decline. Pausing is another issue.
2) With regards to deliberate pausing, I'm quite unsure how to proceed on the topic. We could require TgMAME for those games, but then that requires tracking yet ANOTHER list, and I don't think that skito has written up the modified scoring one yet (or was I supposed to? Not trying to point fingers.) With regards to a separate leaderboard, I am vehemently opposed for a number of reasons:
A) It will shift focus almost exclusively to the new board, potentially leaving regular MARP as a 'dumping ground' for .inps not good enough for the tourney or the TG leaderboard.
B) I would estimate 95-97% of current MARP scores are legit.
C) TgMAME is NOT a valid MAME derivative because of closed source and encryption(flashes of Stig here, but he IS right in this respect), and if the MAME team asked us to withdraw its circulation we would be required to do so.
D) Added focus to TG-only would increase the likelihood of people spending more time hacking it.
E) More work for Zwaxy, because none of us have code access except for gb9, and anytime he plays with the code he breaks the Up Loads screen (sorry GB, that was low :) ) (does Pat have access to code, or only to the game database?)
I would also be somewhat concerned with the 'grandfathering' issue. A lot of initial releases were not working properly or had settings changes, and sooner or later we may have to do a mass revision on these games--I've already found and changed a few. So I don't think these .inps are any more 'exempt' than the others.
If we do the challenge thing, we need to set an acceptable score threshhold, which I would recommend be between 80% to 90% of the score, after talking to others who feel that TG's is too low.
As to Chad's comments about McCarthyism, I will admit that this is also a private concern of mine and the reason that I haven't raised the issue of challenging publicly before. I think the best thing to do would be to set up a panel of judges (5 players), which would comprise myself (since I'm Rules Coordinator) or a substitute if I have a personal interest in the game, 1 other member or the Editors Board, and 3 members of the general MARP population, the challenger and challengee excluded. That should provide plenty of objectivity. I would also recommend the following procedure:
1) E-mails requesting challenges sent to me (again, as Rules Coordinator), and I will notify the MARP Editors of the requested challenge, form a review committee, and a decision will be reached on whether to allow the challenge or not. Challenge e-mail will require a valid challenger (only one at a time) and must point out why the recording is suspicious.
2) If the board decides to proceed in the affirmative, a period of one month is given to the challengee to attain a certain score threshhold on the latest version of TgMAME. No further challenges to the player holding the recording in question will be allowed during this time.
3) If the challengee fails to attain the threshold, the recording, and ALL SUBSEQUENT LOWER PREVIOUS RECORDINGS ARE DELETED.
4) If the challengee attains the score, the score is confirmed (or the new score, if higher) and further challenges on THAT PARTICULAR SCORE are barred.
There, I've rambled on long enough. Eagerly awaiting your commentary.
-- Anonymous, June 27, 2000
you can point fingers about the scoring page, i did it for neo- driftout but i mailed it to you? of course i didn't save what i did, let me know if you don't have it and i'll try to write something else up.
I also think somewhere between 80 and 90 is better than 90. some scores are very hard to get and everyone is not hisa-chan, duplicating scores may be hard for the honest people. but given a month more than 75% should be achieved.
I would also request to halt the challenge introduction untill a the new tgmame comes (unless this isn't even close to being out, from what mark said he almost brought out a m37b3tg4 mame) Unless there can be more than one challenge posed to one player for the same game, so if someone challenges someone to make a tg3 recording, it would be wise to allow additional challenges for that person to make a tg4 recording.
I like QT's comments about keeping the challenge private untill affirmed. However how are we going to find the 3 jury members? how will we know they are unbiased?
Some comments on qt's adamant refusal to accept the possibility of a tg leaderboard... I realise i'm probably in the minority here but i have to address these issues. A) It will shift focus almost exclusively to the new board: Isn't that the point? i think having more focused towards tgmame recordings would eliminate a large amount of pausing and slowdowns, right now they are our major problem. B) I would estimate 95-97% of current MARP scores are legit. Agreed: but i'd say an average marp player can not playback 50% of the recordings because most people don't have the time to track down dos/windows/xmame versions to play them back. with tgmame we can consolidate the recordings so they'll be more playbackable and more accessable to all that come here (one world, one version) C) TgMAME is NOT a valid MAME derivative because of closed source and encryption. We face this same issues with tournaments and tgmame recordings on the regular board, this is not an additional concern. D) Added focus to TG-only would increase the likelihood of people spending more time hacking it.
Tournament's are already focus enough to make this hacking probablity increase; plus, it's a lot easier to hack 8 recordings than to hack 100s that would be required to gain access to the top 10 leaderboard.
E) More work for Zwaxy
zwaxy's not doing any work right now :) This is the only major concern here that it might not be able to be done with the time allocated.
done with my rambling.
-- Anonymous, June 28, 2000
> E) More work for Zwaxy > > zwaxy's not doing any work right now :)
Hmmm... I'm doing the work I'm paid for at the moment, which has tended to take me away from the 'charity' work I do at MARP... :o)
I'm not opposed to having a bash at making a separate leaderboard for TG recordings if that's what people really want. It might take a while, but it's doable.
-- Anonymous, June 28, 2000
I don't really want to go into this, because it's probably going to take me a few hours to write down all my thoughts on this matter, but I'd like to briefly touch on a few things I saw mentioned here:
I think a fixed percentage is too general. Rather, it should be game- based. For some games (T&F, for instance), it's relatively easy to come within 90% of a score, but it hardly proves anything, while for others, your score might be only a fraction of the challenged score, but you still show the same skill, except you missed a 10M bonus for finishing the game.
As for the jury, I think that the jury should always include at least one expert on the game. By the same token, I think that the challenger should be an expert, because what's the use of having someone who doesn't know anything about a game challenge a top score? It's bad, actually, because a real expert can then no longer challenge the score. Don't know how this could be enforced, though, because how do we know the challenger's skills?
Maybe we should have the challenger decide what kind of score he'd find acceptable. After all, he is the one raising doubts in the first place. The jury would then decide on whether that kind of score is acceptable for the challenge, and, if not, propose another score. If the challenger agrees, then the challenge is on, if not, the challenge is voided.
OK. I have to stop now before this gets too long. :-)
Cheers, Ben Jos.
-- Anonymous, June 28, 2000
skito you little Nazi propagandist...grrrrr :)
Your argument is somewhat persuasive along the 'one world, one version' lines, but you fail to realize that by allowing TG MAME only you then eliminate Mac, UNIX, and other OS users, who do not have TG MAME versions available. The only way to get it to them is to open the code. Catch 22.
Not mentioned in my original post is my belief that a TG MAME only MARP will not be effective at attracting new players. With may pick up a few more of the 'premiers,' but noone will come join to try out the new beta games. With some of the established score bases for previous betas at MARP, people won't challenge those either. No influx.
If we implement a new, TG-only leaderboard, we will essentially be saying that the old leaderboard is worthless, and the scores on it are not to be trusted. This is primarily why I am so vehemently opposed to the idea. Some of my .inps, like my own 500K+whatever on Magical Drop 3 (Challenge Mode Normal), are extremely, EXTREMELY hard to do, and I have no wish to try and replicate that score for another leaderboard. Opinion on this topic is of course going to be skewed by those who have less invested in the current leaderboard towards those who have more. While I imagine Mark would think a TG leaderboard to be a great idea, I doubt very much that BBH would. I have hundreds of hours invested in recording to MARP, and I don't wish my scores to be junked simply because a few players can't play fair.
I don't like the attitude that is beginning to prevail with some people at MARP that TGMAME, and even TG, is the answer to all our problems. TGMAME is an ILLEGAL VERSION of an emulator of QUESTIONABLE LEGALITY using ROMS which ARE ILLEGAL. There is no need to compound this problem further by pursuing routes of exclusivity .I can't stress this enough. Yes we use TGMAME in tournaments. Yes we use TGMAME to be sure of a player's skill. But the challenge motion is the last step needed towards proving a score, and it doesn't require TGMAME to be THE MAIN FOCUS, only part of the process. Your argument of "We face this same issues with tournaments and tgmame recordings on the regular board, this is not an additional concern." is the same logic as pretending that the left hand doesn't know what the right hand is doing. Having TG only also puts the power of design for such a thing strictly in the hands of the coder(s). While I would not make any accusations, that's an exclusivity arrangement that is unacceptable to an open community like MARP.
On another track, TG has a far worse track record with verifying scores than we do, and has only just started the process of rectifying this. I am doubly glad for the links now being established between the TG and MARP communities, and that input into the TG process is now accessible to more than just a few 'select' players. Congratulations on this Mark, we owe a lot to your efforts.
I will agree that the Challenge motion SHOULD NOT proceed until the next TG MAME. 'Jury duty' could be as simple as seeing who's in the #marp chatroom, or via e-mail if a specific expert is required.
Challenge e-mail will require a valid challenger (only one at a time) and must point out why the recording is suspicious.
And Chris, if you feel playful one of these days, could you bring back the up/down arrows and weeks/days at position to the main MARP leaderboard (I liked the old, pre-server panic leaderboard). I always thought that it was a really neat feature and encouraged players to improve their position. It would be even cooler if you could add a 'last active' feature that listed when a player had last uploaded, but that might not be viable.
Pretty good debate, Chad. It's so much more fun arguing with you than with Parsley or Lamat (not that that is a fair comparison). I'm a little disappointed at your C) and E) rebuttals, but the rest were good. :)
Cheers all, Q.T.Quazar
-- Anonymous, June 28, 2000
Wow, this has turned into a pretty lively discussion!! What thought provoking dabate!!
I agree with almost everything that has been said, but I would like to add some comments and rebutt some points...
1) Most importantly we need TG MAME on tourneys and challenges. It gives us a common platform for verifing tens, even hundreds of recordings. If we ever want to have a seperate TG LB, the pause key must be disabled. This point cannot be disputed. Same is true for Mark's MAME hiscore site. He can't prove that the pause key WAS NOT USED for rest periods. He can't prove m35 or m36 INPs were not "re-recorded" up to any death. TG MAME eliminates these concerns.
2) At MARP, all the ROMS and TG MAME fall into the same category. Both may be illegal to some extent, but as long as either one is not directly available on MARP, its referenced use is completely acceptable.
3) Restricting TG MAME to DOS and WIN users is OK! Again we need to stay on top of version control. Anyone who uses UNIX or MAC is also somewhat computer versed in DOS or WIN. That one or two that aren't--too bad!! I'll give you a real life example as well. If I want to get the World Record in a arcade game from 1980 (perhaps before some of you were born) I have to find a machine that exists and then play at my best when I finally get to it. In concept, this is no different than requiring m35tg3 and reach at minimum 100% of the claimed high score.
4) On point A--Focus would not necessarily shift toward a TG board. First of all there will never be as many recordings or points available simply because clones won't be allowed. It would function more like Mark's TG MAME hiscore site, which he may be interested in linking with MARP. Only difference being all INPs would be available for review and perhaps undergo a slightly higher level of scrutiny. At regular MARP, INPs are not necessarily poor quality, just that everyone's using different settings and emus, in some cases way off base. Take for example Joust, all scores on regular MARP would stand but on a TG LB none of them are valid. Why? Because everyone decided to use extra men which is dead wrong!!
5) On point D--This can be refuted. Witness Deca2000 over at snipercade.com... There you've got 175+ registered entrants--55+ of which are fully qualified. This has a much larger than any of our tournaments and has a totally different crowd with some overlap. There are sponsorships involved and total prizes worth over $2500. Still, I haven't heard anything about hacks and cheaters...
6) For challenges, the committee should consist of 3 people for simplicity--don't want to make it too political. I suggest 1 editor, 1 confirmer, 1 general MARPer who's an expert at that game and preferablly someone that has posted a score for that game.
7) BTW, the qualifying threshold for TG arcade challenges is 90%, but for us to track each game individually may become too cumbersome.
-- Anonymous, June 28, 2000
This is getting long winded, i doubt anyone will ever read this far... but i've still got some breezes flowing.
About eliminating users by encouraging tgmame: We only eliminate them untill we can teach mark or whom ever how to compile it on that platform. It is easy as heck to compile xmame (probably about a hundred times less complex than dos mame), inserting the encryption code would be another issue but once it's compiled you can distribute the binaries, i know all of the xmame uploaders use linux so that binary will suffice for 100% of the unix users so far. I'll use the C) rebuttal again, this is already an issue with the tournaments, no mac players, no unix players...
yep nearly everything everyone does before they upload to marp is illegal. tgmame is sort of an illegal mame version. even krogman was uploading a version of tetris i knew he didn't own, plus that magix one... We take this risk because we love the games for one reason or another. I think the rom legality is a bit more criminal than the tgmame legality; Although two wrongs definitley don't make a right, one wrong and another minor wrong doesn't make it much more wronger. (i love debating with english majors :)
about not wanting to make recordings again for mag drop and bbh having tons of scores "supposed" he doesn't want to record again, This is what challenges are going to do: make someone honest (or not) attempt to make another bitchingly hard recording. anyone who's spent time in a game WILL be challenged to replay it. A tgleaderboard puts less pressure on the players rather than having their recordings challenged by the marp team. they can upgrade their recordings when they feel like it. although i like the challenge system we are proposing, ben's ideas are great (the minimum requirement is game dependant not a fixed percentage, this would depend on the jury being experts in the game another problem with selecting a jury of peers.)
With a tg leaderboard you don't have to make all your recordings again. There will be no denial of service for people that want to upload non tg recordings, they'd still be welcome as ever. And you'll still have your regular marp score as you would always have, you could even improve upon it.
As for losing your hard worked "invested" points. nothing is lost, there is just more opporunity to play. Everyone starts from scratch (except for the biased few that competed in tournaments or record with tg when ever they can) and i doubt anyone will have as many recordings as they do on the regular board.
this subject take much, csc major need time for english part of brain to cool.
-- Anonymous, June 28, 2000