Reflections on the new Scores at SOL

“Three in a bed” is the first story I’ve sent to SOL since the changes were implemented to the voting system on the site. It’s into it’s third day now and has dropped off the first page of the ‘new stories’ section but currently still shows up in the RSS feed. It’s had nearly 700 downloads in the past two days and shouldn’t think it will get too many more now. That makes it my 19th most downloaded story on the site, and really isn’t too bad for a flash story at a site who’s readers tend to shun flash in favour of much longer stories. It’s also my most downloaded flash. But right now, I’m interested in how the voting on this story is going. I think it’s worth thinking about if the scoring changes have had an effect.
Let’s recap the changes first. They were implemented because the webmaster was concerned about the upward ‘creep’ of voting results on the site. His initial intention was to remodulate the median of votes cast around the value of 6 (out of 10) instead of where it was at almost 9. To this end he devised a complicated algorithm to ‘correct’ the value of votes cast based on the median of votes at the time they were cast. At the same time, he re-worded the voting form to ‘encourage’ the casting of lower votes. The overall effect has been a lowering of scores on the site, but this has been compensated for by the introduction the ‘Q Score’ or ‘Qualified Score’ which is the one based of when the stories were posted, and which is supposedly a fairer reflection of the real score.
On to of this, is the introduction of ‘TPA’ voting. ‘TPA’ stands for Technical merit, Plot, and personal Appeal. Users of the site can choose which type of vote they prefer to cast. Authors can choose to receive only ‘Qscores’, only ‘TPA’ scores, both or neither. I prefer to leave my stories open to both types of votes.
First thing of interest to note is that the ‘Qscore’ is doing its intended job. The ‘order’ of my stories on the site is different if you order them by ‘Qscore’ instead of ‘Score’. You can’t as yet order them by ‘TPA’ score.
Second thing to note is that the new voting form is also doing it’s job. “Three in a bed” has the lowest score I’ve ever received on the site – currently around 6.5. (it’s not got the worst “Qscore” though – another indication that the ‘qscore’ ‘works’) Now, it may be scoring low because it’s shit, but it’s a former Ruthie’s Club story, and the mail I’ve had because of it has all been positive. Flash stories always score low on Sol (a topic for another time perhaps), but this one seems to be doing particularly badly. This seems strange because the current TPA score is 8,7,8. This isn’t bad considering how poorly flashes normally do on SOL. So, is this difference between the “qscore” and the “TPA” score to do with the types of readers who will make the effort to switch to ‘TPA’? Are they ‘better’ readers? More knowledgeable? Or perhaps they just appreciate Flash more. It has been suggested that TPA appeals more to writers than readers, and it has also been said that flash is the same in its appeal to writers rather than readers – so maybe that explains it.
Honestly, I don’t know. And frankly, it’s all a bit academic.

One thing that occurred to me while I was thinking about this blog, was that does the new “qscore” which remember adjusts votes cast based on the median of all votes cast, take account of the possibility that during a given period of time the stories being posted might actually be better than those posted at other times, and that’s why they are getting higher scores? Let me explain.
Lets say that in January, the median of votes is 7.5 but in February the median rises to 8. The “Qscore” will adjust the scores in February downwards by more than the same score cast in January. Eg, I vote a story in January at 8 and this is changed to (I’m guessing here) 6.2, but a story I vote 8 in February will come down to 6. (I’m making these numbers up – it’s just an example). That’s how the system is designed to work as I understand it (I may be wrong). But what if, in February there have been posted a number of truly exceptional stories that actually warrant the higher votes?
This is a totally hypothetical situation, and really, I don’t know if what I’m saying is correct or not, but if I am correct, what can be done about it? Again, I don’t know and it’s academic. But it is interesting.

marcnobbs.com