Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [eclipsecon-na-program-committee] the votes and the stars

Eike,

your approach leads to only 2 different ratings in the 'above avg' area which makes it really hard to differentiate in the end between talks that are quite good (but not excellent submissions) and talks that are 'almost good'. Effectively you remove the '3' from the scale. Personally I don't like that.

Best,
Sebastian

On 06.11.2014, at 06:55, Eike Stepper <stepper@xxxxxxxxxx> wrote:

Am 05.11.2014 um 22:47 schrieb Ian Bull:
0 has one meaning: I haven't formed an opinion. If at the end, I still haven't voted then it means I haven't formed an opinion at all. By using 0 (no vote) in this way, it forces you to come back and revisit the talks without simply giving them a 3. By giving all talks that are out-of-your-expertise a 3, you will influence the average (and possibly pull talks down that the experts would have otherwise selected).
It just seems like a down-pull at the first glance, but that's because of the "zero shift" which is inherent to this 1 to 5 scale where 1 and 2 are really negative. I think my approach would be easier to understand if you look at this scale:

+2: I love the talk
+1: I have a good feeling about the talk
0:   I have no feeling about the talk
-1: I have a bad feeling about the talk
-2: I hate the talk

With this scale it's more obvious that "No feeling" (zero) is just neutral and not a down-pull. Now it's technillay easy to map this scale to the choices 1 to 5 that our submission system offers to us. If we keep in mind that an average vote above 3 means that experts had a positive opinion everything is good and comparable.

Regarding the "being forced to come back and revisit talks", that's less than optimal for me. Hundreds of submissions will come in and I used to look at the new ones each evening. When I decide that I know nothing about the topic and the speaker and that the topic doesn't even create a warm feeling then this decision has usually taken me 5 to 10 minutes and I really don't want to look at that talk again every evening. So, having a clear marker for "Done, I've spent my 10 minutes" is essential for me. And that's nicely included in my approach.

Don't get me wrong, of course I'm willing to do whatever the general consensus is/will be. I just thought I share my opinion ;-)

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper




Cheers,
Ian

On Wed, Nov 5, 2014 at 7:48 PM, Eike Stepper <stepper@xxxxxxxxxx> wrote:
Am 05.11.2014 um 19:35 schrieb Ian Skerrett:

Eike,

 

It you don’t know about a session then I would suggest to not vote at all. If you vote 3 that will impact the overall average of a session so that session could get selected even though you didn’t have an opinion.

That's exactly the purpose. If everybody follows that rule we just need to look at the average vote to see a correct overall ordering. In addition everybody can see who's not voted on what sessions. Of course this mechanism relies heavily on the fact that everybody votes for every session. But with the assumption that it's just okay to vote 3 for "I don't know or don't have an opinion" that should be doable.

This approach avoids two problems:

a) No vote number is overloaded with two meanings (as with Ian's approach where 0 means "Not voted, yet" OR "Don't know" and can't be used to detect missing votes).
b) There's no single ordering of the huge session table that reveals the overall preference of the PC.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper



_______________________________________________
eclipsecon-na-program-committee mailing list
eclipsecon-na-program-committee@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipsecon-na-program-committee



--
R. Ian Bull | EclipseSource Victoria | +1 250 477 7484
http://eclipsesource.com | http://twitter.com/eclipsesource


_______________________________________________
eclipsecon-na-program-committee mailing list
eclipsecon-na-program-committee@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipsecon-na-program-committee

_______________________________________________
eclipsecon-na-program-committee mailing list
eclipsecon-na-program-committee@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipsecon-na-program-committee

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail


Back to the top