Re: [eclipsecon-na-program-committee] Finished voting

["Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>]

I definitely take into account number of votes as well, 4 actually being the threshold where I start to wonder about the result. A talk with 8 votes makes me wonder as well, especially the 3 factor I mentioned earlier. But it all depends on the context. For example 8 people voted for the LTTng tracing talk which is actually an important new area in Linux tools and CDT. I voted 5 and the average sits at 3.25. Not sure what to make of that.

 

Not sure of the public votes. I don’t ignore them, but sometimes I wonder if it’s a measure of desire for the talk, or of the effectiveness the submitters have in rallying their communities.

 

Allocations are here: https://www.eclipsecon.org/wiki/2012_Program_Committee#Rough_Allocation_of_Talks

 

I know of one example, Mobile/Embedded which would be heavily unrepresented if we don’t take tracks into consideration. Once we get an initial cut of the data we’ll see where we stand and discuss what kind of tweaks we need to make.

 

Doug.

 

From: eclipsecon-na-program-committee-bounces@xxxxxxxxxxx [mailto:eclipsecon-na-program-committee-bounces@xxxxxxxxxxx] On Behalf Of David Carver
Sent: Tuesday, November 22, 2011 1:15 PM
To: eclipsecon-na-program-committee@xxxxxxxxxxx
Subject: Re: [eclipsecon-na-program-committee] Finished voting

 

A couple of additional items to take into consideration.

1. The number of PC votes that a submission has.  This will greatly affect the weight of a talk.  If a vote has 4 votes, and another is rated lower but with 8 how does this affect things.

2. While not a lot, but should consider the Public votes as well.

We also need to make sure that we have a balanced program as much as possible.  Doug do we have a weighting for the number of talks for each category, and the allocation?  We used to try to make sure each of the tracks had decent representation based on their community support.

My concern is that the program becomes to representative any one area of eclipse, while not providing a wider overview of the entire ecosystem.

Dave

On 11/22/2011 10:04 AM, Schaefer, Doug wrote:

Yeah, that’s the fear in changing. My main message is to use zero if you don’t really care. 3 is a valid vote to say, if we had infinite talks, I wouldn’t mind seeing this one.

 

We’ll also need to look by categories. I’m finding that you guys aren’t excited about certain categories and that’s skewing things so that few of the talks in that category get selected. It’ll be a balancing act which is why we’ll go several times through the data and make sure we have a very good and balanced program.

 

:D

 

From: eclipsecon-na-program-committee-bounces@xxxxxxxxxxx [mailto:eclipsecon-na-program-committee-bounces@xxxxxxxxxxx] On Behalf Of John Arthorne
Sent: Tuesday, November 22, 2011 8:41 AM
To: Eclipsecon NA program committee discussions
Subject: Re: [eclipsecon-na-program-committee] Finished voting

 

> "Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
> Sent by: eclipsecon-na-program-committee-bounces@xxxxxxxxxxx
> Hey gang, in trying to assign concepts to the vote number, we said
> that 3 means you’re OK with the talk being in. Well looking at the
> current results, it looks like you’ll need a >3.5 in order to get
> accepted. So, as that turns out, a 3 is actually a vote for decline
> as it lowers the average.
>  
> I’m just wondering if that’s a problem or not. Thinking back, I’ll
> want to make sure I didn’t put a 3 vote in for talks that I actually
> didn’t have a strong opinion about that may skew the vote. Not sure
> if we should all do a pass like that.

If we did that it would probably just elevate the average vote but still leave us with exactly the same problem. Personally I think that 3.5 cut-off sounds about right, given the amount of talks we can actually accept. Of course we also have to break it down by category and there could be some cases where lower voted talks are accepted because the category is under-represented by great talks. It's definitely a tough process - I keep thinking elevating my vote on one talk will inevitably cause another talk to fall off the bottom, and maybe that's a great talk too. I guess in the end having too many good submissions is a good kind of problem to have!

John

_______________________________________________

eclipsecon-na-program-committee mailing list

eclipsecon-na-program-committee@xxxxxxxxxxx

http://dev.eclipse.org/mailman/listinfo/eclipsecon-na-program-committee