A couple of additional items to take into consideration.
1. The number of PC votes that a submission has. This will greatly
affect the weight of a talk. If a vote has 4 votes, and another is
rated lower but with 8 how does this affect things.
2. While not a lot, but should consider the Public votes as well.
We also need to make sure that we have a balanced program as much as
possible. Doug do we have a weighting for the number of talks for
each category, and the allocation? We used to try to make sure each
of the tracks had decent representation based on their community
support.
My concern is that the program becomes to representative any one
area of eclipse, while not providing a wider overview of the entire
ecosystem.
Dave
On 11/22/2011 10:04 AM, Schaefer, Doug wrote:
Yeah,
that’s the fear in changing. My main message is to use zero
if you don’t really care. 3 is a valid vote to say, if we
had infinite talks, I wouldn’t mind seeing this one.
We’ll
also need to look by categories. I’m finding that you guys
aren’t excited about certain categories and that’s skewing
things so that few of the talks in that category get
selected. It’ll be a balancing act which is why we’ll go
several times through the data and make sure we have a very
good and balanced program.
:D
>
"Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
> Sent by:
eclipsecon-na-program-committee-bounces@xxxxxxxxxxx
> Hey gang, in trying to assign concepts to the
vote number, we said
> that 3 means you’re OK with the talk being in.
Well looking at the
> current results, it looks like you’ll need a
>3.5 in order to get
> accepted. So, as that turns out, a 3 is actually
a vote for decline
> as it lowers the average.
>
> I’m just wondering
if that’s a problem or not. Thinking back, I’ll
> want to make sure I didn’t put a 3 vote in for
talks that I actually
> didn’t have a strong opinion about that may skew
the vote. Not sure
> if we should all do a pass like that.
If we did that it would
probably just elevate the average vote but still leave
us with exactly the same problem. Personally I think
that 3.5 cut-off sounds about right, given the amount of
talks we can actually accept. Of course we also have to
break it down by category and there could be some cases
where lower voted talks are accepted because the
category is under-represented by great talks. It's
definitely a tough process - I keep thinking elevating
my vote on one talk will inevitably cause another talk
to fall off the bottom, and maybe that's a great talk
too. I guess in the end having too many good submissions
is a good kind of problem to have!
John
_______________________________________________
eclipsecon-na-program-committee mailing list
eclipsecon-na-program-committee@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/eclipsecon-na-program-committee
|
