It has been a while since I took a stats course but I am pretty sure a 1000 responses passes the threshold of being statistically significant.
From: ide-dev-bounces@xxxxxxxxxxx [mailto:ide-dev-bounces@xxxxxxxxxxx] On Behalf Of Mickael Istria
Sent: November-18-13 7:17 PM
Subject: Re: [ide-dev] Getting feedback on IDE
On 11/18/2013 07:06 PM, Doug Schaefer wrote:
But I don’t think we need to put anything in the IDE. We just need a web page and some buzz around it, like a HackerNews mention or Slashdot article, and of course a bunch of tweets and other social media. Anything in the IDE will lead to our first survey: “Do the survey notices bother you?” :).
Creating buzz around a survey is not something easy. See the Eclipse Community Survey: there is much buzz around it, and it gets about 1000 answers every year, mostly from Eclipse contributors. Compared to the 3000000 downloads of Eclipse 4.3.1, it's probably not very representative.
Also, if we want continuous answers to the continuous flow of new questions, we must think about having users returning often on this poll mechanism. That's why I thought of integrating it in IDE.