Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Newcomers » Proposals » [CR] critical mass for relevant statistics
[CR] critical mass for relevant statistics [message #640538] Mon, 22 November 2010 13:01 Go to next message
Maxime Jeanmart is currently offline Maxime Jeanmart
Messages: 35
Registered: November 2010
Member
Hi,

I'd like to know if you already have some ideas of the required number of code samples to have relatively accurate statistics over a specific framework. Do you have ways to compute the standard deviation or something like that?
In other words, I wonder how many developers of a specific framework must share their projects in the knowledge base for the data mining results to be relevant. So do you have some ideas of how I can estimate that?

This of course, get us back to the more general question of bad framework usage. What happens if the knowledge base is mostly populated with bad examples of framework usage? Is it possible to estimate at what time we achieved a number of analysis high enough to trust the results?

Best regards,
Max
Re: [CR] critical mass for relevant statistics [message #640656 is a reply to message #640538] Mon, 22 November 2010 17:47 Go to previous message
Marcel Bruch is currently offline Marcel Bruch
Messages: 229
Registered: July 2009
Senior Member
That's a real tough question to which I don't have a good answer yet. We didn't conduct experiments on how many examples we needed to create "good" results.

Just one example for illustration:

For (mid-size API) SWT Button we found approx. 160 different ways how developers used (i.e., invoked methods on) instances of SWT Button. These observations have been observed in thousands of different method contexts (e.g. PreferencePage#performOk, DialogPage#createContents etc.)

By now, you need at least one example for each method context - assuming that there is only one common way of how to use this type in this method. If you have different ways (like using a Button as check/radio/push button) more examples are needed to identify these patterns.

To be honest: I can't give you a good number yet. We should define some experiments and evaluation statistics on that. Do you (anyone?) have any concrete ideas how to evaluate this? I may find a student to work on this interesting question if we have a concrete vision of what to measure/achieve Smile



tw: @MarcelBruch
tw: @Recommenders
Previous Topic:[CR] support for other languages than Java
Next Topic:[RTP] Introducing the RT Packaging Project
Goto Forum:
  


Current Time: Sun Sep 21 04:14:57 GMT 2014

Powered by FUDForum. Page generated in 0.19483 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software