| Performance benchmarking [message #293193] |
Wed, 19 October 2005 15:22  |
Eclipse User |
|
|
|
Originally posted by: hhaygood.cisco.com
I am trying to do some benchmarking to examine the effect of different
deployment environments on Eclipse. Some of the comparisons I'm
interested in are:
* Local-disk Eclipse installation vs. NFS Eclipse installation
* Workspace on local disk vs. workspace on NFS
* Eclipse running natively vs. Eclipse running in a virtual machine
....you get the idea. I really like the kind of information that is
listed on the performance results page for each release's download, and
would like to know how to do something similar.
A search of the newsgroup came up with a couple postings by Yu Chen in
April, but it looks like that is aimed toward instrumenting performance
testing in code that I am developing. A quick grep of the Eclipse
source doesn't show any reference to the testing methods from
org.eclipse.test.performance.
Is there something I'm missing here? I'd like to simply duplicate the
performance testing done by the release group under different scenarios;
what is the simplest way to accomplish that?
Thanks!
Hal
--
Hal Haygood :: hhaygood@cisco.com :: PGP key: 0x81279B0D
Software Tools and Engineering Productivity, Cisco Systems
Carolina Hurricanes 2005-2006: 6 points (3-2-0, L10: 3-2-0)
|
|
|
|
| Re: Performance benchmarking [message #293288 is a reply to message #293202] |
Thu, 20 October 2005 11:31  |
| Eclipse User |
|
|
|
Originally posted by: hhaygood.cisco.com
Kim,
Thanks for the pointer. I had read that document before, but apparently
missed the part of it that described what I wanted to do.
Now I've got another problem. I can get the test to start up, and I see
some UI stuff appear (the startup progress meter, the main Eclipse
window, etc.), but then everything suddenly disappears and I get this on
the console:
osgi> Error: java.lang.NullPointerException
I prepared the test environment by unpacking the 3.1.1 SDK into a test
directory, creating a workspace area, and unpacking
eclipse-test-framework-3.1.1.zip into the Eclipse installation.
I modified the Windows batchfile to a bash script, and have included it
below. Is there any way to tell what is bombing out? I'm not getting a
stack trace with the NullPointerException.
Thanks for your help! Here's the test script:
#!/bin/bash
export ECLIPSEPATH=/var/tmp/test/eclipse
export JAVA_HOME=/opt/jdk/jdk1.5.0_05
export WORKSPACE=/var/tmp/test/workspace
export TESTPLUGIN=org.eclipse.jdt.text.tests
export \
TESTCLASS=org.eclipse.jdt.text.tests.performace.OpenQuickOut lineTest
export APPLICATION=org.eclipse.test.uitestapplication
export OPTIONS="-console -consolelog -showlocation"
export JVMOPTIONS="-Xms256M -Xmx256M"
echo "Build: $ECLIPSEPATH"
echo "Workspace: $WORKSPACE"
echo "Test: $TESTPLUGIN/$TESTCLASS"
$JAVA_HOME/bin/java $JVMOPTIONS \
-cp $ECLIPSEPATH/startup.jar org.eclipse.core.launcher.Main \
$OPTIONS \
-application $APPLICATION \
-data $WORKSPACE \
-testPluginName $TESTPLUGIN \
-className $TESTCLASS
--
Hal Haygood :: hhaygood@cisco.com :: PGP key: 0x81279B0D
Software Tools and Engineering Productivity, Cisco Systems
Carolina Hurricanes 2005-2006: 6 points (3-2-0, L10: 3-2-0)
Kim Moir wrote:
> Hi Hal
>
> This bug
>
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3
>
> has some pointers to documents that describe how to replicate the
> performances tests that we run with a build.
>
> Kim
|
|
|
| Re: Performance benchmarking [message #293307 is a reply to message #293288] |
Thu, 20 October 2005 13:51 |
| Eclipse User |
|
|
|
Originally posted by: hhaygood.cisco.com
Kim,
Okay, I got it. The document that I was really after is "Automated
Testing of Eclipse Builds", on the web at
< http://download.eclipse.org/eclipse/downloads/drops/R-3.1-20 0506271435/automatedtesting.html>.
That got me to the starting point that I wanted.
Because of the kind of relative testing that I'm performing, I made a
couple modifications to the runtests script. First off, since I'm
interested in the relative performance of local display vs. VNC vs. NX,
I removed the line that hard-sets $DISPLAY to :0.0. I also added some
logic to accept a -workspace argument, so that I can specify the
location of the workspace, since I'm interested in testing the
performance impact of local vs. NFS workspace storage.
Here are my intended test cases. If you see anything missing that might
give interesting/useful information, let me know!
1. Local display, local Eclipse, local workspace (baseline).
2. VNC display, local Eclipse, local workspace.
3. NX display, local Eclipse, local workspace.
4. Local display, NFS Eclipse, local workspace.
5. Local display, NFS Eclipse, NFS workspace.
6. Local display, local Eclipse, NFS workspace.
Hal
Hal Haygood wrote:
[ dumb stuff deleted ]
--
Hal Haygood :: hhaygood@cisco.com :: PGP key: 0x81279B0D
Software Tools and Engineering Productivity, Cisco Systems
Carolina Hurricanes 2005-2006: 6 points (3-2-0, L10: 3-2-0)
|
|
|
Powered by
FUDForum. Page generated in 0.06006 seconds
Home