RE: [cosmos-dev] Performance Tests : QA update #1

["Doma, Srinivas Reddy" <SrinivasReddy.Doma@xxxxxx>]

Title: Performance Tests : QA update #1

Paul,

 

Below tests are not yet captured into QA Test Specification document, which I would do it shortly.

 

These are additional tests as part of ‘load/performance’ for COSMOS Operational Efficiency tests.

 

All below tests are based on SOAP/XML requests, which has been captured and executed individually on all COSMOS components

using Apache JMeter tool.

 

Thanks,

Srinivas

---

From: cosmos-dev-bounces@xxxxxxxxxxx [mailto:cosmos-dev-bounces@xxxxxxxxxxx] On Behalf Of Stratton, Paul
Sent: Wednesday, 12 March 2008 10:08 PM
To: Cosmos Dev
Cc: Bathini, Raghuveer; Adivishnu, Kishore
Subject: RE: [cosmos-dev] Performance Tests : QA update #1

 

Srinivas, are these results for the tests detailed in https://bugs.eclipse.org/bugs/attachment.cgi?id=90146 under Operational Efficiency ?

 

Thanks

 

Paul

 

---

From: cosmos-dev-bounces@xxxxxxxxxxx [mailto:cosmos-dev-bounces@xxxxxxxxxxx] On Behalf Of Doma, Srinivas Reddy
Sent: 12 March 2008 12:22
To: Cosmos Dev
Cc: Bathini, Raghuveer; Adivishnu, Kishore
Subject: [cosmos-dev] Performance Tests : QA update #1

 

Hi,

QA has performed some load testing using ‘Apache JMeter’ tool (open source application) on COSMOS i9 components

and here are the results of  them with load samples of 10, 50,100,150 and 200 users simulation.

Test Environment:

-       COSMOS i9 Components running on Linux (remote machine)

-       Apache JMeter tool running on Windows XP, test execution using SOAP requests for all COSMOS components

Observations:

-       Log reports generation is taking longer time than any other requests

-       For some of the requests, it’s throwing – “Internal Server Error”

-       Unable to simulate higher load samples with remote machine (>200 users), as most of the requests are

taking longer time and throwing timeouts or ‘internal server errors’

Test Results:

a)      Management Domain

Label   #Samples        Average Min     Max     Std Dev Error % Throughput / sec        KB/Sec  Avg Bytes      
SOAP/XML-RPC Request    10      494     471     520     14.092906       0       7.08215297      10.554068       1526   
SOAP/XML-RPC Request    50      779     470     3495    740.660273      0       12.1477162      18.102944       1526   
SOAP/XML-RPC Request    100     742     471     3515    680.803882      0       22.2866057      33.212266       1526   
SOAP/XML-RPC Request    150     797     480     4306    718.451263      0.00666667      28.2592313      42.156113       1527.56667     
SOAP/XML-RPC Request    200     918     470     3746    936.16286       0.03    44.2771751      66.353282       1534.555       

b)      Data Broker

Label   #Samples        Average Min     Max     Std Dev Error % Throughput / sec        KB/Sec  Avg Bytes      
SOAP/XML-RPC Request    50      1961    561     3775    940.0106        0       11.7481203      51.44392        4484   
SOAP/XML-RPC Request    100     2176    511     5027    1249.839        0       19.3124759      84.56752        4484   
SOAP/XML-RPC Request    150     2589    520     6349    1585.648        0       23.5886146      103.3083        4484.69333     
SOAP/XML-RPC Request    200     3379    531     8853    1959.169        0       22.2890895      97.60183        4484   

c)      SML MDR (Sample CMDBf Query Execution)

Label   #Samples        Average         Min     Max     Std. Dev.       Error % Throughput / sec        KB/sec  Avg. Bytes     
SOAP/XML-RPC Request    10      1265    551     4227    1074.661        0       2.02922078      8.699491        4390   
SOAP/XML-RPC Request    50      1725    530     10045   2093.932        0       4.90918017      21.04619        4390   
SOAP/XML-RPC Request    100     833     480     3755    607.3807        0       20.8029956      89.18472        4390   
SOAP/XML-RPC Request    150     1040    511     4126    885.0082        0       34.5065562      147.9334        4390   
SOAP/XML-RPC Request    200     1293    561     4707    996.3588        0.01    34.6140533      147.5056        4363.71

d)      Example MDR (Sample CMDBf Query Execution)

Label   #Samples        Average         Min     Max     Std. Dev.       Error % Throughput /sec KB/sec  Avg. Bytes     
SOAP/XML-RPC Request    10      861     511     3726    955.295038      0       2.19925225      7.82195 3642   
SOAP/XML-RPC Request    50      874     491     4316    808.021782      0       9.90687537      35.2352 3642   
SOAP/XML-RPC Request    100     993     500     4486    825.294205      0       21.6637782      77.05027        3642   
SOAP/XML-RPC Request    150     1548    480     12799   1733.62155      0.066   11.0375276      39.14449        3631.60667     
SOAP/XML-RPC Request    200     1334    521     9554    1350.17049      0.015   20.3376042      71.77317        3613.785       

e)      Statistical Data Manager (Reports)

Label   #Samples        Average         Min     Max     Std. Dev.       Error % Throughput / sec        KB/sec  Avg. Bytes     
SOAP/XML-RPC Request    10      633     520     791     112.9765        0       6.40204866      113.1612        18100  
SOAP/XML-RPC Request    50      1527    511     4647    1214.629        0       9.51112802      168.1166        18100  
SOAP/XML-RPC Request    100     1966    560     14601   1736.601        0       6.64407681      115.2747        17766.4
SOAP/XML-RPC Request    150     2668    581     10976   2210.611        0       12.8899201      206.8429        16432  
SOAP/XML-RPC Request    200     3089    610     11016   2024.09 0.05    17.8635227      276.4992        15849.905      

f)      Logging Data Manager (Reports)

Label   #Samples        Average         Min     Max     Std. Dev.       Error % Throughput /sec KB/sec  Avg. Bytes     
SOAP/XML-RPC Request    10      30034   23393   36042   3859.116        0       0.270679948     173.6536        656943 
SOAP/XML-RPC Request    50      42465   7901    53647   7769    0       0.92084423      578.9759        643834.58      

Thanks,

Srinivas