Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [tracecompass-dev] Stress tests added to ctf testsuite

We can run scripts, yeah. But, as far as I understand it, the ./run.sh scripts require passing a CTF_READER_BIN executable, which we don't have during the testing phase.

Another advantage of using parameterized tests is that it creates one JUnit test for every instance that is created at run-time (here, one per test trace to test). This way they show up individually in test reports [1]. And if one of them fails, at the end we know which one did, and not just "something failed in the script".

I see how the test files can be big, so generating *everything* may not be desirable. Could it be per test then? Or even part of the Python scripts themselves? (generate -> test -> clean, have a default array of sizes to generate, user can specify other sizes?)

Thanks,
Alexandre


[1] For example: https://hudson.eclipse.org/tracecompass/job/tracecompass-master-nightly/31/testReport/org.eclipse.tracecompass.ctf.core.tests.ctftestsuite/CtfTestSuiteTests/

On 2014-11-16 03:49 PM, Mathieu Desnoyers wrote:
Can you modify Junit, or use a different test system,
to allow calling scripts instead ? I expect we will end up having
much more automatically generated test-cases over time, and the more
we will have, the less sense it will make to generate them all at
test start.

Thanks!

Mathieu

----- Original Message -----
From: "Alexandre Montplaisir" <alexmonthy@xxxxxxxxxxxx>
To: "Mathieu Desnoyers" <mathieu.desnoyers@xxxxxxxxxxxx>, "Matthew Khouzam" <matthew.khouzam@xxxxxxxxxxxx>
Cc: "lttng-dev" <lttng-dev@xxxxxxxxxxxxxxx>, "Trace Compass Developer Discussions" <tracecompass-dev@xxxxxxxxxxx>
Sent: Sunday, November 16, 2014 9:39:44 PM
Subject: Re: Stress tests added to ctf testsuite

Hi Mathieu,

Looks great! Preliminary testing shows that Trace Compass also blows up
in many different ways, but hey that's expected ;)

If I may suggest, having a script, either shell or Python, to just
generate/clean the test traces would be very useful.
We don't use the run.sh scripts in CI (we're within JUnit tests, there
is no executable at that point), we just look at the trace directories
and try to run everything in there. If there was a simple way to "create
everything" and then "clean everything" it would make it easier to
integrate in our (and I guess others') build system.

Cheers,
Alexandre


On 2014-11-16 11:41 AM, Mathieu Desnoyers wrote:
Hi!

If you look at
https://github.com/efficios/ctf-testsuite

I just pushed various CTF stress-tests, and did some
general cleanup in the tree (e.g. renaming tests to
meaningful names).

Have fun!

Thanks,

Mathieu





Back to the top