Eclipse Community Forums: SWTBot » Timeout Issues in SWTBot Tests

Help

Home

Home » Eclipse Projects » SWTBot » Timeout Issues in SWTBot Tests(How to handle Timeout Issues in SWTBot Tests)

Show: Today's Messages :: Show Polls :: Message Navigator

Timeout Issues in SWTBot Tests [message #1740978]

Mon, 22 August 2016 05:17

Vivek Veera

Messages: 18
Registered: July 2015

Junior Member

Hi,

I am new to SWTBot UI testing and using version 2.4 for testing my production grade RCP Application. I am using Mars.2 and at present running my tests in eclipse using test run configurations.

I have written tests for Wizards and Menus in my RCP application and noticed that my tests are not stable meaning they sometimes break with 'Timeout Exception' (log attached below). I have increased timeout to 8000 ms as I noticed more timing related failures at 5000 ms. I am using dynamic wait methods bot.waitUntil(Condition) wherever I can identify probable delay in execution but sometimes the tests fail at a point which worked perfectly fine in previous executions. Also, the test works fine when I run it again without making any changes to the test code. This part is bit confusing as to why tests fail sometime.

I don't run any other applications and take my hands off the keyboard after starting the test but still the tests fail sometimes. So, I am trying to understand if we have better options to handle the timing related issues and make tests more stable so that they don't break unexpectedly when we run them repeatedly.

I am trying to avoid static waits like bot.sleep(xx ms) as its difficult to predict the exact response time and may add unnecessary delays in test execution. I thought bot.waitUntil(Condition) is a better option. Please correct me if thats not the case.

Any help is very much appreciated. Please let me know if you have any other questions or need more info. I will not be able to continue using SWTBot for UI Automation unless the timing issues are fixed completely and tests are stable. So, its a go/no-go call for me now.

I have gone through the SWTBot User Guide and references given on site. Please point me to any best practices references or guides to help me solve the timing issues and make tests more stable.

Log:

Caused by: org.eclipse.swtbot.swt.finder.widgets.TimeoutException: Timeout after: 8000 ms.: Could not find widget matching: (of type 'Tree')
at org.eclipse.swtbot.swt.finder.SWTBotFactory.waitUntil(SWTBotFactory.java:522)
at org.eclipse.swtbot.swt.finder.SWTBotFactory.waitUntil(SWTBotFactory.java:496)
at org.eclipse.swtbot.swt.finder.SWTBotFactory.waitUntil(SWTBotFactory.java:484)
at org.eclipse.swtbot.swt.finder.SWTBotFactory.waitUntilWidgetAppears(SWTBotFactory.java:466)
... 25 more

Thanks.
Vivek

Report message to a moderator

Re: Timeout Issues in SWTBot Tests [message #1741087 is a reply to message #1740978]

Mon, 22 August 2016 21:21

Patrick Tasse

Messages: 84
Registered: July 2009

Member

Hi Vivek,

I think you are on the right track to avoid sleeps and instead wait until conditions. In our project (Trace Compass) we have many SWTBot tests and have investigated such timing issues often.

The first thing to know, is if it is a matter of the test machine being slow. I understand you are running the test on your own workstation ("hands off the keyboard"). Your test fails because it doesn't find a Tree. Do you see the Tree when it fails? Does it appear later? Then it would just be fixed by increasing the timeout delay, and there is nothing wrong with SWTBot or your test.

If you see the Tree but the test fails, then maybe the bot that you are using is not searching in the right shell or the right view. Make sure you narrow down the scope as much as possible before trying to find a SWTBot widget. Otherwise it might just use the active shell or view and which one is currently active would be random based on timing.

Otherwise, most likely you are not seeing the Tree when it fails. Then the problem is not really in the condition or the delay, but it is in the previous actions that trigger this widget to appear.

I have seen things like:
- pressing a keyboard shortcut and its effect is not immediate, but the next action requires its effect to have occurred
- changing a selection and a menu or command that depends on this selection is not yet enabled
- an action is done on a widget that does not have focus

Some of these might be fixed by making SWTBot more robust, if we can pinpoint the root issue. It would help if you could, by logging or debugging your application, figure out if the actions that should make your tree appear are indeed executed correctly, and if not, why.

Best regards,
Patrick

Report message to a moderator

Re: Timeout Issues in SWTBot Tests [message #1741098 is a reply to message #1741087]

Tue, 23 August 2016 05:48

Vivek Veera

Messages: 18
Registered: July 2015

Junior Member

Thank you Patrick.

Let me debug based on your suggestions. As you mentioned, my machine speed and background processes may also be a reason. I need to observe a bit more to confirm that.

May I know how do you run your SWTBot Tests for your project. Is it in the headless mode ?
I need to run my Tests in headless mode and integrate with maven/tycho build. I have not tried headless mode yet but I foresee a risk if these tests fail due to timing issues in headless mode. Please share if you have any suggestions and experience with headless mode.

Thanks.
Vivek

Report message to a moderator

Re: Timeout Issues in SWTBot Tests [message #1741190 is a reply to message #1741098]

Tue, 23 August 2016 15:30

Patrick Tasse

Messages: 84
Registered: July 2009

Member

Hi Vivek,

When I run SWTBot tests manually I use product "org.eclipse.platform.ide" to use the product as plug-in extensions. But it works just as well if I use product <our RCP product> (it's just that our SWTBot tests don't verify any RCP-specific menus).

For integration with maven, you can see our pom.xml here:

http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/pom.xml

We also use product 'org.eclipse.platform.ide', see the default configuration of artifact 'tycho-surefire-plugin', and in the profile 'swtbot-test-plugin' we override some values and set some environment variables.

Patrick

Report message to a moderator