Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » SWTBot » Got feedback about SWTBot ?
Got feedback about SWTBot ? [message #43578] Wed, 15 July 2009 16:54 Go to next message
Ketan Padegaonkar is currently offline Ketan Padegaonkar
Messages: 873
Registered: July 2009
Senior Member
Chris from the PDE fame pointed me to a comparison of UI testing tools
that also features SWTBot.
http://wiki.eclipse.org/Automated_Testing#UI_tests

The good news is that we're being watched :)

There is also some nice and straight feedback on some of the cons of
SWTBot which certainly need improvement:
- lack of documentation
- record/playback support
- some long pending open issues
- lack of support for testing native dialogs

I would also like to hear from the swtbot community about what you feel
are the pain points you've experienced when you got started with swtbot.
Do you have any ideas on how *your* experience could have been better ?

I'm looking forward to any sort of feedback you may have to make swtbot
better. You may write directly on this list(preferred) or privately to me.

Cheers!

--
Ketan
http://studios.thoughtworks.com/twist | http://twitter.com/ketanpkr
Re: Got feedback about SWTBot ? [message #43609 is a reply to message #43578] Wed, 15 July 2009 17:40 Go to previous messageGo to next message
Derek  is currently offline Derek
Messages: 30
Registered: July 2009
Member
Ketan Padegaonkar wrote:

> Chris from the PDE fame pointed me to a comparison of UI testing tools
> that also features SWTBot.
> http://wiki.eclipse.org/Automated_Testing#UI_tests

> The good news is that we're being watched :)

> There is also some nice and straight feedback on some of the cons of
> SWTBot which certainly need improvement:
> - lack of documentation
> - record/playback support
> - some long pending open issues
> - lack of support for testing native dialogs

> I would also like to hear from the swtbot community about what you feel
> are the pain points you've experienced when you got started with swtbot.
> Do you have any ideas on how *your* experience could have been better ?

> I'm looking forward to any sort of feedback you may have to make swtbot
> better. You may write directly on this list(preferred) or privately to me.

> Cheers!

The documentation when I first started looking into SWTBot was quite
minimal and it was very difficult for me to get things up and running
given my experience level. Once I got things to a point where they were
running, it became easier to understand.
The documentation now is a bit easier to follow, and with the addition of
Mohammed's tutorial video's, it's very easy to pick SWTBot up and start
running with it. A more advanced look into writing tests would surely be
helpful.

It looks like from reading the comparison of UI testing tools that SWTBot
is the most promising.
I am a tester, so I'm not very experienced with dev work, but if I could
help out with writing some How-To Docs or some other documentation I would
like to help out.

-Derek
Re: Got feedback about SWTBot ? [message #43637 is a reply to message #43578] Wed, 15 July 2009 18:03 Go to previous messageGo to next message
Pascal G is currently offline Pascal G
Messages: 157
Registered: July 2009
Senior Member
Ketan Padegaonkar wrote:
> Chris from the PDE fame pointed me to a comparison of UI testing tools
> that also features SWTBot.
> http://wiki.eclipse.org/Automated_Testing#UI_tests
>
> The good news is that we're being watched :)
>
> There is also some nice and straight feedback on some of the cons of
> SWTBot which certainly need improvement:
> - lack of documentation
> - record/playback support
> - some long pending open issues
> - lack of support for testing native dialogs
>
> I would also like to hear from the swtbot community about what you feel
> are the pain points you've experienced when you got started with swtbot.
> Do you have any ideas on how *your* experience could have been better ?
>
> I'm looking forward to any sort of feedback you may have to make swtbot
> better. You may write directly on this list(preferred) or privately to me.
>
> Cheers!
>

The worst issue I had so far when writing tests with SWTBot is
synchronization issues. It's easy to try to synchronize your testing
thread with the UI thread and doing it the wrong way. I had to do
multiple iteration to achieve something relatively stable and
(hopefully) inside the good practice guidelines. Most of the time this
happened when I had to validate the state of the UI.

I didn't have any kind of problem of picking up the how-to of SWTBot,
but I came in pretty recently. I was even able to set-up an headless
environment with CruiseControl/PDE/SWTBot relatively easily.

--
Pascal Gélinas | Software Developer
*Nu Echo Inc.*
http://www.nuecho.com/ | http://blog.nuecho.com/

*Because performance matters.*
Re: Got feedback about SWTBot ? [message #43664 is a reply to message #43578] Thu, 16 July 2009 08:05 Go to previous messageGo to next message
Benjamin Glaser is currently offline Benjamin Glaser
Messages: 17
Registered: July 2009
Junior Member
> There is also some nice and straight feedback on some of the cons of
> SWTBot which certainly need improvement:

> - lack of documentation

I think lack of documentation is the main problem of SWTBot.
If you try some things out by our own, the only advice you get is:
with great power comes great responsibility ^^

Fact is, SWTBot is pretty good so far.

But wanna know:
Is it possible to test an application that is already running?
Or do I have to launch the aplication with every run of my SWTBot tests?

Run tests on already running apps would be helpful too I think.

Greets
Benjamin
Re: Got feedback about SWTBot ? [message #44334 is a reply to message #43637] Mon, 20 July 2009 23:05 Go to previous messageGo to next message
Steven Chamberlin is currently offline Steven Chamberlin
Messages: 30
Registered: July 2009
Member
> The worst issue I had so far when writing tests with SWTBot is
> synchronization issues. It's easy to try to synchronize your testing
> thread with the UI thread and doing it the wrong way. I had to do
> multiple iteration to achieve something relatively stable and
> (hopefully) inside the good practice guidelines. Most of the time this
> happened when I had to validate the state of the UI.

Hi Pascal / Ketan,

I found this interesting. I wonder if it might be related to my issue.

The biggest pain point I am currently experiencing with SWT-Bot is that I
am experiencing random intermittent failures in my test code that seem to
be related to some kind of timing / synchronization issue. Code that
previously worked in a prior run will fail with

org.eclipse.swtbot.swt.finder.exceptions.WidgetNotFoundExcep tion: The
widget was null.

at different places in the code. ...then I'll run it again and it'll be
fine.

Anyone else experience this and anyone have tips about how to fix these
kinds of intermittent issues? I wonder if it might be because the UI is
locked up due to some busy function/job on the ui thread. I think Eclipse
might get in a busy state, similar to when the UI is temporarily frozen
due to some 'long' running function/job on a UI thread. Is there a utility
for checking or handling such a situation, or could this be built in to
SWT-Bot itself? Why doesn't SWT-Bot keep trying until the timeout period
is expired?

If my testcases fail on their own intermittently because of timing issues
in the framework or Eclipse, then I will have a lot of false negatives and
the solution would not be workable / scalable.

Suggestions very much appreciated ... Thanks!

-- Steve
Re: Got feedback about SWTBot ? [message #44457 is a reply to message #44334] Tue, 21 July 2009 11:41 Go to previous messageGo to next message
Pascal G is currently offline Pascal G
Messages: 157
Registered: July 2009
Senior Member
Steven Chamberlin wrote:
>> The worst issue I had so far when writing tests with SWTBot is
>> synchronization issues. It's easy to try to synchronize your testing
>> thread with the UI thread and doing it the wrong way. I had to do
>> multiple iteration to achieve something relatively stable and
>> (hopefully) inside the good practice guidelines. Most of the time this
>> happened when I had to validate the state of the UI.
>
> Hi Pascal / Ketan,
>
> I found this interesting. I wonder if it might be related to my issue.
>
> The biggest pain point I am currently experiencing with SWT-Bot is that
> I am experiencing random intermittent failures in my test code that seem
> to be related to some kind of timing / synchronization issue. Code that
> previously worked in a prior run will fail with
> org.eclipse.swtbot.swt.finder.exceptions.WidgetNotFoundExcep tion: The
> widget was null.
>
> at different places in the code. ...then I'll run it again and it'll be
> fine.
>
> Anyone else experience this and anyone have tips about how to fix these
> kinds of intermittent issues? I wonder if it might be because the UI is
> locked up due to some busy function/job on the ui thread. I think
> Eclipse might get in a busy state, similar to when the UI is temporarily
> frozen due to some 'long' running function/job on a UI thread. Is there
> a utility for checking or handling such a situation, or could this be
> built in to SWT-Bot itself? Why doesn't SWT-Bot keep trying until the
> timeout period is expired?
>
> If my testcases fail on their own intermittently because of timing
> issues in the framework or Eclipse, then I will have a lot of false
> negatives and the solution would not be workable / scalable.
> Suggestions very much appreciated ... Thanks!
>
> -- Steve
>
>
I had this issue myself with certain of my tests. One of the problem I
had was with the Display.post() method in eclipse 3.4 on Linux/GTK: it
caused a deadlock in the UI thread and I has to restart the tests. It
got fixed in eclipse 3.5 so now I'm running with 3.5.

The other issue I had was with the update of the UI was slower then I
had expected. The biggest pain I had was with a JFace TreeViewer with
columns, which was populated each time my underlying model changed. I
wasn't doing any synchronization prior to check the UI state. For
exemple, each time I changed something in the model, I did this kind of
check:
// Change the model with some action: model.setFoo("bar");
SWTBotTreeItem item = bot.tree().getTreeItem("foo");
assertEquals("The model was not changed!", item.cell(2), "bar");

This sometime worked, sometime it threw WidgetNotFoundException, and
sometime it was SWTError: Widget is disposed. What I do now instead is:
bot.waitUntil(new DefaultCondition()
{
public boolean test() throws Exception
{
SWTBotTreeItem item = bot.tree().getTreeItem("foo");
return item.cell(2).equals("bar");
}

public String getFailureMessage()
{
return "The model was not changed!";
}
}

This method has the advantage of synchronizing with the UI, as the
condition will be tried periodically until the default timeout happens,
which is 5 seconds. If the 5 seconds is not enough, then you can make it
longer by changing SWTBotPreferences#TIMEOUT. Of course, if you have a
long running job in the UI thread then the problem is not really with
your tests, but with the design of that job. I also don't think there is
direct support for checking long running job (with a progress monitor
and all, such as the synchronizing with CVS job) in SWTBot, but I'm
pretty sure there is a quick workaround for this.

Hope this helps
--
Pascal Gélinas | Software Developer
*Nu Echo Inc.*
http://www.nuecho.com/ | http://blog.nuecho.com/

*Because performance matters.*
Re: Got feedback about SWTBot ? [message #44607 is a reply to message #43578] Tue, 21 July 2009 21:55 Go to previous messageGo to next message
Asif is currently offline Asif
Messages: 11
Registered: July 2009
Junior Member
Please ignore this request if we have these information already documented.

- Documentation on usage of EclipseSpy View feature which provide the
component details. This is an important feature for SWTBot test case
developer.

- New feature required to identify the component's label from the eclipse
bundle. This helps to keep the test suite modularized and doesn't have to
modify the test when components label being changed by the component
developer. If this feature is already there with SWTBot, Please provide
the documentation.

--Mohammed
Re: Got feedback about SWTBot ? [message #44699 is a reply to message #44607] Wed, 22 July 2009 04:51 Go to previous messageGo to next message
Ketan Padegaonkar is currently offline Ketan Padegaonkar
Messages: 873
Registered: July 2009
Senior Member
I'll document this sometime during the week.

Could you elaborate more on what you mean by "identify the component's
label from the eclipse bundle" ?

--
Ketan
http://studios.thoughtworks.com/twist | http://twitter.com/ketanpkr

On 22/7/09 03:25, Asif wrote:
> Please ignore this request if we have these information already documented.
>
> - Documentation on usage of EclipseSpy View feature which provide the
> component details. This is an important feature for SWTBot test case
> developer.
>
> - New feature required to identify the component's label from the
> eclipse bundle. This helps to keep the test suite modularized and
> doesn't have to modify the test when components label being changed by
> the component developer. If this feature is already there with SWTBot,
> Please provide the documentation.
>
> --Mohammed
>
>
Re: Got feedback about SWTBot ? [message #545587 is a reply to message #43578] Thu, 08 July 2010 10:39 Go to previous message
gomesr  is currently offline gomesr
Messages: 3
Registered: July 2010
Junior Member
Hi,
I've been using SWTBot for some testing, but my testing framework is built on TestNG, what I tried to do was use both TestNG and JUnit annotations, eclipse handles this well but SWTBot (headless) seems to ignore the test as soon as I introduce TestNG annotations in the same class. Any idea why?
And sorry if this is not the place to post this.

Regards,
Rick Gomes
Previous Topic:how to run SWTBot headless in Rational Software Architect
Next Topic:JUnit TestNG Test
Goto Forum:
  


Current Time: Thu Oct 30 18:13:18 GMT 2014

Powered by FUDForum. Page generated in 0.13170 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software