Home » Eclipse Projects » Technology Project and PMC » Gesture / Voice recognition research?
| Gesture / Voice recognition research? [message #67369] |
Mon, 20 June 2005 17:48  |
Eclipse User |
|
|
|
Hello!
Are there any Eclipse-related research projects in gesture / voice
recognition? I know about the Voice Tools Project, but I'm thinking of
gesture and voice recognition *in* the IDE.
I am doing most of my Java developement work in IDEA (hoping to move to
Eclipse as WTP matures) and there's a really simple but nice Mouse
Gesture plugin available for it. It lets you bind orthogonal mouse
gesture sequences (Left, Right, Left, Up, Down etc.) to IDEA's menu
items. I find this somewhat useful and have bound some of the most
common tasks to gestures, so I can avoid remembering keystrokes or
navigating menus. I find it easier to draw the letter C for Commit, U
for Update, backwards C for rollback etc. than to remember the shortcut
keys. And somewhy I feel like it gives me this sense of power -- can't
really explain.
The IDEA plugin is based on a LGPL Mouse Gesture library (search for
Java Mouse Gestures to find it) that is very simple and easy to use. I
tried porting it from Swing to SWT and got the basic gesture recognition
working in Eclipse, but I'm not progressing very fast towards a usable
plugin, being a beginner in Java and not having touched SWT before. Is
anyone else trying to do something similar?
An extension to this idea would be hand gesture recognition, with help
from a webcam. You could move your hand a bit up from the keyboard and
make a gesture without having to reach for the mouse. There seem to be
some open source computer vision projects that already have code that
can do this, maybe they could be leveraged for an eclipse plugin?
One I found is HandVu, that makes use of Intel's OpenCV project:
http://www.cs.ucsb.edu/~matz/HGI/HandVu.html
Another related interesting area of interaction would be programming by
voice, but I don't have the slightest clue about what advances have been
made there.
I am not really knowledgeable in this field, I've only had a casual
interest in it so far, but it seems to me that about now would be a good
time to experiment with future interaction technologies in various
environments. I want to be using an interface similar to the one in the
movie "Minority Report" by 2010! :)
With best regards,
Erkki "Villane" Lindpere
|
|
|
| Re: Gesture / Voice recognition research? [message #67452 is a reply to message #67369] |
Sun, 03 July 2005 20:10  |
| Eclipse User |
|
|
|
Erkki,
I have mouse gesture recognition framework for SWT. Unfortunately it
isn't easy to integrate gestures into Eclipse platform without special
support in core.
regards,
Eugene
Erkki Lindpere wrote:
> Hello!
>
> Are there any Eclipse-related research projects in gesture / voice
> recognition? I know about the Voice Tools Project, but I'm thinking of
> gesture and voice recognition *in* the IDE.
>
> I am doing most of my Java developement work in IDEA (hoping to move to
> Eclipse as WTP matures) and there's a really simple but nice Mouse
> Gesture plugin available for it. It lets you bind orthogonal mouse
> gesture sequences (Left, Right, Left, Up, Down etc.) to IDEA's menu
> items. I find this somewhat useful and have bound some of the most
> common tasks to gestures, so I can avoid remembering keystrokes or
> navigating menus. I find it easier to draw the letter C for Commit, U
> for Update, backwards C for rollback etc. than to remember the shortcut
> keys. And somewhy I feel like it gives me this sense of power -- can't
> really explain.
>
> The IDEA plugin is based on a LGPL Mouse Gesture library (search for
> Java Mouse Gestures to find it) that is very simple and easy to use. I
> tried porting it from Swing to SWT and got the basic gesture recognition
> working in Eclipse, but I'm not progressing very fast towards a usable
> plugin, being a beginner in Java and not having touched SWT before. Is
> anyone else trying to do something similar?
>
> An extension to this idea would be hand gesture recognition, with help
> from a webcam. You could move your hand a bit up from the keyboard and
> make a gesture without having to reach for the mouse. There seem to be
> some open source computer vision projects that already have code that
> can do this, maybe they could be leveraged for an eclipse plugin?
> One I found is HandVu, that makes use of Intel's OpenCV project:
> http://www.cs.ucsb.edu/~matz/HGI/HandVu.html
>
> Another related interesting area of interaction would be programming by
> voice, but I don't have the slightest clue about what advances have been
> made there.
>
> I am not really knowledgeable in this field, I've only had a casual
> interest in it so far, but it seems to me that about now would be a good
> time to experiment with future interaction technologies in various
> environments. I want to be using an interface similar to the one in the
> movie "Minority Report" by 2010! :)
>
> With best regards,
> Erkki "Villane" Lindpere
|
|
| |
| Re: Mouse gestures [message #67495 is a reply to message #67473] |
Sun, 10 July 2005 18:35 |
| Eclipse User |
|
|
|
Erkki Lindpere wrote:
> Eugene Kuleshov wrote:
>
>> I have mouse gesture recognition framework for SWT. Unfortunately it
>> isn't easy to integrate gestures into Eclipse platform without special
>> support in core.
>
> Is the difficulty related to the case below?
>
> I found (assuming mouse gestures should be started by for example right
> clicking anywhere, as in the IDEA plugin) is that if I register a mouse
> button listener for the Shell, it will only work if I start the gesture
> by clicking in the very edge of the window (and a few other places),
> because if another component (that I click over) handles the event it
> won't reach the shell. The mouse move event seems to work fine in the
> whole window, but it has to start from the window's edge.
IDEA UI is AWT-based (well, Swing built on top of AWT) and you can
register listener for a global even queue. SWT does not have such
luxury. The closest you can do is to register an even filter, but the
problem is that you can't register it early enough without hacking into
platform.
> I haven't looked at how IDEA does this. Maybe event handling is
> different enough in Swing?
>
> The only solution I came up with would be to somehow register listeners
> for every component, which sounds insane, so I haven't even tried.
I've managed to make it working in editors and views but not in edges
and menu areas.
Another issue is that I wasn't able to find how to get to the list of
all registered actions in order to assign gesture mappings.
regards,
Eugene
|
|
|
| Re: Gesture / Voice recognition research? [message #598824 is a reply to message #67369] |
Sun, 03 July 2005 20:10 |
| Eclipse User |
|
|
|
Erkki,
I have mouse gesture recognition framework for SWT. Unfortunately it
isn't easy to integrate gestures into Eclipse platform without special
support in core.
regards,
Eugene
Erkki Lindpere wrote:
> Hello!
>
> Are there any Eclipse-related research projects in gesture / voice
> recognition? I know about the Voice Tools Project, but I'm thinking of
> gesture and voice recognition *in* the IDE.
>
> I am doing most of my Java developement work in IDEA (hoping to move to
> Eclipse as WTP matures) and there's a really simple but nice Mouse
> Gesture plugin available for it. It lets you bind orthogonal mouse
> gesture sequences (Left, Right, Left, Up, Down etc.) to IDEA's menu
> items. I find this somewhat useful and have bound some of the most
> common tasks to gestures, so I can avoid remembering keystrokes or
> navigating menus. I find it easier to draw the letter C for Commit, U
> for Update, backwards C for rollback etc. than to remember the shortcut
> keys. And somewhy I feel like it gives me this sense of power -- can't
> really explain.
>
> The IDEA plugin is based on a LGPL Mouse Gesture library (search for
> Java Mouse Gestures to find it) that is very simple and easy to use. I
> tried porting it from Swing to SWT and got the basic gesture recognition
> working in Eclipse, but I'm not progressing very fast towards a usable
> plugin, being a beginner in Java and not having touched SWT before. Is
> anyone else trying to do something similar?
>
> An extension to this idea would be hand gesture recognition, with help
> from a webcam. You could move your hand a bit up from the keyboard and
> make a gesture without having to reach for the mouse. There seem to be
> some open source computer vision projects that already have code that
> can do this, maybe they could be leveraged for an eclipse plugin?
> One I found is HandVu, that makes use of Intel's OpenCV project:
> http://www.cs.ucsb.edu/~matz/HGI/HandVu.html
>
> Another related interesting area of interaction would be programming by
> voice, but I don't have the slightest clue about what advances have been
> made there.
>
> I am not really knowledgeable in this field, I've only had a casual
> interest in it so far, but it seems to me that about now would be a good
> time to experiment with future interaction technologies in various
> environments. I want to be using an interface similar to the one in the
> movie "Minority Report" by 2010! :)
>
> With best regards,
> Erkki "Villane" Lindpere
|
|
|
| Mouse gestures [message #598831 is a reply to message #67452] |
Tue, 05 July 2005 11:34 |
| Eclipse User |
|
|
|
Eugene Kuleshov wrote:
> I have mouse gesture recognition framework for SWT. Unfortunately it
> isn't easy to integrate gestures into Eclipse platform without special
> support in core.
Is the difficulty related to the case below?
I found (assuming mouse gestures should be started by for example right
clicking anywhere, as in the IDEA plugin) is that if I register a mouse
button listener for the Shell, it will only work if I start the gesture
by clicking in the very edge of the window (and a few other places),
because if another component (that I click over) handles the event it
won't reach the shell. The mouse move event seems to work fine in the
whole window, but it has to start from the window's edge.
I haven't looked at how IDEA does this. Maybe event handling is
different enough in Swing?
The only solution I came up with would be to somehow register listeners
for every component, which sounds insane, so I haven't even tried.
Erkki
|
|
|
| Re: Mouse gestures [message #598838 is a reply to message #67473] |
Sun, 10 July 2005 18:35 |
| Eclipse User |
|
|
|
Erkki Lindpere wrote:
> Eugene Kuleshov wrote:
>
>> I have mouse gesture recognition framework for SWT. Unfortunately it
>> isn't easy to integrate gestures into Eclipse platform without special
>> support in core.
>
> Is the difficulty related to the case below?
>
> I found (assuming mouse gestures should be started by for example right
> clicking anywhere, as in the IDEA plugin) is that if I register a mouse
> button listener for the Shell, it will only work if I start the gesture
> by clicking in the very edge of the window (and a few other places),
> because if another component (that I click over) handles the event it
> won't reach the shell. The mouse move event seems to work fine in the
> whole window, but it has to start from the window's edge.
IDEA UI is AWT-based (well, Swing built on top of AWT) and you can
register listener for a global even queue. SWT does not have such
luxury. The closest you can do is to register an even filter, but the
problem is that you can't register it early enough without hacking into
platform.
> I haven't looked at how IDEA does this. Maybe event handling is
> different enough in Swing?
>
> The only solution I came up with would be to somehow register listeners
> for every component, which sounds insane, so I haven't even tried.
I've managed to make it working in editors and views but not in edges
and menu areas.
Another issue is that I wasn't able to find how to get to the list of
all registered actions in order to assign gesture mappings.
regards,
Eugene
|
|
|
Goto Forum:
Current Time: Mon Nov 03 22:24:10 EST 2025
Powered by FUDForum. Page generated in 0.04480 seconds |
Home