I need a little help.  Since getting my first tablet at the end of last summer I've been bummed that the floating TIP doesn't work in Java swing apps.  Why?  For one I use a number of Java apps that use the Swing windowing toolkit, but I'm also a developer of some of these apps.

Whats the problem?  Well Swing does not use "native" UI controls.  It renders all of it's controls on it's own.  On Windows XP TE, text controls in Swing aren't recognized as Windows text controls.  So you don't get the little floating button that allows you to open the floating TIP.  This means that you have to resort to using the bottom-of-the-screen TIP which sacrifices screen realestate, sometimes forces apps to resize without restoring their original sizes when it's closed.  Basically it's a hassle to use these apps in this way.

I was finally inspired to start working on a fix for this issue since Sun has been a no show in providing good Windows tablet support in Swing.  I think I'm onto a solution and want to get some feedback as to whether it works for others running XP and even Vista on their UMPCs and other tablet PCs.

I put an early test application here:

http://www.eeaston.com/tabletpc/2007-04-09/SwingOnTablet.jar

It's an executable jar that should run by double clicking if you have a Java 1.2 or newer Sun JVM installed on your Windows box or by executing

java -jar PATH\TO\SwingOnTablet.jar

specifying the appropriate path to the jar file.



Assuming you are running on a tablet with an active digitizer or have  UMPC running floatip.exe, all it does is open a window with three text fields:
- a native text field that should offer the floating TIP button as expected
- a regular Swing text field that doesn't (i.e. is broken)
- a fixed Swing text field that offers the TIP button (but no right click copy/cut/paste menu yet)

If you decide to help me test it, please report if it works or not as well as the OS and Java versions it reports (or a screen shot).

My initial goal is to allow a Java developer using Swing to  explicitly tablet-enable their app on Windows Tablet systems.  Then at some later point find a way to enable tablet support for all Swing apps.

BTW, even though Java is oriented at being cross platform, this is meant for Windows tablet systems.  It might run on non-Sun non-Windows JVM as implemented, but will probably act erratically until I rig it to only add the special processing on Windows tablets.  So if you are running on a Mac, a Linux system, or something other than a Windows tablet, please don't bother experimenting with it.  It'd be sort of pointless.

Thanks in advanced for anyone who offers their testing help.

Hi,

I tested it on a Samsung Q1P (Windows XP Tablet PC 2005), didn't work :(

Cursor on first field

 

 

Cursor on third field

Some follow up questions:
1) was the blinking cursor even visible in the last text field?  It's not in the capture, but perhaps happened to have blinked off when the capture was taken.
2) can you edit text in the third field using the non floating TIP at the bottom of the screen?
3) did anything odd happen when you selected the third field or did it act normal but simply didn't show the floating TIP button?

Thanks.

BTW, I published it as an in browser applet here for those that don't want to muck with downloading it and getting Java to run it.

worked fine on r2h with xp tablet and ie7... i tested your browser applet.

( r2h has HID support, q1 does not )

first field ,  floating tip, editing and right click menu

second field, no tip

third field, tip and editing worked, no right click menu

 

update: same results with i7209 without hid support.

jkk,
Thanks.  Good to hear. 

I didn't implement the right click menu on the swing components yet.  I can make it pull up the regular menu, but if a developer wants to use a non-native look and feel, it looks out of place.  So I was going to provide Swing equivalents...eventually.

I'm still not understanding why HID support is required for this to work on the Q1.  If you have floattip running and you can get a floating TIP button on a regular (i.e. native) text field, then I expected it to be showing up with my test code.  Basically what it's doing is creating a tiny native text field, giving it focus, and forwarding all text events to the swing component so it looks as if the swing field has focus when in reality the native field does.  Then it makes the native field follow the caret around so that the button moves appropriately.  So unless there are some focus bugs, the focus should be on a native component when you navigate to the third field.

well... there is a lot of wierd stuff on q1 implementions of touch... i7209 has same hw as q1, but works different....

Eek, sounds like it may be awkward to control the focus.

Have you considered superclassing the Swing controls and simply adding your own floating keyboard popup that triggers a native call (never tried SWIG on a Windows platform but that may be something to consider for the wrapping) to pop up the keyboard?

Adopting this approach may enable you to produce a classpath jar that lets you run any Swing app with floating keyboard support.

-Andy

iamasmith:

Eek, sounds like it may be awkward to control the focus.

It is awkward, but it's working as well as native apps on the systems I've tested on (I found a person at work to try it too) and for everybody who's tested it except Michaelangelo.  If it's not working for others, I haven't heard about it yet.

Have you considered superclassing the Swing controls and simply adding your own floating keyboard popup that triggers a native call (never tried SWIG on a Windows platform but that may be something to consider for the wrapping) to pop up the keyboard?

I wanted to avoid native calls.  Native code is just a pain for developers to deploy.  I'll resort to them eventually if that's what's needed to make it work reliably.  But since it's working for most right now, it's a step up from not working at all.  To get it working on other JVM implementations might require I go native since each implementation seems to have it's own subtle event dispatching quirks.

Did you mean subclassing?  I want to do something that would work for any JTextComponent, be it a JTextField, JTextPane, JTextArea or any of their subtypes in swing or that a developer might have made on their own.  i.e. don't require any one to change their code for it to work.

Lastly, I want it to work regardless of the look and feel that a developer is using.  This means I'll probably have to use some byte code modification APIs to alter the UIManager.getUI code and attach my code as a listener to every JTextField as it connects the widget to the LAF's UI.  Why a bytecode modification API instead of WrapLF?  WrapLF would works but uses code that's generated off of the Java 1.4 swing classes.  I'd like this work work on all Java 1.2 and newer java implementations.

Now lets say I can manage to do all the above, the real kicker is enabling it for all Java apps and not just ones that developers explicitly add support for.  After all we don't want to just enabled the TIP on apps that are implemented by a Java developer who is aware of the challenges on a tablet PC.  We need something that will tie into JVM invocation on tablet PCs to enable the behavior...this is clearly going to require native code.  If you have ideas, let me know.  Or maybe Sun can just fix Java to do the right thing :(

BTW Andy,
Does it work for you or not?

ree:

Some follow up questions:
1) was the blinking cursor even visible in the last text field?  It's not in the capture, but perhaps happened to have blinked off when the capture was taken.
2) can you edit text in the third field using the non floating TIP at the bottom of the screen?
3) did anything odd happen when you selected the third field or did it act normal but simply didn't show the floating TIP button?

1) yes, there is a blinking cursor

2) yes

3) nothing special, same behavior as field 2

and I got the same results using browser applet (java version 1.6.0_01)

 

michelangelo,
Thanks.  I plan to put together a "debug" version of the applet that logs gobs of event processing details so that I can figure out what's going wrong.  I'll post details here when I do, if you don't mind trying again at that time, I'd really appreciate it.

Also, I wrote up a more detailed "article" on this to try to drum up more interest/testers.  It's here.

Sorry ree, I haven't had time to test it yet.

No, I was talking about Superclassing rather than Subclassing.

I have to say that this isn't something that I have investigated in Java but from what I know it sounds like Java supports superclassing.

Superclassing relates to taking a Subclass object and modifying or changing the behaviour of the object that it subclasses. In this manner if you have several controls that inherit from an object called 'Control' for example, by superclassing 'Control' at the time that the objects are instantiated you can change their overall behaviour. This obviously tackles the issue of byte code modification of the standard classpath stuff.

I think a Java Guru is needed to comment on what Java supports in this manner.

-Andy

FWIW, I have a new release of this code available here:

Enabling Floating TIP for Java Swing Apps on Tablet PCs (Part 2)

Notable updates:

• Fixed some race conditions on a couple JVM implementations
• Works with text areas now
• Figured out how an application developer can easily enable floating TIP for ALL Swing JTextComponents in their app
• Demo is now a Java Webstartable app
• Published Sources for review by the curious

My next steps include installing it as a JRE extension so that all Swing apps you run on your tablet PC get the benefit of the floating TIP.

Feedback by UMPC/TabletPC Java app users would be greatly appreciated.