32-bit or 64-bit application on 64-bit OS?

If your application provides no improvements for 64-bit host operating systems and is compatible with your 32-bit releases then I see no immediate need to provide it. However, most, if not all new system are based on x64 architecture where I advocate that also 64-bit software should be the natural default. This needs grows stronger the closer you get to hardware level.

I can't tell you how awkard it is to run a virtual operating just to support some 32-bit VPN client. Promoting the 64-bit client would probably impact your download statistics if you would decide to make it the preferred choice.

Most 32-bit JVMs are limited to around 1.2-1.5 GB. If you find your application uses much more memory with 64-bit JVM, try the -XX:+UseCompressedOops which tells the 64-bit JVM to use 32-bit references but can still access 32 GB of memory.

If the application won't ever need that much memory, 64 bit installer / JVM adds no value. On the contrary, it is a poor option because (as you observed) it simply uses more memory and (probably) runs slower as a result.(Actually, the actual limit will be less than 4GB. Some parts of the 32-bit address space will be unusable, because of hardware architecture issues.

) I suggest you withdraw the 64 bit version, but provide the user with the ability of using a JVM that they have downloaded and installed separately.(Indeed, you should probably do the latter anyway. Embedded copies of the JRE tend to get overlooked when people upgrade to get the latest JVM security fixes ...).