That works, but I've discovered a much, much easier way.
The -storetype argument to keytool allows you to translate key material. In fact, jarsigner can take the argument
-storetype pkcs12, meaning you don't need to keep your signing key in a JCE keystore at all. But I digress.
If you can make a pkcs12 file with your private key and certificate - which should be easy with openssl's pkcs12 command - then you can use the following command to turn that into a java keystore:
keytool -importkeystore -srckeystore file.p12 -srcstoretype pkcs12 -destkeystore file.jceks -deststoretype JCEKS