Cannot sign jar with .jnilib file inside it

[b-zurg]

Every time I run electron-osx-sign (through electron-forge) and then the notarization process I get the following errors:

{
  "logFormatVersion": 1,
  "jobId": "****",
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "statusCode": 4000,
  "archiveFilename": "myapp.zip",
  "uploadDate": "****",
  "sha256": "****",
  "ticketContents": null,
  "issues": [
    {
      "severity": "error",
      "code": null,
      "path": "myapp.zip/myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "i386"
    },
    {
      "severity": "error",
      "code": null,
      "path": "myapp.zip/myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "i386"
    },
    {
      "severity": "error",
      "code": null,
      "path": "myapp.zip/myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "myapp.zip/myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "myapp.zip/myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64h"
    },
    {
      "severity": "error",
      "code": null,
      "path": "myapp.zip/myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64h"
    }
  ]
}

What's basically happening is that the .jar has this .jnilib file that's not being signed for some reason, but everything else is.

I would really appreciate some help with this issue.

The .jar can be found here (I had to zip it to upload it to github)
jna-4.5.1.zip

[sethlu]

@b-zurg The latest electron-osx-sign should be able to identify binary files and codesign them automatically, and it looks like the latest electron-forge does include the recent updates from electron-osx-sign.

Since you're using electron-forge, I think some configuration like the following lets you specify additional binary files that electron-osx-sign might have missed.

{ 
  "config": {
    "forge": {
      "electronPackagerConfig": {
        "osxSign": {
          "binaries": ["myapp.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib"]
        }
      }
    }
  }
}

[sethlu]

Sorry I just realized that electron-osx-sign and the macOS sees the JAR as a single file and doesn't allow traversing its contents. Here's a related post on Stack Overflow and it suggests code-signing the components before creating the JAR 🤔

https://stackoverflow.com/questions/53439639/notarize-java-app-for-distribution-on-mac-app-store

[b-zurg]

Ohhh interesting. How unfortunate 😆 I wonder if the following approach would work:

1. unzip the .jar so that all of its contents are in a folder.
2. Sign its contents
3. re-zip into a .jar file.
4. Sign the .jar (not sure if necessary)

This could potentially be an optional process with an array of known .jar files with native libraries packed inside them.

[sethlu]

@b-zurg Does this manual workaround work on your end? 🤔
We can probably support some automation for this but it's also a very specific use case.

[b-zurg]

I did figure it out. I used the following commands to test it out.

mkdir .repackage
cp resources/elastic/lib/jna-4.5.1.jar .temp
cd .temp
unzip jna-4.5.1.jar
codesign --force --deep --sign "Developer ID Application: ...." com/sun/jna/darwin/lib/libjnidispatch.jnilib
zip -r -u jna-4.5.1.jar com META_INF
codesign --force --deep --sign "Developer ID Application: ...." jna-4.5.1.jar
cp jna-4.5.1.jar jna-4.5.1.zip
xcrun altool --notarize-app --primary-bundle-id "myapp.org" --username "user" --password "pass" --file jna-4.5.1.zip

The reason I renamed it to a zip file was just for testing purposes as the notarization tool doesn't accept anything but zip, dmg, and pkg I think.

So having the basics of the process down I was able to automate it using the following hook in my electron-forge config:

    prePackage: () => {
      const run = (cmd, cwd) => execa.commandSync(cmd, { shell: true, cwd });

      const jar = "jna-4.5.1.jar";

      const jarDir = "resources/elastic/lib";
      const workingDir = ".temp";

      try {
        if(process.platform === "darwin") {        
          run(`mkdir ${workingDir}`);
          run(`cp ${jarDir}/${jar} ${workingDir}`);
  
          run(`unzip ${jar}`, workingDir);
          run(`codesign --force --deep --sign "${process.env.APPLE_CERT_IDENTITY}" com/sun/jna/darwin/libjnidispatch.jnilib`, workingDir);
          run(`zip -r -u ${jar} com META-INF`, workingDir);
  
          run(`/bin/cp ${workingDir}/${jar} ${jarDir}/${jar}`);
          run(`rm -R ${workingDir}`);
          console.log(`[forge/hooks/prePackage] - successfully repacked ${jarDir}/${jar} to handle signing inner native dependency.`);
        }
      } catch(error) {
        run(`rm -R ${workingDir}`);
        console.error(`Could not repackage ${jar}.  Please check the "prePackage" hook in forge.config.js to ensure that it's working. This jar has to be treated specially because it has a native library and apple's codesign does not sign inner native libraries correctly for jar files`);
        throw error;
      }
    },

It works pretty well.

I think it would be good if others didn't have to go through the same journey I did, and I could imagine this coming up for anyone else who's packaging a java application alongside their electron app. I'm actually surprised it hasn't come up before but perhaps having native libs in a .jar file is not a widespread practice.

If you did want to automate it I could imagine a separate routine that took an array of jar files and then after unzipping ran a glob pattern looking for native dependencies like jnilib or dylib files and signed them before rezipping. Since all it's using is the native zip utility then there's no extra dependencies there.

What do you think? Would that be too much added code?

[sethlu]

@b-zurg Thanks for coming up with the automated script! ❤️ This is great!
I played around with the code today and integrated your procedure with the existing signing workflow.

The latest changes are available here: https://github.com/electron/electron-osx-sign/tree/traverse-archives

This should handle almost all zip-like archives (including jar files) and nested archives. All files in archives are treated mostly the same way as if they are not inside of an archive by electron-osx-sign, so the binary-like files should be picked up automatically (including dynamic libraries and shared objects). We can get nice logs with file paths like my.app/Contents/Resources/app.asar.unpacked/node_modules/elasticsearch/lib/jna-4.5.1.jar/com/sun/jna/darwin/libjnidispatch.jnilib (even though here libjnidispatch.jnilib doesn't technically exist this way).

---

Currently it's disabled by default but one can opt-in the automation with the added flag --traverse-archives.
I haven't tested it by uploading any app to the notarization service. It will be great if you can help test it out 🙌

[b-zurg]

@sethlu Amazing! I'll give it a try in the coming days.

[b-zurg]

Ok I came across something very interesting:

WARNING: Code sign failed; please retry manually. Error: Command failed: unzip -d /var/folders/yx/vzstvm8j40q6ddn2d0_0hwy40000gn/T/tmp-785-0-uncompressed /var/folders/yx/vzstvm8j40q6ddn2d0_0hwy40000gn/T/electron-packager/darwin-x64/myApp-darwin-x64/myApp.app/Contents/Resources/elastic/jdk/Contents/Home/lib/security/public_suffix_list.dat
error:  cannot create /var/folders/yx/vzstvm8j40q6ddn2d0_0hwy40000gn/T/tmp-785-0-uncompressed/verm+�gensberater
        Illegal byte sequence
error:  cannot create /var/folders/yx/vzstvm8j40q6ddn2d0_0hwy40000gn/T/tmp-785-0-uncompressed/verm+�gensberatung
        Illegal byte sequence
error:  cannot create /var/folders/yx/vzstvm8j40q6ddn2d0_0hwy40000gn/T/tmp-785-0-uncompressed/+�++
        Illegal byte sequence
error:  cannot create /var/folders/yx/vzstvm8j40q6ddn2d0_0hwy40000gn/T/tmp-785-0-uncompressed/��
        Illegal byte sequence

I tried out this file public_suffix_list.dat just running unzip on it and it's not a zip file. I think this procedure should only try out zip files that the unzip utility can verify as being valid zip files.

The best way to do this seems to be to run unzip -t FILE where FILE is the path to the file to test.

An example run:

zurg@zurgs-Mac .temp % unzip -t public_suffix_list.dat 
Archive:  public_suffix_list.dat
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of public_suffix_list.dat or
        public_suffix_list.dat.zip, and cannot find public_suffix_list.dat.ZIP, period.
zurg@zurgs-Mac .temp % echo $?
9
zurg@zurgs-Mac .temp % unzip -t HdrHistogram-2.1.9.jar 
Archive:  HdrHistogram-2.1.9.jar
    testing: META-INF/MANIFEST.MF     OK
    testing: META-INF/                OK
    testing: META-INF/maven/          OK
...
No errors detected in compressed data of HdrHistogram-2.1.9.jar.
zurg@zurgs-Mac .temp % echo $?                        
0

In this we can see that the first file exits with code 9 and the valid zip-like .jar file validates with exit code 0.

What do you think?

[sethlu]

@b-zurg Ah yes, thanks for finding this! I think the case where some files begin like a zip file wasn't well handled.
I tweaked the script a little so it should skip traversing inside an archive if it can't unzip it (unzip exit code 0 seen as success).
Can you help give this another try? 😄