code signing, entitlements, bundles, sandboxes, hardened runtime, notarisation, app store security

Some people had mentioned troubles with code signing, i thought a brain dump of my learnings on it might be informative and helpful as I've learned a lot in developing a mac app (the s4a IDE).  Some of these concepts are used in ios too but this post is about mac apps.

First up, terminology.

Code signing.

***

This is a variant of normal PKI (public key infrastructure) signing. A hash is made of the binary you're signing (the actual macho executable file), then encrypted with your private key to make a signature that anyone with your public key can decrypt and verify to check your binary hasn't been altered in any way since you signed it. Code signing could in theory be done with any tool, (many of these concepts are NOT mac/ios only concepts but follow general unix/windows/etc. principles and ideas widely used across industry), but in reality, people on a mac use the built in (to macOS) tool "codesign" which is a command line tool that can create, alter, remove and verify signatures. The signature is added to the macho as an extra load command (I believe).

The primary purpose of code signing is to prevent malicious modification of code.

The code signatures can be checked on app run (at least) for all signed programs. That used to be only GUI programs, but now includes (optionally) command line tools. In theory you could code sign /usr/bin/grep and /usr/bin/wc and each time they were used in a pipe command like grep XX myFile |wc ... the code signature would be checked, although I don't think that's usually done to avoid it slowing the tool startup times.

To run a program on an iOS device at all, it must be code signed, this is not strictly true for macOS, although increasingly it is becoming so.

Code signing was fairly uncontroversial in some ways but Apple tended to require an Apple approved signing certificate, which meant having a developer account and was seen by some as a $100 a year "Apple tax" on development targeting macOS platforms.

entitlements

***

Part of the security feature/benefits of code signing, on top of tamper proofing, is you can add a list of required subsystem privileges in a special plist called the entitlements. This is added into the code signature, so when the OS runs the program, it has access to this and knows if the program should have access to (for example) the user's address book. This increases security because it means if you write a program that includes third party code that surreptitiously tries to access user resources that you don't intend, it will fail because you haven't approved that access, and similar attack vectors. Far from perfect but a good line of defence and essential on iOS, which is a high security environment.

bundles

***

Most people will know this, but apps on a mac (and on iOS) are not simple macho executables, instead they come in a folder with structure. On macOS it will have one subfolder "Contents" with many sub-folders "Resources" (containing images, xibs, fonts, sounds, movies, etc.), "MacOS" (containing all macho executables and dynamic linked libraries... .so files).

How does this interact with code signing? It's fairly seamlessly handled for you and thus not well documented. As best I can work out, when you sign a macho file, the signature is added as an LC to the macho. When you sign a bundle, it must sign everything in the bundle to be secure, otherwise how could you be sure someone hadn't modified a resource, such as a xib file in a malicious way? The macho files can all be code signed individually, but resources do not have a place in their structure to put the signature, so they get signed and the signature placed in a special _CodeSignature sub folder in the bundle.

As a last step, I believe the whole bundle is usually code signed... but this is not well documented. The resulting signature is possibly added in the main (executable) macho file in place of the simple code signature that was there, after all code signatures are always the same (rough) size. This is called "inside out" signing as I understand it, each part is signed, then an overall signature calculated, all are checked on app startup.

sandboxes

***

This is an unrelated technology to all the previous ones. In the iOS world we get confused on this because all of these technologies are mandatory. Sandboxes are a widely used term and technology. In iOS and macOS it is something similar to a docker container. The process has its own, separate mini file system that looks like the full file system (you can see them in $(HOME)/Library/Containers for mac apps). Many other features are isolated/namespaced. Again the details are not fully documented in the way they would be on Linux for example, but I'd assume that the file namespace is fully isolated (just like a docker container), the process space and some degree of networking space, such as IPC handles, unix sockets, macho/darwin ports, etc.

This is an important technology for security because it means a rogue or faulty process has limited scope to damage the wider system, also as a process author it makes your life simpler, in the same way docker does, because you know other processes won't mess with "your" filesystem and overwrite your data.

However it also trips macOS app developers up. What happens when you want to load or save files outside the container? It's possible but... complex.

Sandboxing of macOS apps is required for them to be released in the mac app store but, for example, the "normal" S4A IDE from the website is not sandboxed because it makes interaction with the build system complex and means developers have much more limited access to play with files, it's harder to do.

Sandboxing is enabled by adding a specific entitlement when code signing then the OS automatically loads the process in a sandbox.

hardened runtime

***

You may not have heard of this technology. It does a few things to increase security, one of the cleverest is live/in-memory code signature verification. It requires code signing on your executable before you can use it and it's added as another entitlement.

Pages/data loaded into memory must have been code signed (they're loaded from a code signed executable). Once in memory they are also tamper proofed. The in-memory code cannot be altered (without special exemptions) and alterations will have their code signature checked.

As I understand it, the hardened runtime is required for notarisation.

notarisation

***

Introduced relatively recently in macOS and being updated all the time, notarisation covers another attack vector. Previously users were strongly warned if a (GUI) application was not code signed and increasingly it became much harder to run unsigned applications. However, this just meant that someone had made an application and signed it to prove it was them. You could verify a signature but how could you know they were not a malicious actor, either willingly or accidentally (via subversive malware)?

Notarisation is a seal of approval from Apple. To get it, you must structure your app in a fairly normal way, sign it with a mac approved signature AND the hardened runtime enabled and, finally, create a zip file and upload it to Apple's special notarisation server for verification/approval. This is done by automated servers at Apple, which unzip the application, check all its signatures/entitlements, etc., scan it for malware and various other automated checks. You must have a developer account, the application must have an app id they know about and the certificate etc. must match ones known by Apple on your developer account and structured for "DeveloperID" applications (their confusing trade name for this technology).

If approved or rejected, you'll get an email to your designated account telling you, there will also be some logs available online temporarily about the process.

You can then deploy your app as you see fit... email it to people, put it on your website, burn it to a CD, put it on USB thumb drives, deploy via Amazon S3, etc.

When the user downloads your app and tries to run it for the first time, macOS will check the code signature as usual, then see that it's notarised and contact Apple's same notarisation servers over the internet and query if the app is approved. If the app is not validated, the user will get an error and be unable to start it. Increasingly Apple are strongly pushing that all applications must at least have this type of approval if they're not deployed via the App Store. It is still possible to run apps that don't have it, and I think it always will be for political reasons, it just looks too bad otherwise, but Apple make it extremely difficult and off-putting for normal users to work around this security.

Note: this is NOT related to the MacOS App Store and does not go through any of the same approval steps. Nor does your application have to adhere to App Store terms and conditions.

The App Store

***

Final and most secure location, this builds on the above technologies and security layers and adds some more.

Firstly, all applications must be code signed to be uploaded to the App Store. Furthermore, when you sign for the app store, your signature is really "transient". You sign it as described above, then upload it to the App Store waiting rooms (nothing to do with notarisation, totally separate). But when Apple actually approve and deploy it, they remove your code signature and add their own. Apps downloaded from the app store must be signed by Apple themselves. Indeed on iOS they will not even run if they are not.

Sandboxing is required. Notarisation is NOT required and is NOT compatible with app store builds! It uses different types of certificates and is a different way to deploy apps.

Surprisingly, I believe hardened runtime is NOT required, but I'm not sure. And I think it's still recommended.

Lastly, code signing is more stringent. ALL executable files of ANY kind (including command line tools and ANY dynamic libraries they use) must be stored in the MacOS folder (which is very difficult in many cases). The only exception is if you have an XPC Service, it has its own bundle that is stored under the XPC Services folder... but that bundle contains a Contents folder, and all usual sub-folders and any tool inside there must have binaries stored in its MacOS folder so you really have the same issue.

The code signatures are then done inside out... binaries signed, wrappers signed outside that and on to the outermost layer.

I'm not sure why code signing is so impossibly awkward on the App Store... probably it makes life easier for Apple more than anything else. But the fact remains if you include binaries outside the MacOS folder, even if they are code signed and even have the hardened runtime enabled, your app will be automatically rejected.

The final barrier everyone knows about on the App Store is apple app store review, where stringent and often highly controversial rules are applied and all apps are at least partially manually validated.