Let’s talk about security. One of the criteria I wanted to bake into this project early on was security. I’ve worked in security off and on again all over my career. Which I say only to add emphasis that writing about embedded security is extremely obtuse. It exists in a state somewhere between “non- existent” and “imagine Ikea instructions for a car engine.” I struggled for days on how to layout most of this information and I’m still not thrilled with it… so there’s likely going to be a lot of additional writing and probably a revision coming. Anyway, moving right along.
Recap
I started with the Raspberry Pi because I knew it would be a good launchpad to do something more interesting and pursue this idea. I got a basic version working as expected. I could manage a Raspberry Pi from a central management machine, up to and including replacing the OS on the unit entirely via the network. Though not exactly how it would work in something like a managed datacenter, the process was remarkably similar.
If you didn’t read the entire previous article, the bootstrapping process goes something like this:
- new machine ARPs for address, DHCP offers a new address along with a ‘next-server’ to begin netbooting over TFTP (maybe PXE)
- PXE, or something behaving like PXE, starts looking for configuration files over TFTP.
- Machine finds a boot configuration and starts booting
- Some sort of inventory + imaging process kicks off and lays down a new OS on the disk
- Maybe some initial configuration is done for “first boot” into the new OS
- Machine is integrated into the fleet
The first thing I should mention is that any compromise of the management network is a total catastrophe. All of these services assume a secure network and provide little-to-no risk mitigation. For instance, if an attacker’s rogue server is simply faster than the management machine you can offer DHCP first and direct all machines to boot whatever you ‘d like. Or, if you have physical access to the SD card in the Raspberry Pi, it doesn’t matter what’s on that SD card because it can just be replaced and the network skipped entirely.
So why do we care & what’s the difference?
The fundamental difference between a shippable datacenter and a traditional one is physical tamper exposure. A traditional datacenter is a fortress. For example, a provider I worked with recently had five biometric checkpoints between the parking lot and the cage with servers. There was a Mantrap door between the parking lot and the lobby you had to enter just to provide your ID so you could authenticate further. If you were unannounced you never even made it to the lobby. Like the scene from Sneakers: “The whole building says ‘Go Away.’”
Physical attacks against this kind of infrastructure usually involve going for the supply chain. Finding a way to tamper with a device bound for the cage that’s going to ship from a manufacturer directly to the datacenter without crossing the office. While definitely possible, relatively speaking, it’s a very expensive & complicated attack.
This shippable datacenter on the other hand has no fortress. It’s everything moving as a unit over some sort of logistics freighting system. Additionally, the precept was that it’s installed in the field without typical solid infrastructure. The possibility someone would have physical access is significantly higher.
The Requirements
With this framing, let’s talk about how we can build a more secure architecture to meet our needs. Is there a way we can ensure that these machines only run the system that we want them to run? We have three major concerns right now:
- Attacker replaces SD card
- Attacker places TFTP-capable server on the network and begins offering boot info, system image, etc
- Remote verification/attestation
Phrased differently: Is the boot media in the machine the media I expect? Is the replacement content for that boot media served by the network the content I expect? Can I verify the state of that content at arbitrary points?
The rest of this particular article, and indeed the start of the next several articles, is going to concern a survey of the major secure boot options available.
What is secure boot?
I’m going to be slightly reductive and say secure boot is the process where a trusted piece of firmware verifies cryptographic signatures on every successive stage of booting. The firmware verifies the bootloader, and the bootloader verifies the OS.
Raspberry Pi (In)Security
While the Raspberry Pi is a superb educational tool and a great prototype platform, it provides absolutely no security. None. If there ‘s even a chance that a system compromise would have negative consequences with real-world fallout, it would be unethical to deploy the Raspberry Pi. Key takeaways, assorted notes, and what I mean by no security:
- The Raspberry Pi bootloader has no notion of authentication. If it finds
bootcode.binandstart.elfon the SD card it’s going to boot from it. Similarly, if the unit has been programmed for USB boot it will try the network. If a TFTP server offersbootcode.binandstart.elfit will load those. bootcode.binandstart.elfare, themselves, not authenticated. If someone reverse engineers these (currently closed source) they can control the boot process. Since Raspberry Pi technically boots the CPU from the GPU the CPU will start in a potentially unknown state.- There’s no way to specify the boot order. First thing found offering
bootcode.binandstart.elfgets to dictate the rest of the process. SD card is always checked first. - The SoC (System on a chip) is the Broadcom BCM2837B0. All the security features that exist in the underlying ARM Cortex-A53 cores have been disabled in silicon. https://github.com/christinaa/rpi-open-firmware/issues/37#issuecomment-388551489
- No TPM (trusted platform module) for key storage. Encrypting SD card is useless since it would have to contain the key in plain text for booting. Requiring a password 1) defeats the purpose of remote deployment and 2) lack of something like SOL (Serial over LAN) makes inputting it remotely difficult.
What are the alternatives in the space? Does any single-board computer offer a compelling security story? As of this writing there are a few different competing specifications for implementing secure boot on ARM, or even x86, based architectures.
The Options (Sort of Specification-wise)
This list, while not exhaustive, covers the major options for securely booting a single board computer (SBC).
- ARM Trusted Firmware / Trusted Board Boot
- High Assurance Boot (Freescale iMX SoCs only)
- U-Boot (Sort of…)
- UEFI Secure Boot
ARM Trusted Board Boot (TBB)
Technically this is a thing, but in practice, it’s mostly just a document. ARM’s ecosystem, much like the Android ecosystem, is severely fragmented. ARM manufactures the Cortex series cores that are used in nearly every mainstream SoC available today. Despite those cores implementing the features required to support things like ARM TrustZone and TBB, they require cooperation from both the SoC manufacturer and the SBC manufacturer. The SoC manufacturer has to wire them in and integrate them correctly. The SBC manufacturer has to work with the SoC manufacturer to make sure their bootloader can initialize everything correctly and securely. As you’ll recall, above, I said the Broadcom SoC in the Raspberry Pi has mostly disabled the required security features.
Additionally, before you can have TBB, you must meet TBBR (Trusted Board Boot Requirements)… Which is another onerous specification. So assuming the manufacturer has done everything for TBBR, they can begin cloning+modifying ARM’s TrustedFirmware reference starting point <https://github.com/ARM- software/arm-trusted-firmware> . This reference spec is itself a living breathing thing, so unless a manufacturer is keeping their proprietary fork up to date, as pieces become available in the formal spec, the specific manufacturers firmware releases won’t support them.
A handful of manufacturers exist claiming to offer various levels of implementation of TBB like Xilinx, but finding out if a specific SBC has support is a documentation nightmare. The spec has been around for years, but unless people start clamoring for it, adoption looks like it will remain low. Matteo Carlini has a good presentation at Linaro Connect talking about the spec http://connect.linaro.org/resource/sfo17/sfo17-201/ that also includes mentioning how UEFI (below) can co-exist with it.
Theoretically this specification allows for remote attestation provided that the ARM Trusted Firmware implementation sufficiently implements TrustZone. Unfortunately, given that it’s up to every individual manufacturer, this has to be evaluated on a case-by-case basis.
High Assurance Boot
First off, it’s a proprietary option made by Freescale (which merged with NXP and then purchased by Qualcomm, so maybe we’ll see it elsewhere?) and thus only works on boards using the i.MX SoC. Under the hood it works like most of the options out there- whatever you want to boot requires a signature that we validate before execution. What makes HAB nice is that the keys are write- protected behind one-time programmable (OTP) fuses and we can permanently fuse the keys to the board. Others might disagree, but I like this no-take-backsies approach.
Things I’m less fond of:
- the best reference site out there is actually a blog post from BoundaryDevices https://boundarydevices.com/high-assurance-boot-hab-dummies/ related to their Nitrogen series SBCs.
- Proprietary toolchain for signing binaries.
- No mention of a TPM so no clear strategy yet on remote attestation.
U-Boot (Sort of)
I’m including this, despite it not being a specification, because there’s a handful of devices out there that basically make the pitch “you always have the option of compiling u-boot for your particular board and using that for secure boot.” Unless there’s a way to:
- permanently fuse your compiled version of u-boot with keys into something like onboard SPI flash
- guarantee it’s the only boot option
- make sure it always executes no matter what
… then it’s not much better than the Raspberry Pi option. Secure boot is only viable if the attacker can’t replace the bootloader. This is why Apple pays a $200K bug bounty on bootloader vulnerabilities.
While it can use a TPM it doesn ‘t require it, so remote attestation remains tenuous.
UEFI Secure Boot
In my opinion this is the best option available today. It also has the added benefits of being the de-facto standard on servers for years as well as being the standard advocated for by several industry juggernauts (Microsoft, Intel, HP, etc). UEFI in general has been around for nearly 20 years and has a full ecosystem around it. I’m not sure when secure boot showed up specifically but it was common enough that in 2011 Microsoft was comfortable mandating it as a requirement for installing Windows 8.
Unfortunately, as far as SBCs go, it’s only implemented on a few very select boards, most of which are manufactured by Intel. This is because Tiano, the reference UEFI implementation, is also made by Intel. One major thing to keep in mind: UEFI secure boot requires a TPM to verify the bootloader. So boards like BeagleBoard supporting EDK2/Tiano are available, but actually implementing all of secure boot requires some sort of a DIY TPM add-on.
The Best Option So Far
Is there a board out there that actually implements a reasonable security story? The best one I’ve managed to locate yet is the Minnowboard Turbot.
It’s manufactured by Intel and uses the reference Tiano/EDK2 UEFI firmware. Additionally, it supports the newer firmware TPM (fTPM) 2 specification so actually includes a proper chain for both making sure the board only boots what we want and verifying the firmware remotely afterward. Surprise surprise, an interesting side note on this one, it ‘s one of the few SBCs that actually doesn ‘t use ARM and uses Intel Atom E3826 or Intel Atom E3845 depending on version.
I’ve gone ahead and setup the dual-core Turbot board. After working with the Raspberry Pi (which I still think is great!), the Turbot feels absolutely luxurious. It more or less boots and behaves like a rack mounted server thanks to the full UEFI implementation. Further, it doesn’t require any special steps in constructing boot media, so using the Ubuntu base server image on USB was seamless. Additionally, getting secure boot working with the fTPM, while not trivial, was at least straightforward.
Next up!
Next in the sequence I’ll talk about getting the Minnowboard’s UEFI secure boot process setup along with TPM basics so we can start building a closed chain of trust.
Appendix & Related Materials
If you liked the above and want to see more examples and how they interrelate then this section is for you.
The raw slides from the Linaro Connect presentation
^^ Worth reading to get a grasp of how gnarly these specifications can be on the inside.
^^ Though it mostly applies to handsets, the underlying tech is very similar. General purpose computers/servers are lagging behind what Android is doing in embedded security.
UEFI and ARM living together discussion
^^ Solid discussion of how they live together and can support one another.
^^ Fun presentation by Job de Haas on how many ways you can do secure boot incorrectly and render it useless. Great survey of attacks.
^^ Provides a great starting place with TianoCore. Particularly interesting if you want to see how the sausage is made regarding UEFI.
Take Control of Your PC with UEFI Secure Boot, Linux journal
^^ Provides an excellent discussion on the individual administrative steps of UEFI and how to do them with OpenSSL.
Easing access to ARM TrustZone OP-TEE and Rpi3
^^ Great presentation on the basics of ARM TrustZone and OP- TEE. It’s a great set of educational slides but makes the usual caveats on slide 7 that because of implementation details, none of this is actually secure on RPi3.
Verified Boot: From ROM to Userspace
^^ Another excellent presentation. Marc Kleine-Budde discusses verified boot using HAB on a RioTBoard.
Trusted Boot on the Olimex OLinuXino LIME2
^^ Getting trusted boot working with a DIY TPM by Merlijn Wajer.
