Running Docker inside Nitro Enclaves has always been a critical research goal. A lot of users and protocols alike are interested in being able to package applications up into containers that can run inside enclaves, not to mention the existing broad ecosystem of containerized applications. It allows users to use standard Docker containers without really worrying about quirks of enclave images. Oyster should be able to provide a standard Docker enclave image that can host these containers. The above repository is the first working example from the efforts in this direction.
It uses a nix based pipeline to build images instead of the original Docker based one. It allows us to sidestep thorny Docker-in-Docker problems and install Docker in the image using the nixpkgs versions. It also uses the new 6.6 series kernels released by AWS recently. I suspect this might have resolved some compatibility issues seen in previous attempts. The enclave image itself only has Docker installed, the images are successfully pulled in at runtime.
There are still a few lingering issues though:
The iptables rules that Docker sets at the start don’t work yet. Hence, dockerd is configured to start without setting those rules. This means that there is no bridge network being set up by Docker.
Containers that run in the enclave have to use host networking because of the above in order to be reachable from outside. This means the inability to map ports differently as well as potential for conflicts between different containers.
There are a few errors that are thrown when Docker starts up as well as when containers are being run. They don’t seem to affect the applications I have tried, but it likely means that Docker is running with a restricted set of features that might break some applications.
Nevertheless, there’s now a good foundation to base research in this direction upon and turn it into a production ready environment.
Does this mean that in the near future, we could simply provide our Docker container configuration without having to do all the enclave setup? For example, launching a front-end application without all the configuration? Simpler is better. What kind of application could break ?
Let’s be honest, in my own opinion, I would love to use an easy-to-use deployment to automatically deploy or redeploy enclaves. Developers are lazy, just like I am.
Nix version of docker installs iptables-nft which does not work. Currently, there’s a very hacky way of working around it without having to build Docker from source.
The existing iptables rules to implement transparent proxies conflict with Docker’s rules and are higher priority. I have worked around it with a somewhat hacky solution that puts a custom rule to route packets to Docker first.
A bridge network exists that’s created by Docker during start up. I’m able to map ports from the container to the same ports on the host but not different.
Does this mean that in the near future, we could simply provide our Docker container configuration without having to do all the enclave setup?
That’s the goal, yes.
What kind of application could break ?
There are a bunch errors at the start about Docker not loading in some optional modules, like aufs and other snapshotters. Not sure what types of applications it affects, would have to experiment a bit.
Enclave Builder helps build enclave images easily, a frontend application can be deployed pretty easily as specified in this blog. Further improvements to the UX with Enclave Builder are coming out soon with improved V2 which provides a way to run enclaves simply by providing docker configuration without any extra setup.
There is also some work towards deploying enclaves with a single command with a couple of prompts. This tools automates building enclave images, uploading them to IPFS, sending tx to deploy enclave and getting the IP address of enclave once deployed.