An Outsider Perspective on Unikernels

I have read The Rise and Fall of the Operating System by Dr. Antti Kantee as well as Unikernels are unfit for production by Bryan Cantrill. As an outsider I have much to learn about Unikernels. I also don't have a horse in the race yet. For me this is a very academic debate on the pros and cons, today and in the next decade, of Unikernels.

Kantee writes,

Technology should encapsulate complexity and be optimized for
the common case, not for the worst case, even if it means we, the
software folk, give up the illusion of being in control of hardware.

This is a hard idea to digest for me. In my Quality Assurance roles I feel a deep desire to explore the worst case scenarios to get an understanding of when and how systems fail and their impact when they fail. For me the happy path, the "common case" as Kantee writes, is to make sure systems don't regress during active development. The common case deserving more optimization goes against the grain of my approach.

Systems don't exist in isolation anymore. In a networked world there are many more chances an individual system will fail because it interacts with the outside world. I'm inclined to optimize for recovering gracefully and also providing enough information to figure out the why and how of the failure.

Kantee continues,

In other words, since the operating system does not protect
the user from evil or provide powerful abstractions, it fails its
mission in the modern world. Why do we keep on using such
systems?

This is a very interesting observation. Cantrill makes a similar point about Unikernels,

The organs that provide this kind of functionality have been deliberately
removed from unikernels in the name of weight loss; any unikernel that
provides sufficiently sophisticated debugging tooling to be used in
production would be violating its own dogma.

They're both making the point that since a particular system does not adhere to its own purpose or design it crosses the boundary into hypocrisy. Are we to throw out entire bodies of work because they are not pure enough?

Kantee concludes,

Minimally implemented
application support is a few thousand lines of code plus the drivers,
as we demonstrated with the Rumprun unikernel. Therefore,
there is no reason to port and cram an operating system into
every problem space. Instead, we can split the operating system
into the “orchestrating system” (which also has the catchy OS
acronym going for it) and the drivers. Both have separate roles.
The drivers define what is possible. The orchestrating system
defines how the drivers should work and, especially, how they
are not allowed to work. The two paths should be investigated
relatively independently as opposed to classic systems development
where they are deeply intertwined.

This conclusion is important to keep in mind when reading Cantrill's post or thinking about Unikernels in general.

Unikernels are a think-outside-the-box approach to solving the problem of adapting to the various use cases (bare metal, cloud, mobile, desktop, etc.) by doing the minimum work necessary to get things running. As Cantrill points out, this sacrifices a lot of the tooling and experience gained in operations used to solve the same problems today.

As evidenced by Rumprun, running existing applications unmodified requires sacrifices from the Unikernel. For example, POSIX stuff is still required for many applications. In addition, in a perfect Unikernel-only world a vast majority of applications will need to be rewritten. As Cantrill points out, "There are no processes" and "Good news: apps kinda work! Bad news: did we mention that they need to be ported?".

Who out there is willing to learn a new paradigm of deploying applications that also requires them to rewrite them? Windows XP is still in use throughout the world even though it has been End of Life for a while now. Change is not easy, especially when there are huge monetary investments to be protected.

This leads me to believe that Unikernels will be successful in niches. The size of these niches may be significant and that may attest to their success with time. Nevertheless, I don't see them taking over the world in the next decade.

Unikernels also throw out the baby with the bath water. Instead of building on the advances made in Operating Systems (OS) and tools, they rely on every application deciding how much of the OS work it is willing to reimplement. For example, debugging, monitoring, and other operational considerations are now the responsibility of each application, instead of the same applications using specialized work in the OS ecosystem in those areas.

When I look at the maker and Internet of Things world, the world of Raspberry Pis and Arduinos, I like the idea of Unikernels a lot. They potentially provide only the things I need to get my single application running. This is why I think Unikernels are bound to shine in highly specialized niches. They are not going to replace my OS X anytime soon.

Many of us have to run multiple applications simultaneously many of which share libraries. These libraries are abstractions built upon other abstractions. Unikernels aim to get rid of these layers of abstractions and interfaces and provide a single-layered surface where, as I understand it, it's a giant ball of code compiled together. The application is the OS and the OS is the application. I still can't wrap my head around the changes required in software development practices with this sea change.

Unikernels have an uphill battle in convincing software practitioners that this radical idea will provide handsome dividends at some point. They are at an unfortunate disadvantage because they either have to win at everything or they will be declared a failure. They are also burdened with casting off any crutches to make current applications run as soon as they can or else the purity of the entire idea will be questioned. It will be said, "If they need this idea and that thing and that other stuff from traditional OSes what's the big idea here anyways?"

I'm taking a wait and watch approach. I would love to be proven wrong in my assumptions and predictions. What I do know for sure is that as Unikernels mature and gain wider adoption the debate will not only intensify but we will also witness their evolution. We will get a better idea of how some theoretical things are executed in practice and how well they are received. I look forward to following this technology.

Update: Episode 69 of DevOps Cafe with Jon Carmack is great to learn more about Unikernels. It certainly made me rethink how I look at them. One thing's for sure: there's a lot of cool research and development in this area and it'll be fun watching as progress is made.