«Shaya Potter Submitted in partial fulﬁllment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences ...»
Virtualization Mechanisms for Mobility, Security
and System Administration
Submitted in partial fulﬁllment of the
requirements for the degree
of Doctor of Philosophy
in the Graduate School of Arts and Sciences
All Rights Reserved
Virtualization Mechanisms for Mobility, Security and System Administration
Shaya Potter This dissertation demonstrates that operating system virtualization is an eﬀective method for solving many diﬀerent types of computing problems. We have designed novel systems that make use of commodity software while solving problems that were not conceived when the software was originally written. We show that by leveraging and extending existing virtualization techniques, and introducing new ones, we can build these novel systems without requiring the applications or operating systems to be rewritten.
We introduce six architectures that leverage operating system virtualization. *Pod creates fully secure virtual environments and improves user mobility. AutoPod reduces the downtime needed to apply kernel patches and perform system maintenance.
PeaPod creates least-privilege systems by introducing the pea abstraction. Strata improves the ability of administrators to manage large numbers of machines by introducing the Virtual Layered File System. Apiary builds upon Strata to create a new form of desktop security by using isolated persistent and ephemeral application containers.
Finally, ISE-T applies the two-person control model to system administration.
By leveraging operating system virtualization, we have built these architectures on Linux without requiring any changes to the underlying kernel or user-space applications. Our results, with real applications, demonstrate that operating system virtualization has minimal overhead. These architectures solve problems with minimal impact on end-users while providing functionality that would previously have required modiﬁcations to the underlying system.
Contents Contents i List of Figures vii List of Tables ix Acknowledgments xi 1 Introduction 1
1.1 OS Virtualization Security and User Mobility............. 3
1.2 Mobility to Improve Administration................... 5
1.3 Isolating Cooperating Processes.............
My deepest thanks go to my advisor, Jason Nieh, for his continual support and guidance. His constant questioning, demanding of explanations and objective evaluation has helped develop ideas that I would not have been able to reach on my own, while also teaching me skills that I hope remain with me. I am constantly amazed by how many diﬀerent studies, projects and papers he is able to juggle while retaining the ability to ask insightful questions. He has provided the model that I aspire to be.
There are many people at Columbia who have been a signiﬁcant part of my graduate experience. My oﬃcemates, Dinesh Subhraveti, Dan Phung and Dana Glasner have been good friends, acted as sounding boards, provided valuable feedback, and, in general, made the graduate experience an enjoyable one. I’ve worked on many projects together with Ricardo Baratto and Oren Laadan and I am always amazed by their abilities. Stelios Sidiroglou-Douskos, Mike Locasto, Carlo P´rez and Gong Su e provided valuable feedback and friendship. I’d also like to thank Angelos Keromytis, Steven M. Bellovin for providing help and guidance in my research. In addition, I’d like to thank Erez Zadok, Gail Kaiser and Chandra Narayanaswami for serving on my Ph.D. committee. Finally, I’d be remiss if I did not thank the administrative staﬀ in the Computer Science Department, including Alice Cueba, Twinkle Edwards, Elias
Introduction Computer use is more widespread today than it was even 10 years ago, but we are still using software designs from 20 or 30 years ago. Although these designs are well tested and understood, they were created to solve the problems of that time.
Today’s users face diﬃculties that the original software designers did not imagine.
We can redesign the operating system and applications to attempt to address these problems, but this creates new, relatively untested software and designs and may force users and administrators to learn fundamentally new models of usage. This dissertation demonstrates that many problems can be solved not by redesigning and rewriting the applications, but instead by virtualizing the interfaces through which existing applications interact with the operating system.
Virtualization is the creation of a layer of indirection between two entities that previously communicated directly. For example, in hardware virtualization [28, 34, 142, 147], a virtual machine monitor (VMM) places a layer of indirection between an operating system and the underlying hardware. A VMM provides a complete virChapter 1. Introduction 2 tualized hardware platform for an operating system, enabling any operating system supporting that platform to run as though on physical hardware. Hardware virtualization has been shown to enable operating systems to take advantage of hardware for which they were not designed. The Disco project  demonstrated how to run an operating system not designed for ccNUMA architectures on those architectures by using a VMM.
Operating systems can also be virtualized in multiple ways, most commonly by providing each process with its own virtualized and protected memory mappings. Instead of letting a process directly access the machine’s memory, the operating system, with hardware support, places a layer of indirection between the processes and physical memory, creating a virtualized mapping between the process’s memory space and the physical machine’s memory space. This provides security, eﬃciency and ﬂexibility. The processes’ memories are isolated from one another, but memory can still be shared among processes.
Memory, however, is not the only operating system interface that can be virtualized. Zap  and FiST  demonstrated that an operating system’s kernel state and ﬁle systems can be virtualized as well. Kernel virtualization operates by virtualizing the system call interface, that is, by placing a layer of indirection between processes and the system calls they use to access the operating system kernel’s functionality and ephemeral state. Similarly, ﬁle system virtualization works by placing a layer of indirection between processes and the underlying physical ﬁle systems, or the operating system’s persistent state. Instead of accessing the machine’s kernel and ﬁle system directly using built-in system call and ﬁle system functions, the application running in the virtualized operating system executes a function within the virtualization layer. The virtualization layer can modify the parameters passed to it, perform
work required by the desired virtualization, call built-in kernel and ﬁle system funcChapter 1. Introduction 3tions to perform the desired real work, and modify the return value passed to the calling process.
This dissertation demonstrates that by leveraging diﬀerent forms of operating system virtualization, we can use commodity operating systems and software in novel ways and solve problems that the original developers could not have anticipated. By virtualizing the interfaces, we do not change the applications or operating system, but instead create specialized environments that enable us to solve problems. Although virtualized environments, from the perspective of processes, look and behave like the system they are virtualizing, they can look and behave very diﬀerently to the systems on which they are hosted. This decoupling of execution environment and host environment lets us create tools that run on the host and solve new problems without modifying a well-tested operating system and application code. For example, we can create virtual private namespaces for applications distinct from the namespace of the physical computer. To the processes running within the virtualized environment, the environment looks like a regular machine, provides the same application interface, and does not require applications to be rewritten. Similarly, because operating system virtualization only interposes itself between the application and the underlying operating system kernel, the underlying kernel’s binary and source code do not have to be modiﬁed either.
1.1 OS Virtualization Security and User Mobility
Some forms of operating system virtualization [85, 100] are limited to isolating a single user’s processes and are not designed to provide any security constraints. This is especially noticeable for processes that run with elevated privileges, such as those provided to root on Unix systems. Without secure virtualization, operating system
Chapter 1. Introduction 4virtualization can only solve single user problems, substantially limiting its use. To enable secure virtualization, we have enabled each virtualized environment to have a unique set of virtualized users. Virtualizing the set of users gives each environment an isolated set of privileges. However, unlike hardware virtualization, where each virtual machine has a full operating system instance and therefore its own isolated privileged state, operating systems generally only have a single set of privileged states.
Therefore, in addition to providing unique sets of virtualized users, we also restrict the abilities of virtualized root users. If the virtualized root users were not restricted, they could be treated equivalently to the root user of the underlying system, enabling them to break the virtualization abstraction. This dissertation demonstrates how operating system virtualization can be used to simply virtualize the set of users while restricting the abilities of the privileged but virtualized root user.
We then show that operating system virtualization can be combined with checkpoint/restart functionality to improve mobile users’ computing experience. Many users lug around bulky, heavy computers simply to have access to their data and applications. To solve this problem, we created *Pod devices. A *Pod is a physical storage device, such as a portable hard disk or USB thumb drive, containing a complete application-speciﬁc environment, such as a desktop or web environment.
*Pod devices run their applications on whatever host computer is available at the user’s current location. By storing the entire environment on the portable device, users can move it between computers while retaining a common usage environment.
Operating system virtualization, coupled with process migration technology, enables users to move their running processes and data between physical machines, much like a laptop can be suspended and resumed when changing locations. We have built a number of *Pod devices that enable users to carry an application [109,110,113] or an entire desktop  with them.
Chapter 1. Introduction 5
1.2 Mobility to Improve Administration
Building on *Pod, we demonstrate how operating system virtualization and checkpoint/restart ability can improve system maintenance, much of which requires taking the machine oﬄine and shutting down all active processes. Among other problems, this prevents the kernel from being patched quickly. as it requires the machine to be rebooted for the patch to take eﬀect, thereby killing all running processes on the machine. To address this, we developed AutoPod , a system that enables unscheduled operating system updates while preserving application service availability.
AutoPod leverages *Pod’s virtualization abstraction to provide a group of processes and associated users with an isolated machine-independent virtualized environment decoupled from the underlying operating system instance. This enables AutoPod to run each independent service in its own isolated environment, preventing a security fault in one from propagating to other services running on the same machine. This virtualized environment is integrated with a checkpoint/restart system that allows processes to be suspended, resumed and migrated across operating system kernel versions with diﬀerent security and maintenance patches. AutoPod incorporates a system status service to determine when operating system patches need to be applied to the current host, then automatically migrates application services to another host to preserve their availability while the current host is updated and rebooted. AutoPod’s ability to migrate processes across kernel versions also increases *Pod’s value by making it possible for users to move their *Pod between machines that are not running the exact same kernel version.
Chapter 1. Introduction 6
1.3 Isolating Cooperating Processes
AutoPod envisions virtual computer usage growing rapidly as users create and use many task-speciﬁc virtual computers, as is already occurring with the rise of virtual appliances. But more computers mean more targets for malicious attackers, making it even more important to keep them secure. Operating system virtualization, as in a pod, provides namespaces that isolate processes from the host, enabling a level of least-privilege isolation as single services are constrained to independent pods.
Today’s services, however, are complex applications with many distinct components.
Even within a pod, each component of the service has access to all resources required by every component within the system, which is not a true least-privilege system.
To solve this problem, we developed PeaPod , which combines the pod with a pea (Protection and Encapsulation Abstraction). As AutoPod demonstrates, pods can be used to isolate services into separate virtual machine environments. The pea is used within a pod to provide ﬁner-grained isolation among application components of a single service while still enabling them to interact. This allows services composed of multiple distinct processes to be constructed more securely. PeaPod enables processes to work together while limiting the resources each process can access to only those needed to perform its job.
1.4 Managing Large Numbers of Machines