The ubiquity and acceptance of the Internet in our society has encouraged the development of many applications that are inherently real-time in  nature. Examples include multi-source streaming media delivery, interactive distance learning, webcasting, group simulations, live auctions, and stock brokerage applications. These applications require quality of service(QoS) guarantees on the data exchanged between processes on different hosts. The QoS requirements on this data are typically in terms of throughput, loss, delay and jitter. Moreover, the resource demands to meet these QoS requirements may vary at run-time. For example, a target tracking application may require more or less CPU cycles to identify an object of interest in a graphical image, depending on the content of that image.

Existing general-purpose operating systems are ill-equipped to meet the QoS requirements of applications like those described above.  Consequently, many classes of real-time applications have been developed to compensate for inadequate system support, or to run on special-purpose systems. Alternatively, many researchers have provided middleware solutions that bridge the `semantic gap' between the needs of applications and the services provided by the system. Unfortunately, middleware solutions lack fine-grained control over system resources, thereby limiting the rate at which resource allocations can be adapted. For example, many of the CORBA-based (middleware) adaptation frameworks now being developed are subject to the costs of monitoring and adapting system resource usage via processes and system calls, that have limited capabilities.

It is desirable to implement QoS management abstractions at the kernel-level, as opposed to using middleware, so that resources are managed more efficiently and better service is provided to real-time applications. Moreover, if these abstractions are implemented in general-purpose systems, improved service is possible for many desktop applications with QoS constraints (e.g., RealNetworks RealPlayer, or Windows Media Player). It is therefore beneficial to extend the behavior of existing commodity `off-the-shelf' operating systems, by incorporating new service abstractions and interfaces.  Unfortunately, most commodity systems are structured around monolithic kernels that have not been designed to support extensibility or specialization of system behavior. Although some systems support limited extensibility, by allowing device drivers to be loaded at run-time (e.g., Linux and Solaris), they lack support for code extensions that override or alter the behavior of system-wide service policies.

As a consequence, we are developing a combined kernel- and user-level adaptive QoS management infrastructure, called Linux Dionisys (see Figure 1). In this infrastructure, application processes link with a Dionisys library to create service extensions. Service extensions can be passed around the system to SafeX daemon processes, where they are compiled and linked into the address space of the target kernel or user-level sandboxes. In the original Dionisys prototype, these service extensions were either monitors , handlers or service managers. We are now developing support for high-level sandboxed service extensions for application data management, such as stream processing agents (SPAs).

A service manager is an encapsulation of a resource management subsystem, and has a corresponding policy for providing service of a specific type. For example, a CPU service manager may have (potentially configurable) policies for CPU scheduling and synchronization, while a network service manager might have policies for flow control and/or packet scheduling. Low-level service managers (e.g., for CPU, network or memory management) are  typically linked into the kernel address space, whereas high-level  (e.g., for application data management) are typically linked into the address space of user-level sandboxes. Sandboxed service extensions may include stream processing agents, for relaying/routing, filtering, transforming, splitting or merging application data streams, exchanged between hosts.  In effect, it is possible to deploy sandboxed agents on strategic intermediate hosts for the purpose of end-system multicasting and QoS-constrained routing (in a manner similar to Narada or Pastry). One of the main targets of this work, therefore, is to provide scalable end-system solutions to the real-time transport and delivery of data streams from publishers to potentially thousands of subscribers (with independent QoS constraints).

Kernel Service Managers

Observe that kernel service managers (see Figure 2) run as bottom half handler routines that are invoked periodically or in response to events within the system. This enables extensions to execute asynchronously to application processes, so an application may influence the way resources are allocated to it even when not scheduled.


Each service manager has two queues for monitor and handler functions. A monitor function, M, bound to the queue of service manager, S, is executed periodically on behalf of an application, A. Using the interfaces provided by the SafeX daemons, M observes the service provided to A by S. Similarly, a handler function, H, that is bound to the queue of S and operating on behalf of an application, A' , is executed in response to events from a corresponding  monitor function associated with A'. H uses SafeX interfaces to influence changes to the service provided by S, so that improved service (or guaranteed QoS) is provided to A'. Moreover, applications can create event channels (using the MEDEA subsystem) between their monitor and handler functions, so that deficiencies in one monitored service can trigger compensatory changes in the same or other service managers.

Monitors and handlers operate on attribute classes. These are data structures that hold the names of various QoS attributes and their corresponding values. For example, a name-value pair referring to CPU scheduling priority would be (priority-numeric_value). As another example, an application-specific attribute for a video application might be (frame_rate-numeric_value) .  These attribute classes are created by applications and linked into the kernel address space of a given host by a SafeX daemon.  Service extensions and applications get and set these attributes by name, as long as they have the necessary access rights.

Each host has a separate attribute store for each application, that is identified by a class descriptor. Application processes and service extensions on a given host may only retrieve and set attributes in the corresponding attribute class on the local host (as identified by a class descriptor). As with service extensions, an application process can create an attribute class for deployment on a remote host. Access to this class is granted to remote processes that acquire permission from the class creator. Observe that the nameserver in  the figure shown above maintains a database of bindings of class descriptors to attribute classes on each host, event channel ids to actual channels, and service extension names to their actual locations.

Each service manager is equipped with a guard function that is automatically generated by the code generator in a SafeX daemon process running on the same host. A guard function is responsible for the mapping of attributes, contained in attribute classes, to kernel policy-specific structures. It ensures that attributes are within valid ranges and will not affect the QoS guarantees to the corresponding application, or other applications. Observe that each SafeX daemon process implements a code generator and linker/loader, for compiling and linking extension code into the host's kernel address space. Moreover, each SafeX daemon is capable of generating code for run-time safety checks of extensions, thereby guaranteeing they have bounded execution time.

Further details regarding sandboxed extensions can be found on the project page for user-level sandboxing.

Related Projects