Search This Blog

Sunday, March 16, 2014

OpenStack nova boot server call diagram

The OpenStack architecture consists of multiple distributed services which often work together to carry out a single logical operation. Given the nature of this architecture, getting up to speed on the call flows and interactions can be a daunting task for developers and operational admins alike.

Not so long ago, I had to pleasure of digging into one of the more common flows in OpenStack nova compute -- the nova 'boot server' operation. As we all know the boot server operation provisions a new nova compute Virtual Machine (VM) on an underlying hypervisor such as KVM, ESXi, etc.. As part of the boot server operation, a number of OpenStack components are involved including:



  • REST API controllers.
  • The nova compute scheduler.
  • Nova compute manager and virt driver (a per hypervisor component).
  • Numerous OpenStack clients which interface with other OpenStack services (glance, neutron, etc.).
  • Conductor for database persistence.
  • RPC via AMQP.
  • etc..


For a single boot server operation consider some of the high level logistics which must be carried out:

  • A REST API controller must validate, sanitize and initiate a request operation.
  • An initial 'shell' server must be persisted in the database (the operation is mostly async and thus a 'shell' server response must be returned to caller so they can poll status).
  • The scheduler must choose and reserve a hypervisor / host for the provision operation.
  • A VM image must be downloaded from glance (if not already cached on the hypervisor) to boot the server from.
  • Volumes must often be allocated and realized from OpenStack Cinder.
  • Network related aspects must be allocated and realized.
  • The VM must be booted, prepped and plugged (volume(s), network info, etc.).

The list given above is a very abstract and simplistic view provided to illustrate the complexity and breadth of operations which must occur to fulfill a single request. For this reason I found it easiest to visualize the components and interactions using a diagram which is provided below.

The ad-hoc diagram below depicts the high level call flow of an OpenStack nova compute 'boot server' operation. In this diagram each of the boxes illustrate a major component of OpenStack and the lines represent interactions between components. This diagram is not intended to show each and every interaction / operation, however it provides incite into the major operations and interactions which occur to full the provisioning of a new nova VM. Finally note this diagram was constructed in late July of 2013 and thus reflects the state of OpenStack at that point in time -- internals may have shifted.


Legend: Component (box) color coding

Box Color
Component Type
Yellow
Controller classes for nova API
Pink
Nova scheduler classes
Green
Nova manager (framework) classes
Cyan
Nova virt driver (implemented per hypervisor type)
Grey
Clients used by the various nova classes to interface with other services via REST API
Red
Other OpenStack services



Legend: Interaction (line) color coding

Line Type
Interaction Type
Solid black
Python API call
Black dashed
REST API call
Reddish solid
AMQP

Diagram: OpenStack nova compute 'boot server' call flow