Deconstructing Cisco NX-OS Part 1: Exploding Kickstart

I have worked for quite some time with SAN-OS, which later became the basis for NX-OS. Being the tinkerer that I am, I really like to get inside things and figure how they work. My past experience with UNIX based OS’s made it ideal for me to be able to further explore the innards of NX-OS. During this discovery, I found multiple security vulnerabilities, which were reported to Cisco via their PSIRT process and has hopefully made NX-OS a more secure and better product. The angle of this post is more from an engineering perspective, not security. Having access to the NX-OS filesystem is not really a security risk, but it is certainly interesting. There are many ways to achieve this. Like with IOS-XR, it is possible to gain access to the “shell” of the NX-OS system via its normal CLI interface, however we will go about it in a much more straight forward approach.

In this tutorial I am working with CentOS 5.5, but really just about any version of linux should work fine for these tasks.

NX-OS is based on MontaVista Linux (HardHat Linux). Cisco distributes the kickstart and system binary files which are used to run NX-OS. Kickstart is actually a composite file that contains multiple different sections and is used to get the system up and running. The system binary contains the majority of the actual daemons and processes that run the system.  You will obviously have to have access to NX-OS images to do this.  I have used SAN-OS images in the past as well.  Below we are working with the latest release of the Nexus 7000 code:

 

The kickstart image contains a header of variable length. The header maps out a lot of data, notablythings like the version, image offset, image type, etc. The kickstart image is a NBI image.

 

Ok, so we have established this is a standard NBI mode 2 file (Network Bootable Image). This is a standard and you can read about NBI images here Draft Net Boot Image Proposal 0.3
We can take a look at what segments we have in our kickstart file:

 

So we have six basic segments to this file. Segment 7 doesn’t really count, as its Image Length is 0, and we can’t have that now can we :). So we explode these out into our filesystem:

 

binmode() on unopened filehandle 0 at /usr/bin/disnbi line 156.

 

You can safely ignore the error at the end, of course it can’t read the data, if its 0 in size that’s rather impossible now isn’t it? Note that when you do this, what was called “Segment 1” in the disnbi output gets saved as segment0, and what was called “Segment 2” gets saved as segment1, and so forth. The actual nbi header gets saved as nbidir. It contains all the information about each file such as the Load Address, Image Length, Memory Length, Position, and Vendor tag. So for example, looking at the nbidir file:

So we can get back to our original kernel as well as initrd with the following:

So we are making progress. Hopefully this doesn’t seem to complex, I can imagine coming from the network side of things, all this linux jargon can be mind blowing.are getting close to some cool stuff though.

This is basically an initrd image (Initial Ramdisk) and its compressed, we can rename it correctly and take care of it:

Ok, so we are inside the kickstart filesystem!!!!  The binaries are executable:

You may run into things you can’t run, because you have to set your libraries up properly with LD_LIBRARY_PATH, but basically you can do some really cool stuff.  You can even use the NBI tools to “repackage” your own NX-OS with modifications :).

So there is much to look at!  This was just a splash dive into the kickstart, there is still much to discuss on how it actually boots, switches to the system image, and the role it plays.  The “system” image is handled a bit differently than the kickstart.  We will explore that in our next segment and in fact use some of the tools in the kickstart image to allow us to do this.