Deconstructing Cisco NX-OS Part 2: Exploding the System Image

Previously I wrote on how to look inside of the NX-OS kickstart image.  This post will be how to look inside the NX-OS system image.  The system image is not “booted” like the kickstart image.  Its loaded into place by the kickstart.  So unlike the kickstart image, the system image is not a NBI (Network Bootable Image) or designed to be booted natively:

[root@cluebox nx-os]# ls -al n7000-s1-dk9.5.1.3.bin
-rw-r–r– 1 root root 146701191 Mar 24 00:48 n7000-s1-dk9.5.1.3.bin
[root@cluebox nx-os]# file n7000-s1-dk9.5.1.3.bin
n7000-s1-dk9.5.1.3.bin: data

Looks like just “data”, but looks can be deceiving.  Let’s see what we can find out with the NX-OS tools:

[root@cluebox bin]# ./imghdr show /tmp/nx-os/n7000-s1-dk9.5.1.3.bin

**********************************************

HEADER CONTENTS
**********************************************
Header version: 1.0
Len: 4912 byte
Image length:146701191 byte
Magic number: 21326
Platform type: 7
Verification type: 1
Software family: 2
Image type: 3
Debug attribute: 1
Hardware type: 0
Compression type: 2
Run time location: 1
Packaged by: 0
Memsize: 256
Timestamp: 1299832655
Version string: 5.1(3)
Interim version string: 5.1(3)
Image full name: n7000-s1-dk9.5.1.3.bin
Features:
Build ID: S39
**********************************************
Cisco NX-OS(tm) n7000, Software (n7000-s1-dk9), Version 5.1(3), RELEASE SOFTWARE Copyright (c) 2002-2010 by Cisco Systems, Inc.

Ok so this tells us there is a header that is 4912 in length.  What’s after that?

[root@cluebox bin]# hexdump -C -s 4912 /tmp/nx-os/n7000-s1-dk9.5.1.3.bin
00001330  1f 8b 08 00 51 df 79 4d  00 03 ec 9a 0b 3c 94 d9  |….Q.yM…..<..|

So those of you who have worked with filetypes and hex editors probably are immediately aware of the classic “1f 8b 08” signature.  This is a zipped file 🙂

So all we have to do is try to assemble the file without the header, no problem, that is what dd is for:

[root@cluebox nx-os]# dd if=n7000-s1-dk9.5.1.3.bin bs=16 skip=307 of=system-filesystem.tar.gz
9168517+1 records in
9168517+1 records out
146696279 bytes (147 MB) copied, 18.9374 seconds, 7.7 MB/s
[root@cluebox nx-os]# ls -al system-filesystem.tar.gz
-rw-r–r– 1 root root 146696279 Mar 24 19:49 system-filesystem.tar.gz
[root@cluebox nx-os]# file system-filesystem.tar.gz
system-filesystem.tar.gz: gzip compressed data, from Unix, last modified: Fri Mar 11 03:37:37 2011
[root@cluebox nx-os]# gunzip system-filesystem.tar.gz
[root@cluebox nx-os]# file system-filesystem.tar
system-filesystem.tar: POSIX tar archive
[root@cluebox nx-os]# mkdir system-filesystem
[root@cluebox nx-os]# cd system-filesystem
[root@cluebox system-filesystem]# tar -xvf ../system-filesystem.tar

./
./isan/
./isan/etc/
./isan/etc/crapcli/
./isan/etc/crapcli/obfl_do_cli.cli_
./isan/etc/imghdr.bin
./isan/etc/schema/
./isan/etc/climib/
./isan/plugin_img/
./isan/plugin_img/eth_plugin
./isan/plugin_img/img.conf
./isan/plugin_img/core_plugin

 

This is much different then say the way SAN-OS use to work.  The system image use to be a complete filesystem, much like the kickstart was.  It appears that with NX-OS they have these “plugins”, but we can take a look and inspect them:

[root@cluebox plugin_img]# ls -al
total 144436
drwxr-xr-x 2 root root     4096 Mar 11 03:37 .
drwxr-xr-x 4 root root     4096 Mar 11 03:37 ..
-rwxr-xr-x 1 root root 65249301 Mar 11 03:37 core_plugin
-rwxr-xr-x 1 root root 82478519 Mar 11 03:37 eth_plugin
-rwxr-xr-x 1 root root      250 Mar 11 03:37 img.conf
[root@cluebox plugin_img]# file *
core_plugin: data
eth_plugin:  data
img.conf:    ASCII English text
[root@cluebox plugin_img]# cat img.conf
#
# This img.conf specifies all plugins included in the image. It is packaged
# under ./isan/plugin_img/img.conf. When the image boots up, img.conf is
# read to figure what plugins should be loaded.
#
+ LIST_BEGIN
core_plugin
eth_plugin
+ LIST_END

 

 

Ok so it loads these plugins.  Its just a neater way of packing things that they seem to be doing in NX-OS, sort of separating things.  We can look at each of these plugins:

[root@cluebox plugin_img]# hexdump -C core_plugin |more
00000000  64 01 53 4e 06 dc 00 00  63 6f 72 65 5f 73 75 70  |d.SN….core_sup|
[root@cluebox plugin_img]# hexdump -C eth_plugin |more
00000000  64 01 53 4e 06 40 00 00  65 74 68 5f 73 75 70 64  |d.SN.@..eth_supd|

So notice how each of these has a signature of “64 01 53 4e “.  That is the signature of NX-OS/SAN-OS images, they all start off that way.  So this tells us we have more to “unravel”, quite the rabbit hole but we can continue:

[root@cluebox plugin_img]# /tmp/nx-os/kickstart/kickstart-filesystem/isanboot/bin/imghdr show core_plugin
**********************************************
HEADER CONTENTS
**********************************************
Header version: 1.0
Len: 1756 byte
Image length:65249301 byte
Magic number: 21326
Platform type: 1
Verification type: 1
Software family: 2
Image type: 3
Debug attribute: 1
Hardware type: 0
Compression type: 2
Run time location: 1
Packaged by: 0
Memsize: 256
Timestamp: 1299832452
Version string: 5.1(3)
Interim version string: 5.1(3)
Image full name: core_supdc3x.bin
Features:
Build ID: S39
**********************************************
Cisco NX-OS(tm) core_supdc3x.bin, Software (core_supdc3x), Version 5.1(3), RELEASE SOFTWARE Copyright (c) 2002-2010 by Cisco Systems, Inc.

Ok, same drill.  We pull out dd and let it take a whack at this header:

[root@cluebox plugin_img]# dd if=core_plugin bs=1 skip=1756 of=core_plugin_fixed.tar.gz
65247545+0 records in
65247545+0 records out
65247545 bytes (65 MB) copied, 108.441 seconds, 602 kB/s
[root@cluebox plugin_img]# file core_plugin_fixed.tar.gz
core_plugin_fixed: gzip compressed data, from Unix, last modified: Fri Mar 11 03:37:17 2011
[root@cluebox plugin_img]# gunzip core_plugin_fixed.tar.gz
[root@cluebox plugin_img]# file core_plugin_fixed.tar
core_plugin_fixed.tar: POSIX tar archive
[root@cluebox plugin_img]# mkdir core
[root@cluebox plugin_img]# cd core
[root@cluebox core]# tar -xvf ../core_plugin_fixed.tar

<a ton of files are unarchived>

[root@cluebox core]# ls
etc  isan  usr  var

and we rinse and repeat for the eth_plugin:

[root@cluebox plugin_img]# /tmp/nx-os/kickstart/kickstart-filesystem/isanboot/bin/imghdr show eth_plugin
**********************************************
HEADER CONTENTS
**********************************************
Header version: 1.0
Len: 1600 byte
Image length:82478519 byte
Magic number: 21326
Platform type: 1
Verification type: 1
Software family: 2
Image type: 3
Debug attribute: 1
Hardware type: 0
Compression type: 2
Run time location: 1
Packaged by: 0
Memsize: 256
Timestamp: 1299832452
Version string: 5.1(3)
Interim version string: 5.1(3)
Image full name: eth_supdc3x.bin
Features:
Build ID: S39
**********************************************
Cisco NX-OS(tm) eth_supdc3x.bin, Software (eth_supdc3x), Version 5.1(3), RELEASE SOFTWARE Copyright (c) 2002-2010 by Cisco Systems, Inc.

We call dd again, gunzip, untar:

[root@cluebox plugin_img]dd if=eth_plugin bs=1 skip=1600 of=eth_plugin_fixed.tar.gz
82476919+0 records in
82476919+0 records out
82476919 bytes (82 MB) copied, 136.03 seconds, 606 kB/s
[root@cluebox plugin_img]# file eth_plugin_fixed.tar.gz
eth_plugin_fixed.tar.gz: gzip compressed data, from Unix, last modified: Fri Mar 11 03:35:14 2011
[root@cluebox plugin_img]# gunzip eth_plugin_fixed.tar.gz
[root@cluebox plugin_img]# file eth_plugin_fixed.tar
eth_plugin_fixed.tar: POSIX tar archive
[root@cluebox plugin_img]# cd eth
[root@cluebox eth]# tar -xvf ../eth_plugin_fixed.tar

alot of files are unarchived…..

[root@cluebox eth]# ls
boot  isan

At this point, the system image is fully deconstructed and there is much to look at and explore 🙂  Basically the eth_plugin contains all the stuff related to networking like routing protocols, OTV, etc.  And core contains everything else.  I will be posting more on navigating your way around the various files later on as well as some more interesting things to look at.

This entry was posted in Nexus, NX-OS and tagged , , . Bookmark the permalink.

One Response to Deconstructing Cisco NX-OS Part 2: Exploding the System Image

  1. Pingback: Technology Short Take #12: Networking Edition - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers

Leave a Reply