Installing & Maximising the NetApp ONTAP 8.1 Simulator

UPDATE 7 September 2012:  NetApp has released its Virtual Storage Appliance, Data ONTAP Edge which is a licensed functional VSA.
One of the strengths of NetApp’s storage offering is that its controller operating system called Data ONTAP is the same across every NetApp Filer storage product they sell. This is hugely significant as it means if you know how to administer and configure the smallest FAS2000 Series Filers you are pretty much up to speed to administer and configure Filers up to their biggest and beefiest FAS6200 Series Filers so you don’t have to learn another management interface when you upgrade.
NetApp has also produced a Data ONTAP Simulator which in their words is a “A tool that gives you the experience of administering and using a NetApp storage system with all the features of Data ONTAP at your disposal.”
There have been various versions of the simulator over the years which initially could be installed on a simple RedHat or SuSE Linux box using emulated disks and have the same look and feel as a real NetApp Filer (without the rack space requirement or electricity bill!). Things progressed over the years and you could use more Linux distros. Nowadays there is a pre-built VMware virtual machine so you don’t have to install RedHat or SuSE beforehand.
Unfortunately the simulator is only available to existing NetApp Customers and Advantage Partners and requires a login to the NetApp Support Site, http://now.netapp.com
I was about to start a serious rant about this limited availability when surprisingly whilst writing this post Vaughn Stewart sent out a tweet that NetApp are in fact looking at the possibility of opening up access to the simulator for version 8.1 which is fantastic news.
I fiercely believe opening up access to learning and training tools allows so many more people to learn about your technology in their own time and if they know about your technology they are more likely to buy!

So, although the current status is the simulator is not available to all, hopefully this will change soon and this has saved me and you from a far longer rant!
There are however some limitations to the simulator:

• There isn’t any official support by NetApp and it works more on a community support model using NetApp’s community support forums.
• There is still quite a bit of configuration to do to get the simulator up and running, it’s not a plug and use VSA.
• The simulator is limited in the number of disks it can support and the size of the disks. In previous versions of the simulator it was provided with 2 x simulated shelves of 14 x 1 GB Disks with only 20 GB usable space which with some hacking could be extended by 2 more shelves. The current version 8.1 is provided with the same disk configuration but I’ll show you how this can be changed to use up to 4 x shelves of 4 GB disks with 180 GB available space which is a vast improvement and far more usable but you will need VMware Workstation to do this as ESXi doesn’t support changing the size of IDE disks.
• The simulator isn’t as robust as other true VSAs as it replicates a real Filer where you would have redundant PSUs so doesn’t play nicely when you just power it off! This is unfortunate as previous versions did seem to be a little more robust but the 8.1 version needs to be handled with care.
• You can’t by default connect to multiple simulators with NetApp OnCommand System Manager as the built in serial numbers are the same and System Manager doesn’t play nicely but I’ll show you how this can be changed.
• You can’t clone a fully configured simulator as you can’t change the serial number after the initial configuration.
• The simulator isn’t suited to running anything that is performance heavy as it is aimed at testing the software features rather than any IOPS benchmarking.

I’m going to go though the steps to amend the simulator disk size to the maximum available disks and configure the basic simulator settings so you can use it as a VSA using the following steps:

1. Download the Simulator
2. Add to VMware workstation or ESX(i)
3. Increase the simulator VM disk size
4. Amend the default serial number
5. Run the initial configuration
6. Add 2 new shelves of 14 GB Disks
7. Move the current config to the new shelves
8. Delete the 2 x shipped 1 GB disk shelves
9. Add 2 x 4 GB Disk shelves to replace the shipped 1 GB disk shelves

Hopefully the steps I’m going to go through in this post will one day be put into the simulator as delivered by NetApp to save you the hassle!
Download the Simulator
The simulator can be downloaded from http://now.netapp.com/NOW/download/tools/simulator/ontap/8.0/ (you will need a NetApp login which is only available if you are a Customer or Advantage Partner)
Check that you can meet the hardware and software requirements listed on the download page.
I am going to set up and configure the 7-mode version rather than the cluster mode and on VMware workstation but you can upload the VM to ESX(i) but only after the disk size has been amended as you can’t amend an IDE disk in ESX(i).
Download the 7-mode version which is a .ZIP file either for VMware Workstation, VMware Player and VMware Fusion or the version for VMware ESX(i).
Add to VMware workstation or ESX(i)
Extract the .ZIP file. Copy the extracted files to a folder on your workstation or upload them to a datastore if you are going to be using it on ESX(i) but remember the issue with increased an IDE disk on ESX(i).
Double click on the .vmx file to add to VMware Workstation or browse the ESX(i) datastore and add to inventory.
You should now have a new VM called vsim-7m.

Rename the VM to what you will ultimately call your simulator to avoid confusion.

Edit the VM Settings.
As this is a testing simulator for a virtual hosting environment I am going to only have 1 network card as I don’t need to test any network failover configuration but if you do need some more network functionality leave them in and configure them as you would a physical filer.
Set the Network Adapter to a Network connection or port group on your network.

Increase the simulator VM disk size
The shipped VM comes with a 48 GB VM disk which needs to be expanded to accommodate the new data in the emulated simulator disks when you write data to them.
You need to increase the VM disk size and then extend the partitions and also slices within the simulator to take advantage of the increased VM disk space.
The current 48 GB VM Disk contains 28 x emulated 1 GB disks and then has another 20 GB so if we are going to have 56 x emulated 4 GB disks + 20 GB then we will need a 244 GB VM Disk. I’m not sure if this is actually the correct figure but it works for me. You can make this VM disk thin-provisioned so you don’t have to have all 244 GB available until you write to it.
Edit the VM Settings and increase the HardDisk from 48 GB to 244 GB.

If you want to run the simulator on ESX(i) you can now upload it from VMware workstation with the amended IDE disk size.
The simulator VM runs on FreeBSD and uses the UFS file system.  Unfortunately the usual Linux partition manager GParted can’t see UFS partitions so it isn’t much help so you need to use a FreeBSD boot disk.
I downloaded the FreeBSD 8.2 i386 LiveFS .ISO from freebsd.org.
ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/ISO-IMAGES/8.2/FreeBSD-8.2-RELEASE-i386-livefs.iso
Edit your simulator VM settings and add a new CD/DVD Drive as the simulator doesn’t come with one.
Attach the FreeBSD LiveFS .ISO
View the console so you can see the simulator boot process.
Boot the VM and change the BIOS boot order to boot first from CD-ROM.

Press Enter to Boot FreeBSD [default].

Select your country and press Enter.

Select your keyboard layout if prompted and press Enter.

Enter F and Enter to launch the Fixit repair mode option.

Select CDROM/DVD and press Enter.

You will now enter “fixit” mode.
Find the VM disk files:

1	dmesg | grep VMware

You will see there are two VMware Virtual IDE Hard Drive disks, ad0 which is 249856 MB and ad1 which is 1024 MB.

So, ad0 is the disk which is 244 GB.
To view the partitions on the disk:

1

fdisk ad0

You will see that partition 4 which is 44007 MB must be the partition which holds the emulated simulator disks and this is the partition that needs to be extended.

To extend the partition using the interactive slice editing process:

1	fdisk –u ad0

Press Enter to not change the BIOS info.

Press Enter to bypass editing the partitions until you get to partition 4 then type Y.
Press Enter to keep the default “sysid”.
Press Enter to keep the default “start”.
Now you need to enter the new sector size. I have worked out what the number needs to be in a completely non-scientific way using a whole host of different boot disks and a million different options! If someone else knows how to work this out in a far better way I would love to know how!
Enter 507513006 as the “size” and press Enter.
Press Enter to skip beg/end address.
You will now see a summary of the partition changes that fdisk will apply.
Press Y to be happy with the entry.
Press Enter to not change the active partition.

You will see a summary of the new partition table showing the new size of 247809 MB.

Press Y to write the new partition table.
You may get a message fdisk: Class not found but this can be ignored.
You then need to extend the slice in the partition.
To view the slices use:

1	gpart list

You can then find the index of the partition in the slice. You can see under the Geom name: ad0s4 that ad0s4b is the slice which is 39 GB and that corresponds to index 2.

gpart by default will make the partition use all the available space. –i is the index number.
Extend the slice with the following command:

1	gpart resize –i 2 ad0s4

The disk, partition and slice has now been extended and you can disconnect the mounted .ISO from the VM and shut down the simulator.
Amend the default serial number
Boot the simulator again.
When you see Hit [Enter] to boot immediately, or any other key for command prompt, hit Ctrl-C.
You will then enter the SIMLOADER prompt.
I’ve used the same steps and number scheme from this NetApp Communities post.
Enter the following commands to set your unique serial number. (I’ve used 1111111101 – That’s 8 x 1s + 01)

1 2 3

set bootarg.nvram.sysid=1111111101 set SYS_SERIAL_NUM=1111111101 boot

Run the initial configuration
The simulator will continue to boot.
When you see Press Ctrl-C for Boot Menu, hit Ctrl-C.
Enter option (4) Clean configuration and initialize all disks.

When the simulator says Zero disks, reset config and install a new file system, type Y.
Type Y to confirm erasing all the data on the disks.
The simulator will set up the disks and reboot.
Once it has rebooted the configuration will continue.

When prompted enter the new hostname. Mine is lonfiler01.
Press Enter to accept the default [n] for IPv6.
Press Enter to not configure interface groups.
Enter the IP address for your network.
Enter the Subnet Mask.
Press Enter to accept the default media type.
Press Enter to accept the default flow control.
Press Enter to accept the default of no jumbo frames.
Press Enter to not continue setup through the web interface.
Enter the default gateway IP Address.
Press Enter to bypass configuring an administrative host.
Enter a timezone.
Enter a location name.
Press Enter to accept the default HTTP root file directory.
Press Enter to not run the DNS resolver.

Press Enter to not run the NIS client.
Press Enter to continue, auto support will be turned off later.
Press Enter to not run the Shield Alternate Control Path Management interface for SAS shelves.
Enter and confirm the root password.
The initial configuration will continue. Once complete, press Enter to bring up a login prompt and enter the root password you had entered.
It’s now best to continue configuration through an SSH connection with Putty so you can copy and paste the configuration commands.
Some of the ONTAP functions are licensed already in the simulator but some aren’t so you may as well add all the options that aren’t added from this list.

01 02 03 04 05 06 07 08 09 10 11 12

license add BSLRLTG #iscsi license add BQOEAZL #nfs license add ANLEAZL #flex_clone license add DFVXFJJ #snapmirror license add DNDCBQH #snaprestore license add JQAACHD #snapvalidator license add BCJEAZL #snapmanagerexchange license add RKBAFSN #smdomino license add ZOJPPVM #snaplock license add PTZZESN #snaplock_enterprise license add PDXMQMI #sv_ontap_sec license add RIQTKCL #syncmirror_local

Add 2 new shelves of 14 GB Disks
Now you are ready to add the new shelves.
Enter advanced mode and unlock the diagnostic user. Enter a password and confirm.

1 2 3

priv set advanced useradmin diaguser unlock useradmin diaguser password

Launch the systemshell and login as diag and enter the password you have just set.

1	systemshell

The next step will use the makedisks script to add 14 x 4 GB disks as shelf 2 and 3 of type 31 which is:
31     NETAPP__ VD-4000MB-FZ-520 4194,304,000 B 4289,192,960 B     Yes     520

1 2 3 4

setenv PATH "${PATH}:/sim/bin" cd /sim/dev sudo makedisks.main -n 14 -t 31 -a 2 sudo makedisks.main -n 14 -t 31 -a 3

Then exit the systemshell and lock the diaguser.

1 2	exit useradmin diaguser lock

Set admin mode.

1	priv set admin

Reboot the filer.

1

reboot

Once the filer has come back up, connect and login again.
Assign all the disks that are unowned to the filer.

1	disk assign all

Move the current config to the new shelves
We will move the current root volume vol0 to a new aggregate and new root volume on the new shelves.
First, create a new aggregate with the new shelves. Turn of checking for spare disks (which will still error in the logs but at least allow you to create a bigger aggregate) and set the raid group size to 28.

1 2	options disk.maint_center.spares_check off aggr create aggr1 -r 28 28@4G

Wait until the aggregate has been created and then create a new flexible volume vol1 on aggregate aggr1 of size 850Mb.

1	vol create vol1 aggr1 850m

Then we need to ndmp copy the contents of the old root volume into the new root volume.

1 2	ndmpd on ndmpcopy /vol/vol0 /vol/vol1

Wait until the ndmp copy has finished.
Set the new volume as the root volume.

1	vol options vol1 root

Reboot the filer.

1

reboot

Once the filer has come back connect with Putty and login again.
Something seems to go funny with the SSL certificates so you cannot also connect with OnCommand System Manager. The easiest way is to just recreate them.

1 2	options ssl.enable off secureadmin setup ssl

Enter Y to proceed and then you can just accept all the defaults or if you are more security minded you can enter more accurate values for your certificate requirements.
Then turn back on SSL.

1	options ssl.enable on

Delete the 2 x shipped 1 GB disk shelves
Now we can remove the old vol0 and aggr0 as they are no longer holding the root volume.

1 2	vol offline vol0 vol destroy vol0

Y to confirm deletion.

1 2	aggr offline aggr0 aggr destroy aggr0

Y to confirm deletion.
Now we can remove the old disks and shelves.

1 2 3

options disk.auto_assign off priv set advanced disk remove_ownership v5.*

Y to Confirm.

1	disk remove_ownership v4.*

Y to Confirm.
We may as well rename the root volume and aggregate back to what they were originally just to keep things clearer.
Rename the existing root volume.

1	vol rename vol1 vol0

Rename the existing aggregate.

1	aggr rename aggr1 aggr0

Add 2 x 4 GB Disk shelves
Now we can go ahead and add 2 more 4 GB disk shelves to replace the 2 x 1 GB Disk shelves that have just been removed.
Enter advanced mode and unlock the diagnostic user. Enter a password and confirm.
Launch the systemshell and login as diag and enter the password you previously set.

1 2	useradmin diaguser unlock systemshell

Now we need the remove the old simulated disks from shelf 0 and 1.

1 2 3 4 5

setenv PATH "${PATH}:/sim/bin" cd /sim/dev/,disks sudo rm v0* sudo rm v1* sudo rm ,reservations

Again use the makedisks script to add 14 x 4 GB disks as shelf 0 and 1 of type 31 which is:
31 NETAPP__ VD-4000MB-FZ-520 4194,304,000 B 4289,192,960 B Yes 520

1 2 3

cd /sim/dev sudo makedisks.main -n 14 -t 31 -a 0 sudo makedisks.main -n 14 -t 31 -a 1

Then exit the systemshell.

1

exit

Lock the diaguser and set admin mode.

1 2	useradmin diaguser lock priv set admin

Reboot the filer.

1

reboot

Once the filer has come back connect and login again.
You may as well turn off the pesky autosupport warnings and then assign all the new disks that are unowned to the filer.

1 2 3 4

options autosupport.support.enable off options autosupport.enable off options disk.auto_assign on disk assign all

You can now add these disks to the existing aggregate.

1	aggr add aggr0 28@4G

Y to Confirm adding the disks and again for the low spare warning.
You now have an aggregate spanning 4 x simulated shelves of 14 x 4 GB disks giving you 180.97 GB remaining available disk space after the root volume which is much better than the 20 GB you had initially!
Now is a good time to halt the filer and take a snapshot so you can easily revert if you have an issue.
You can also now go ahead and create your volumes and exports, LUNs, CIFS shares or whatever you want to use your simulator to test.
Remember, never just power off the simulator as you will land up with complicated wafl consistency issues which are difficult and sometimes impossible to fix.
Hopefully all these steps will be redundant when NetApp publicly release their simulator with the maximum possible disk size already configured and available as a simple to import virtual appliance.