Month: January 2016

How to find and online all of your SteelFusion Core LUNs on a NetApp filer

Recently, I offlined a bunch of LUNs that had belonged to a SteelFusion Core in the lab that I had forgotten about. Needless to say, I had some unhappy users. The good news though is that I was able to get the LUNs back up and connected to the Core within minutes. This is how I did it.

The first thing I needed to do was find out which LUNs the Core was using. I did this by logging into the Core via SSH and running the following commands:

enable
conf t
terminal length 0
show storage luns iscsi

I output this to a file /tmp/core30luns.txt. An entry looks like this:

Total LUNs: 9
Locally Assigned Serial: P3PdB/-GFigd
Configuration status : Ready
Alias : avamar_restore
LUN Size : 150.00 GB
LUN Type : iscsi
Online : yes
IOPs acceleration : Enabled
Failover Enabled : yes
Prefetch : Enabled
Edge mapping : pod3-3100b
Target mapping : iqn.2003-10.com.riverbed:oh1mt0017065c.000
Origin portal : 10.33.192.174, 10.33.192.175
Origin target : iqn.1992-08.com.netapp:sn.135037602
Backend session status : Connected
Use iSCSI Reservation : Yes
LUN Edge data session : Connected
Client type : other
Original LUN vendor : NetApp
Original LUN serial : P3PdB/-GFigd
Pinned : no
Prepop : Disabled
Smart prepop : Enabled
Prepop status : N/A
MPIO policy : roundrobin
iSCSI Reservation status : LUN reserved

Prepop schedules:
Mapped igroups:
all

Mapped initiators:

The next thing was to find out what LUNs are on the NetApp to do some matching. You can do that by running this command:

lun show -v

I output this to a file /tmp/netapp_luns.txt. An entry looks like this:

/vol/NewYork_rvbd_d_e7cc5c29_f400_4c52_b1d4_f87da1b62652_1451278801/lun_RDM 10g (10737418240) (r/w, offline)
Serial#: P3PdB/9ytT31
Share: none
Space Reservation: disabled
Multiprotocol Type: vmware

Now with the 2 files, I could do some matching. I first want to extract the serial numbers from the LUNs. I do this by running:

grep serial /tmp/core30luns.txt | cut -f2 -d: > /tmp/core30lunlist.txt

From that, I would just get a list of serial numbers like this:

P3PdB/-GFigd

Next, I will loop through my list of LUNs to find the volumes I will need to put back online. I do this by running:

for i in `cat /tmp/core30lunlist.txt`; do grep -2 $i /tmp/netapp_luns.txt >> /tmp/netappvolumes.txt; done

This would give me a list like this:

/vol/NewYork_rvbd_d_8f3a7b69_05f7_4be8_b3a6_14a689c2b3b0_1452834001/lunC11 60.0g (64445480960) (r/w, offline)
Comment: “Cdrive”
Serial#: P3PdB/-KWreM
Share: none
Space Reservation: disabled

With that list, I can cut the volumes out with the following command:

grep -v : /tmp/netappvolumes.txt | cut -f1 -d' ' > /tmp/volumes.txt

This would give me a list like this:

/vol/NewYork_rvbd_d_8f3a7b69_05f7_4be8_b3a6_14a689c2b3b0_1452834001/lunC11

Now that I have a list of volume names from the NetApp, I can just put them all online with a loop:

for i in `cat /tmp/volumes.txt`; do echo "lun online" $i >> /tmp/online_vols.txt ; done

You can just take the /tmp/online_vols.txt file now and just paste it into your NetApp SSH session and you’ll have all of your LUNs online again.

 

How to expand a LUN with SteelFusion

 Why would you want to do this?

No matter how well you plan, resources are almost never in abundance. There could be a few reasons why you want to expand a LUN. Perhaps you want to add more vmdks to the same VMFS volume. Another reason could be that you might want to increase the size of the partition of the guest VM. In either case, you might want some of the workarounds listed at the end of the article before continuing. If you’re convinced that expanding the LUN is what you’re going to do, here is how to do it.

Check feasibility. First check at the maximum size and expandability of the partitions involved.

  • Check the guest OS partition. If NTFS, the maximum partition size for NTFS is 256TiB according to http://en.wikipedia.org/wiki/NTFS. You probably don’t need to go beyond that.
  • Check the partition size of VMFS. Consult VMware documentation http://www.vmware.com/support/pubs/ to ensure that you do not exceed maximum size for your partition. For example, if your partition is VMFS-3 and formatted with a 1MB block size, your maximum is 2TB-512bytes. If you’re already at the maximum, see workarounds further in this document.

These are the steps in a nutshell:

  • Resize/expand the LUN on the backend array.
  • If the SteelFusion Core is a virtual machine and LUN is connected via fiber, run a Rescan/Refresh on the HBA on the ESXi server where the Core is hosted.
  • On SteelFusion Core, perform a LUN rescan
  • Following VMware documentation, resize the VMFS volume http://kb.vmware.com/kb/1752
  • Resize or map the device in Windows

Resizing the LUN on the backend array.

Consult your array’s documentation for resizing the LUN. Here’s an illustration on how it’s done with a Netapp array.

1) Locate the LUN you want to expand, right click and select Edit.

01locate_lun

2) In the Edit LUN window, assign the new LUN size, and click save and close.

02editlun

Seeing the new size on the SteelFusion Core

The SteelFusion Core may automatically detect the change in LUN size depending on how busy the LUN is. If there are read/write operations from the backup with a “LUN configuration changed” status, the Core will be forced to rescan and retry and it will automatically change. There’s also a periodic rescan every 5 minutes that would cover change it. If you need the change immediately, you can do a manual rescan and force the change. The size will highlight in orange when it changes.

03addlun

Once the Core sees the change, the new size will be propagated to the Edge. If the LUN was pinned, the Edge will approve or deny the size increase based on Blockstore capacity.

Extending the VMFS volume size

Reference this http://kb.vmware.com/kb/1752 VMware KB article for extending your VMFS volume. Here’s how it’s done with the vSphere Web Client if you’re using vCenter:

 

Find the Datastore, right click and select Increase Datastore Capacity.

04increasedscap

Select Use Free space to expand the Datastore.

05usefree

If you’re using the deprecated Windows client, you can effectively do the same in the Datastore view here:

07increase

Select the Datastore you are expanding, go to the Configuration tab and select the Properties of the Datastore. In the Properties box, Select Increase.

08epxand

Follow the wizard and it will expand the volume.

Expanding the size of the virtual disk on the VMFS volume

If the UI doesn’t allow you perform the following steps, refer to this VMware KB:

http://kb.vmware.com/kb/1004047

Find the VM settings and change the size of the VMDK.

09find

In the Windows client, it’s in the same area.

10find

Expanding the partition in Windows

Lastly, go into the Windows guest and expand the volume size.

First, go to the Disk Management in the Computer Management application, right click and select Rescan Disks.

11rescan

Upon rescanning, you should see the newly added space as unallocated.

12unallocated

Right click on the primary partition of the disk and select Extend Volume.

13extend

At completion, you now should have your expanded disk.

Consideration

Consider inflating the VMDK file after doing the extension during non-peak hours. Because the added capacity that you just added to the VMDK file is sparse, you may want to fill it up with zeroes. The reasoning behind this is so that you are not struck with first write or double-write penalty (in all copy-on-write disks) when your guest writes to the disk. You can read a little about the penalty here: https://blogs.vmware.com/vsphere/2014/05/thick-vs-thin-disks-flash-arrays.html

http://blogs.vmware.com/vsphere/2012/03/thin-provisioning-whats-the-scoop.html

You can do this in the Datastore browser. Just find the VMDK file of your VM, right click and select Inflate.

14inflate

Functionality also is available in the Windows client.

15inflate

Workarounds

Here are some workarounds to consider LUN expansion is not feasible:

  • Consider moving swap files to another VMFS volume, preferably on a Local LUN. You can do this in the Options tab of a VM’s settings. This will require your VM to be shutdown, but the benefit of having this on a Local LUN is that it does not need to be replicated.
  • Consider moving guest page file, swap files, and print spools to another drive. Like workaround #1, these things do not need to be replicated, so moving them to a Local LUN can help with your WAN traffic.

Not-recommended workaround

A workaround that may sound logical, but is not recommended is to Storage vMotion the disk to another LUN. This should be your last resort. This may be something that vSphere administrators do commonly, but because the Blockstore on the Edge is a cache and not ordinary storage, this will result in a lot of WAN traffic and can be potentially dangerous.