How to set the NTP server on all ESX hosts in vCenter at the same time with PowerCLI

I was frustrated with a number of my lab hosts have the wrong times/timezones. Rather than going to each ESXi host to update the time servers, I ran a simple script to do it with PowerCLI.

As before running any script, you must authenticate with the vCenter or the host by running

Connect-VIServer <hostname>

Then, we’ll cycle through each of the ESXi hosts and add an ntp server. Obviously, use the time server you want i.e. or ūüôā

Get-VMHost | Add-VMHostNtpServer -ntpserver ""

With that set, you can then move on to punching the hole through the firewall to request the time from the servers.

get-vmhost | Get-VMHostFirewallException | where {$_.Name -eq "NTP client"} | Set-VMHostFirewallException -Enabled:$true

#Start NTP client service and set to automatic
get-vmhost | Get-VmHostService | Where-Object {$_.key -eq "ntpd"} | Start-VMHostService
get-vmhost | Get-VmHostService | Where-Object {$_.key -eq "ntpd"} | Set-VMHostService -policy "automatic"

How to use PowerCLI to register or unregister a bunch of templates at a time …

I recently resignatured a datastore that housed all of my templates and ran into an issue with a bunch of datastores that wouldn’t unmount and a slew orphaned templates. To get rid of them, I had to unregister them all and then register them, but had a ton, so I¬†didn’t want to do them one at a time.

Unregistering the VM templates is easy. Just select them all, right click and click “remove from inventory”. The registering part is harder – need to browse the datastore and then right click and add to inventory and give it a name.¬†So, I did it with PowerCLI.

The first thing we want to do is get a list of all of the vmtx files in the datastore that the templates reside in. I do it like this:

$vmtxs = gci vmstore:\$((Get-Datacenter -name "DCpod"))\templates -Recurse -Include *.vmtx | select -exp datastorefullpath

You can list it and you should see something like this:

PowerCLI C:\Program Files\VMware\Infrastructure\vSphere PowerCLI> $vmtxs
[templates] Win8.1_Template/Win8.1_Template.vmtx
[templates] Windows 10 template/Windows 10 template.vmtx
[templates] Win2K8R2_Template_1/Win2K8R2_Template.vmtx
[templates] Ubuntu_Server_Template/Ubuntu_Server_Template.vmtx
[templates] Win2K12R2_Template/Win2K12R2_Template.vmtx
[templates] webservertemplate2/webservertemplate2.vmtx

With the list of files, you can loop through and register them all on a particular host like this:

PowerCLI C:\Program Files\VMware\Infrastructure\vSphere PowerCLI> foreach($vmtx in $vmtxs)
$currentname = ($vmtx -split "/|\.")[1]
$vm = new-template -name $currentname -templatefilepath $vmtx -vmhost


ESX 6.0 – The operation is not allowed in the current connection state of the host.


The things in the posting did NOT work. According to IBM’s website, the fix is by removing a package:

esxcli software vib remove -n intel-cdmv2-provider

I got this message when trying to power on a vm, trying to deploy a template, and something else.

So I read this
and then I go to the host and try restarting hostd (/etc/init.d/hostd restart) and it will no longer restart.

Looking into /var/log/hostd.log, it appears that /tmp is full:

2016-02-09T06:01:01.648Z info hostd[79EC2B70] [[email protected] sub=Vimsvc.ha-eventmgr] Event 620 : The ramdisk ‘tmp’ is full.
As a result, the file /tmp/auto-backup.6942719/local.tgz could not be written.

So I look into the /tmp directory and find a file that’s 256MB – cimple_log_err_messages

cat cimple_log_err_messages
2016/02/09 23:28:57:LOG FILE ERROR: log.cpp(610) : No CIMPLE_HOME env var defined. Looking for CIMPLE_HOME
2016/02/09 23:28:57:LOG FILE ERROR: log.cpp(610) : No CIMPLE_HOME env var defined. Looking for CIMPLE_HOME

The first thing I do is to clear up the file – just run:
> cimple_log_err_messages

Then I’m able to start hostd, but how do I stop the file from filling up again?

I find this kb:

Not that it helps! It’s for 5.x. I go to the IBM url and it just leaves me more confused.

I look into disabling CIM. I find instructions here:

ummm it’s for 5.x again. ūüôĀ
Run the instructions and the sfcb watchdog just restarts by itself!

Finally, I poke around in the vSphere client and notice in the profile that there is some CIM stuff there. For the heck of it, I try disallowing communication with IBMIMM.

Screen Shot 2016-02-10 at 11.45.09 AM

I believe the log messages continued after disabling it, so I continued to disable CIM as well. After disabling CIM, the log messages stopped.

Screen Shot 2016-02-29 at 12.14.11 PM


Linux self-service firewall with Apache, Perl, IPtables, and UFW

I was recently in a situation where I was offering DNS service to some people. There just happened to be some records that were different from their ISPs DNS servers so I set up the server on my public IP address. I did not want to offer DNS to the world because last time I tried that, I got queries from all kinds of places for all kinds of records. I was initially opening up IP tables when people asked for the service and give me their IP address. After getting about 10 texts, I quickly got tired of collecting the IP addresses, so I made a webpage and with the perl script to write them to a list. With that, I would have a cron jobs go through the list and use UFW to update the IPTables to allow them access.

Here are the files inside of the directory where I’m creating the list.

There’s a simple index.html file in the directory. It’s basically a form that asks for:
Name – who the person is. duh!
IP address – I want them to enter the IP address they want to authorize just in case they’re submitting someone else’s IP address.
Password – I don’t want just anyone to come in and get access to my DNS server.

The addip.cgi basically just writes all of those inputs and the IP address they’re coming in from into /tmp/iplist.txt in CSV format. I record the IP address they’re coming in from $ENV{‘REMOTE_ADDR’} just in case I get abuse or something.

The root user then has a cron job that runs through the iplist.txt file every 10 minutes. Here’s my file:

if [ -f /tmp/iplist.txt ]; then
DATE=$(date +%Y%m%d)
cp /tmp/iplist.txt /home/alton/dnsservice/iplist.txt.$DATE
for i in `grep rice /tmp/iplist.txt | cut -f1 -d','`; do /usr/sbin/ufw insert 1 allow proto udp from $i to any port 53; done
grep rice /tmp/iplist.txt >> /home/alton/dnsservice/authorized_dns_ips.txt
grep -v rice /tmp/iplist.txt >> /home/alton/dnsservice/cheaters.txt
rm -rf /tmp/iplist.txt

Obviously, rice was my password. I just looped through the file and authorized anyone that used the right password. I also logged anyone that used the wrong password in /home/alton/dnsservice/cheaters.txt.

Hope this was useful! I welcome any comments. Obviously, this was quick and dirty. I’m sure there is a more secure way of doing this, but this is what came easy to me. Would love to hear your thoughts!

Getting VMware PowerCLI 6 to work on Windows 10

UPDATE: PowerCLI 6.3R1¬†has been released and fully supports¬†¬†Windows 10, so there’s no need to do the hack anymore!


PowerCLI 6 is not supported on Windows 10 yet, but I just thought I would give it a try anyway. Upon installation of PowerCLI 6 on Windows 10, I was greeted with this unwelcome screen:


Luckily and very quickly, I found this link:

Just an environment variable. Need to add PSModulePath with “C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Modules” as the value and reboot.

Screen Shot 2016-02-18 at 12.31.24 PM

Lenovo T400 blank screen issue on Windows 10

Someone who owns a very well SEO company came to me with a freshly installed Windows 10 OS on a Lenovo T400 and showed me that he would get a black screen at start up. He would suspend the machine (close the lid), and it would come back to normal. He would reboot, and the same problem would reproduce again.

This didn’t happen when he first installed the OS. Only happened after the OS was installed and Windows update was run. I was suspicious that it might be a driver issue, but Lenovo’s website didn’t show much.

Then looked into the device manager and saw this:


What’s interesting here is that the laptop actually has 2 video cards. I think it’s for power savings. I took a gamble and disabled the Intel video card. The problem went away!

Update: Booted into the BIOS and under video settings, found that the Lenovo could install a Switchable graphics driver along with their power management software so that it could save power. So, my hunch was correct. So, no need to disable the Intel card. Just boot into the BIOS and set it to discrete or the other one, whatever it is. I just set mine to discrete.

How to find and online all of your SteelFusion Core LUNs on a NetApp filer

Recently, I offlined a bunch of LUNs that had belonged to a SteelFusion Core in the lab that I had forgotten about. Needless to say, I had some unhappy users. The good news though is that I was able to get the LUNs back up and connected to the Core within minutes. This is how I did it.

The first thing I needed to do was find out which LUNs the Core was using. I did this by logging into the Core via SSH and running the following commands:

conf t
terminal length 0
show storage luns iscsi

I output this to a file /tmp/core30luns.txt. An entry looks like this:

Total LUNs: 9
Locally Assigned Serial: P3PdB/-GFigd
Configuration status : Ready
Alias : avamar_restore
LUN Size : 150.00 GB
LUN Type : iscsi
Online : yes
IOPs acceleration : Enabled
Failover Enabled : yes
Prefetch : Enabled
Edge mapping : pod3-3100b
Target mapping :
Origin portal :,
Origin target :
Backend session status : Connected
Use iSCSI Reservation : Yes
LUN Edge data session : Connected
Client type : other
Original LUN vendor : NetApp
Original LUN serial : P3PdB/-GFigd
Pinned : no
Prepop : Disabled
Smart prepop : Enabled
Prepop status : N/A
MPIO policy : roundrobin
iSCSI Reservation status : LUN reserved

Prepop schedules:
Mapped igroups:

Mapped initiators:

The next thing was to find out what LUNs are on the NetApp to do some matching. You can do that by running this command:

lun show -v

I output this to a file /tmp/netapp_luns.txt. An entry looks like this:

/vol/NewYork_rvbd_d_e7cc5c29_f400_4c52_b1d4_f87da1b62652_1451278801/lun_RDM 10g (10737418240) (r/w, offline)
Serial#: P3PdB/9ytT31
Share: none
Space Reservation: disabled
Multiprotocol Type: vmware

Now with the 2 files, I could do some matching. I first want to extract the serial numbers from the LUNs. I do this by running:

grep serial /tmp/core30luns.txt | cut -f2 -d: > /tmp/core30lunlist.txt 

From that, I would just get a list of serial numbers like this:


Next, I will loop through my list of LUNs to find the volumes I will need to put back online. I do this by running:

for i in `cat /tmp/core30lunlist.txt`; do grep -2 $i /tmp/netapp_luns.txt >> /tmp/netappvolumes.txt; done

This would give me a list like this:

/vol/NewYork_rvbd_d_8f3a7b69_05f7_4be8_b3a6_14a689c2b3b0_1452834001/lunC11 60.0g (64445480960) (r/w, offline)
Comment: “Cdrive”
Serial#: P3PdB/-KWreM
Share: none
Space Reservation: disabled

With that list, I can cut the volumes out with the following command:

grep -v : /tmp/netappvolumes.txt | cut -f1 -d' ' > /tmp/volumes.txt

This would give me a list like this:


Now that I have a list of volume names from the NetApp, I can just put them all online with a loop:

for i in `cat /tmp/volumes.txt`; do echo "lun online" $i >> /tmp/online_vols.txt ; done

You can just take the¬†/tmp/online_vols.txt file now and just paste it into your NetApp SSH session and you’ll have all of your LUNs online again.