Should the image contain hotfixes or not?

One more post in my WSUS/Hotfix series of blogposts. I’ve been asked a couple of times how we approve Hotfixes and if we include them in the images.

I’ve made an Autoapproval Rule where we approve all Hotfixes automatically to the various Computer Groups with a Deadline, like this.

wsus16

And this is how the details looks like;

wsus17

First of all, any server that could cause problems if it automatically rebooted doesn’t have a Deadline, thats servers like Hyper-V Hosts and SOFS Nodes. Those servers are managed by SCVMM’s (System Center Virtual Machine Manager) Patch Management. VMM has a feature to put a cluser node in maintenance mode, automatically drain the node of VM’s, patch it, and then bring the node back online again before it takes the next node.  So we handle all patching of clustered servers from SCVMM. While we let the WSUS Client handle all other servers. We might add SCCM to the mix some day and let it handle all of the servers, but as most of our customers don’t want to run SCCM to manage their Fabric, this is the way we do it now.

By putting a deadline, we know the hotfix will be installed sooner or later. And if there is a Patch Tuesday before that date, it will also install the hotfixes at the same time.

Notice that the hotfix is NOT approved for All Computers and NOT for Unassigned Computers. How come?

When we build a VM image for any OS, it’s done automatically through MDT. Those VM’s are ending up in Unassigned Computers as they don’t have a role yet and we don’t want any Hotfixes in the images. Of course, if there is a mandatory hotfix whish is needed to make the image or deploy it, that one will be included!

The reasons we don’t want any hotfixes in an image is quite simple if you think about it. There are two main reasons really.
The first one is that if we make an image in august, which contains hotfixes. When we deploy that image 3 months later, there is a big chance that the hotfix we had in the image is replaced by a proper update from Microsoft so there was no use for the hotfix in the first place.
Second, when we create an image, we don’t add Clustering, Hyper-V and other roles and features to the image, right? So Windows will then only install the hotfixes for the core OS. And when the image is later deployed and someone adds the Hyper-V Role, it would install hotfixes for that role then. So the server wouldn’t be fully patched anyway so adding 5 or 15 hotfixes automatically after deployment doesn’t really make much of a difference.
Third, a minor reason is also that we normally use the same images for Fabric, Workload and Tenants and we like to keep them quite generic.

Here is a great blogpost about making reference images from my colleague Mikael Nystrom.

 

Semi-Automatic Hotfix import into WSUS

One of my blogreaders, Andreas Fjellner, came up with a way to make the import of hotfixes a bit faster than copy and paste.

You can download a XML file with all the Hotfixes I’ve got imported so you don’t have to do a findstr or excel filtering from the previous blogpost, the XML file contains the same list as shown here List of Private Cloud related Hotfixes – 2016-02-03

Download: XML File (notice the Download button at the top so you don’t have to copy and paste). Save the file as c:\temp\details.xml on your WSUS Server and then run this script;

It will spawn one internet explorer for each Hotfix with the correct URL. Just click ADD to basket. Close the IE Window and pick the next window.
When you are done with the first batch of 20 hotfixes, use the “Import updates” link as described here: Importing Hotfixes and Drivers directly into WSUS and you will now be able to import all hotfixes into your WSUS. And now press Y in the powershell window to take the next batch of hotfixes. Repeat until done.

Another way is to use AutoIT to make a script that moves the mouse and clicks on the right place doing the import semi-automatic, as another blogreader pointed out. There is always a way!

Live (VSM) migration fails with mirror operation failed and access is denied error

When doing a Live Migration from SCVMM (System Center Virtual Machine Manager) with VSM, moving a Virtual Machine from one Cluster to another Cluster and at the same time also to a new Storage Location, you are getting an error message similar to this:

The strange thing is that there is a destination folder in the new location, it’s just does not copy content to that folder and aborts with the Access Denied error. But If you shutdown the VM first, so it’ s just a migration over the Network, it works!

The solution is to give the SOURCE Cluster Write Access on the DESTINATION Storage. When you do a VSM Migration, the destination Hyper-V host, creates the Directory on the SOFS Node, but it’s the Hyper-V Host that owns the VM that copies the VHD’s files to the destination storage. And as the current owner, by default does not have access to write there, it will fail. One could think that VMM should grant permissions to a host when VMM knows that the host needs to write in the location?

Maybe it’s fixed in the next version, but until then, there are two ways to do this.
Solution 1) In VMM add the Destination SOFS Shares as Storage on the Source VM Hosts like this. That will make VMM add the VM Hosts with Modify Permissions in the SOFS Shares so it can write there.

sofs2

This works quite fine, if the Hyper-V Clusters and all Storage is located in roughly the same location. But if you have one compute cluster with storage in one location, and another compute cluster with storage in another location. There is then a risk that you may be running VM’s cross the WAN link.

Solution 2) This is the one we used. By not using VMM to grant permissions to the shares, but rather do it manually we achieve the same solution as above but with the added benefit that a new VM will always be provisioned on the local storage and there is no (or a lot less) risk of running a VM cross the WAN link. Yes, it’s still technically possible to do it, but no one will by accident provision a VM that uses storage in the other datacenter.

You can either add each node manually, so we have created a “Domain Servers Hyper-V Hosts” security Group in AD where we add ALL Hyper-V hosts to during deployment. And then added that group to the Share and NTFS Permissions. All Hyper-V hosts will then automatically have write access to all locations they may need.

I wrote these two short scripts to query the VMM Database for the available SOFS Nodes and use powershell to grant permissions to the share, and to NTFS.

As all our SOFS Shares were called vDiskXX or CSVXX (where XX is a number) I just used a vDisk* and CSV* to do the change on all those shares. You might have to modify it a little to suit your name standard.

Updated Script (2016-02-04):
I got a report that the script was getting an error on some servers, which I managed to reproduce. Here is an alternative version where it will connect to the server and execute the ACL change locally via invoke-command. It’s also only changing permissions on Continuously Available (SOFS) shares.

 

 

List of Private Cloud related Hotfixes – 2016-02-03

I’ve posted my list of resources for finding Hotfixes previously here.  And this is a list of hotfixes we’ve imported in our WSUS server for our and our customers Private Clouds.
I usually want to install a hotfix to avoid getting a known problem, than try to find a solution to a problem after it has already happened and affected the users and customers.

wsus3

I’ve used the script I posted here to make the list. I’m sorry for the format below, but there is no good way to extract the info from WSUS and as I don’t really know if anyone is interested in this besides myself, I won’t spend hours on fixing a nice output right now or I would never get this blogpost published.  Sorry!

I’m using the MUUri to paste into the WSUS IE to search and locate the hotfixes fast, instead of manually search for each. There is unfortunately no way to script the import according to Microsoft PM’s, so it has to be done manually. Sigh…

Continue reading “List of Private Cloud related Hotfixes – 2016-02-03”

Export information from WSUS about Hotfixes or Updates

I want to export information about all our Hotfixes in our WSUS Server, to share with the community as it’s sometimes hard to find up to date info of which hotfixes to apply in an environment.

Here is a quick and dirty script. No, scratch that, it’s not quick in any way but very dirty. The problem is that the WSUS Database does not contain the Description or Title of a Hotfix, so it’s not possible to export that info. Thus, I’ve to use a scripted Internet Explorer, to navigate to the URL of each Hotfix and grab the Title. Which makes the process veeeeeeery slow but I’ve been unable to come up with any better solution than that.

This is how the output looks like:

wsus2

And here is the full script, I will be really grateful if you share any changes you do as I’m sure there is a lot of ways to improve this script.

 

Working with Virtual NIC’s in Windows

At times when I’m for example at a customer and need to connect my Laptop to different VLAN’s it’s really nice to add new virtual Network Cards (vNIC’s) on the fly, and be connected to multiple networks at the same time. vnics

By transforming the Network Cards in your computer, into a virtual switch, and then add Virtual Network Cards connected to that switch, it’s possible to do a bit of network magic.

Here is a part of the script that I run each time I reinstall my PC’s to create the vNIC’s that I need and use the most. The script is also installing the software I need and doing some other minor changes (always a work in progress).

Pre-Requisits: Hyper-V Role installed

Thanks to my friend and colleague Mikael Nyström who showed me this a few years ago.

 

List all expiring certificates on all domain joined servers

A colleague asked me if I could list all expiring certificates on all Domain Joined servers in the environment.
– Sure!

A few minutes later, a script that will connect to all Servers and list certificates that will expire in less than 90 days. I’m sure there are a thousand of scripts out there who does the same, and here is script number 1001.

The script will get all Windows Server Computer Accounts, that are not expired and is not a virtual object (like a Cluster Service).
It will then connect to those servers and list all Certificates that will expire in less than 90 days.

Short and easy to use, and we did find 2 certificates that need to be replaced ASAP!

Script to change from Dynamic to Static MAC Address on all VMs

A customer had a lot of VM’s with Dynamic MAC address, rather than the preferred method of using Static MAC addresses.
Here is a small powershell script that will shutdown each of the VM’s with a Dynamic MAC Address, change to a Static MAC Address and then start the VM.
I’m running the script on the System Center Virtual Machine Manager (SCVMM) Server and to make sure VMM does not shutdown itself, I’ve added an exclude for the SCVMM Server.

The MACAddress 00:00:00:00:00:00 will automatically be transformed into a real static address from VMM’s mac address pool.

URL Rewrite (redirect) of HTTP to HTTPS with Powershell script

When deploying Web Application Proxy as a frontend to for example ADFS and Windows Azure Pack, or other services, the current version of Web AppProxy only supports HTTPS urls. It’s possible to use the “URL Rewrite” module for IIS to redirect users from HTTP to HTTPS. There are plenty of guides on internet on how to do that.
But I wanted to add that configuration to my WebApplication Proxy configuration script, and couldn’t find any powershell examples, so here is the script I’ve made.

It will use Web Platform installer to install the URL Rewrite module, then add the IIS Web Management tools, and in the end create a Global Rule redirecting all HTTP requests to HTTPS without the user noticing it.

 

 

Automatically Assign Availability Set Names to VMs with Powershell

This blog-post is about using System Center Virtual Machine Manager (SCVMM) Availability Sets to spread similar VM’s to different Hyper-V Hosts to increase reliability both when using Failover Clustering, and when using stand-alone Hyper-V hosts.

First of all, what is Availability Sets?
In SCVMM 2012 SP1, Microsoft added Availability Sets. Failover Cluster Manager users are probably familiar with AntiAffinityClassNames, and Availability Sets are a very similar concept. This allow the user to specify a set of VMs which they would prefer to keep on separate hosts, and the Intelligent Placement engine works hard to make sure that all our features respect that preference.

Attempting to place multiple VMs with the same Availability Set onto a single host will generate a placement warning, meaning that the host will be prioritized last in the placement dialog

  • When placing a VM with an Availability Set into a cloud placement or as part of a service will avoid hosts with another VM from the same Availability Set, and warn the user if that was the only choice.
  • Dynamic Optimization will never move 2 VMs from the same Availability Set onto the same host. It will also actively attempt to separate any VMs with the same Availability Set that are on the same host.
  • Power Optimization will never power off a host that would lead to 2 VMs with the same Availability Set sharing a host.
  • Putting a host in maintenance mode will attempt to spread VMs with the same availability set to different target hosts.
  • If your VMs are highly available and hosted on a Hyper-V failover cluster, VMM will create AntiAffinityClassNames on the VMs with an Availability Set, so that even during cluster failover, SCVMM opt to failover to different hosts, if possible.

You can manually create ASCVMM2vailability Sets through SCVMM by selecting Properties on a VM.
Just click Create to make a new Name and assign it to the VM’s you want to keep on separate Hosts. When a Availability Set is not assigned to a VM any longer, the Availability Set will be deleted automatically, thus cleaning up the list for you.

For example, for your SQL Server Cluster, you may want to create a Availability Set name called SQL and assign it to your SQL Server Nodes. Easy!
Also, if you are using Service Templates, you can opt in to automatically create Availability Set names for your services.

Though I like to control things like SCVMM1that automatically. Depending on your naming convention for your Virtual Servers, this might or might not be possible for you.
In our case we have a strict naming policy to name servers with:
PREFIX FUNCTION NUMBER as seen in this picture:

Which makes it very easy for me to define that all servers called CLAZSQ* are similar and should be kept on different servers.

But, if all servers were called SRV0001-SRV9999 it would not be possible to utilize the ServerName for setting Availability Set names, and you would have to query the CMDB for info first.

Also, in our environment we have multiple Tenants, who could each have servers called DomainController01 and DomainController02. So just having a availability set called DomainController, would not be enough. I have to make it DomainController_TenantName or something similar.

I wrote this quick and short Powershell script to automatically assign a Availability Set to all VM’s. It will remove Numbers from the VM Name, and use the VMName + UserID (Tenant Subscription id) as the Availability Set Name. Clean, simple and easy, just schedule it to run regularly, or even make a SMA Job to trigger when a VM is created through AzurePack.

And then trigger an Host Cluster Optimization of all Clusters in the Environment if you don’t want to wait for the normal one.