Creating a small footprint, base image Part 3 | SysPrep and Compacting Images

New-WindowsImage -size small | Test-Lab

One of the time consuming steps to deploying new VMs is the time spend managing Images and and applying patches. I’m not big on Golden images. I tend to use a fully patched VHDX or VMDK  and let DSC handle the configuration and software. This is not the fastest, and at scale you need to create more then one image based on what saves the most time.  (IIS, SQL, Exchange, etc…). In this series, I’m going to go over how I create a a baseline image of 2012r2 with PowerShell. Also because Install-WindowsFeature has issues when the target VM has been patched, we are also going to create a fully patched WIM to use as -source.

Blogs in this Series

SysPrep and Compacting Images

Sysprep the Core Image

Now I need to sysprep the core image before it’s usable. but I don’t want to repeat this whole process from scratch next month. So I’m going to copy the VHDX and create a new VM that I will sysprep.  So back to the host

copy .\CorePatched.vhdx CoreSysprep.vhdx
new-vm -Name CoreSysprep -MemoryStartupBytes 512MB -BootDevice VHD -VHDPath G:\temp\CoreSysprep.vhdx -Generation 2 -SwitchName TestLab
Name        State CPUUsage(%) MemoryAssigned(M) Uptime   Status
 ----        ----- ----------- ----------------- ------   ------
 CoreSysprep Off   0           0                 00:00:00 Operating normally
start-vm coreSysprep
vmconnect localhost coresysprep

Now on the new VM run

c:\windows\System32\Sysprep\sysprep.exe /quiet /oobe /generalize /shutdown

Once the VM is shutdown it’s time to look at our progress on the host.

Compacting the Images

dir core*
    Directory: G:\temp
Mode                LastWriteTime         Length Name
 ----                -------------         ------ ----
 -a----        5/14/2015   9:20 PM     5641338880 CoreFromIso.vhdx
 -a----        5/15/2015   4:10 PM     9265217536 CorePatched.vhdx
 -a----        5/15/2015   4:27 PM     9265217536 CoreSysprep.vhdx

Wow, almost 10 gigs. not exactly small. Well we are not done. We need to convert both our images to WIM and the core one back to VHDX. The transfer from VHDX to WMI and back to VHDX does a better job then using a utility to write all 0’s to the disk and then compact it. This is partly due to the swap file. Back on the host we need to create a folder to mount the VHDX into and then create the WIM.

mkdir g:\mount
    Directory: G:\
Mode                LastWriteTime         Length Name
 ----                -------------         ------ ----
 d-----        5/15/2015   8:54 PM                mount
Mount-WindowsImage -ImagePath G:\temp\CoreSysprep.vhdx -Path G:\Mount -Index 1
Path           : G:\Mount
 Online         : False
 Restart Needed : False
New-WindowsImage -CapturePath G:\Mount -Name 2012r2_Core -ImagePath G:\Temp\2012r2Core.wim -Description "2012r2 Core Patched May 2015" -Verify
LogPath : C:\Windows\Logs\DISM\dism.log
Dismount-WindowsImage -Path G:\mount -Discard
LogPath : C:\Windows\Logs\DISM\dism.log
Mount-WindowsImage -ImagePath G:\temp\GUIPatched.vhdx -Path G:\Mount -Index 1
Path           : G:\Mount
 Online         : False
 Restart Needed : False
New-WindowsImage -CapturePath G:\Mount -Name 2012r2_Source -ImagePath G:\Temp\2012r2Source.wim -Description "2012r2 Source Patched May 2015" -Verify
LogPath : C:\Windows\Logs\DISM\dism.log
Dismount-WindowsImage -Path G:\mount -Discard
LogPath : C:\Windows\Logs\DISM\dism.log

Converting the Core WIM back to VHDX is going to require that we make a change to Convert-WindowsImage.ps1. The logic of the script looks at the WIM and if there is only one image it uses that one. But it does so using metadata that does not exist when a WIM is created using New-WindowsImage. So down on line 4020 we nee to comment out a few lines

        # If there's only one image in the WIM, just selected that.
        #if ($openWim.Images.Count -eq 1) { 
        #    $Edition   = $openWim.Images[0].ImageFlags
        #    $openImage = $openWim[$Edition]
        #} else {

            if ([String]::IsNullOrEmpty($Edition)) {
                Write-W2VError "You must specify an Edition or SKU index, since the WIM has more than one image."
                Write-W2VError "Valid edition names are:"
                $openWim.Images | %{ Write-W2VError "  $($_.ImageFlags)" }
                throw
            } 

            if ([Int32]::TryParse($Edition, [ref]$null)) {
                $openImage = $openWim[[Int32]$Edition]    
            } else {
                $openImage = $openWim[$Edition]
            }        
        #}

Now it will work using -edition 1 (There is only one OS inside the WIM)

G:\temp\Convert-WindowsImage.ps1 -SourcePath G:\temp\2012r2Core.wim -VHDPath G:\temp\2012R2Core.vhdx -SizeBytes 40gb -VHDType Dynamic -VHDFormat VHDX -VHDPartitionStyle GPT -Edition 1 -Verbose
Windows(R) Image to Virtual Hard Disk Converter for Windows(R) 8
 Copyright (C) Microsoft Corporation.  All rights reserved.
 Version 6.3.9600.7.amd64fre.fbl_core1_hyp_dev(mikekol).140217-3000 Release to Web
 VERBOSE: isUserAdmin? True
 VERBOSE: isWindows8? True
 VERBOSE: Temporary VHDX path is : G:\temp\faf6d4fa-fb64-41d7-95f3-5a1ab3825940.vhdx
 INFO   : Image 1 selected ()...
 INFO   : Creating sparse disk...
 INFO   : Attaching VHDX...
 INFO   : Disk initialized with GPT...
 INFO   : Disk partitioned with two Volumes...
 INFO   : System Volume formatted (with DiskPart)...
 INFO   : Boot Volume formatted (with Format-Volume)...
 INFO   : Access path (L:\) has been assigned to the System Volume...
 INFO   : Access path (M:\) has been assigned to the Boot Volume...
 INFO   : Applying image to VHDX. This could take a while...
 INFO   : Signing disk...
 INFO   : Image applied. Making image bootable...
 VERBOSE: Running bcdboot.exe M:\Windows /s L:\ /v /f UEFI
 VERBOSE: Return code was 0.
 INFO   : Drive is bootable. Cleaning up...
 INFO   : Closing VHDX...
 INFO   : Closing Windows image...
 INFO   : Done.
dir *core* | sort -Property LastWriteTime
    Directory: G:\temp
 Mode                LastWriteTime         Length Name
 ----                -------------         ------ ----
 -a----        5/14/2015   9:20 PM     5641338880 CoreFromIso.vhdx
 -a----        5/15/2015   4:10 PM     9265217536 CorePatched.vhdx
 -a----        5/15/2015   8:00 PM     9265217536 CoreSysprep.vhdx
 -a----        5/15/2015   9:39 PM     2376356767 2012r2Core.wim
 -a----        5/15/2015  10:15 PM     4936695808 2012R2Core.vhdx
dir *gui*, *source* | sort -Property LastWriteTime
    Directory: G:\temp
 Mode                LastWriteTime         Length Name
 ----                -------------         ------ ----
 -a----        5/14/2015   9:29 PM     9130999808 GUIFromIso.vhdx
 -a----        5/15/2015   3:56 PM    16143876096 GuiPatched.vhdx
 -a----        5/15/2015  10:03 PM     6153716725 2012r2Source.wim

As you can see our Source.WIM is 6.1Gig and our Core VHDX is only 4.9gig. fully patched it’s smaller then the baseline Core image. We can place the WIM on a share that is accessible to new VM’s and they can use that for adding features. In another blog I will go over how to update the WIM file automatically each month.

Advertisements

Creating a small footprint, base image Part 2 | Patching and Cleanup via PowerShell

New-WindowsImage -size small | Test-Lab

One of the time consuming steps to deploying new VMs is the time spend managing Images and and applying patches. I’m not big on Golden images. I tend to use a fully patched VHDX or VMDK  and let DSC handle the configuration and software. This is not the fastest, and at scale you need to create more then one image based on what saves the most time.  (IIS, SQL, Exchange, etc…). In this series, I’m going to go over how I create a a baseline image of 2012r2 with PowerShell. Also because Install-WindowsFeature has issues when the target VM has been patched, we are also going to create a fully patched WIM to use as -source.

Blogs in this Series

Patching and Cleanup via PowerShell

Patching The Core Image

Starting with our Core image we are going to Patch and clean up extra files. At the command prompt I’m going to type: start Powershell Now I could use sconfig to patch, but we are doing this via PowerShell. Using James Oniel’s blog as a reference. I have cut the code down to just the needed commands because the paste keystrokes has a limited size. And sometimes opening a fresh VM does not give me the options for an Advanced Session. I’m going to take the code below and “Paste from clipboard” into PowerShell to download the all patches, including optional patches

$Criteria = "IsInstalled=0 and Type='Software'"
$updateSession = New-Object -ComObject 'Microsoft.Update.Session'
$updates = $updateSession.CreateupdateSearcher().Search($Criteria).Updates
$downloader = $updateSession.CreateUpdateDownloader()   
$downloader.Updates = $updates  
"Downloading $($downloader.Updates.count) updates"
$Result = $downloader.Download() 
if (($Result.Hresult -eq 0) -and (($Result.resultCode -eq 2) -or ($Result.resultCode -eq 3)) ) 
{
    $updatesToInstall = New-Object -ComObject 'Microsoft.Update.UpdateColl'

    $updates |
    Where-Object -FilterScript {
        $_.isdownloaded
    } |
    ForEach-Object -Process {
        $null = $updatesToInstall.Add($_)
    }
    $installer = $updateSession.CreateUpdateInstaller()
    $installer.Updates = $updatesToInstall


    "Installing $($installer.Updates.count) updates"
    $installationResult = $installer.Install()
    $Global:counter = -1
    $installer.updates | Format-Table -AutoSize -Property Title, EulaAccepted, @{
        label      = 'Result'
        expression = {
            $resultcode[$installationResult.GetUpdateResult($Global:counter++).resultCode ]
        }
    } 
}

$installationResult

This takes a while. Once it’s done I’m going to reboot the VM and paste the script again until there are 0 updates. It can take 2-3 times depending on the age of your ISO.

Restart-Computer

Once we have updated Core with every patch, it’s time to remove unneeded file. If you going install any features, special software or setup boot time scripts. now is the time to do so. I’m not going to this time. I will start by removing the source file for adding windows features. We will be creating a WIM from our GUI image for that purpose.

Get-WindowsFeature | where-object{$_.Installed -eq 0 -and $_.InstallState -eq 'Available'} | uninstall-windowsfeature -remove
dism /online /cleanup-image /StartComponentCleanup /ResetBase
defrag c: /UVX
  • First we removes the source from SxS
  • Second we remove any “superseded” items in SxS. So any older patches that have been updated with a newer one will be removed.
  • Third, we defrag the drive and consolidate free space (this probably is not necessary)

Preparing and Patching the GUI Image

With that done lets move to the GUI VM. I’m going to first make sure every feature is available

get-windowsfeature | Where InstallState -eq Removed
Display Name                                            Name                       Install State
 ------------                                            ----                       -------------
     [ ] .NET Framework 3.5 (includes .NET 2.0 and 3.0)  NET-Framework-Core               Removed
     [ ] Windows PowerShell 2.0 Engine                   PowerShell-V2                    Removed

So .NET 3.5 and PowerShell 2.0 are removed. If your never going to use them that is ok. I find that .NET 3.5 is frequently needed so to be safe I’m going to add them both to SxS.

get-windowsfeature | Where InstallState -eq Removed | Install-WindowsFeature -Source D:\sources\sxs
Success Restart Needed Exit Code      Feature Result
 ------- -------------- ---------      --------------
 True    No             Success        {.NET Framework 3.5 (includes .NET 2.0 and...
 WARNING: Windows automatic updating is not enabled. To ensure that your newly-installed role or feature is
 automatically updated, turn on Windows Update.

Because we want the WIM to be a source when adding to VM’s created from our Core Image, and we want the feature to be patched from the get-go. They need to be installed and patched on this VM. If they are not all installed, that is OK, it will still work as a source so long as the installed features in the WIM are the same version or newer then the VM your adding the feature to.So if we don’t mind patching the new items after the fact when we can just remove the GUI, patch and move on. I’m going to keep the GUI, and add some features I know I will be using in my lab. (now before you go and think I’ll just run Get-WindowsFeature | Add-WindowsFeature, that does not work because some features are mutually exclusive)

Install-WindowsFeature DNS, DHCP, File-Services, RSAT -IncludeAllSubFeature
Install-WindowsFeature AD-Domain-Services, AD-Certificate, ADCS-Cert-Authority

There will be a few warnings but everything should install Now I’m going to use the same code used on the Core VM to run updates. Once that is patched and shutdown it’s ready for Part 3.

Creating a small footprint, base image Part 1 | VHDX from ISO

New-WindowsImage -size small | Test-Lab

One of the time consuming steps to deploying new VMs is the time spend managing Images and and applying patches. I’m not big on Golden images. I tend to use a fully patched VHDX or VMDK  and let DSC handle the configuration and software. This is not the fastest, and at scale you need to create more then one image based on what saves the most time.  (IIS, SQL, Exchange, etc…).

In this series, I’m going to go over how I create a a baseline image of 2012r2 with PowerShell. Also because Install-WindowsFeature has issues when the target VM has been patched, we are also going to create a fully patched WIM to use as -source.

Blogs in this Series

VHDX from ISO

First you need a 20112r2 ISO, at work you would get this from the Volume License Service center. Every time Microsoft releases a roll-up, they usually update the ISO file so it’s a good idea check every few months for an updated ISO. Now for a home lab, unless you have a personal MSDN, your going to need an ISO from another source.  I’m just going with the latest trial from the TechNet Evaluation Center. Don’t download the VHD. We are going to need both Core and GUI from the ISO. While we could get there from the VHD, It’s not the path I’m taking.

I’m saving the ISO in c:\ISO\ and renaming it Server2012R2.ISO

Now what I’m going to do is create two VHDX one core and one with a GUI. I could use the ISO to manually install but I want to limit my use of the mouse, (and screen shots are a pain to insert in the blog)

Once I have that downloaded I can mount it on My windows 8.1 hyper-v host that contains the Test Lab. The reason for this is I will need the VHDX on the host. The WMI file can be on any share accessible by the VM’s.

PS C:\> Mount-DiskImage -ImagePath c:\ISO\Server2012R2.ISO

This will mount it as it’s own drive. in my case I:

Next I’m going to need a way to extract the contents of Install.wim and create a properly formatted VHDX. Now many of the examples of this online are for Gen1 machines. I want Gen2. so I’m going to get a copy of Convert-WindowsImage.ps1
Unfortunately this is not a module but and advanced script.  I’m not going to go in-depth on how Convert-WindowsImage works. Check it’s help and open it up if your curious.

Unblock-File G:\temp\Convert-WindowsImage.ps1
G:\temp\Convert-WindowsImage.ps1 -SourcePath I:\sources\install.wim -VHDPath G:\temp\CoreFromIso.vhdx -SizeBytes 40gb -VHDType Dynamic -VHDFormat VHDX -VHDPartitionStyle GPT -Edition 3 -Verbose
Windows(R) Image to Virtual Hard Disk Converter for Windows(R) 8
 Copyright (C) Microsoft Corporation.  All rights reserved.
 Version 6.3.9600.7.amd64fre.fbl_core1_hyp_dev(mikekol).140217-3000 Release to Web
 VERBOSE: isUserAdmin? True
 VERBOSE: isWindows8? True
 VERBOSE: Temporary VHDX path is : c:\temp\a585fb17-2bfe-4db6-8440-034c561e53ce.vhdx
 INFO   : Image 3 selected (ServerDataCenterEvalCore)...
 INFO   : Creating sparse disk...
 INFO   : Attaching VHDX...
 INFO   : Disk initialized with GPT...
 INFO   : Disk partitioned with two Volumes...
 INFO   : System Volume formatted (with DiskPart)...
 INFO   : Boot Volume formatted (with Format-Volume)...
 INFO   : Access path (J:\) has been assigned to the System Volume...
 INFO   : Access path (K:\) has been assigned to the Boot Volume...
 INFO   : Applying image to VHDX. This could take a while...
 INFO   : Signing disk...
 INFO   : Image applied. Making image bootable...
 VERBOSE: Running bcdboot.exe K:\Windows /s J:\ /v /f UEFI
 VERBOSE: Return code was 0.
 INFO   : Drive is bootable. Cleaning up...
 INFO   : Closing VHDX...
 INFO   : Closing Windows image...
 INFO   : Done.
G:\temp\Convert-WindowsImage.ps1 -SourcePath G:\sources\install.wim -VHDPath G:\temp\GUIFromIso.vhdx -SizeBytes 40gb -VHDType Dynamic -VHDFormat VHDX -VHDPartitionStyle GPT -Edition 4 -Verbose
Windows(R) Image to Virtual Hard Disk Converter for Windows(R) 8
 Copyright (C) Microsoft Corporation.  All rights reserved.
 Version 6.3.9600.7.amd64fre.fbl_core1_hyp_dev(mikekol).140217-3000 Release to Web
 VERBOSE: isUserAdmin? True
 VERBOSE: isWindows8? True
 VERBOSE: Temporary VHDX path is : G:\temp\dd0ac2d0-b05c-461f-b071-d73b59522bb4.vhdx
 INFO   : Image 4 selected (ServerDataCenterEval)...
 INFO   : Creating sparse disk...
 INFO   : Attaching VHDX...
 INFO   : Disk initialized with GPT...
 INFO   : Disk partitioned with two Volumes...
 INFO   : System Volume formatted (with DiskPart)...
 INFO   : Boot Volume formatted (with Format-Volume)...
 INFO   : Access path (K:\) has been assigned to the System Volume...
 INFO   : Access path (L:\) has been assigned to the Boot Volume...
 INFO   : Applying image to VHDX. This could take a while...
 INFO   : Signing disk...
 INFO   : Image applied. Making image bootable...
 VERBOSE: Running bcdboot.exe L:\Windows /s K:\ /v /f UEFI
 VERBOSE: Return code was 0.
 INFO   : Drive is bootable. Cleaning up...
 INFO   : Closing VHDX...
 INFO   : Closing Windows image...
 INFO   : Done.
disMount-DiskImage -ImagePath c:\ISO\Server2012R2.ISO
dir
Directory: C:\temp
Mode                LastWriteTime     Length Name
 ----                -------------     ------ ----
 -a---         5/14/2015   3:16 PM     187332 Convert-WindowsImage.ps1
 -a---         5/14/2015   3:29 PM 5104467968 CoreFromIso.vhdx
 -a---         5/14/2015   3:42 PM 8157921280 GUIFromIso.vhdx

 We now have two VHDX files however they are still a little out of date, and we can still trim them up a bit.

I’m going to copy the VHDX files, just so we can see how the size changes on each step.

copy .\CoreFromIso.vhdx CorePatched.vhdx

copy .\GUIFromIso.vhdx GuiPatched.vhdx

Now we will create VM’s and attach the two drives.  I will also add the ISO to the GUI version as we need it for making sure all windows features are ‘Available”

new-vm -Name CorePatch -MemoryStartupBytes 1024MB -BootDevice VHD -VHDPath G:\temp\CorePatched.vhdx -Generation 2 -SwitchName TestLab
Name      State CPUUsage(%) MemoryAssigned(M) Uptime   Status
 ----      ----- ----------- ----------------- ------   ------
 CorePatch Off   0           0                 00:00:00 Operating normally
new-vm -Name GuiPatch -MemoryStartupBytes 1024MB -BootDevice VHD -VHDPath G:\temp\GuiPatched.vhdx -Generation 2 -SwitchName TestLab
Name     State CPUUsage(%) MemoryAssigned(M) Uptime   Status
 ----     ----- ----------- ----------------- ------   ------
 GuiPatch Off   0           0                 00:00:00 Operating normally
Add-VMDvdDrive -Path C:\ISO\Server2012R2.ISO -VMName guipatch

Start-VM *patch

vmconnect localhost corepatch
vmconnect localhost guipatch

Those last two command connect me to the servers.

Now I’m going to go on the assumption that you have experience setting up server and you can handle the Out of Box experience on your own.

In Part 2 we will cover Windows Updates with PowerShell, along with cleaning up unneeded files.

Test-Lab | Update-GitHub

Posts in this series

  1. Test-HomeLab -InputObject ‘The Plan’
  2. Get-Posh-Git | Test-Lab
  3. Get-DSCFramework | Test-Lab
  4. Invoke-DscBuild | Test-Lab
  5. Test-Lab | Update-GitHub

Last time we were able to get the sample DSC config to build. The problem is we had to modify the sample script and the instructions were not clear on the setup of the files.

So to day I plan to update the SampleBuild.ps1, SampleConfiguration.psm1 and readme.md with my changes and submit a pull request.

The first thing I’m going to modify is SampleConfiguration.psm1, now this file worked as is with the previous blog. but some of the things we did were to get arround an issue with this file.

The line i’m talking about is here

 Import-Module DscConfiguration -ErrorAction Stop

This imports a module that is already loaded by SampleBuild.ps1 but is also not in the path at this point of Invoke-DscBuild, unless the DscConfiguration module is added to a path in -ModulePath used by Invoke-DscBuild.  Now if your going to run import-module SampleConfiguration crate the ConfigurationData hash table your self and call SampleConfiguration then it makes sense to have that here. I don’t imagine anyone doing that but stranger things have happen, so I’m going to keep that line but wrap it in something to avoid the requirement of placing extra modules in DSC_Tools

if (-not (Get-Module DscConfiguration)) {
 Import-Module DscConfiguration -ErrorAction Stop
}

This checks to see if DscConfiguration is already loaded into memory and skips the import is it is.

Now I don’t need any modules in DSC_Tooling. Unless I have modules used by my scripts that I dont import prior to calling Invoke-DscBuild

Next up is SampleBuild.ps1, I dont have any changes to make from what I did last time. so i will add that to my git repo in just a bit.

Now Readme.md in the examples folder needs some serious work.

Example DSC Build
------

This folder contains some very basic examples of what a DSC configurationData folder structure, script, and call to Invoke-DscBuild might look like.  If you want to execute SampleBuild.ps1, there are a few dependencies you need to set up ahead of time:

- You must install all of the DSC tooling modules (content of \Tools minus the example folder) from this repository into your PSModulePath (typically into C:\Program Files\WindowsPowerShell\Modules\)
- You must also copy the Tooling\Examples\SampleConfiguration folder to the PSModulePath.
- You must copy [Pester](https://github.com/pester/Pester) (version 3.0.0 or later) and [ProtectedData](https://github.com/dlwyatt/ProtectedData) (version 2.1 or later) into the PSModulePath.
- You should create a DSC_Resources folder in the same directory as SampleBuild.ps1 and DSC_Configuration.  Copy the following modules into that DSC_Resources folder:
  - [StackExchangeResources](https://github.com/PowerShellOrg/StackExchangeResources)
  - [cWebAdministration](https://github.com/PowerShellOrg/cWebAdministration)
  - [cSmbShare](https://github.com/PowerShellOrg/cSmbShare)

Create a folder to place all the files into. i.e. c:\DSC, inside that folder create folders named BuildOutput, DSC_Configuration, DSC_Resorces, DSC_Script, DSC_Tooling. 

the folder structure should look like this
C:\DSC                # copy SampleBuild.ps1 here
+---BuldOutput        # Where the MOF files and ziped modules end up
+---DSC_Configuration # Copy \Tooling\Examples\DSC_Configuration\*  here
+---DSC_Resources     # copy StackExchangeResources, cSmbShare and cWebAdministration here
+---DSC_Script        # copy \Tooling\Examples\SampleConfiguration here
+---DSC_Tooling       # This is for any modules that may be used in a Configuration script, in the case of SampleConfiguration it would be empty.

If you plan on modifying SampleConfiguration.psm1 inside of DSC_Script you will also want to add the content of DSC_Modules to C:\Program Files\WindowsPowerShell\Modules\ but that is not necessary if your just building configurations that are authored on another machine. 

Once these dependencies are set up, you can execute SampleBuild.ps1.  It will run tests against the 3 modules in your DSC_Resources folder, compile your configuration into MOF documents, produce zip files for the resource modules, generate checksums for everything and copy them into BuildOutput

_Note:  The SampleBuild.ps1 file currently just dumps DSC_Tooling modules into the temporary folder, since I wasn't using that feature.  We'll build on these examples soon to show off some of the other functionality in the DscBuild and DscConfiguration modules, such as encrypting credentials in source control._

If you compare with my last post you will notice I removed the line about placing SampleConfguration into C:\Program Files\WindowsPowerShell\Modules\. this path is no longer in the psmodulepath when inovke-DscBuild runs, as it used to be. This is due to needing to keep psmodulepath clean so that DSC_Resorces are accessible for configuration building but not necessarily needed.

I also added a section showing the required folder structure and what each folder is used for, along with what needs to be done if your going to author on the same machine you build on.

Now that that is done. I’m going to copy the files into C:\GitHub\PshOrgDSC\Tooling\Examples one at a time and add and commit changes.

copy SampleConfiguration.psm1 first. Notice the prompt change thanks to posh-git, (you will not see the color change)

C:\GitHub\PshOrgDSC\Tooling\Examples\SampleConfiguration [development +0 ~1 -0]> git add *

C:\GitHub\PshOrgDSC\Tooling\Examples\SampleConfiguration [development +0 ~1 -0]> git commit -m 'Fixed issue with loading scamplescript moduel not loading if module is already loaded but no longer in path'
[development a5301a8] Fixed issue with loading scamplescript moduel not loading if module is already loaded but no longer in path
 1 file changed, 3 insertions(+), 1 deletion(-)

C:\GitHub\PshOrgDSC\Tooling\Examples\SampleConfiguration [development]> 

next up SampleBuild.ps1

C:\GitHub\PshOrgDSC\Tooling\Examples [development +0 ~1 -0]> git add *

C:\GitHub\PshOrgDSC\Tooling\Examples [development +0 ~1 -0]> git commit -m 'updated SampleBuild.ps1 to include missing value to invoke-DSCBuild. and added -verbose'
[development 5c35462] updated SampleBuild.ps1 to include missing value to invoke-DSCBuild. and added -verbose
 1 file changed, 4 insertions(+), 3 deletions(-)

C:\GitHub\PshOrgDSC\Tooling\Examples [development]>

last up readme.md

C:\GitHub\PshOrgDSC\Tooling\Examples [development +0 ~1 -0]> git add *

C:\GitHub\PshOrgDSC\Tooling\Examples [development +0 ~1 -0]> git commit -m 'updated \Tooling\Example\Readme.md to matche the current state of development, and included seciton showing required folder structure'
[development 3aa570d] updated \Tooling\Example\Readme.md to matche the current state of development, and included seciton showing required folder structure
 1 file changed, 14 insertions(+), 2 deletions(-)

C:\GitHub\PshOrgDSC\Tooling\Examples [development]>

finaly I push the commits to my fork

C:\GitHub\PshOrgDSC\Tooling\Examples [development]> git push
git : To https://github.com/BladeFireLight/DSC.git

Now I can subit a pull request.

I head over to github https://github.com/BladeFireLight/DSC/tree/development

compaireandpull

Clicking Compare and pull request gives me a diff of each file and it’s changes.

exampleDiff

It also tells me there are no issues merging my commits into the upstream Development branch.

ExamplePull

I’m submitting the pull request and will update this post with the results.

Invoke-DscBuild | Test-Lab

Posts in this series

  1. Test-HomeLab -InputObject ‘The Plan’
  2. Get-Posh-Git | Test-Lab
  3. Get-DSCFramework | Test-Lab
  4. Invoke-DscBuild | Test-Lab
  5. Test-Lab | Update-GitHub

In the last post we forked a copy of the Powershell.org DSC tools and cloned a copy locally

Today I’m going to get the example configuration working.

Now looking at the README.md under examples seems straightforward, but I know for a fact that it’s missing a few steps.

This folder contains some very basic examples of what a DSC configurationData folder structure, script, and call to Invoke-DscBuild might look like.  If you want to execute SampleBuild.ps1, there are a few dependencies you need to set up ahead of time:

– You must install all of the DSC tooling modules from this repository into your PSModulePath (typically into C:\Program Files\WindowsPowerShell\Modules\)
– You must also copy the Tooling\Examples\SampleConfiguration folder to the PSModulePath.
– You must copy [Pester](https://github.com/pester/Pester) (version 3.0.0 or later) and [ProtectedData](https://github.com/dlwyatt/ProtectedData) (version 2.1 or later) into the PSModulePath.
– You should create a DSC_Resources folder in the same directory as SampleBuild.ps1 and DSC_Configuration.  Copy the following modules into that DSC_Resources folder:
– [StackExchangeResources](https://github.com/PowerShellOrg/StackExchangeResources)
– [cWebAdministration](https://github.com/PowerShellOrg/cWebAdministration)
– [cSmbShare](https://github.com/PowerShellOrg/cSmbShare)

Once these dependencies are set up, you can execute SampleBuild.ps1.  It will run tests against the 3 modules in your DSC_Resources folder, compile your configuration into MOF documents, produce zip files for the resource modules, generate checksums for everything and copy them into C:\Program Files\WindowsPowerShell\DscService\

Looks like we are going to need a few more DSC resources. I’m going to go and fork/clone them just like before. (don’t forget to update the URL’s to match your own fork. )

# Download required files via Git
#Next line not needed if you were following along with my last blog
git clone https://github.com/BladeFireLight/DSC.git c:\GitHub\PshOrgDSC --branch development
git clone https://github.com/BladeFireLight/StackExchangeResources.git c:\GitHub\StackExchangeResources
git clone https://github.com/BladeFireLight/cWebAdministration.git c:\GitHub\cWebAdministration
git clone https://github.com/BladeFireLight/cSmbShare.git c:\GitHub\cSmbShare 
git clone https://github.com/BladeFireLight/Pester c:\GitHub\Pester
git clone https://github.com/BladeFireLight/ProtectedData.git c:\GitHub\ProtectedData

Now I’m going to create some folder structure and place all the files where they need to go

#Create Folders
mkdir c:\DSC
mkdir C:\DSC\BuldOutput
mkdir C:\DSC\DSC_Configuration
mkdir C:\DSC\DSC_Resources
mkdir C:\DSC\DSC_Script
mkdir C:\DSC\DSC_Tooling

#Copy Files
copy C:\github\PshOrgDSC\Tooling\* C:\DSC\DSC_Tooling\ -Exclude 'examples', 'readme.md' -Recurse
copy C:\github\PshOrgDSC\Tooling\* 'C:\Program Files\WindowsPowerShell\Modules' -Exclude 'examples', 'readme.md' -Recurse
copy C:\github\* 'C:\Program Files\WindowsPowerShell\Modules' -Include 'Pester','ProtectedData' -Recurse 
copy C:\github\* C:\DSC\DSC_Tooling\ -Include 'Pester','ProtectedData' -Recurse 
copy C:\github\PshOrgDSC\Tooling\Examples\SampleBuild.ps1 c:\dsc\SampleBuild.ps1
copy c:\github\* C:\DSC\DSC_Resources -Include 'cSmbShare', 'cWebAdministration', 'StackExchangeResources' -Recurse
copy c:\github\* 'C:\Program Files\WindowsPowerShell\Modules' -Include 'cSmbShare', 'cWebAdministration', 'StackExchangeResources' -Recurse
copy C:\github\PshOrgDSC\Tooling\Examples\DSC_Configuration\* C:\DSC\DSC_Configuration -Recurse
copy C:\github\PshOrgDSC\Tooling\Examples\* C:\DSC\DSC_Script -Include 'SampleConfiguration' -Recurse

#delete unneeded folders. 
dir -Path c:\dsc -Include '.git' -Recurse | del -Recurse -Force #-Confirm:$false
dir -Path 'C:\Program Files\WindowsPowerShell\Modules' -Include '.git' -Recurse | del -Recurse -Force

Then result should give us a folder structure like this

C:\DSC
├───BuldOutput
├───DSC_Configuration
│   ├───AllNodes
│   ├───Services
│   └───SiteData
├───DSC_Resources
│   ├───cSmbShare
│   │   └───DscResources
│   │       └───PSHOrg_cSmbShare
│   ├───cWebAdministration
│   │   ├───DSCResources
│   │   │   ├───PSHOrg_cAppPool
│   │   │   └───PSHOrg_cWebsite
│   │   └───Examples
│   └───StackExchangeResources
│       ├───DSCResources
│       │   ├───StackExchange_CertificateStore
│       │   ├───StackExchange_FirewallRule
│       │   ├───StackExchange_NetworkAdapter
│       │   ├───StackExchange_Pagefile
│       │   │   ├───StackExchange_en-US
│       │   │   └───StackExchange_nl-NL
│       │   ├───StackExchange_PowerPlan
│       │   │   └───StackExchange_en-US
│       │   ├───StackExchange_ScheduledTask
│       │   ├───StackExchange_SetExecutionPolicy
│       │   │   └───StackExchange_en-US
│       │   └───StackExchange_Timezone
│       └───test
│           ├───integration
│           │   └───StackExchange_PageFile
│           │       └───pester
│           └───unit
│               └───StackExchange_Pagefile
│                   └───pester
├───DSC_Script
│   └───SampleConfiguration
└───DSC_Tooling
    ├───cDscDiagnostics
    ├───cDscResourceDesigner
    ├───dscbuild
    ├───DscConfiguration
    ├───DscDevelopment
    ├───DscOperations
    ├───Pester
    │   ├───bin
    │   ├───en-US
    │   ├───Examples
    │   │   ├───Calculator
    │   │   └───Validator
    │   ├───Functions
    │   │   └───Assertions
    │   ├───Snippets
    │   └───vendor
    │       └───tools
    │           ├───OneGet
    │           │   └───Etc
    │           └───PowerShellGet
    │               └───en-US
    └───ProtectedData
        └───en-US

C:\PROGRAM FILES\WINDOWSPOWERSHELL\MODULES
├───cDscDiagnostics
├───cDscResourceDesigner
├───cSmbShare
│   └───DscResources
│       └───PSHOrg_cSmbShare
├───cWebAdministration
│   ├───DSCResources
│   │   ├───PSHOrg_cAppPool
│   │   └───PSHOrg_cWebsite
│   └───Examples
├───dscbuild
├───DscConfiguration
├───DscDevelopment
├───DscOperations
├───Pester
│   ├───bin
│   ├───en-US
│   ├───Examples
│   │   ├───Calculator
│   │   └───Validator
│   ├───Functions
│   │   └───Assertions
│   ├───Snippets
│   └───vendor
│       └───tools
│           ├───OneGet
│           │   └───Etc
│           └───PowerShellGet
│               └───en-US
├───ProtectedData
│   └───en-US
└───StackExchangeResources
    ├───DSCResources
    │   ├───StackExchange_CertificateStore
    │   ├───StackExchange_FirewallRule
    │   ├───StackExchange_NetworkAdapter
    │   ├───StackExchange_Pagefile
    │   │   ├───StackExchange_en-US
    │   │   └───StackExchange_nl-NL
    │   ├───StackExchange_PowerPlan
    │   │   └───StackExchange_en-US
    │   ├───StackExchange_ScheduledTask
    │   ├───StackExchange_SetExecutionPolicy
    │   │   └───StackExchange_en-US
    │   └───StackExchange_Timezone
    └───test
        ├───integration
        │   └───StackExchange_PageFile
        │       └───pester
        └───unit
            └───StackExchange_Pagefile
                └───pester

now lets fix C:\DSC\SampleBuild.ps1

end
{
    Import-Module Pester -ErrorAction Stop
    Import-Module dscbuild -ErrorAction Stop
    Import-Module dscconfiguration -ErrorAction Stop

    $params = @{
        WorkingDirectory = (Get-TempDirectory).FullName
        SourceResourceDirectory = "$PSScriptRoot\DSC_Resources"
        SourceToolDirectory = "$PSScriptRoot\DSC_Tooling"
        DestinationRootDirectory = "$PSScriptRoot\BuldOutput"
        DestinationToolDirectory = $env:TEMP
        ConfigurationData = Get-DscConfigurationData -Path "$PSScriptRoot\DSC_Configuration" -Force -verbose
        ModulePath = "$PSScriptRoot\DSC_Script"  , "$PSScriptRoot\DSC_Tooling"
        ConfigurationModuleName = 'SampleConfiguration'
        ConfigurationName = 'SampleConfiguration'
        Configuration = $true
        Resource = $true
    }

    Invoke-DscBuild @params -verbose
}

begin
{
    function Get-TempDirectory
    {
        [CmdletBinding()]
        [OutputType([System.IO.DirectoryInfo])]
        param ( )

        do
        {
            $tempDir = Join-Path -Path ([System.IO.Path]::GetTempPath()) -ChildPath ([System.IO.Path]::GetRandomFileName())
        }
        until (-not (Test-Path -Path $tempDir -PathType Container))

        return New-Item -Path $tempDir -ItemType Directory -ErrorAction Stop
    }
}

So what is different?

  • DestinationRootDirectory = “$PSScriptRoot\BuldOutput”
    • Changed to point to a relative output. this is a test, so there is no need to place it into the common location for a pull server, although in production you may want to. I prefer to copy the files after a build, as it’s usually built on a machine other then the pull server
  •  ConfigurationData = Get-DscConfigurationData -Path “$PSScriptRoot\DSC_Configuration” -Force -verbose
    • added verbose so I can see how its progresses and help with troubleshooting
  • ModulePath = “$PSScriptRoot\DSC_Script”  , “$PSScriptRoot\DSC_Tooling”
      This is the big one. ModulePath and SourceResourceDirectory are going to be the only path’s in $psmodulepath when the configuration module (the module referenced by ConfigurationModuleName ) is loaded and when the configuration in ConfigurationName  is executed. This feature was added to resolve a problem when your modules used to build .mof files may be newer then the ones used to configure the machine running them. Something I ran into and Dave Wyatt was kind enough to solve
  • Invoke-DscBuild @params -verbose
    • Added -verbose. I’m a bit of a verbose junky.

With the updates to SampleBuild.ps1 we are ready to build our first sample set of .mof files

I do this in a clean environment so I’m opening PowerShell as an administrator and running C:\DSC\SampleBuild.ps1. After a long bit of scrolling if it works the last few lines should look like this.

VERBOSE: Moving 718aec80-e8fe-41b5-ac31-fbcd5d0186b1.mof to C:\DSC\BuldOutput\Configuration
VERBOSE: Moving b4519959-9724-40d5-ab62-5c4f82bbcd80.mof to C:\DSC\BuldOutput\Configuration
VERBOSE: Moving fc107c0b-1fc8-45fb-9991-a0a1f0fd6c21.mof to C:\DSC\BuldOutput\Configuration

Congratulations you have your first set of .mof files working with the PowerShell.org DSC tools.

Next up. Creating a pull request to update the readme.md and SampleBuild.ps1

Get-DSCFramework | Test-Lab

Posts in this series

  1. Test-HomeLab -InputObject ‘The Plan’
  2. Get-Posh-Git | Test-Lab
  3. Get-DSCFramework | Test-Lab
  4. Invoke-DscBuild | Test-Lab
  5. Test-Lab | Update-GitHub

In the last post we got the posh-git installed, Now we are going to fork the Powershell.org DSC tools development branch and clone that locally.

I already have an account with GitHub, You will need one to be able to contribute.

I had over the the repository and click the fork button

fork

with that done next is to get a copy of the clone URL.

cloneurl

I create a folder to store the repository in

C:\> mkdir github
    Directory: C:\
Mode                LastWriteTime     Length Name                                                                                                              
----                -------------     ------ ----                                                                                                              
d----          5/1/2015   5:25 PM            github                                                                                                            

C:\> cd github
C:\github> 

now i have everything I need to make a clone.

C:\github> git clone https://github.com/BladeFireLight/DSC.git PshOrgDSC --branch development
git : Cloning into 'PshOrgDSC'...
At line:1 char:1
+ git clone https://github.com/BladeFireLight/DSC.git PshOrgDSC --branch developme ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Cloning into 'PshOrgDSC'...:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 

C:\github> cd .\PshOrgDSC

C:\github\PshOrgDSC [development]> dir

    Directory: C:\github\PshOrgDSC

Mode                LastWriteTime     Length Name                                                                                                              
----                -------------     ------ ----                                                                                                              
d----          5/1/2015   5:38 PM            Tooling                                                                                                           
-a---          5/1/2015   5:38 PM        605 .gitattributes                                                                                                    
-a---          5/1/2015   5:38 PM        366 .gitignore                                                                                                        
-a---          5/1/2015   5:38 PM       1099 LICENSE.txt                                                                                                       
-a---          5/1/2015   5:38 PM       1231 README.md                                                                                                         
-a---          5/1/2015   5:38 PM       7305 README.old.md                                                                                                     

C:\github\PshOrgDSC [development]> 

I’m not sure why PowerShell thought it was an error but the clone worked.

Next up will be getting the example config to build.

C:\github\PshOrgDSC [development]> cd .\Tooling\Examples

C:\github\PshOrgDSC\Tooling\Examples [development]> dir

    Directory: C:\github\PshOrgDSC\Tooling\Examples

Mode                LastWriteTime     Length Name                                                                                                              
----                -------------     ------ ----                                                                                                              
d----          5/1/2015   5:38 PM            DSC_Configuration                                                                                                 
d----          5/1/2015   5:38 PM            SampleConfiguration                                                                                               
-a---          5/1/2015   5:38 PM       1770 README.md                                                                                                         
-a---          5/1/2015   5:38 PM       1203 SampleBuild.ps1