Showing posts with label PowerShell. Show all posts
Showing posts with label PowerShell. Show all posts

Friday, February 12, 2021

PowerShell script to remove old Docker Images without deleting the base OS image

 By Steve Endow

If you are creating Business Central Docker Containers and regularly updating the artifacts and images for your containers, you may be familiar with this problem.

When you type "docker images" in PowerShell, you are greeted with a pile of old unused images consuming tons of disk space.

A digital junk drawer

I want to delete the old, unused Docker images.  But I do not want to delete the 10.4 GB "base OS" image that is listed at the very bottom.  As long as my OS version remains the same, that image can be used repeatedly, and if I deleted it, I would have to download it again.

Saturday, August 15, 2020

Updating from NavContainerHelper to the new BcContainerHelper

 By Steve Endow

UPDATE: If you're interested in a convenient way to run the BcContainerHelper PowerShell commands, check out Krzysztof's Azure Data Studio Notebook with sample BcContainerHelper commands.  Jupyter Notebooks are a great way to include documentation and commands in a single convenient format.

On August 11, 2020, Freddy Kristiansen announced that he had released BcContainerHelper.

You can read his blog post here:

          https://freddysblog.com/2020/08/11/bccontainerhelper/

You will definitely want to read the entire blog post carefully, and perhaps read it a few times, as he shares quite a few important details about the new BcContainerHelper.

Same Great Container Helper, But Updated and Upgraded!


I made a video showing how I updated from NavContainerHelper to BcContainerHelper.  The process went flawlessly.


Here are the PowerShell Docker commands that I used to perform my update.

Monday, August 3, 2020

Automatically Update BcContainerHelper - Improved Script

By Steve Endow

UPDATE: I have updated the script to work with the newer BcContainerHelper rather than the older NavContainerHelper


UPDATE: If you're interested in a convenient way to run the BcContainerHelper PowerShell commands, check out Krzysztof's Azure Data Studio Notebook with sample BcContainerHelper commands.  Jupyter Notebooks are a great way to include documentation and commands in a single convenient format.


I previously wrote about creating a simple scheduled task to automatically update NavContainerHelper.

While that simple script does work, it didn't have any logging or notification. I never knew when NavContainerHelper / BcContainerHelper was updated.

I recently created a fancier PowerShell script to automatically build my Business Central Docker Containers every day, and while working on that script, I learned how to record elapsed time, log all activity, and send an email notification.

Based on that new learning, I wanted to upgrade the script that updates BcContainerHelper.

I wanted logging, elapsed time, and email notification if BcContainerHelper was updated to a new version.

Email notification when an update occurs


Here's what I came up with.  You'll need to adjust the file paths and email address, and generate your own email password file.

If you think of any other features the update could have, let me know!

#v2.0 - June 22, 2021
#
$server = $env:COMPUTERNAME
#Define date values
$simpleDate = Get-Date -Format "M/d/yy"
$dateTime = Get-Date -Format "M/d/yy HH:mm"
$fileDateTime = Get-Date -Format "yyyy-MM-dd HHmm"

#Define the module name
$moduleName = 'BcContainerHelper'

$body = ""
$message = ""
$updated = $false

#Specify file location for activity transcript log file
$transcriptFile = "D:\BCPowerShell\Logs\" + $fileDateTime + " " + $server + " " + $moduleName + " Update Log.txt"

#Start recording transcript
Start-Transcript -Path $transcriptFile

#Record the start time
$StartTime = $(get-date)


#Define email configuration
$emailFrom = "myemail@gmail.com"
$emailTo = "myemail@gmail.com"
$smtpServer = "smtp.gmail.com"
$port = "587"
#Email password file created using:  Read-Host -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath D:\BCPowerShell\Gmail.securestring
$passwordFile = "D:\BCPowerShell\Gmail.securestring"
$securePassword = ConvertTo-SecureString (Get-Content -Path $passwordFile)
$smtpCred = New-Object -TypeName PSCredential ($emailFrom, $securePassword)

$subject = $server + ": " + $simpleDate + " " + $moduleName + " Update Log";

$started = "Start time: " + $StartTime
$body += $started + "`n`n"

Try
{
    #Get currently installed version
    $currentVersion = (Get-InstalledModule $moduleName -ErrorAction SilentlyContinue).Version.ToString()
}
Catch 
{
    $currentVersion = "(not installed)"
}

$oldVersion = $currentVersion

#Get the latest version available
$latestVersion = (Find-Module $moduleName).Version.ToString()


#If we don't have the latest version installed
If ($currentVersion -ne $latestVersion)
{

    $message = "Current version: " + $currentVersion + "`nLatest version: " + $latestVersion;
    $message += "`n`nUpdating to version " + $latestVersion;
    #Remove existing version(s)
    Uninstall-Module $moduleName -Force -AllVersions -ErrorAction SilentlyContinue

    #Install latest version
    Install-Module $moduleName -Force

    $updated = $true;

    #Verify version number
    $newVersion = (Get-InstalledModule $moduleName).Version.ToString()
    $message += "`nUpdate complete";
    $message += "`nCurrent version is now " + $newVersion;
        
    $body += "`n" + $message;

    Write-Output $message
}
Else
{
    #If we are on the latest version, log that info
    $message = "You are on the latest version of " + $moduleName + ": " + $newVersion;
    Write-Output $message
}



#Record end time
$EndTime = $(get-date)
$output = "`nEnd time: " + $EndTime
$body += "`n" + $output 
Write-Output $output 

#Calculate elapsed time
$elapsedTime = $EndTime - $StartTime
$totalTime = "{0:HH:mm:ss}" -f ([datetime]$elapsedTime.Ticks)
$output = "Elapsed time: " + $totalTime
$body += "`n" + $output 
Write-Output $output

$body += "`n`nFull log file is attached" 

#Finish the transcript recording
#The Transcript must be stopped before trying to send the email,
#otherwise the log file will be locked, causing the email to fail
Stop-Transcript
        
if ($updated)
{
#Send the email
Send-MailMessage -From $emailFrom -To $emailTo `
    -Subject $subject `
    -Body $body `
    -Attachments $transcriptFile `
    -SmtpServer $smtpServer `
    -Credential $smtpCred `
    -Port $port `
    -UseSsl
}  


Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server.

You can also find him on Twitter and YouTube

https://www.precipioservices.com

Saturday, July 25, 2020

Fully automate the Business Central Container build process

By Steve Endow

I don't like maintenance tasks.  I don't like having to remember to do something on a regular basis.  I don't like having to do low value tasks constantly.

While it has been valuable for me to manually run NavContainerHelper the last several months, I think I am finally at a point where I sufficiently understand how it works and what it does.  I'm now ready to automate my Business Central Container build process so that I don't have to do it manually.

Just tell me how it went...

This isn't anything new--I'm just finally getting around to doing it for me with the features that I want.  If there are other ways or better ways to do this, let me know.

I share the background, my thinking, and my journey creating the script in this video.



In short, I've just wrapped the call to New-BcContainer in some additional PowerShell script to enable full logging, some basic error handling, and email notification.

Friday, July 24, 2020

Recording Business Central NavContainerHelper benchmarks

By Steve Endow

As I've been using NavContainerHelper, it's become clear that there can be a lot of waiting as it downloads gigabytes of data for the images and hundreds of megabytes for the artifacts.  It then has to mash all of that together to build a Business Central Container.  It can take a while.

Sometimes it seems the process is pretty quick.  But sometimes the process seems to take forever.  When it's in the middle of doing its thing, I can't tell whether it's taking longer than normal, or if it just feels like it's taking a long time.

So I'm now measuring how long it takes for NavContainerHelper to create containers.

Ya gotta measure things

If the container setup process takes longer than normal, I'll want to know why.  Was it a performance issue downloading images or artifacts?  Or is there a performance issue with the VM?  Or did some error cause the script to fail prematurely?


Here's the PowerShell script I've setup to record the start time, end time, and elapsed time of the New-BcContainer process.  This is a sample that uses the values for building a BC container in my environment--you'll want to adjust the New-BcContainer parameter values to your preferred values.


 $StartTime = $(get-date)   
  Try {   
   $output = "Started: " + $StartTime   
   Write-Output $output    
   $containerName = 'bc1'   
   $password = 'P@ssw0rd'   
   $securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force   
   $credential = New-Object pscredential 'admin', $securePassword   
   $auth = 'UserPassword'   
   $artifactUrl = Get-BcArtifactUrl -type 'Sandbox' -version '' -country 'us' -select 'Latest'   
   New-BcContainer `   
    -accept_eula `   
    -containerName $containerName `   
    -credential $credential `   
    -auth $auth `   
    -artifactUrl $artifactUrl `   
    -imageName 'myimage1' `   
    -dns '8.8.8.8' `   
    -updateHosts   
   }   
   Catch {   
    Write-Error "Um, something didn't go well:"   
    Write-Error $_   
   }   
   Finally {   
    $EndTime = $(get-date)   
    $output = "End time: " + $EndTime   
    Write-Output $output    
    $elapsedTime = $(get-date) - $StartTime   
    $totalTime = "{0:HH:mm:ss}" -f ([datetime]$elapsedTime.Ticks)   
    $output = "Elapsed time: " + $totalTime   
    Write-Output $output   
   }   


At the top of the script, I record the start time, then output that time in the log so that I can see when the script started.

I added a try/catch block so that even if the script fails, my script outputs the end time and elapsed time.

In the finally block, I record the time again, then calculate elapsed time.  The elapsed time is then output to the log.

Building a new named image vs. using an existing named image

In this example, I see that when the script had to build a new named image, it took 13 minutes to create the named image and then create the container.  But when it was able to use an existing named image, it only took 1 minute.

Good to know!

Now I can use this script to see how my dedicated physical server compares to my VMs.  I assume that the VMs will be slower, but I'm curious to see how much slower.


Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server.

You can also find him on Twitter and YouTube

http://www.precipioservices.com



Automatically Updating NavContainerHelper

By Steve Endow

One of the challenges I've had using the Business Central NavContainerHelper module has been managing the versions and updates.  Freddy Kristiansen is constantly updating NavContainerHelper to add features or fix issues, so a new version might be released within 24 hours of the prior release.

It seems like there is at least one update per week.  As a result, I always forget to update NavContainerHelper and often end up using an older version, which can sometimes result in errors.

I wanted to automate the update process so that I always know that I have the latest version of NavContainerHelper installed.  And ONLY the latest version--making sure that I don't have any older versions installed side-by-side.

I don't know if this is the best method, but I think it should work.  If you know of a better way, let me know!

Here's the script I'd like to run automatically every day:

 $MyModuleName = 'NavContainerHelper'  
 Uninstall-Module $MyModuleName -Force -AllVersions  
 Install-Module $MyModuleName -Force  

The Script

I want to install the latest version, but I want to make sure that I don't have an older version still installed.  There are a few ways to do this, but I figure with this script, it removes all existing versions, and then installs the latest version.

I normally don't recommend Install-Module -Force if you are running the script manually, but in this case, I believe I need -Force in order to bypass the verification dialog that appears during installation.  Also, since I know that I've just uninstalled all versions before installing the latest version, I think using -Force should be okay in this particular case.

I saved the above 3 line script to a .ps1 file.

I then opened Windows Task Scheduler using Run As Administrator.

Windows Task Scheduler
I created a new folder called Steve to make it easier to find my tasks.

I then created a new task (not a Basic task).

Task - General Info

I gave the task a name.  Then I set the task to use the Local Administrator user.  This is on one of my dev VMs, and I know the PowerShell script will need Admin rights, so I chose Administrator just to avoid any hassles.  If you are on a company network / domain, you'll need to figure out which domain user / rights you need to allow the script to run as a task.

Nightly update
On the Triggers tab, I setup a schedule to run Daily at 8pm.

On the Actions tab, I created an action to call powershell with the -File argument, based on this blog post.



The argument I used looks like:

     -File C:\BCPowerShell\UpdateNavContainerHelper_v1.ps1


I then saved the task and tested it. 

It seems to work, so I'll keep an eye on it for a week or so to verify that NavContainerHelper is always on the latest version and that I don't have any old versions installed side-by-side.


Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server.

You can also find him on Twitter and YouTube

http://www.precipioservices.com

Wednesday, July 22, 2020

Basic Business Central Docker Commands That I Use

By Steve Endow

On July 1, 2020, I made a video showing the basic Docker commands that I use to manage Business Central Docker Containers.

PowerShell isn't too bad

Here is a complete list of the commands that I've collected to manage Business Central Containers, as of July 2020.  If you have any others that you recommend, please comment below and let me know!


Thanks to Daniel and Tanya and Džoka for some additions!


   
 Open Powershell / Powershell ISE on HOST machine using Run As Administrator  
   
 
 NOTE: When referring to a Container ID or Image ID, you can use the first 
       few characters of the ID if they are unique--you do not have to always
       use the full ID.


 Check NavContainerHelper Version  
   
      Get-InstalledModule NavContainerHelper  
   
   
 Fresh Install of NavContainerHelper  
   
      Install-module navcontainerhelper  
      Import-Module navcontainerhelper  
   
   
 Install Latest NavContainerHelper  
   
 If you KNOW you don't have multiple versions installed:  
   
      Update-Module NavContainerHelper -Force  
   
 If you MIGHT have multiple versions installed:  
   
      $MyModuleName = <modulename>  
      Uninstall-Module $MyModuleName -allversions   
      Install-Module $MyModuleName
   
      NOTE: I do not recommend using the -Force option with Install-Module. 
            It can bypass valuable warnings that you want to know about.


 Alternate method to Remove Old Versions of NavContainerHelper  
   
      $ModuleName = 'navcontainerhelper';  
      $Latest = Get-InstalledModule $ModuleName;   
      Get-InstalledModule $ModuleName -AllVersions | ? {$_.Version -ne $Latest.Version} | Uninstall-Module -WhatIf  
   
   
   
 Container Management Commands  
   
   	Compact Container listing:
    
      docker ps --format "table {{.Names}}\t{{.ID}}\t{{.CreatedAt}}\t{{.Status}}"
      
    Full listing:
    
      docker ps  
      docker ps -a  
        
      docker start <id>  
      docker stop <id>  
   
      
      docker container inspect 


 Remove Container  
   
    Recommended method:

      Remove-BCContainer <containername>


    Manual Docker method:

      docker stop <id> 
      docker rm <id> 
      Manually clean up hosts file
      Manually remove desktop icons
      Manually remove extensions from C:\ProgramData\NavContainerHelper\Extensions\
   

 Stop all running containers:  
        
      docker stop $(docker ps -aq)  
   
   
 Images  
   
       docker images  
       docker images -a  
   
 Remove Image  
   
      docker rmi <id>  
        
      docker rmi <repository>:<tag>  
   
   Remove all images:

      docker rmi $(docker images -aq)

   
 Clean up old images  
   
   Remove orphaned images with "< none >" identifiers:

      docker images -q -f dangling=true | % { docker rmi $_ }  


   Remove orphaned images:
   
   	  docker image prune -f
      
   Remove orphaned AND UNUSED images:
   
   WARNING: Be careful with this: It will remove generic OS images used by NavContainerHelper!

      docker image prune -a -f
   

   
 Get Artifact Info for an existing BC Container:  
   
      Get-BcContainerArtifactUrl containername  
        
      Get-NavContainerArtifactUrl containername  
   
   
 Diagnostics  
   
      Get-BcContainerEventLog  
   
   
   
 Create New BC Container  
   
      New-BcContainerWizard  
   
   
 https://blog.baudson.de/blog/stop-and-remove-all-docker-containers-and-images  
   




Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server.

You can also find him on Twitter and YouTube

http://www.precipioservices.com

Friday, July 10, 2020

The New-BcContainerWizard! And working through Docker errors...

By Steve Endow

UPDATE: If you're interested in a convenient way to run the BcContainerHelper PowerShell commands, check out Krzysztof's Azure Data Studio Notebook with sample BcContainerHelper commands.  Jupyter Notebooks are a great way to include documentation and commands in a single convenient format.

Have you used the new New-BcContainerWizard tool?  If not, settle in!  And I'll throw in a story about a puzzling Docker error and how Freddy helped me work through it.

Here's a video where I walk through my recent Docker learning:



Yesterday I planned on working through Erik Hougaard's video about calling an external web API from AL and processing the JSON response.  

Since I needed a new BC Docker Container to work on the demo code, I thought it would be a good opportunity to test Freddy's new BC Container Wizard tool.  So I made sure I had the latest version of NAV Container Helper, and typed New-BcContainerWizard.

The wizard is a really nifty PowerShell utility that asks you a series of questions that walk you through the different options available when using the NewBcContainer command.  After I completed the questions, it provided me with this script.

 $containerName = 'sandbox1'  
 $password = 'P@ssword'  
 $securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force  
 $credential = New-Object pscredential 'admin', $securePassword  
 $auth = 'UserPassword'  
 $artifactUrl = Get-BcArtifactUrl -type 'Sandbox' -version '' -country 'us' -select 'Latest'  
 $licenseFile = 'c:\zdisks\business central\2020-06bcv16license.flf'  
 New-BcContainer `  
   -accept_eula `  
   -containerName $containerName `  
   -credential $credential `  
   -auth $auth `  
   -artifactUrl $artifactUrl `  
   -updateHosts `  
   -assignPremiumPlan `  
   -licenseFile $licenseFile `  
   -dns '192.168.25.21' `  
   -imageName 'mybc16'   
 Setup-BcContainerTestUsers -containerName $containerName -Password $credential.Password -credential $credential  

This is a really great tool.  When I first started using NavContainerHelper, it seemed very confusing and I had no idea which options to include and which values made sense for my needs.  This wizard resolves that and will be a great resource for people who are new to Docker.

Now that I had my nifty PowerShell script, I ran it.  

Sad Trombone
Sad Trombone

"This request is not supported"

Um.  What?

After a bit of googling, I eventually made my way to the NavContainerHelper GitHub repository to see if I could learn anything from the Issues list.  I saw another issue that also had "CreateComputeSystem", but the specific error was different.

So I submitted a separate issue on GitHub.  This was the first GitHub issue I've ever submitted, so I didn't know how or what to include.  I used another issue as an example, and I stumbled through the process, including what I was trying to do, a copy of my New-BcContainer script, and as much information about the error that I had available.

Freddy responded within a few hours asking for the full build log from a successful container build (excluding the imageName parameter).  I was able to attach that full build log as a text file.

Within a few minutes, he replied, explaining that the log showed I was using a very old version of windows.

The last few digits are the important ones

Sure enough, Windows Update had a few failed updates.  Those failed updates had prevented ANY updates from being applied to my server.  Once I cleared those errors and updated my server to the latest windows build, the New-BcContainer script worked successfully with the imageName parameter.

I then looked through the full container build log that I had submitted to the issue, and sure enough, there it was.

I'm talkin' to you!


Among the hundreds of lines that scrolled by during the build process, the warning was right there, showing that I had a very old Windows build and that the generic image didn't match.

While this did delay me by over a day in my AL learning process, this side adventure was very valuable, as it helped me understand a little bit more about Docker and the NavContainerHelper build process.


Wednesday, July 8, 2020

I'm no PowerShell expert: Discovering Update-Module for NavContainerHelper

By Steve Endow

I wrote a post in April, "Make sure to update NavContainerHelper", in which I described some issues I had that were caused by having several old versions of the NavContainerHelper module installed.

Today, while updating the NavContainerHelper module on my Docker VM, I learned a few things, and I think I finally have a basic understanding of the PowerShell Module install and update process.

If you're a PowerShell pro or a NavContainerHelper pro, this will probably seem obvious, but as a newbie with both tools, it was a "learning moment" for me that explained the issues I had previously with multiple NavContainerHelper versions.

Let's start with a screen shot.

Install-Module -Force Warning

Wednesday, July 1, 2020

Business Central Basic Docker Commands

By Steve Endow

If you haven't used Docker before, and if you're not a PowerShell expert, welcome to the club.

I am new to both, and the only reason I'm using Docker and PowerShell is to use Business Central Docker Containers.  I know just enough to do the bare minimum that I need to manage a few Business Central Docker Containers for development and testing.

I have a live stream video demonstrating the commands:

https://www.youtube.com/watch?v=u-mlyIqNl5c





Here are the basic Docker commands that I have been using.


 #Containers  
   
 docker ps  
 docker ps -a  
   
 docker start <id>  
 docker stop <id>  
   
 docker rm <id>  
   
 #Stop all running containers:  
 docker stop $(docker ps -aq)  
   
   
 #Images  
   
 docker images  
 docker images -a  
   
 docker rmi <id>  
   
 docker rmi <repository>:<tag>  
   
   
 #Check NavContainerHelper version  
 Get-InstalledModule  
   
   
 #Install Latest NavContainerHelper  
 $MyModuleName = <modulename>  
 Uninstall-Module $MyModuleName -allversions   
 Install-Module $MyModuleName -Force  





Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server.

You can also find him on Twitter and YouTube


How many digits can a Business Central Amount field actually support?

 by Steve Endow (If anyone has a technical explanation for the discrepancy between the Docs and the BC behavior, let me know!) On Sunday nig...