(This blog post is inspired by a forum question. Credit to Arvydas for the question and sharing the solution.)
What if I made the mistake of storing some "worthwhile" data in a Business Central Docker container over the course of several months? Perhaps it was test data or data imports that I didn't think I would ever need--but then 3 months later I realized that the accumulated data was valuable.
In theory, no big deal. If you realize that you want to keep the data from a working container, Stefan Maroń has a blog post with instructions on converting that Container into an Image using "docker commit". Just create an image and you can make new containers from that image. You could also backup the SQL databases from the Docker Container to the local file system.
But, what if the Business Central Docker Container with your valuable data stops working due to a Windows Update? And you only realize you need that data after the container stops working. If the Container can't be started at all, how can you extract the Business Central data?
This solution appears to allow me to copy the SQL mdf and ldf files from the broken container to my local file system, then copy the mdf and ldf files to a new working container to effectively "restore" them.
This may not work for all situations where a BC container has issues, but should work if the container is intact, but just won't start up due to a Docker or Windows issue or error.
Let me know if there are any issues or caveats with using this method to restore the data from a container.
If you are creating Business Central Docker Containers and regularly updating the artifacts and images for your containers, you may be familiar with this problem.
When you type "docker images" in PowerShell, you are greeted with a pile of old unused images consuming tons of disk space.
A digital junk drawer
I want to delete the old, unused Docker images. But I do not want to delete the 10.4 GB "base OS" image that is listed at the very bottom. As long as my OS version remains the same, that image can be used repeatedly, and if I deleted it, I would have to download it again.
While using Business Central with Google Chrome, I started to notice strange words appearing in Business Central fields while I entered data in the browser.
Why are you doing this?
In this example, the word "test" was appearing when I clicked on the drop down list for bank accounts in Business Central. The suggested value was in a text box displayed over the standard Business Central list box.
Update: I originally wrote this post referencing my Lenovo ThinkPad P1 laptop, but it appears that similar Intel Graphics software is used on several different laptop brands and models, causing the same issue. The name or appearance of the specific Intel Graphics software may vary, but the concept still applies.
If you use Dynamics 365 Business Central, you should be aware of the Page Inspection feature.
"Business Central Twitter" is in constant conversation. Business Central community members from around the globe, quite literally, are asking questions, answering questions, posting updates, blogs, videos, and MS Docs articles.
I pull up my web browser, and I click a shortcut on the browser shortcut bar.
Single click
And you probably know what I see next.
I've arrived
A glorious, modern, rich, interactive, attractive, customizable user interface. Take a moment to appreciate this significant feature of Business Central.
A "native" web interface. A native web application. Pretty amazing.
I didn't have to install an application on my computer. I didn't have to configure anything on my computer. I didn't have to setup a web server or install a web client. In fact, I can access Business Central from any computer. I just need a web browser. Or I can install the Business Central mobile application.
"What? Are you crazy, Steve? Not having access to SQL is a limitation! It's a weakness! That's not amazing!"
I am not trying to change anyone else's mind--I'm just sharing a few thoughts why I think that NOT having access to SQL Server is amazing.
I've been working with on-prem ERP systems since 1996, when I implemented Solomon IV for Windows, then MAS 90, then Peoplesoft, then Dynamics GP. I'm aware of all the arguments for wanting or needing access to SQL. I get it.
But that direct access comes at significant cost. And responsibility. And risk. And those things are not amazing. Mid-market companies want to sell their product or service, to provide customer satisfaction, and to make revenue.
They don't want to be in the ERP database administration business. Or the ERP database backup business. Or the ERP disaster recovery business. Or the ransomware defense business.
Do you really want to be in the database administration and maintenance business?
After seeing a few very cool Business Central features the other day, I was so impressed and in such a good mood that I might have posted a slightly overly enthusiastic tweet.
Can't a guy be in a really good mood?
Does Business Central do my dishes? My laundry? Paint my house?
Sadly, it does not. Yet. 😉
But it does do lots of awesome things and it has some amazing features and benefits. And after posting my happy tweet, I think a few people are wondering why I think Business Central is so amazing.
So now I feel obligated to explain why I think it's amazing, and in the process, share my enthusiasm for the new ERP platform that I'm learning about.
UPDATE: If you're interested in a convenient way to run the BcContainerHelper PowerShell commands, check out Krzysztof's Azure Data Studio Notebook with sample BcContainerHelper commands. Jupyter Notebooks are a great way to include documentation and commands in a single convenient format.
On August 11, 2020, Freddy Kristiansen announced that he had released BcContainerHelper.
You will definitely want to read the entire blog post carefully, and perhaps read it a few times, as he shares quite a few important details about the new BcContainerHelper.
Same Great Container Helper, But Updated and Upgraded!
I made a video showing how I updated from NavContainerHelper to BcContainerHelper. The process went flawlessly.
Here are the PowerShell Docker commands that I used to perform my update.
If you are new to Dynamics 365 Business Central development, join me on a journey to learn how to develop Business Central Extensions using VS Code and the AL language.
I have been developing business software for many, many years, but I am completely new to Business Central, AL, and VS Code. And I am also new to Docker and PowerShell.
Basically, I'm having to learn everything from scratch.
I'm writing blog posts and making videos to share the learning resources I'm using and share my learning journey.
UPDATE: After researching this type of Windows hosts file name resolution issue, it seems there are dozens of possible reasons why the problem can occur. I suspect my case was one of the simpler versions, and fortunately flushdns worked for me. Here is a forum thread with examples of several rather odd causes and solutions.
This morning I tried to create a new Business Central AL project in VS Code. I used AL Go! to create the project, set the server name in the launch.json file, and then tried to Download Symbols.
Very simple, very easy, right?
Nope.
No soup for you!
What? I performed these exact same steps 2 days ago on this same machine, and I was able to download symbols without any issues.
Here is the full video of the troubleshooting process:
UPDATE: I have updated the script to work with the newer BcContainerHelper rather than the older NavContainerHelper
UPDATE: If you're interested in a convenient way to run the BcContainerHelper PowerShell commands, check out Krzysztof's Azure Data Studio Notebook with sample BcContainerHelper commands. Jupyter Notebooks are a great way to include documentation and commands in a single convenient format.
I previously wrote about creating a simple scheduled task to automatically update NavContainerHelper.
While that simple script does work, it didn't have any logging or notification. I never knew when NavContainerHelper / BcContainerHelper was updated.
I recently created a fancier PowerShell script to automatically build my Business Central Docker Containers every day, and while working on that script, I learned how to record elapsed time, log all activity, and send an email notification.
Based on that new learning, I wanted to upgrade the script that updates BcContainerHelper.
I wanted logging, elapsed time, and email notification if BcContainerHelper was updated to a new version.
Email notification when an update occurs
Here's what I came up with. You'll need to adjust the file paths and email address, and generate your own email password file.
If you think of any other features the update could have, let me know!
#v1.0 - March 27, 2021
#
$server = $env:COMPUTERNAME
#Define date values
$simpleDate = Get-Date -Format "M/d/yy"
$dateTime = Get-Date -Format "M/d/yy HH:mm"
$fileDateTime = Get-Date -Format "yyyy-MM-dd HHmm"
#Define the module name
$moduleName = 'BcContainerHelper'
$body = ""
$message = ""
$updated = $false
#Specify file location for activity transcript log file
$transcriptFile = "D:\BCPowerShell\Logs\" + $fileDateTime + " " + $server + " " + $moduleName + " Update Log.txt"
#Start recording transcript
Start-Transcript -Path $transcriptFile
#Record the start time
$StartTime = $(get-date)
#Define email configuration
$emailFrom = "myemail@gmail.com"
$emailTo = "myemail@gmail.com"
$smtpServer = "smtp.gmail.com"
$port = "587"
#Email password file created using: Read-Host -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath D:\BCPowerShell\Gmail_password.securestring
$passwordFile = "D:\BCPowerShell\Gmail_password.securestring"
$securePassword = ConvertTo-SecureString (Get-Content -Path $passwordFile)
$smtpCred = New-Object -TypeName PSCredential ($emailFrom, $securePassword)
$subject = $server + ": " + $simpleDate + " " + $moduleName + " Update Log";
$started = "Start time: " + $StartTime
$body += $started + "`n`n"
Try
{
#Get currently installed version
$currentVersion = (Get-InstalledModule $moduleName -ErrorAction SilentlyContinue).Version.ToString()
}
Catch
{
$currentVersion = "(not installed)"
}
#Get the latest version available
$latestVersion = (Find-Module $moduleName).Version.ToString()
#If we don't have the latest version installed
If ($currentVersion -ne $latestVersion)
{
$message = "Current version: " + $currentVersion + "`nLatest version: " + $latestVersion;
$message += "`n`nUpdating to version " + $latestVersion;
#Remove existing version(s)
Uninstall-Module $moduleName -Force -AllVersions -ErrorAction SilentlyContinue
#Install latest version
Install-Module $moduleName -Force
$updated = $true;
#Verify version number
$currentVersion = (Get-InstalledModule $moduleName).Version.ToString()
$message += "`nUpdate complete";
$message += "`nCurrent version is now " + $currentVersion;
$body += "`n" + $message;
Write-Output $message
}
Else
{
#If we are on the latest version, log that info
$message = "You are on the latest version of " + $moduleName + ": " + $currentVersion;
Write-Output $message
}
#Record end time
$EndTime = $(get-date)
$output = "`nEnd time: " + $EndTime
$body += "`n" + $output
Write-Output $output
#Calculate elapsed time
$elapsedTime = $EndTime - $StartTime
$totalTime = "{0:HH:mm:ss}" -f ([datetime]$elapsedTime.Ticks)
$output = "Elapsed time: " + $totalTime
$body += "`n" + $output
Write-Output $output
$body += "`n`nFull log file is attached"
#Finish the transcript recording
#The Transcript must be stopped before trying to send the email,
#otherwise the log file will be locked, causing the email to fail
Stop-Transcript
if ($updated)
{
#Send the email
Send-MailMessage -From $emailFrom -To $emailTo `
-Subject $subject `
-Body $body `
-Attachments $transcriptFile `
-SmtpServer $smtpServer `
-Credential $smtpCred `
-Port $port `
-UseSsl
}
Steve Endow is a Microsoft MVP in Los Angeles. He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server. You can also find him on Twitter and YouTube https://www.precipioservices.com
I don't like maintenance tasks. I don't like having to remember to do something on a regular basis. I don't like having to do low value tasks constantly.
While it has been valuable for me to manually run NavContainerHelper the last several months, I think I am finally at a point where I sufficiently understand how it works and what it does. I'm now ready to automate my Business Central Container build process so that I don't have to do it manually.
Just tell me how it went...
This isn't anything new--I'm just finally getting around to doing it for me with the features that I want. If there are other ways or better ways to do this, let me know.
I share the background, my thinking, and my journey creating the script in this video.
In short, I've just wrapped the call to New-BcContainer in some additional PowerShell script to enable full logging, some basic error handling, and email notification.
As I've been using NavContainerHelper, it's become clear that there can be a lot of waiting as it downloads gigabytes of data for the images and hundreds of megabytes for the artifacts. It then has to mash all of that together to build a Business Central Container. It can take a while.
Sometimes it seems the process is pretty quick. But sometimes the process seems to take forever. When it's in the middle of doing its thing, I can't tell whether it's taking longer than normal, or if it just feels like it's taking a long time.
So I'm now measuring how long it takes for NavContainerHelper to create containers.
Ya gotta measure things
If the container setup process takes longer than normal, I'll want to know why. Was it a performance issue downloading images or artifacts? Or is there a performance issue with the VM? Or did some error cause the script to fail prematurely?
Here's the PowerShell script I've setup to record the start time, end time, and elapsed time of the New-BcContainer process. This is a sample that uses the values for building a BC container in my environment--you'll want to adjust the New-BcContainer parameter values to your preferred values.
At the top of the script, I record the start time, then output that time in the log so that I can see when the script started.
I added a try/catch block so that even if the script fails, my script outputs the end time and elapsed time.
In the finally block, I record the time again, then calculate elapsed time. The elapsed time is then output to the log.
Building a new named image vs. using an existing named image
In this example, I see that when the script had to build a new named image, it took 13 minutes to create the named image and then create the container. But when it was able to use an existing named image, it only took 1 minute.
Good to know!
Now I can use this script to see how my dedicated physical server compares to my VMs. I assume that the VMs will be slower, but I'm curious to see how much slower.
Steve Endow is a Microsoft MVP in Los Angeles. He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server. You can also find him on Twitter and YouTube http://www.precipioservices.com
One of the challenges I've had using the Business Central NavContainerHelper module has been managing the versions and updates. Freddy Kristiansen is constantly updating NavContainerHelper to add features or fix issues, so a new version might be released within 24 hours of the prior release.
It seems like there is at least one update per week. As a result, I always forget to update NavContainerHelper and often end up using an older version, which can sometimes result in errors.
I wanted to automate the update process so that I always know that I have the latest version of NavContainerHelper installed. And ONLY the latest version--making sure that I don't have any older versions installed side-by-side.
I don't know if this is the best method, but I think it should work. If you know of a better way, let me know!
Here's the script I'd like to run automatically every day:
I want to install the latest version, but I want to make sure that I don't have an older version still installed. There are a few ways to do this, but I figure with this script, it removes all existing versions, and then installs the latest version.
I normally don't recommend Install-Module -Force if you are running the script manually, but in this case, I believe I need -Force in order to bypass the verification dialog that appears during installation. Also, since I know that I've just uninstalled all versions before installing the latest version, I think using -Force should be okay in this particular case.
I saved the above 3 line script to a .ps1 file.
I then opened Windows Task Scheduler using Run As Administrator.
Windows Task Scheduler
I created a new folder called Steve to make it easier to find my tasks.
I then created a new task (not a Basic task).
Task - General Info
I gave the task a name. Then I set the task to use the Local Administrator user. This is on one of my dev VMs, and I know the PowerShell script will need Admin rights, so I chose Administrator just to avoid any hassles. If you are on a company network / domain, you'll need to figure out which domain user / rights you need to allow the script to run as a task.
Nightly update
On the Triggers tab, I setup a schedule to run Daily at 8pm.
On the Actions tab, I created an action to call powershell with the -File argument, based on this blog post.
It seems to work, so I'll keep an eye on it for a week or so to verify that NavContainerHelper is always on the latest version and that I don't have any old versions installed side-by-side.
Steve Endow is a Microsoft MVP in Los Angeles. He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server. You can also find him on Twitter and YouTube http://www.precipioservices.com
Here is a complete list of the commands that I've collected to manage Business Central Containers, as of July 2020. If you have any others that you recommend, please comment below and let me know!
Thanks to Daniel and Tanya and Džoka for some additions!
Open Powershell / Powershell ISE on HOST machine using Run As Administrator
NOTE: When referring to a Container ID or Image ID, you can use the first
few characters of the ID if they are unique--you do not have to always
use the full ID.
Check NavContainerHelper Version
Get-InstalledModule NavContainerHelper
Fresh Install of NavContainerHelper
Install-module navcontainerhelper
Import-Module navcontainerhelper
Install Latest NavContainerHelper
If you KNOW you don't have multiple versions installed:
Update-Module NavContainerHelper -Force
If you MIGHT have multiple versions installed:
$MyModuleName = <modulename>
Uninstall-Module $MyModuleName -allversions
Install-Module $MyModuleName
NOTE: I do not recommend using the -Force option with Install-Module.
It can bypass valuable warnings that you want to know about.
Alternate method to Remove Old Versions of NavContainerHelper
$ModuleName = 'navcontainerhelper';
$Latest = Get-InstalledModule $ModuleName;
Get-InstalledModule $ModuleName -AllVersions | ? {$_.Version -ne $Latest.Version} | Uninstall-Module -WhatIf
Container Management Commands
Compact Container listing:
docker ps --format "table {{.Names}}\t{{.ID}}\t{{.CreatedAt}}\t{{.Status}}"
Full listing:
docker ps
docker ps -a
docker start <id>
docker stop <id>
docker container inspect
Remove Container
Recommended method:
Remove-BCContainer <containername>
Manual Docker method:
docker stop <id>
docker rm <id>
Manually clean up hosts file
Manually remove desktop icons
Manually remove extensions from C:\ProgramData\NavContainerHelper\Extensions\
Stop all running containers:
docker stop $(docker ps -aq)
Images
docker images
docker images -a
Remove Image
docker rmi <id>
docker rmi <repository>:<tag>
Remove all images:
docker rmi $(docker images -aq)
Clean up old images
Remove orphaned images with "< none >" identifiers:
docker images -q -f dangling=true | % { docker rmi $_ }
Remove orphaned images:
docker image prune -f
Remove orphaned AND UNUSED images:
WARNING: Be careful with this: It will remove generic OS images used by NavContainerHelper!
docker image prune -a -f
Get Artifact Info for an existing BC Container:
Get-BcContainerArtifactUrl containername
Get-NavContainerArtifactUrl containername
Diagnostics
Get-BcContainerEventLog
Create New BC Container
New-BcContainerWizard
https://blog.baudson.de/blog/stop-and-remove-all-docker-containers-and-images
Steve Endow is a Microsoft MVP in Los Angeles. He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, .NET, Dynamics GP, and SQL Server. You can also find him on Twitter and YouTube http://www.precipioservices.com
UPDATE: If you're interested in a convenient way to run the BcContainerHelper PowerShell commands, check out Krzysztof's Azure Data Studio Notebook with sample BcContainerHelper commands. Jupyter Notebooks are a great way to include documentation and commands in a single convenient format.
Have you used the new New-BcContainerWizard tool? If not, settle in! And I'll throw in a story about a puzzling Docker error and how Freddy helped me work through it.
Here's a video where I walk through my recent Docker learning:
Yesterday I planned on working through Erik Hougaard's video about calling an
external web API from AL and processing the JSON response.
Since I needed a new BC Docker Container to work on the demo code, I thought
it would be a good opportunity to test Freddy's new BC Container Wizard
tool. So I made sure I had the latest version of NAV Container Helper,
and typed New-BcContainerWizard.
The wizard is a really nifty PowerShell utility that asks you a series of
questions that walk you through the different options available when using the
NewBcContainer command. After I completed the questions, it provided me
with this script.
This is a really great tool. When I first started using NavContainerHelper, it seemed very confusing and I had no idea which options to include and which values made sense for my needs. This wizard resolves that and will be a great resource for people who are new to Docker.
Now that I had my nifty PowerShell script, I ran it.
Sad Trombone
"This request is not supported"
Um. What?
After a bit of googling, I eventually made my way to the NavContainerHelper GitHub repository to see if I could learn anything from the Issues list. I saw another issue that also had "CreateComputeSystem", but the specific error was different.
So I submitted a separate issue on GitHub. This was the first GitHub issue I've ever submitted, so I didn't know how or what to include. I used another issue as an example, and I stumbled through the process, including what I was trying to do, a copy of my New-BcContainer script, and as much information about the error that I had available.
Freddy responded within a few hours asking for the full build log from a successful container build (excluding the imageName parameter). I was able to attach that full build log as a text file.
Within a few minutes, he replied, explaining that the log showed I was using a very old version of windows.
The last few digits are the important ones
Sure enough, Windows Update had a few failed updates. Those failed updates had prevented ANY updates from being applied to my server. Once I cleared those errors and updated my server to the latest windows build, the New-BcContainer script worked successfully with the imageName parameter.
I then looked through the full container build log that I had submitted to the issue, and sure enough, there it was.
I'm talkin' to you!
Among the hundreds of lines that scrolled by during the build process, the warning was right there, showing that I had a very old Windows build and that the generic image didn't match.
While this did delay me by over a day in my AL learning process, this side adventure was very valuable, as it helped me understand a little bit more about Docker and the NavContainerHelper build process.
I wrote a post in April, "Make sure to update NavContainerHelper", in which I described some issues I had that were caused by having several old versions of the NavContainerHelper module installed.
Today, while updating the NavContainerHelper module on my Docker VM, I learned a few things, and I think I finally have a basic understanding of the PowerShell Module install and update process.
If you're a PowerShell pro or a NavContainerHelper pro, this will probably seem obvious, but as a newbie with both tools, it was a "learning moment" for me that explained the issues I had previously with multiple NavContainerHelper versions.
