Tuesday, September 28, 2021

Using $metadata and $expand with Business Central Web APIs

By Steve Endow

In June 2021, I discussed my circuitous journey to learn about and understand the $metadata and $expand OData "query options" and how to use them with the Business Central web API.

BC Web API OData Entity Model

Here is that video:

Today I had to research the $expand option for a BC API, but completely forgot how to use $metadata and find the related record "multiplicities" or "NavigationProperties" available for BC records exposed by APIs.

Since I had forgotten everything I discussed in my video from June, I figured it was time to write down a few notes in a blog post so that I can come back to this post in the future to refresh my memory.

The $metadata Option

When working with the Business Central web API, you can add $metadata to the end of the base API URL to get information about all of the available endpoints as well as related data elements that can be accessed using the $expand option.

For example, when working with my BC Docker Container named "dev18", I would use this URL:


When working with a BC SaaS environment:  (substitute your tenant and environment)


You can read all the gory details in the OData documentation here:


If you access the above Business Central API URL with the $metadata option, the request should return a massive pile of XML that looks something like this:

BC Web API OData Entity Model

Okay, so what are you supposed to do with those 4,000 lines of XML?

First, find the BC API object that you are working with.  For example, let's suppose we want to know which additional data elements can be expanded on the Customer object.

If you search in this XML for "customer", you'll find lots of entries.  Look for the line that says EntityType Name="customer".

This shows the customer object and the properties (fields) that are returned by the API.

If you scroll down past the Properties, you will start to see "NavigationProperty" items.

These are the "child records" or "related records" that you can include with the Customer records using the $expand option.

The $expand Option

Now that we know the NavigationProperty objects that can be retrieved for a Customer, we can use the $expand option in our API request.

You can read all the gory OData details about $expand here:


For example, if I want to retrieve a specific customer, and I want to also retrieve the Contacts associated with the customer, my request would look like this:

Sample Docker container URL:


Sample BC SaaS URL:


Using this $expand option includes contactsInformation in the customer data:

If I repeat this process for BC items, I go back to my $metadata XML and search for EntityType Name="item".  I can then see the NavigationProperty values that are available for BC items.

In the case of items, there are only 5 navigation properties available for use with $expand.  Which is a bummer, because my customer would like to retrieve item quantities by location from the BC API.  Based on this metadata, it looks like quantities by location are not available, and it will likely require a custom API.

Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, and .NET.

You can also find him on Twitter and YouTube

Thursday, June 17, 2021

Automatically Create BC Docker Container - PowerShell Script version 3

 By Steve Endow

I created this PowerShell script to automatically re-create my Business Central Docker Containers every morning.

Version 3 of the script records BC Image version information to a log file and to the notification email.

(Thanks to Kishor Mistry for the improved method to get the BC image version!)

Email notification with full log

 #v3.0 - June 17, 2021  
 #Define the host, container, and image names  
 $server = $env:COMPUTERNAME  
 $containerName = "dev18"  
 $imageName = "i" + $containerName  
 $dnsIP = ""  
 #Define date values  
 $simpleDate = Get-Date -Format "M/d/yy"  
 $dateTime = Get-Date -Format "M/d/yy HH:mm"  
 $fileDateTime = Get-Date -Format "yyyy-MM-dd HHmm"  
 $currentDate = Get-Date -Format "yyyy-MM-dd"  
 $currentTime = Get-Date -Format "hh:mm:ss"  
 #Specify file location for activity transcript log file  
 $transcriptFile = "D:\BCPowerShell\Logs\" + $fileDateTime + " " + $containerName + " Container Build Log.txt"  
 #File location for BC version tracking  
 $versionFile = "D:\BCPowerShell\Logs\" + $containerName + " BC Version Log.txt"  
 #Start recording transcript  
 Start-Transcript -Path $transcriptFile  
 #Record the start time  
 $StartTime = $(get-date)  
 #Define email configuration  
 $emailFrom = "precipioazure@gmail.com"  
 $emailTo = "steveendow@gmail.com"  
 $smtpServer = "smtp.gmail.com"  
 $port = "587"  
 #Email password file created using: Read-Host -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath D:\BCPowerShell\Gmail.securestring  
 $passwordFile = "D:\BCPowerShell\Gmail_sendow.securestring"  
 $securePassword = ConvertTo-SecureString (Get-Content -Path $passwordFile)  
 $smtpCred = New-Object -TypeName PSCredential ($emailFrom, $securePassword)  
 $subject = $server + ": " + $simpleDate + " BC Build Log for Container: " + $containerName  
 $started = "Start time: " + $StartTime  
 $body = $started + "`n"  
 $errFlag = $false  
 $warnFlag = $false  
 #Output startup info  
 Write-Output $subject  
 Write-Output $started  
 Try {  
   #Get image version from current container  
   try {   
     $currentImageNum = (Get-BcContainerAppInfo -containerName $ContainerName | Where-Object -Property Publisher -EQ "Microsoft" | Where-Object -Property Name -EQ "Application").Version.ToString()   
     $currentImageNum = 'No Container'  
   Remove-Module BcContainerHelper -ErrorAction SilentlyContinue  
   Import-Module BcContainerHelper -ErrorAction SilentlyContinue  
   #Remove old artifacts and images  
   Flush-ContainerHelperCache -cache bcartifacts -keepDays 7  
   #Remove existing container if it exists  
   Remove-BCContainer $containerName  
   #Remove old images without deleting base OS image  
   docker images --format "{{.Repository}}\t{{.Tag}}\t{{.ID}}" |    
    Select-String "businesscentral" -notMatch |   
    ConvertFrom-CSV -Delimiter "`t" -Header ("Repository","Tag","ID") |   
    Sort-Object Tag | % ID | % { docker rmi $_ }   
   #Call New-BcContainer - Use New-BCContainerWizard to generate the script/params for your environment  
   $password = 'P@ssw0rd'  
   $securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force  
   $credential = New-Object pscredential 'admin', $securePassword  
   $auth = 'UserPassword'  
   $artifactUrl = Get-BcArtifactUrl -country us -select Latest -type Sandbox  
   New-BcContainer `  
     -accept_eula `  
     -containerName $containerName `  
     -credential $credential `  
     -auth $auth `  
     -artifactUrl $artifactUrl `  
     -dns $dnsIP `  
     -updateHosts `  
     -licenseFile "D:\BCPowerShell\BCLicense\BCv18.flf" `  
     -assignPremiumPlan `  
     -includeTestToolkit `  
     -includeAL `  
     -shortcuts None `  
     -imageName $imageName  
   Publish-BcContainerApp -appFile "D:\zDisks\BCCL\BCCL_1.15.0.125\installer\Extension\Hougaard.com_Business Central Command Line_1.15.0.125.app" -containerName dev18 -install -packageType Extension -sync -syncMode Add  
   Catch {  
     #If the script fails or has an error, output to error stream  
     Write-Error "Um, something didn't go well:"  
     Write-Error $_  
     #Capture error in email body   
     $body += "`nERROR:`n`n" + $_ + "`n`n"  
     $errFlag = $true  
   Finally {  
     #Get image version info from new container  
     try {   
       $newImageNum = (Get-BcContainerAppInfo -containerName $ContainerName | Where-Object -Property Publisher -EQ "Microsoft" | Where-Object -Property Name -EQ "Application").Version.ToString()   
       $newImageNum = 'No Container'  
     #Create line to write to BC Version log file  
     $versionEntry = $currentDate.ToString() + "`t" + $currentTime.ToString() + "`t" + $currentImageNum + "`t" + $newImageNum  
     #Write entry to BC Version log file  
     Add-Content -Path $versionFile -Value $versionEntry  
     #Add BC Versions to email body  
     $body += "`n" + "Prior BC version: " + $currentImageNum  
     $body += "`n" + "New BC version: " + $newImageNum + "`n"  
     #PROTOTYPE - This is fairly fragile and needs improvement  
     #Search for the Container OS Build number and compare it to the Host OS build number  
     [string]$containerOSVersion = Select-String -Pattern "Container OS Version: " -SimpleMatch -Path $transcriptFile  
     $trimBefore = $containerOSVersion.IndexOf("Version: ")  
     #This is pretty crude parsing, so there is presumably a more elegant method  
     $containerOSBuild = $containerOSVersion.Substring($trimBefore+9, ($containerOSVersion.Length-($trimBefore+9))).Split(" (")[0].Split(".")[3]  
     $osVersion = (Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion" -Name UBR).UBR  
     If ($containerOSBuild -ne $osVersion)  
       $warnFlag = $true  
       $buildWarning = "`nWARNING: Container OS Build " + $containerOSBuild + " does NOT match Host OS Build " + $osVersion  
       Write-Error $buildWarning  
       $body += "`n" + $buildWarning + "`n"  
       $output = "Container OS Build " + $containerOSBuild + " matches Host OS Build " + $osVersion  
       Write-Output $output  
       $body += "`n" + $output + "`n"  
     ipconfig /flushdns;  
     #Record end time  
     $EndTime = $(get-date)  
     $output = "End time: " + $EndTime  
     $body += "`n" + $output   
     Write-Output $output   
     #Calculate elapsed time  
     $elapsedTime = $EndTime - $StartTime  
     $totalTime = "{0:HH:mm:ss}" -f ([datetime]$elapsedTime.Ticks)  
     $output = "Elapsed time: " + $totalTime  
     $body += "`n" + $output   
     Write-Output $output  
     $body += "`n`nFull log file is attached"   
     If ($errFlag)  
       $subject = "ERROR: " + $subject  
       $greeting = "Greetings human.`n`nIt appears there was an error building your Business Central Docker Container: " + $containerName + "`n`n"  
     elseif ($warnFlag)  
       $subject = "WARNING: " + $subject  
       $greeting = "Greetings human.`n`nI prepared your Business Central Docker Container: " + $containerName + ", but detected potential issues.`n`n"  
       $greeting = "Greetings human.`n`nI have prepared your Business Central Docker Container: " + $containerName + "`n`n"  
     $body = $greeting + $body  
     #Finish the transcript recording  
     #The Transcript must be stopped before trying to send the email,  
     #otherwise the log file will be locked, causing the email to fail  
     #Send the email  
     Send-MailMessage -From $emailFrom -To $emailTo `  
       -Subject $subject `  
       -Body $body `  
       -Attachments $transcriptFile `  
       -SmtpServer $smtpServer `  
       -Credential $smtpCred `  
       -Port $port `  

Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, and .NET.

You can also find him on Twitter and YouTube

Tuesday, June 1, 2021

Restoring data from a broken Business Central Docker Container

 By Steve Endow 

(This blog post is inspired by a forum question. Credit to Arvydas for the question and sharing the solution.)

What if I made the mistake of storing some "worthwhile" data in a Business Central Docker container over the course of several months?  Perhaps it was test data or data imports that I didn't think I would ever need--but then 3 months later I realized that the accumulated data was valuable.

In theory, no big deal.  If you realize that you want to keep the data from a working container, Stefan MaroĊ„ has a blog post with instructions on converting that Container into an Image using "docker commit".  Just create an image and you can make new containers from that image.  You could also backup the SQL databases from the Docker Container to the local file system.  

But, what if the Business Central Docker Container with your valuable data stops working due to a Windows Update?  And you only realize you need that data after the container stops working.  If the Container can't be started at all, how can you extract the Business Central data?

This solution appears to allow me to copy the SQL mdf and ldf files from the broken container to my local file system, then copy the mdf and ldf files to a new working container to effectively "restore" them.

This may not work for all situations where a BC container has issues, but should work if the container is intact, but just won't start up due to a Docker or Windows issue or error.  

Let me know if there are any issues or caveats with using this method to restore the data from a container.

Sunday, March 14, 2021

Software Algorithms to Solve Messy Data Problems

 By Steve Endow

In the software / consulting world, we're sometimes given data that is messy, and we, the software developers, consultants, or report writers, are tasked with making sense of that data and using it in a business process.  Sometimes we have to do "creative" and often unintuitive things to that data to make it usable.

For example, I was once asked to import employee time clock data into a payroll system.  Simple, right?  In the ideal world, an employee clocks in for work in the morning, and clocks out for work in the afternoon.  But in the real world, it's never this easy.

First, the customer's horrible timeclock system did not differentiate between a "clock in" or a "clock out".  I was simply given two fields:  Employee ID and Date Time.  It was up to me to figure out which entries were "clock in" and which entries were "clock out".

Second, sometimes the employees would take breaks or leave for lunch. They would therefore "clock out" when leaving, and "clock in" when returning to work. So an employee could have up to four records for a single shift.

Sunday, February 21, 2021

Automated PowerShell Script v2.1 to Recreate My Business Central Docker Containers

 By Steve Endow

UPDATE:  I have posted version 3 of the script here

In July 2020 I posted an earlier version of this script.  This updated version has a few refinements, such as removing and importing BcContainerHelper to ensure the latest version is being used, adding Flush-ContainerHelperCache to remove old helper files, and removing old Docker images.

Automate, Automate, Automate

I setup a "Dev Container" task in Windows Task Scheduler to run this script every morning at 6:45am so that I always have a fresh BC container based on the latest BC artifacts.  I call my main Container "dev17", based on the current 17.x version of Business Central.  Do NOT name your container just "dev"--see my video here on why that will cause problems.

Tuesday, February 16, 2021

Changing the network location from Public to Private on Windows Server

By Steve Endow

This is just a post for me to store this information for Future Steve, as he will most certainly need it, just as Present Steve has needed it a dozen times over the last year.

Any time I replace my router, replace a piece of Ubiquiti network gear, or reconfigure my network, all of my Windows Server VMs decide they've been kidnapped and change their network location to Public network.

This blocks all traffic on the servers until I can login directly to each server and fix the problem.

Which requires an annoying process using gpedit or PowerShell, which I can never remember after 6 months.

So here is the post with the information to fix the problem:


Using the GUI:

  1. Hit Winkey + R to open Run prompt and type gpedit.msc
  2. Navigate to: Computer Configuration/Windows Settings/Security Setting /Network List Manager Policies.
  3. Choose your Network name in the right pane.
    Note: To show networks not currently connected, right-click Network List Manager Policies in the left pane and choose Show All Networks.
  4. Go to Network Location tab and change the Location type from Public to Private.

Using PowerShell...

One line version:

Set-NetConnectionProfile -InterfaceAlias Ethernet -NetworkCategory Private

Multi-line version:

PS C:\>$Profile = Get-NetConnectionProfile -InterfaceAlias Ethernet1

PS C:\>$Profile.NetworkCategory = "Private"

PS C:\>Set-NetConnectionProfile -InputObject $Profile

Steve Endow is a Microsoft MVP in Los Angeles.  He works with Dynamics 365 Business Central, Microsoft Power Automate, Power Apps, Azure, and .NET.

You can also find him on Twitter and YouTube

Friday, February 12, 2021

Creating a simple text file using AL in Business Central - Sample Code

 By Steve Endow

On January 23, 2021, I recorded a video showing how to create a simple text file using AL in Business Central.

Here is the video:


Below is the sample code from that video.

Filter by Date Range with Business Central Web API v2.0 - OData date filter syntax

by Steve Endow I just spent 10 minutes trying to figure out the proper syntax for date filtering with Business Central Web API v2.0 endpoint...