I wanted to share the resolution to a crazy hair-pulling issue I’ve been having since late mid-July with Microsoft Support unable to determine the cause.

We had been using PXE booting from our locations for a while now. Got everything setup by our infrastructure teams with IP Helpers, no DHCP scope options. I tuned the registry settings for RamDiskTFTPBlockSize and RamDiskTFTPWindowSize for our VPN tunnels. Everything was working great for months.

Then all of a sudden PXE booting stopped working… But only for UEFI devices… Legacy clients were still booting correctly.

Error:

File: \Boot\BCD
Status: 0xc000000f
Info: The Boot Configuration Data for you PC is missing or contains errors.

Everything still worked great at our local office. So we figured something had changed on the networking side. But then we thought to test booting an unknown device and UEFI PXE works… Huh? Since Unknown Computers work, that basically rules out Networking. From any subnet that has to traverse a WAN connection to an ConfigMgr PXE server, UEFI PXE booting fails for computers that are in the ConfigMgr database. ‘Unknown’ computers boot fine. Legacy PXE works from all scenarios.

Computer Type	Client Location	Server Location	UEFI PXE	Legacy PXE
Known Computer	Local Office	Local Data Center	Pass	Pass
Known Computer	Local Office	Remote Data Center	Fail	Pass
Known Computer	Remote Office	Local Data Center	Fail	Pass
Known Computer	Remote Office	Remote Data Center	Fail	Pass
Unknown Computer	Local Office	Local Data Center	Pass	Pass
Unknown Computer	Local Office	Remote Data Center	Pass	Pass
Unknown Computer	Remote Office	Local Data Center	Pass	Pass
Unknown Computer	Remote Office	Remote Data Center	Pass	Pass

I tried every bit of advice the internet had to offer:

• DHCP options 60, 66, 67 are NOT set on the subnet(s)
• IP Helpers are in place on the router config to forward DHCP broadcasts
• TFTP Options have been set to ensure that the block size is small enough for our VPN tunnels and prevent fragmentation
  • [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP]
    • “RamDiskTFTPBlockSize”=dword:00000550 (1360)
    • “RamDiskTFTPWindowSize”=dword:00000004 (4)
  • [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WDSServer\Providers\WDSTFTP]
    • “MaximumBlockSize”=dword:00000550 (1360)
• Tried different task sequences with difference boot images.
• Tried updating boot images
• Disabled and re-enabled PXE on the DPs to rebuild the PXE role.

I asked for some help from the networking team and I was able to get a router setup like one of our locations with a mirrored port.

I did some Wireshark captures from the client. The only difference in the Wireshark captures from a working and non-working client is the DCHP ACK contains option 243 and option 252 with the BCD file path:

Non-working client:

Working client:

So what gives?

After looking at the SMSPXE.log though I noticed that there were two attempts made to update the Device Cached LastAdv Time. On clients that were failing to boot, the ‘Client boot action reply’ found no advertisements:

So this got me thinking, maybe there was some sort of timeout happening after the first time the Device Cached LastAdv Time was updated.

I starting to poke around the ConfigMgr DB at the stored procedures, filtered down to ones with just ‘PXE’ in the name.

In the SQL stored procedure dbo.NBS_GetPXEBootAction I noticed the following SELECT statement:

select * from LastPXEAdvertisement as lpa where lpa.MAC_Addresses = @MACAddress AND lpa.LastPXEAdvertisementID = sp.OfferID
AND LastPXEAdvertisementTime < DATEADD(SECOND, -40, @CurrentTimeInUTC

Based on this, it look like ConfigMgr is only allowing up to 40 seconds for a device to complete the download of smsboot\x64\wdsmgfw.efi. Since our locations take longer than 40 seconds to download the NBP, they would then fail to show up in this query when the second request for this device is made, therefore breaking the boot process.

So I tried changing -40 to -120, as the slowest location we have takes about 90 seconds to download this file:

select * from LastPXEAdvertisement as lpa where lpa.MAC_Addresses = @MACAddress AND lpa.LastPXEAdvertisementID = sp.OfferID
AND LastPXEAdvertisementTime < DATEADD(SECOND, -120, @CurrentTimeInUTC

Computers are now UEFI PXE booting as expected!

It’s still possible that something changed on the networking / server side that’s causing these initial downloads to be slower, but at this time, I’m going to say…