Introduction
Recently I performed an upgrade of an SCCM site from 2103 to 2107 and the organization also upgraded the site systems' operating systems from Windows Server 2012 to Windows Server 2019. And this went well but with one issue. A number of servers did not upgrade the client agent - approximately 25% of all servers were in this state.
I advised the operations engineer to manually uninstall the old client and reinstall the new client. This failed using the client push installation method. It did work by copying the client source files locally, running ccmsetup.exe /uninstall and then running ccmsetup.exe smssitecode=xxx from the source files directory.
But although the new client installed it would not communicate with either of the two management points.
The client CcmMessaging.log file showed a number of errors such as
Post to https://xxxx.xxx.xxx/ccm_system/request failed with 0x87d00231. CcmMessaging 28/10/2021 10:01:51 17280 (0x4380)
and
Failed in WinHttpSendRequest API, ErrorCode = 0x2efe
and
Post to https://xxx.xxx.xxx/ccm_system_windowsauth/request failed with 0x87d00231.
In the SCCM console these devices had a strange ? icon next to their entry in the Devices node.
The Things I Tried
I won't turn this post into a final year dissertation by detailing everything I tried to get the SCCM client communicating with the management points. But in summary:
1) Attempted every suggestion in every related google article returned from a search of any of these error return codes.
2) Requested the networks team to investigate the possibility of any network related issues that might be blocking the flow of traffic.
3) Re-installed the Management Points.
4) Re-installed IIS and the Management Points.
5) Ensured .net 4.8 was installed on the Management Points.
6) Reinstalled the certificates - client and Management Points.
7) Repaired the WMI repository on a failing client.
8) Checked the DCOM configuration on a failing client.
9) Tried a TCP/IP reset on a failing client.
10) Disabled Microsoft Endpoint Protection anti-virus scanning.
11) Analysed group policy settings for differences between a communicating and non communicating client.
12) Ensured BITS service was running.
13) Checked permissions on the IIS virtual directories
14) Ensured webdav was enabled on the Management Point IIS configuration.
15) Scanned the IIS logs for clues - the failing client's IP addresses did not appear in these logs.
16) And many many more things.
Nothing would make these server SCCM clients register or communicate with the Management Points and therefore they were, for all real purposes, orphaned and unmanaged.
It was a very difficult problem and I did get some inspiration from Churchill's famous quote - Never give in. Never give in. Never, never, never
The IIS Crypto 3.2 Tool Breakthrough
One change that had taken place in this organisation, previously and before I began providing this organisation with my services, was the disabling of the TLS 1.1 and 1.0 protocols - and enabling the TLS 1.2 protocol.
I did check the SCHANNEL key under: HKLM\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL
I found no differences between a working SCCM client and a failing SCCM client.
Finally I downloaded the IIS Crypto 3.2 tool from Nartac.
https://www.nartac.com/Products/IISCrypto/Download
I ran this tool on a working server and a failing server and it showed no differences on the Schannel blade.
I then clicked on the Cipher Suites blade and I did notice that a failing server had a number of Cipher Suites not selected - Cipher Suites that were selected on the working servers.
Namely, the following Cipher Suites were not enabled on the failing servers.
And so I did enable these cipher suites, clicked on apply and rebooted the server. And after the reboot everything started working, and the client registered itself correctly in the SCCM database and was now fully manageable from within the SCCM console.
