___________________________________________________________________________________________________________________________
IMPORTANT ANNOUNCEMENT FOR OUR READERS!
AskPFEPlat is in the process of a transformation to the new Core Infrastructure and Security TechCommunity, and will be moving by the end of March 2019 to our new home at https://aka.ms/CISTechComm (hosted at https://techcommunity.microsoft.com). Please bear with us while we are still under construction!
We will continue bringing you the same great content, from the same great contributors, on our new platform. Until then, you can access our new content on either https://aka.ms/askpfeplat as you do today, or at our new site https://aka.ms/CISTechComm. Please feel free to update your bookmarks accordingly!
Why are we doing this? Simple really; we are looking to expand our team internally in order to provide you even more great content, as well as take on a more proactive role in the future with our readers (more to come on that later)! Since our team encompasses many more roles than Premier Field Engineers these days, we felt it was also time we reflected that initial expansion.
If you have never visited the TechCommunity site, it can be found at https://techcommunity.microsoft.com. On the TechCommunity site, you will find numerous technical communities across many topics, which include discussion areas, along with blog content.
NOTE: In addition to the AskPFEPlat-to-Core Infrastructure and Security transformation, Premier Field Engineers from all technology areas will be working together to expand the TechCommunity site even further, joining together in the technology agnostic Premier Field Engineering TechCommunity (along with Core Infrastructure and Security), which can be found at https://aka.ms/PFETechComm!
As always, thank you for continuing to read the Core Infrastructure and Security (AskPFEPlat) blog, and we look forward to providing you more great content well into the future!
__________________________________________________________________________________________________________________________
Hi this is Michael Koeppl from the Support for Mission Critical team again. This time I wanted to walk you through a recent troubleshooting scenario I had at my customer.
14 out of 2150 Server 2016 systems reported and error when my customer tried to install the latest monthly rollup hotfix.
Normally, to be honest with you, we wouldn’t even bother as the number of systems is so low that it can almost fly under the radar. The Customer usually re-installs the machines in question but this time as all of them showed the same error they send me the list of affected machines and I started to investigate.
As I knew there’s an issue with Patch installation, I instantly turned to DISM an checked the CBS Store for any kind of corruption:
C:>dism /online /cleanup-image /scanhealth
Deployment Image Servicing and Management tool
Version: 10.0.14393.0
Image Version: 10.0.14393.2457
[==========================100.0%==========================] No component store corruption detected.
The operation completed successfully.
Repair a Windows Image
https://docs.microsoft.com/de-de/windows-hardware/manufacture/desktop/repair-a-windows-image
Okay no joy, what next?
I manually tried to install the January rollup (KB4480977) as this is the one my customer tried to install as well. For a list of all available monthly rollup packages visit: https://support.microsoft.com/en-us/help/4000825 for S2016
Here’s the error message we were getting
Log Name: Setup
Source: Microsoft-Windows-WUSA
Date: 26.03.2019 12:47:41
Event ID: 3
Task Category: None
Level: Error
Keywords:
User: DomainAdminUser
Computer: MyPC.domain.de
Description:
Windows update “Update for Windows (KB4480977)” could not be installed because of error 2147956481 “The referenced assembly could not be found.” (Command line: “”C:Windowssystem32wusa.exe” “C:tmpwindows10.0-kb4480977-x64.msu” “)
Event Xml:
<Event xmlns=”http://schemas.microsoft.com/win/2004/08/events/event“>
<System>
<Provider Name=”Microsoft-Windows-WUSA” Guid=”{09608C12-C1DA-4104-A6FE-B959CF57560A}” />
<EventID>3</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime=”2019-03-26T11:47:41.355709400Z” />
<EventRecordID>109</EventRecordID>
<Correlation />
<Execution ProcessID=”4824″ ThreadID=”9044″ />
<Channel>Setup</Channel>
<Computer>MyPC.domain.de</Computer>
<Security UserID=”S-1-5-21-4188125556-1267690402-888888790447-0000″ />
</System>
<EventData>
<Data Name=”UpdateTitle”>”Update for Windows (KB4480977)”</Data>
<Data Name=”ErrorCode”>2147956481</Data>
<Data Name=”ErrorString”>The referenced assembly could not be found.</Data>
<Data Name=”CommandLine”>”C:Windowssystem32wusa.exe” “C:tmpwindows10.0-kb4480977-x64.msu” </Data>
</EventData>
</Event>
Okay, let’s have a look at the CBS.log file which can be found at windowslogscbs
2019-03-26 12:40:12, Error CSI 0000001a@2019/3/26:11:40:12.185 (F) onecorebasewcpcomponentstorecsd_locking.cpp(200): Error STATUS_SXS_ASSEMBLY_MISSING originated in function CCSDirectTransaction::LockComponent expression: (null)
[gle=0x80004005]
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCBS.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190326072401.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190324034222.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190324001230.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190323234238.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190323230433.log to WER report.
2019-03-26 12:40:12, Info CBS Not able to add pending.xml to Windows Error Report. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Info CBS Not able to add pending.xml.bad to Windows Error Report. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Info CBS Not able to add poqexec.log to Windows Error Report. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Error CSI 0000001b (F) STATUS_SXS_ASSEMBLY_MISSING #1760302# from CCSDirectTransaction::OperateEnding at index 0 of 1 operations, disposition 2[gle=0xd015000c]
2019-03-26 12:40:12, Error CSI 0000001c (F) HRESULT_FROM_WIN32(ERROR_SXS_ASSEMBLY_MISSING) #1760150# from Windows::ServicingAPI::CCSITransaction::ICSITransaction_PinDeployment(Flags = 0, a = cd4484f3ed27edafc9d99e613bb62654, version 10.0.14393.2457, arch amd64, nonSxS, pkt {l:8 b:31bf3856ad364e35}, cb = (null), s = (null), rid = ‘Package_7763_for_KB4462917~31bf3856ad364e35~amd64~~10.0.1.7.4462917-16739_neutral’, rah = ‘3’, manpath = (null), catpath = (null), ed = 0, disp = 0)[gle=0x80073701]
2019-03-26 12:40:12, Info CBS Failed to pin deployment while resolving Update: Package_7763_for_KB4462917~31bf3856ad364e35~amd64~~10.0.1.7.4462917-16739_neutral from file: (null) [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
2019-03-26 12:40:12, Info CBS Failed to bulk stage deployment manifest and pin deployment for package:Package_9131_for_KB4480977~31bf3856ad364e35~amd64~~10.0.1.7 [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
2019-03-26 12:40:12, Info CBS CommitPackagesState: Started persisting state of packages
2019-03-26 12:40:12, Info CBS CommitPackagesState: Completed persisting state of packages
2019-03-26 12:40:12, Info CSI 0000001d@2019/3/26:11:40:12.294 CSI Transaction @0x27526657710 destroyed
2019-03-26 12:40:12, Info CBS Perf: Resolve chain complete.
2019-03-26 12:40:12, Info CBS Failed to resolve execution chain. [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
2019-03-26 12:40:12, Error CBS Failed to process single phase execution. [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
2019-03-26 12:40:12, Info CBS WER: Generating failure report for package: Package_for_RollupFix~31bf3856ad364e35~amd64~~14393.2759.1.7, status: 0x80073701, failure source: Resolve, start state: Absent, target state: Installed, client id: WindowsUpdateAgent
2019-03-26 12:40:12, Info CBS Not able to query DisableWerReporting flag. Assuming not set… [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCBS.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190326072401.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190324034222.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190324001230.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190323234238.log to WER report.
2019-03-26 12:40:12, Info CBS Added C:WindowsLogsCBSCbsPersist_20190323230433.log to WER report.
2019-03-26 12:40:12, Info CBS Not able to add %windir%winsxspoqexec.log to WER report. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Info CBS Not able to add %windir%winsxspending.xml to WER report. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Info CBS Not able to add %windir%winsxspending.xml.bad to WER report. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:12, Info CBS Reboot mark cleared
2019-03-26 12:40:12, Info CBS Winlogon: Simplifying Winlogon CreateSession notifications
2019-03-26 12:40:12, Info CBS Winlogon: Deregistering for CreateSession notifications
2019-03-26 12:40:12, Info CBS FinalCommitPackagesState: Started persisting state of packages
2019-03-26 12:40:12, Info CBS Reporting package change for package: Package_for_RollupFix~31bf3856ad364e35~amd64~~14393.2759.1.7, current: Absent, pending: Default, start: Absent, applicable: Installed, target: Installed, limit: Installed, hotpatch status: StillGoing, status: 0x0, failure source: Resolve, reboot required: False, client id: WindowsUpdateAgent, initiated offline: False, execution sequence: 161, first merged sequence: 161, reboot reason: REBOOT_NOT_REQUIRED, RM App session: -1, RM App name: N/A, FileName in use: N/A, release type: Update, release quality: final, OC operation: False, download source: 0, download time (secs): 4294967295, download status: 0x0 (S_OK), Express download: False, Download Size: 0
2019-03-26 12:40:13, Info CBS SQM: Package change report datapoints not populated because SQM is not initialized or not running online.
2019-03-26 12:40:13, Info CBS Reporting package change completion for package: Package_for_RollupFix~31bf3856ad364e35~amd64~~14393.2759.1.7, current: Absent, original: Absent, target: Installed, status: 0x80073701, failure source: Resolve, failure details: “(null)”, client id: WindowsUpdateAgent, initiated offline: False, execution sequence: 161, first merged sequence: 161, pending decision: InteractiveInstallFailed, primitive execution context: Interactive Flight: False
2019-03-26 12:40:13, Info CBS The store corruption status report is incomplete. [HRESULT = 0x80070002 – ERROR_FILE_NOT_FOUND]
2019-03-26 12:40:13, Info CBS Resolve time performance datapoint is invalid. [HRESULT = 0x80070490 – ERROR_NOT_FOUND]
2019-03-26 12:40:13, Info CBS Stage time performance datapoint is invalid. [HRESULT = 0x80070490 – ERROR_NOT_FOUND]
2019-03-26 12:40:13, Info CBS Execute time performance datapoint is invalid. [HRESULT = 0x80070490 – ERROR_NOT_FOUND]
2019-03-26 12:40:13, Info CBS SQM: Package change report datapoints not populated because SQM is not initialized or not running online.
2019-03-26 12:40:13, Info CBS FinalCommitPackagesState: Completed persisting state of packages
2019-03-26 12:40:13, Info CBS Enabling LKG boot option
2019-03-26 12:40:13, Info CBS Exec: Processing complete. Session: 30729156_3451904436, Package: Package_for_RollupFix~31bf3856ad364e35~amd64~~14393.2759.1.7 [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
2019-03-26 12:43:05, Info CBS Trusted Installer is shutting down because: SHUTDOWN_REASON_AUTOSTOP
2019-03-26 12:43:05, Info CBS TiWorker signaled for shutdown, going to exit.
2019-03-26 12:43:05, Info CBS CbsCoreFinalize: ExecutionEngineFinalize
2019-03-26 12:43:05, Info CBS Ending the TiWorker main loop.
Ohhhhh, okay well…..so lot’s of ASSEMBLY errors as well. Well, that at least matches what we saw in the event log …but where to start?
It’s particularly hard to read here as we do have more line breaks as in the original cbs.log file but I’ve marked the interesting things above.
Look at the package name / number we tried to install
Failed to bulk stage deployment manifest and pin deployment for package:Package_9131_for_KB4480977~31bf3856ad364e35~amd64~~10.0.1.7 [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
This is the January KB we tried to install (KB4480977) and the error is ERROR_SXS_ASSEMBLY_MISSING
Okay, but can you spot an identical error just before?
2019-03-26 12:40:12, Error CSI 0000001b (F) STATUS_SXS_ASSEMBLY_MISSING #1760302# from CCSDirectTransaction::OperateEnding at index 0 of 1 operations, disposition 2[gle=0xd015000c]
2019-03-26 12:40:12, Error CSI 0000001c (F) HRESULT_FROM_WIN32(ERROR_SXS_ASSEMBLY_MISSING) #1760150# from Windows::ServicingAPI::CCSITransaction::ICSITransaction_PinDeployment(Flags = 0, a = cd4484f3ed27edafc9d99e613bb62654, version 10.0.14393.2457, arch amd64, nonSxS, pkt {l:8 b:31bf3856ad364e35}, cb = (null), s = (null), rid = ‘Package_7763_for_KB4462917~31bf3856ad364e35~amd64~~10.0.1.7.4462917-16739_neutral’, rah = ‘3’, manpath = (null), catpath = (null), ed = 0, disp = 0)[gle=0x80073701]
2019-03-26 12:40:12, Info CBS Failed to pin deployment while resolving Update: Package_7763_for_KB4462917~31bf3856ad364e35~amd64~~10.0.1.7.4462917-16739_neutral from file: (null) [HRESULT = 0x80073701 – ERROR_SXS_ASSEMBLY_MISSING]
As you can see, the very same error is reported for an already installed patch, in this case the October 2018 (KB4462917) rollup.
The solution for those of you who are eager to know was to uninstall the patch which reported the error first – KB4462917 this can be done either through the control panel à view installed updates à uninstall or in our case we used DISM again to automate this.
1st get a list of all installed packages
dism /online /get-packages /format:table
2nd from this list find the package name for the one you want to uninstall
DISM.exe /Online /Remove-Package /PackageName:Package_for_KB2870699~31bf3856ad364e35~amd64~~6.2.1.1 /quiet /norestart
After that we were able to install any of the monthly rollup patches again without any issues.
Other related but not purposeful geek trivia for you to know.
I first had the idea that just some RegKey is missing or corrupt, so I checked the referenced key from the CBS.log.
Registry :
HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionComponent Based ServicingPackagesPackage_7763_for_KB4462917~31bf3856ad364e35~amd64~~10.0.1.7
As you can see the LastError “0x80073701” is matching with the one from the CBS.log and the CurrentState which is 112 dec means installed.
| Applicable/Current State |
Hex |
Dec |
| Absent | 0 | 0 |
| Uninstall Pending | 0x5 | 5 |
| Resolving | 0x10 | 16 |
| Resolved | 0x20 | 32 |
| Staging | 0x30 | 48 |
| Staged | 0x40 | 64 |
| Superseded | 0x50 | 80 |
| Install Pending | 0x60 | 96 |
| Partially Installed | 0x65 | 101 |
| Installed | 0x70 | 112 |
| Permanent | 0x80 | 128 |
For a complete list and more information’s visit https://blogs.technet.microsoft.com/tip_of_the_day/2015/10/12/tip-of-the-day-cbs-servicing-states-chart-refresher/
This didn’t help at the end and the only one and proper solution is to remove the patch which is reporting the assembly error first and either re-install that one or if available a newer one.
Thanks for reading
Michael