Update global Crowdstrike outage | News item

News item | 19-07-2024 | 10:18

The workaround that CrowdStrike shared this morning seems to work in most cases. As a result, the problems at organizations are starting to decrease. The expectation is that many organizations will be able to fully restore their systems during the course of the day. The NCSC will continue to monitor the situation and share updates via this page.

Update: 16:52 19/07/24

This morning it became clear that an update of CrowdStrike agent causes technical problems in systems at organizations. The problems are worldwide and also in the Netherlands several sectors are affected.

What we know now

• The issues caused by the CrowdStrike agent update only affect Windows systems. Linux and Mac systems are not affected.
• Windows systems that came online following 05:27 UTC this morning (19/07/2024) are not affected.
• The NCSC has now understood from many organizations that the workaround offered by Crowdstrike has been effective and systems are slowly becoming operational once more. In some cases, a variant of the workaround is required. We share the variants known to us below in the adjusted action perspective. The NCSC can confirm with reasonable certainty that executing this workaround will not affect the operation of the Crowdstrike agent.

Action perspective

The version causing problems is: Channel file ‘C-00000291*.sys’ with timestamp ‘0409 UTC’. Versions of this file with a timestamp of ‘0527 UTC’ or later are known to be good versions.

• For systems that have performed the update, first try restarting the system to automatically download a new version of the channel file.
• If this does not work and the system enters a ‘loop crash’, Crowdstrike recommends taking the following steps to perform a manual intervention:

  1. Boot Windows into Safe Mode
  2. Navigate to C:WindowsSystem32driversCrowdStrike directory in Explorer
  3. Locate file “C-00000291*.sys” file, click right mouse button and delete the file or rename it to “C-00000291*.renamed”
  4. Boot the host

In some cases, a variant of the workaround is needed. The variants known to us are shared below:

• For physical laptop/desktop systems with BIOS storage set to ‘RAID’, this setting may first need to be changed from ‘RAID’ to ‘AHCI/NVMe’ before the C: drive will be visible in Safe Mode.
• If Bitlocker is used, the following actions can be taken prior to the workaround described above. Local admin rights on the host are required:

Use the ‘advanced restart options’ to open a command prompt.

Skip the request for a bitlocker key when prompted.

Run het commando: ‘bcdedit /set {default} safeboot minimal’.

The system should now boot into Safe Mode and you can follow the steps above to rename the .sys file in question.

After this the system will boot into Safe Mode following a restart and ‘msconfig’ should be run.

On the boot tab, safeboot needs to be turned off once more.

Finally, the system must be rebooted to exit Safe Mode.

• Amazon has a action perspective published for users of AWS Windows EC2 Instances or Windows Workspaces.
• Microsoft has action perspective published for users of Windows VMs running on Azure.[3]
• Google has action perspective published for users of Windows VMs in Google Cloud.

The NCSC will continue to monitor the situation and will keep you informed of further updates.