Troubleshoot rogue PowerShell processes running from C:\Windows\CCM\SystemTemp

A new customer called us in to troubleshoot an issue where multiple PowerShell processes would gradually appear. This was impacting CPU performance, with each process listed in the top handful of processes for CPU use. These processes would not stop unless manually terminated.

Rogue PowerShell services could also be a sign of security breach, so it was a critical issue to resolve.

Here’s a rundown of our troubleshooting process, since we saw many forum posts reporting similar issues over past years, with no easy way of identifying the source.

Step 1. Investigate with ProcMon

1. Download Microsoft/Sysinternals Process Monitor, and launch.
2. Filtering on powershell.exe we didn’t see any activity.
3. Run Tools > Process Tree
4. Click on the PowerShell processes, and get the command line that’s displayed.
   This view is helpful because it also shows when the process was started, along with parent/child processes. We could see no parents for our powershell.exe’s. We could see that the processes were being launched approximately a week apart.

We couldn’t find the source script files that were being launched. We could see a GUID-like filename with the path C:\Windows\CCM\SystemTemp. So we knew it was likely a ConfigMgr task. But what script was it?

Step 2. Search CCM logs

We found a helpful reddit post that suggested using Findstr against the ConfigMgr logs to identify what component was triggering the script. Unfortunately on our sample machines this turned no results either.

1. Open a command prompt as Admin

2. cd c:\windows\ccm\logs

3. FINDSTR /i /s /m "dd3d5a73" *.*

4. Use CMTrace to examine the log files listed.

Step 3. Search Event logs

We found some useful logs in Event Viewer that were a big help.

1. Open Event Viewer
2. Browse to Applications and Services > Microsoft > Windows PowerShell > Operational
3. Review events around the process start times noted with ProcMon in Step 1

We saw some 4104 Warning events for “Executing a Remote Command”. The details include the full script content along with the file path. So we could confirm these are the processes that were still running, and we could verify the script content.

Step 4. Find the offending script

Now back to the ConfigMgr/SCCM Console. Having just been scanning ConfigMgr for any problems, we recognised the PowerShell script shown in Event Viewer. We were able to confirm that it matched a Configuration Baseline / Item that had been in use for some time. We disabled that Configuration Baseline so that it could be revised/tested or decommissioned.