There are times as a security professional you have to roll up the sleeves and get your hands dirty to make sure some of the basics are applied to the environment we’re looking after. As a common example, most of us have had to patch the odd Windows machine, or three, to help out a friend to make sure they’re safe and up to date from the various nasties out there.
 

What happens when you’re presented with forty seven Windows XP computers: all networked, in a Windows workgroup, have varying levels of patches installed, hardly any internet connectivity and a limited time frame to get them to a current patch level? Now throw in every machine is infected and the infections is causing embarrassing and crippling problems to the users.

Here’s my solution; if you have a better one, or helpful pointers, feel free to comment.

- Assess the situation, explain the discovered risks to the business and come up with a plan of attack.

Uncovered background on the problem

• Tech support for the network is one poor soul that “is good with computers” but it’s not their primary job
• The same antivirus software (AV) was on all machines, but the definitions were totally out of date
• The admin password for all of the machines is the same
• Switched fast Ethernet network linked the machines
• The internet link was very expensive, very limited and only used for email
• The machines were riddled with malware -  Conficker being the most obvious
• The file server was another XP machine used to store all the data
• This is a favour to a friend so no budget and had to be done over an evening
• Business owner had signed off on the risk of patching everything in one go
• These machine were on four different floors (lots of running around), but only one network
• Random application software installed
• Did I mention no budget for anything IT, including support or training

Leaving aside the bigger picture of no security policies or procedures and a total lack of fundamental IT management, it was important to get to a measured and consistence baseline where the users could actually work.

The aim was to bring a standard, baseline patch level to the Windows machines, avoid full rebuilds and purge the main malware problems.

Here's my quick five steps :

1. Back up the file server, verifying data copy is malware-free and valid
2. Get the current service pack installed
3. Get all the current hotfixes installed
4. Check that all the machines are patched
5. Get all machines to the current AV definition level, scan and clean any malicious activity on the machines

Faced with a very hostile network, trusting one of the existing machines was not any option.

Thankfully virtualisation provides a great option to plug in a machine and dispose/revert it if this type of situation arises.  I also happen to have a Windows server virtual machine (VM) with Windows Server Update Service (WSUS) [5] installed on my laptop, that had been recently synced with the latest updates.*
 

Step 1: Backup the critical data off the XP file server is a copy an external USB drive

The drive was then plugged in to secured machine with a current AV. The autorun nasties were removed from the drive, the data was scanned and cleaned from all know problems. Then someone from the company confirmed the data was good.
 

Step 2: Create a share on the virtual machine for XP SP3 and deploy it to all machines

The wonderful PSEXEC [1] comes to the rescue as it can be used to deploy and execute the SP3 patch from the VM's share. Smarter scripting techniques [2] with PSEXEC mean you can automate this process for deployment.
As an example, this command copies SP3 to the target machine, then silently starts the installation and forces a reboot once the SP3 has been installed.
psexec computer  -c -f -s servershareWindowsXP-KB936929-SP3-x86-ENU.exe / quiet /forcerestart

Step 3: Deploy all current patches

Having a WSUS server as a virtual machine means fast, portable patch management with reporting. Using PSEXEC to deploy registry keys [3] to point all the XP machines to my WSUS VM, this forced all the machines to register and download the current updates. This proved a log of all the machines that connected, and what Windows patch level they were at.
 

Step 4: Check that all the machines are patched

Microsoft’s free tool Microsoft Baseline Security Analyzer (MBSA) [5] is a quick and effective way to verify that all the machines are up to the correct patch level, as it can reference the portable WSUS server at the patch baseline for each machine.
 

Step 5: Update AV and force a full scan

Copy the current AV definition to the same share as SP3. PSEXEC strikes again to copy to each machine and a simple batch script to kick of a full scan and confirm the scan has run successfully by dumping the results to the VM's share. This allows for a quick and easy way to check every machine has run AV, plus see what the AV detected on the machines.

End Result

This got the all the machines to the same consistent state and removed the problematic malware - all over one long evening.

Notes:

• If you do use the WSUS trick, remember to remove the registry keys afterward; using PSEXEC with a batch script will to do this nicely .
• WSUS could have been used to deploy SP3, but I find forcing patch deployment with PSEXEC is a lot faster. Had there only been a few patches missing, a WSUS server is somewhat over kill. A simple batch file utilizing hotfix chaining [4] and PSEXEC is a very fast way to deploy patches remotely.

[1] http://technet.microsoft.com/en-us/sysinternals/bb897553
[2] http://ss64.com/nt/psexec.html
[3] http://support.microsoft.com/kb/328010
[4] http://support.microsoft.com/default.aspx?scid=kb;en-us;296861
[5] http://technet.microsoft.com/en-us/security/cc184924
[6] http://technet.microsoft.com/en-us/windowsserver/bb332157
 

*Doesn’t everyone? Well if you’re building machines and travelling to places with poor internet access all the time, it makes patching a darn sight easier!
 

Chris Mohan --- Internet Storm Center Handler on Duty