10 Indispensa
Some days you control the NOC. Some days when the NOC controls you.
For the bad days, there are tools to turn them around, or to keep an eye on ops to minimize life’s stress. We’ve used these tools in the lab regularly and cannot imagine running a Windows operation center without them. (Also see our companion piece on similar tools for Linux admins.)
One of the tools is based on Linux, but you’ll see in a moment why we keep this tool around: It’s a lifesaver.
SysRescCD.ISO
SysRescCD.ISO can make you a hero, or a heroine, or a conspirator, as it’s the System Rescue CD for Windows, used to rescue Windows systems where the password has somehow been forgotten or mangled. Download the file and burn it to a CD or, better still, a USB drive.
Make sure you can legally use this file. This tool specifically allows you to replace the administrator/most-privileged user account password. You may need very specific permission to do this; laws on password cracking vary from region to region. Check yours first.
Boot the password-rejected machine with the CD or USB drive. The instructions are inside the ISO download, or you can refer to its website for the docs on how to rescue everything from Windows 2000, Windows XP, to Windows 2003, and interim editions.
You won’t need this tool often, but it’s a help-desk must-have. Inside the NOC, where things are busy and documentation can be forgetful, it’s the only way to crack the administrative password. Keep this secret, and remember that nothing protects a machine if a user – you – has physical access to it.
WireShark
How many times has WireShark been able to pull me out of the drink? Plenty. Here’s what it does: It listens to promiscuous Ethernet devices (Wi-Fi, too) and assembles packets. You can export them, sort them, pair conversations, and ultimately sniff the wire and air. WireShark is a sniffing tool that lets you find all of the traffic, which can be overwhelming, so that you can minimize the captured packet volume to something manageable.
There are plenty of tricks to WireShark and other packet-trace tools. Most Layer 2/3 switches allow you to mirror ports, so that you can capture traffic on segments that you’re not logically connected to. (Although the wisdom of port reflection is up to the astute user. Meaning: Be careful.)
With WireShark, you can diagnose and verify the fix for numerous server maladies, ranging from understanding server traffic overloads to decoding packet and segment-related error messages at the protocol (rather than stupid operating system) level. Then you can do things like whip DHCP servers back into shape, find rogues, malware phone-homes, and a myriad other problems. To use it well, expect to go back to the book from your class on network protocols, the one you slept through; but you’ll have great fun.
Remember that your organization may be guided by extreme privacy principles, and the data you see may not be for your eyes. Use of a tool like this implies that you know the rules, what you’re doing, and you will use it for good and not for evil.
Windows 2003 Server Resource Kit and 2008 Administration Tools Pack
The Windows 2003 Administration Tools Pack is just a 12MB download. It works on Windows 2000 servers (please retire them) and Windows 2003 servers (now getting long in the tooth). Microsoft makes these tools for free use, yet we rarely find them when we’re forced to parachute into a NOC war zone. We find this surprising, given the tools’ breadth and coolness factor.
Numerous tools in the package do small things, such as migrate older Windows server versions (yes, we see Windows NT and Windows 2000 servers still chugging away, perhaps now wheezing, looking desperately to become a VM). You need to search tools for specific jobs in Microsoft’s knowledge base for Tool Pack glitches, gotchas, and exception handling before you use some of the more esoteric items in this tool pack, like the Windows NT migration package; an ounce of prevention is worth a pound of unemployment line.
There are plentiful tools, and many are updated to match libraries and newer .NET conventions. Check versions frequently, and use a central computer for the utilities so that updates, patches, and fixes are sure to be applied on that system rather than a spotty administration over numerous server machine repositories.
ADFind for Those Complex Directories
You’re lost in a sea of Active Directory (AD) variables, the kind where some goofball decided to modify the schema years ago, and put in all of these great fields—some of which joined together LDAP variables with AD overlays. Now it’s a disaster.
For nice and fast AD query results, JoeWare ADFind is a killer command-line search tool that works against the Active Directory components to spill the dirt. The query tool is something that will be soon found in Windows 8 PowerShell commands, whenever they arrive.
The rest of the free tools on the Joeware site are interesting as well, and strictly CLI, and (big warning here) they may or may not twig management applications that look at queries on such things as security violations, not-permitted or private information-based Active Directory components, and so forth. These are the ones you keep on your USB stick, but use only when you know that organizational policies permit them. Otherwise, be careful.
TFTP for your PxE
The Pre-boot Execution (PxE, pronounced “pixie”) is a boot on network method for remote program load. If you’re an administrator, you recognize this as a netboot/netload method that allows you to wake up a compatible machine on a very local network, and have that system get its OS plus program load images from another server on that LAN. We use PxE all the time to do server and VM loads. While it’s not tough to understand, it needs the TFTP protocol available on the image host.
TFTP is one of the more insecure downloading methods you can use. It shouldn’t be used on routable segments, or otherwise at all—because it needs no passwords. The folks at SolarWinds have a nice TFTP because its version, unlike others, can handle multiple requesters at the same time, nicely. Sadly, PxE still can’t use passwords, but no matter.
The net effect of multiple concurrent TFTP thread handling is that you can do multiple servers, VMs, whatevers, all at the same time. They have other free and paid-for tools on their site as well. It’s recommended for imaging a number of servers from bare metal or to wipe them and populate them afresh with juicy operating system loads.
OpenLDAP for Windows
It sounds immoral, maybe illegal, but LDAP and Active Directory can be compatible with Linux without Microsoft’s love, sanction, or blessing. The OpenLDAP protocol that’s a part and piece of virtually all modern Linux implementations can run on Windows networks in a more formal way than hoping SAMBA is working today.
It’s not part of a toolkit, precisely; it’s more of an infrastructure choice to run OpenLDAP officially atop Windows AD networks. Doing so can bring a wealth of cross-platform authentication strength, but it’s not a casual endeavor. You need to understand directory services, authentication methodologies, and how you intend to implement OpenLDAP to get the best use from it. Where it’s practical in our labs, it’s used extensively. It may defy the policies of other organization policies, so don’t drop this into a network casually.
Xming X Server/Client for Windows
If you come from Unix environments (ranging from Old Workstations to Macs to Linux to BSD), there is a common window manager and execution environment you may be very familiar with: X. There’s a great port of X to Windows, replete with familiar X utilities (like Xkill for aberrant windows) that can be used to make Microsoft Windows clients and servers useful on an X-based platform. Both 32- and 64-bit versions available for Windows are cross-compiled, so updates coming from X.org ought to be soon or readily compatible for bug fixes, and new fresh code.
And while all of this sounds like a throwback into the 1990s, a lot of X runs still in a lot of places, just like Windows 2003, Windows XP, and other long-in-the-tooth versions of working systems. “If it’s not broken, don’t fix it” is a great motto to live by. This code makes Windows operating systems a useful member of an X platform, or it can serve as a useful and very-compatible teaching tool for X. There’s also a secure connection method available through PuTTY. Speaking of which –
PuTTY
PuTTY is the original and still favored ssh client for Windows, developed by Simon Tatham. It is a key linking tool between Windows and the Unix/Linux/BSD world, and is actually illegal in some jurisdictions because it can encrypt. The world is better off for it, no matter what you think of national security agencies.
A recent update adds code for other ssh server updates to bring things up to speed. In the same folder, you also want PSCP (scp for Windows) and PSFTP for those secure ftp sessions that are ssh-enabled.
PuTTY also works as “TTY” program for telnet (tell us you’re not still using it!) as well as rlogin. There’s also a nice convertor program for OpenSSH and ssh.com private key files, so that if you change keys often, or must use special ones, you can digest them correctly into PuTTY for use in architected systems. Remember, this is the client app, and not the server. We might be shot dead if we gave you an SSH server tip.
BartPE to the Rescue
This one’s been around for a while. It’s Bart’s Preboot Execution environment. It’s Windows XP on a CD that can be booted to scrape malware and other detritus from an WinXP installation. Originally, Bart Lagerweij built the systematic approach to boot a system the way he wanted it.
Since virtually all Windows NOCs have notebooks and laptops (and soon: more tablets) with Windows on them, they need to be cleansed periodically. One organization we know of routinely takes Windows XP systems and boots BartPE CD by policy. Their version goes through and plants a known-good kernel and registry, and then goes through three different passes of malware and virus applications until the operating system and environment is known to be clean. Then, and only then, can the computer be put back into service.
It seems draconian to put users — especially administrators — into a method where their machines are dry cleaned periodically, but administrators are known to operate loose and fast during a crisis. They also get their hands and machines dirty in the process. Using a CD-boot, primitive as it is, means that a resident piece of badware or rooted kernel can be methodically cleansed.
AppCrash Viewer
If you’re a help desk, admin, systems engineer, or programmer, you know that it’s now possible to get full crash details from many applications on Windows Vista+ applications. An organization called NirSoft makes a Windows .WER file application crash forensic viewer called AppCrashView that works with Windows Vista+ (Windows 2008, too).
The AppCrashView app digests the crash file and gives you loads of details about the system’s environment when the application crashed, so that you can debug the environment or find things like Registry settings that blew up code, and so on.
Procedurally, it makes the old Dr Watson look deaf and blind. Users send the .WER files to you, you look inside, and you don’t need to bother users for details again. This is because the environmental variables, other apps running, and detailed information can be digested via text, HTML, CSV, and XML files.
AppCrashView doesn’t eliminate bugs, but it goes a long way towards finding environmental forensic variables after crashes so that code checkers, QA, developers, and help desk people can assist users when apps go ugly. There’s also a BlueScreenView digestion tool, and a short list of other tools on their site as well. The very first time we used this, it provided needed clues that lead to an ugly app-exit problem fix.
What other tools do you consider must-have? Tell us about the in the comments, so we can add to the community’s body of knowledge.
See also:
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Email to a Friend
- Printer Friendly Page
- Report Inappropriate Content
