Over the weekend, I got an email from Kurt, our customer who functions as the reluctant administrator of his network. After reading comments about his performance in my last post, pro and con, Kurt wanted to let you know how he came to be the network manager.
If you haven’t been following this conversation, read the Prologue. Otherwise, jump on down to “Bad Management.”
Prologue.
Last Tuesday, I told you about rootkit prototypes that can takeover a 64-bit AMD or Intel computer at the chip level, allowing the rootkit to feed the OS anything it wants.
Some commenters cried “foul” because the prototypes require root level access to do their dirty work. For example, land0 wrote, “…this will only effect Nix admins who use the root account as their personal login. Can I see a show of hands of who does this on a Nix system? …Crickets chirping… thought not.”
I responded with “Lots of new and frustrated admins are committing early Windows administrative gaffes… especially when installing new software. Don’t kid yourself. The sins of Windows are being visited on Nix.”
From there, we were off to the races. On Thursday, I posted my interview with Kurt, an office manager in a small business, who had been handed the company’s network administrative chores, on top of his normal duties: Confessions of a Real-World Linux Admin: “I Always Login As Root.”
Some thought I was too hard on Kurt… which made me a putz. Others thought I was too easy on him… which made me a putz. One even thought I had developed some exotic aversion to self-taught administrators. Of course, few of the outraged drop-ins had bothered to read the prior story and comments, so they had no way of knowing where I was coming from… or the sarcastic streak running through the author.
So why am I telling you this? Because this post springboards from the prior two. If you’re not going to bother to read those for insight into an ongoing conversation, please refrain from commenting here, because the chance that you know what we’re talking about is close to zero.
Instead, go back to slashdot or digg or reddit or wherever, and mouth off where others might be hoodwinked into thinking you have a Great Insight that everyone here missed. In other words, as we Iowans say, “Thanks for stopping by. Don’t let the door hit you on your way out.”
Bad Management.
As Kurt sees it, his network problems started with the Internet. Before that, he happily ran Windows nodes on a Novell ethernet network, which demanded more Windows-with-screwdriver time than network administration. A guy from his local computer dealer dropped in when the going got too tough.
Then he bought the Windows server pitch, which was quickly followed by acquiring an Internet presence and switching to TCP/IP. For Kurt, that was like switching from driving cars to building them from scratch. He needed help. Fast.
After interviewing some twenty applicants, three of whom committed some requisite experience to resume, Kurt hired Gustaf. For the first year, Kurt thought Gustaf knew what he was doing. Gustaf simplified this, and smoothed out that. Moved email to sendmail, web services to Apache. The guy was always busy fixing, migrating or tweaking something.
“My guru,” Kurt beamed.
But as time passed, gremlins started creeping out: a sales guy couldn’t get to needed accounting recs, the receptionist couldn’t print, the boss lost his email.
By the time Gustaf found his new job at a temp agency, Kurt’s company was nearly broken. Servers blue screened daily… The Customer Relations Management database required re-indexing several times a day… Five frustrated sales people had quit… and only one person could access company financial records at a time.
Someone needed to grab hold of that network for good. Kurt and his boss decided to rebuild it themselves, as opposed to bringing in an outside gunslinger, so they would, he says, “know where the levers are.”
Left with no logs or journals… and few passwords, Kurt and the boss burned the midnight oil, shutting down what they couldn’t understand or access, and replacing it with software that made sense to them.
After seven heroically sleepless nights (to hear Kurt tell it), his LinWin network was up and running… and running… and running.
Sixty clean running days later, the Boss gave Kurt a modest raise and pronounced him Network Adminstrator.
And, as far as Kurt knows, it has run pretty much the same way since. He tries not to twiddle it too often.
Now you know why.

6 comments
Comments feed for this article
August 19th, 2006 at 7:37 pm
Pingback from E-Bitz - SBS MVP the Official Blog of the SBS “Diva” : Rule one - know which window you are on
August 14th, 2006 at 8:56 pm
Mike
We have an acronym at work: TSDW (The S#!$ Don’t Work). Unfortunately, with 500 remote Windows servers, TSDW is the norm, not the exception. If it is not some process mysterousely hanging, it is a BSOD because the latest version of Symantic AV corrupted the registry. Or my batch files can’t tell if a copy worked because there is no return code that I can rely on. We installed 30 patches with 12 reboots to our servers last month, with yet another set of critical patches we need to install this week.
Meanwhile, at home, my Linux desktop just runs and runs and runs#da_4&9){~^vswORA-03113: end of file on communication channel
Aug 14 19:08:47 mike kernel: EXT3-fs error (device sda1) in start_transaction: Journal has aborted
Aug 14 19:08:47 mike last message repeated 2 times
Aug 14 19:08:47 mike kernel: scsi0 (0:0): rejecting I/O to offline device
Aug 14 19:08:47 mike kernel: EXT3-fs error (device sda1): ext3_get_inode_loc: unable to read inode block - inode=294914, block=589832
Aug 14 19:08:47 mike kernel: scsi0 (0:0): rejecting I/O to offline device
Aug 14 19:09:14 mike kernel: scsi0 (0:0): rejecting I/O to offline device
:)
August 15th, 2006 at 8:52 am
BJ Gillette
Hi Mike.
I couldn’t possibly agree more… I think.
August 15th, 2006 at 11:57 am
Art
New. Instant. Classic.
August 15th, 2006 at 2:51 pm
Chris
Excellent story. I believe in the philosophy of not fixing things that aren’t broke. Sounds like they had the right idea. Always go with what you know.
August 15th, 2006 at 5:13 pm
BJ Gillette
Hi Art and Chris.
Thanks for your kind words. Combined with the two companion articles, I hope I’ve painted a fairly three-dimensional picture of the challenges and realities of network management for many businesses.
There’s solid method behind what some consider to be Kurt’s madness. He had a business to save, and he did it. (Did you get that, Kurt?)