I nearly crashed my whole church network recently. You’ll laugh when you find out how. But, I’m gonna make you work for the one thing that you should do right now to prevent it. You’re going on a journey with me…

Friday

Friday is my day off. I’m not at work on Friday. Except today, because the printer mysteriously stopped scanning stuff to its network destinations. I decided that I’d just have people scan to USB stick for the day and do a deep dive sort of investigation on Monday.

Monday

Printer still not working correctly. I reboot it. Several times… Still nothing. “Perhaps it has something to do with accessing the network share,” I think. So I go to a Mac machine that is used by our receptionist. At the computer I attempt to login to the network share. I’m strangely locked out. The Admin account isn’t logging in correctly. “That’s weird” I mutter. “That’s not what I want to hear!” she says.

So now I’m walking down the hall to the Network Closet. After unlocking the door, I move some cables around so that I can have a screen for the server. I log into the server. I’m thinking, “No problem here.” Then I thought, “Maybe something is jacked up with the active directory settings.”

So, I google how to access my active directory stuff. Strangely I cannot get the menus to say what they say on Google… So I start clicking through stuff and discover/remember that I’ve got to login to the Virtual Machine that runs all of that stuff. So I pull up the login screen and enter my credentials. It failed.

Bad sign.

I try it again. Same thing.

Now my pulse starts racing a little. I message my network consultant and ask, “what happens if I lose my domain controller VM?”. “Really bad stuff” he responds. Turns out I’m locked out of the thing that directs all of our dns requests, handles all of our network shares, and active directory stuff, etc. The big problem is that thing called DNS. Computers need something to convert google.com into 8.8.8.8 and that is what the DNS does. So if I can’t access it and it crashes… nothing on our network can get to the internet. Basically this is a ticking time bomb. At some point in the future something will need to be adjusted and I won’t be able to do it.

“What’s the login for the domain controller?” I send in a text to my network guy. He sends me the info. “Hmm… that isn’t working” I send back.

He calls.

Bad sign.

“I’m going to try to remote in” he says. Pause. “It isn’t working for me” he says. Then he politely asks, “Did you maybe change something on there?” At this point I’m in full panic mode. Pause and take a breath. I try to sound decisive. “I’ll call back in a little while and let you know what I’m going to do here” I tell him. Hours later he decides to call me since I never responded. He lists several expensive sounding options for re-gaining access to the system. I tell him I’ll let him know. Then I immediately text every network person I’m friends with to see if they know how to break into a server… nothing… Out of desperation I start trying old passwords one after another.

Finally, one works!!

I text everyone that “I was just kidding.” I text the network guy, “turns out I’m not a total idiot – I remembered the password.” He responds with the only thing he can probably muster an emoji :).

The Point: Do This Right Now!

The point of this story is: MAINTAIN A LIST OF ALL OF YOUR PASSWORDS. No I’m serious write down a list of all the critical passwords in a document somewhere. Then print it out and put it in a folder that a few other people know about. This stuff is real. And I was that close to losing a ton of data, and a bunch of money, and possibly the respect of everyone on my team.

Go make that document. Modify it whenever you change anything. The risks are too high for you not to do it.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.