Why Disaster Recovery Procedures Matter

Check out this gem of an email that I received the other morning from a high profile, and well funded, US based startup to whom’s software as a service I subscribe to:

Yesterday morning we had a major server outage affecting our 1.0 customers. We completely lost one of our database servers. The day was spent rebuilding and restoring everything we possibly could.

There were a handful of accounts that the restore completely failed on. Yours was one of those accounts. We have exhausted all available avenues for restoring the account data with no positive results.

Quite simply, I find this astounding! I would have thought that should have had a mirrored/clustered database server, and at the very least held an offline backup of my information. Obviously not.

In my previous life prior to YouDo, I spent countless hours with auditors going over BCP and DR plans, and countless hours making sure we’d avoid this kind of screw up ever happening to us. I used to hate it. Really hate it.

But now I’ve been on the receiving end of what seems like an inadequate disaster recovery plan, and can relate to the pain that our customers could feel – it’s going back near the top of my agenda.

20% Product, 80% Execution

They say you might have the best idea/product in the world, but it’s all about execution. Get that wrong and it’s game over.

The All Blacks found that out the hard way this morning.

(Disclaimer – I follow the team in white)

Decisions Decisions Decisions

I need a new mobile phone.   None of the keys on my Telecom Apache no longer work, so I’m reduced to stylus only operations (or highly inaccurate big fat thumb operations).  It sucks.

I had my heart set on an iPhone after Sandy told us all you could get them from Tauranga.  But with Apple playing silly buggers with the new software release breaking the hack, I’ve got reservations.

I’m after the following:

• a nice slimline profile, so I can carry the phone in one of may pockets without it looking like I’m carrying a brick
• decent call quality (unlike the Apache)
• easy access to my gmail (and ideally Outlook)
• good and bandwidth efficient surfing ability to my GMail, GCal, Google Reader, and BBC pda
• ability to sync up contacts
• photos/music are nice, but not essential

Now I know there are a gazillion phones out there that could probably do the job, but I (at present) have my heart set on one of:

• the iPhone (cos it’s so bloody sexy)
• the Blackberry Pearl (cos it’s a bit cheaper, but still quite sexy)
• the Blackberry Curve (cos it’s sexier than the Pearl)

As you can see, I like sex!!

If you’ve have any comments or experiences with any of the above, or know a screamingly good cellphone that’d do the trick then please let us know.

I’ll blog with an update as to which road fate leads down.

Calendar Carnage

Today was the first day back at work since New Zealand’s new daylight savings regime kicked in.

Around the blogosphere there appears a scene of utter carnage:

Rod’s Blackberry is out of sync,  Wellington Mac Users were in a state disbelief,  and I was one hour early to my first external appointment of the day!

This is clearly not good.

Whilst I took every effort to patch my related systems, there were simply some that had to revert to the good old “wind it forward” technique.  And unless everyone you deal with looked after their side of the bargain it all falls to pieces with regards to calendars.

And then there are the piggy in the middle providers – I have a nagging suspicion that Google Calendar wasn’t aware of the new daylight saving change either.

In other words, my fellow bloggers and readers, we’re totally buggered.

For the next week I’m going back to the trusted dog and bone (telephone) and double confirming all appointments this week.

Better than a big fat egg on your face.   And in a strange kind of way, its kinda nice to talk…

Lamb Chopped!

Having two little children, you really learn to appreciate the benefits of supermarket internet shopping.  No waiting in checkout queues, no cleaning up after your loved ones destroy each aisle, no whinging for chocolate and sweeties (not that my little cherubs ever get up to that kind of mischief!!).  Gotta love it.   Sure, internet shopping is pain in the arse to get up and running to begin with (this is an issue in itself), but once you have your “base shop” in place, it’s relative plain sailing from there on in.

Or so I thought…

I ordered a shop the other day, and fancied a bit of lamb.  Imagine my surprise, then, when I got home to find a shelf and a half of my fridge fully dedicated to lamb!  Yes, I’d accidentally ordered 80 lamb cutlets instead of 10.  I’d ordered 8 x 10 pack of cutlets instead of what I thought was 8 lamb cutlets…  Doh!
Okay, so I’m a stupid oaf, but you would have thought that they would have put in place some checks to make sure that daft people like me don’t order silly things.  That’s not a difficult thing to do.  You know, just a simple “are really sure you want 80 lamb cutlets?” would suffice.   80 lamb cutlets is not “normal behaviour” is it?

This is not the first time (proving once again my oafness).  In the past, I’ve accidentally ordered 20 cigarette lighters!!  Yes, twenty!!  Don’t ask me how.  It just happened.

There must be a better online grocery experience…..

Lamb cutlets?  Lighters?  Barbeque round at my place anyone?

CRM: Leading by Example

I can’t recount how many times I have had the following conversation.

Caller from existing IT supplier: “Hi.  I’m Joe Bloggs.  Let me introduce myself.  I’m your new account manager from ACME Company.  Can I tee up a time for a coffee?”

Me: “Sure.  How about next Monday 9am?”

Caller from existing IT supplier: “Great.  See you then.”

Me: “Oh, can you email me your details just in case I need to reschedule?”

Caller from existing IT supplier: “Erm.  Actually.  Can I have your email address?”

Me: “You should have this in your CRM!”

Caller from existing IT supplier: “Yeah, but….(range of lame excuses)”

It’s disturbing that so many IT companies, many of which sell and consult on CRM systems,  can’t recall my email address.

They say that everything has to come from the top.  Well, if our CRM suppliers can’t even recall an email address, what hope is there for others to get their CRM nailed.

OK, I’m kinda exaggerating, but I hope you get my point…

PDF robbery stifles standards

ITIL version 3 was recently launched this month.  Whilst the fundamentals largely appear the same, they’ve rejigged it to adopt a more pragmatic “business” view as opposed to “process silo” view which caused so many issues in the previous version.  ITIL is good.

They’ve released six new books, which supposedly cover all the aspects required.  They’re downloadable from the TSO Bookstore.  These replace the old books of version two (of which there were plenty).   They are BIG, BIG books so I can just about see how they justify their £299 (yes, pounds) price tag!!  The printing cost of the books would be pretty significant.   They’re equivalent to a small tree.

But check this out…  Their PDF version is a whopping £351.  Yes – 52 quid more expensive!  And the PDFs are DRMed!  Check it out!  Astounding.  How can a DRMed PDF cost more than six whopping tree resembling books?

I thought that standards were out there for people to adopt.  Why the f**k should we have to pay £351 for a individual PDF license?  Jeeps!  It’s even a British Government standard.

Maybe I’m missing something here, but if Governments et al (British, Kiwi, whoever) want everyone to adopt these standards then why charge like a wounded bull.  Make them free.  Drive adoption.

Mercury Rising

With all the Mercury coverage recently, I thought I’d mention that my power went off at home one day last week.  From 9am to 4pm.  The whole street was out for Lines work.  No one told me.  No prior notice.  Nothing.    Same deal with the neighbour.

Makes you think what could have…..

Broadband Shock

I almost sh*t myself today. I got an email through this afternoon from TelstraClear:

You have now used 66 additional block[s] of usage, on your TelstraClear HighSpeed Internet plan for the period to 4/06/2007.

Additional usage blocks are charged at $2.95 per block of 1,024 megabytes, regardless of the number of megabytes you use from that block.

So I thought I’d check out what the deal was on their Internet Usage tool. And got this – thankfully.

Apparently, I am not alone. I literally would have sh*t myself if I got the shock that Mauricio did!

Sure, people make mistakes, but why have they not sent me an email update, telling me not to crap myself. Easy customer service stuff. And am I gonna call them? Not with their call waiting times!

Production Spreadsheets. Banish Thee!

As an ex-operations manager, “production” spreadsheets used to really nag me.

You know what I mean.  A mission critical business function developed in Excel using botched together macros and datasources (if you were lucky).  Often meant to be a short term thing, but end up being around for eons.  These are usually developed by the spreadsheet gurus that, whilst far more intelligent than I will ever be, don’t know how to make these “production strength”, or appreciate why they need to.  Inevitably they move on, taking all knowledge of their spreadsheet mission critical business function with them. Either that or no-one know where the spreadsheet is.  Lots of other problems.  Duplicated data.  Users having freedom to write their own queries.  Drivers stop working.  Passwords change.  The list goes on.

So why is the production spreadsheet phenomena so widespread?  It’s usually one or more of the following:

• Poor response times: A request is made to IT, who say that it will take at least a couple of months before they around to it.
• Too expensive: IT respond and tell the business it’s gonna cost millions.
• Two left hands: IT is not in partnership with the business.
• No visibility: “IT?  Who?.”
• Short term view: “It’s the easiest thing to do in the short-term”

With todays web technologies and frameworks there is simply no excuse for production spreadsheets.  Anything that is performed on a regular basis should be part of a system, not part of someones personal drive.

Particularly with web apps, moving production spreadsheets online reduces risk,  provides easier ways to share information and allows for automation.  Long term benefits.  It’s a no-brainer if your IT dudes are on to it.  Which is not always the case, unfortunately.  It’s our responsibility as IT professionals to sort this shit out.

So for those suffering production spreadsheet mania, I challenge you to step up to the plate and replace them.  Banish Thee!