ForumsNewsAnd we're back! (from a very prolonged outage)


And we're back! (from a very prolonged outage)
Author Message
p1brunne

Posted: Jun 11, 2009
Score: 0 Reference
Just a great job to bring back gigabytes with only 1 second possible loss.

And a great tool too.

Cheers from Zurich


This message was edited Jun 11, 2009.
amalis

Posted: Jun 11, 2009
Score: 1 Reference
I haven't seen it mentioned yet here in this forum - keep in mind that you can export your Todos and Notebook to CSV files for your own personal backup (and peace of mind).
coachmac

Posted: Jun 11, 2009
Score: 0 Reference
Et voilĂ ! Merci!
Rick Kennedy

Posted: Jun 11, 2009
Score: 1 Reference
Jake, You claim you never anticipated both the main database and the backup to fail simultaneously -- but you did anticipate it! You had nightly backups and real time logs so that you were able to reconstruct things if they both failed. Give yourself a little more credit!

Welcome back. :-) And thanks for everything.
TheGriff_2

Posted: Jun 11, 2009
Score: 1 Reference
Jake (Toodledo),

I think you're being too hard on yourself. Honestly, I do.

I place the problem squarely at the feet of Rack Space. Their hardware was faulty. If the switch had worked as it should have we would have experienced no downtime.

Sure, you made a mistake with the 1 second delayed flush. Sure, it corrupted the database. Point is you should not have experienced the server going down in the first place!

I now question why Rack Space doesn't first have all servers connected to a UPS. A UPS is essential to carrying the server between power-fail and when the generator kicks in. ESSENTIAL!

Just be thankful you aren't the HP technician that swapped two fiber cables on the RAID for an Exchange server I used to administer. That was a 24 hour long rebuild process.

Now add a task to your Toodledo list to stop stressing. Star it and make it priority one.
carsten

Posted: Jun 11, 2009
Score: 0 Reference
Great work - you rock. Well done and these things and your kind of honest and good communication will make us all fans of you!!!
Brad_11

Posted: Jun 11, 2009
Score: 0 Reference
You handled this very professonally. It takes guts to take responsibility, even if it would have been easy to blame others. And most of my tasks were well overdue anyway, so another few hours didn't bother me much!
David_1386078507

Posted: Jun 11, 2009
Score: 0 Reference
Thanks for the honesty and lack of the typical finger pointing that occurs in these situations. Have a ginger ale to settle your stomach. As others have said, "we're here for the long haul". Take care.
dshalgishira

Posted: Jun 11, 2009
Score: 0 Reference
I understand how you feel. I can tell you that now I appreciate you even more.
You did everything right.
I stay, and I will let my friends know about you as well.
How about better Hebrew support?
:)

Thank you.
Daniel
giovanni.ferro

Posted: Jun 11, 2009
Score: 0 Reference
Thank you!!
tavman

Posted: Jun 11, 2009
Score: 0 Reference
I've been using the trial pro account for the last couple of days while I decide if TD is right for me. Your response and open communication about the outage convinced me that your company deserves my business. I've just signed up for the pro account and remain a happy TD user!

Thanks,
Dave Tavener
Linden

Posted: Jun 11, 2009
Score: 0 Reference
I've made a note in another thread, but I'll echo the crux of it here: thank you for doing such a good job and keeping us up to speed while you fixed things. These things are guaranteed to happen at some point or another, and you've both fixed it and learned from it.

Again, thank you for everything that you do here. Toodledo is the first tool I've ever paid for online, and I don't regret a penny of it -- even after last night. You've got a solid system and do amazing work in making it better every day.

Take a deep breath. :)
Adam Lyons

Posted: Jun 11, 2009
Score: 1 Reference
I do agree that this does expose the need for an offline solution such as Google Gears. If we had that, this whole thing would have been relatively seamless to the end user. In the end this was handled well, though as somebody else pointed out the hosting company really screwed the pooch. I dont ever think its a good idea to proactively switch to the generator. The switch over can cause problems (which this one did because of the mechanical failure) and why expose yourself to that risk if you do not have to.

Im a pro user and I do NOT plan on asking for a refund. This happened once and to be honest was very inconvenient for me. However, since its a 1 time thing that has been handled professionally Im not about to freak out.
gaylon

Posted: Jun 11, 2009
Score: 0 Reference
Thanks for the openness. It gives me confidence in keeping my Pro-Plus account open with Toodledo. And, being a database programmer for 30 years, I can relate in some measure to the angst it caused. We had a similar crash at our data center (large eastern company) a few years ago, also related to failover to generators, but unfortunately not all the data recovery efforts faired as well... Good job!
phillips1021

Posted: Jun 11, 2009
Score: 3 Reference
I have the free Toodledo account. After reading about how your company handled this outage and the passion you showed in getting your customer's back online I'm purchasing a Pro subscription.

Thanks for caring so much about your customers. It's a rarity.


This message was edited Jun 11, 2009.
Kim

Posted: Jun 11, 2009
Score: 0 Reference
Thank you for all the efforts and thank you for the additional month of Pro Service... I work in the IT industry and failures occur. Your explanations and post mortem of the problem were absolutely top notch.

An example of truly wonderful customer service and customer care.

Kim
Estoril

Posted: Jun 11, 2009
Score: 1 Reference
Don't worry, I'm going to stick with you as yours is the best task list there is!

The main point is that you got it all back up and running, and probably more important than that, that you actually care!

Cheers,
Darren.


This message was edited Jun 11, 2009.
dsteinke63

Posted: Jun 11, 2009
Score: 3 Reference
I am a paying customer, and now after this, a much more dedicated and loyal customer. Was I disappointed? Yes. Did I understand? Absolutely. You learn more about someone during difficult times than in good. Your customer service is excellent, your communication was spot on, and your demonstrated actions to make things right, exemplary. Toodledo is a fantastic product. The iPhone App kept me going during the outage. Oh what a wonderful world it would be if all companies were this customer centric, dedicated, and honest. Keep up the great work!
Thank you,
David


This message was edited Jun 11, 2009.
envirochristian

Posted: Jun 11, 2009
Score: 0 Reference
I work for an engineering firm that, among other things, does data center services. May I send you a brochure? If so, to what email address?

Best regards (and thanks for working so diligently!)
Meri

UPDATE: I just created a support ticket and attached the brochure.


This message was edited Jun 11, 2009.
wayne

Posted: Jun 11, 2009
Score: 0 Reference
Thanks for keeping us updated. Amazing what a little honesty will do for your customer goodwill. Keep up the good work and welcome back!
You cannot reply yet

U Back to topic home

R Post a reply

Skip to Page:  1   2   3   4   5   6   7   8   9   10   11      Next

To participate in these forums, you must be signed in.