The Twitterverse exploded this morning with Dreamhost customers freaking out about high charges, account suspensions, and overdrawn bank accounts. Currently, the DH panel is unavailable, as well as an increased load on all hosted sited. What the hell happened?

• “Last night, this guy ran a standard billing cycle to clean up stragglers from 2007. Unfortunately, the biller was ran for 2008 (December 31st, 2008 to be exact). This caused everyone to be billed as if today was 2008-12-31, wreaking the havoc that we are so sorry you had to be put through.”

I’m on MediaTemple for my personal endeavors, but we use Dreamhost as a hosting solution for WNYMedia.net. Not only do we pay out of our ass for service, but support is simply shit. There is no way to contact them except for a shoddy threaded support panel, and the company number will answer with a message to tell you to visit dreamhost.com.

I think this was the last straw for a lot of folk who are sick of dealing with Dreamhost. We’d have moved months ago if it wasn’t for the sheer logistical nightmare of moving everything over, downtimes and dns issues being something we like to avoid. However, I think nows the perfect time to start more aggressively looking for a new host. If only Mediatemple had a faster mySQL system, but at least you can call them 24/7.

• Are you currently getting rough fucked by the Dreamhost? Is your account touching a negative thanks to Josh? Let me know how you feel. Also, if Dreamhost has spared you and you guys go to the movies and shit, I wanna know what a success story sounds like.

• P.S. – Dreamhost is “*extremely* sorry for the mishap.” Golly, those asterisks make me feel so much better.

• Dreamhost Status

Updates

• [11:15] – Panel still down, and now so is the Dreamhost Status blog.
• [2:23] – Just got word that we were overcharged $1200 and it went through… waiting on the reversal. Dreamhost says we shouldn’t worry because “we’re fixing it right as you read this (and maybe already have for your account)! The best move for you is to just sit back and pretend you never got those weird billing email(s) talking about owing lots of money!” HAHAAHA! Yea.. thats totally hilarious, we’ll just laugh this off you fucktards! Panel is back up though.

So what happened and what did we they learn?

It seems the total looting from the faulty script was valued at about $7.5 Million. However Josh, the man who we all hate right now, takes the blame on himself, and maintains that the billing system is completely robust and secure. Rubust and secure, but not human proof, which is where today’s Fubar came from. See, Josh was merely trying to catch up on old bills from 2007, and rerun the payment systems to retry for those that never went through on the last days of January due to a systems upgrade.  When he put the dates down, he put the year as 2008, instead of 2007. What happened was the secure and robust system ran exactly as he told it to, billing for December 2008 and making almost everyone’s account appear 11 months behind.

• The end to this story is that of course, I’m very very sorry, we’re very very sorry, and I’m sure you’re very very sorry this happened.

  A new service is running right now (in parallel on all the controllers) that fixes all those future charges, and if your credit card was automatically rebilled, refunds the payment automatically. You don’t have to contact us or your bank, and you’ll get an email when your account is finished fixing up. It’s going to take several hours to complete. There are a lot of you these days!

  If, because of this billing mistake, you somehow incurred some fees from your bank or credit card company, please let us know after tomorrow (today we are just replying to all 10,000+ billing messages with a generic explanation) and we’ll do our best to make it right for you.

  The moral of this story is that “flexibility” is rarely desired in programming! The less a program will accept/the less a program will do/the less options and preferences it has, the more usable it is/the more understandable it is/the more stable it is.