Early Friday afternoon, the websites of NectarTECH customers became inaccessible. NectarTECH’s registrar, Go Daddy, had renamed ns1 and ns2.nectartech.com as NSx.SUSPENDED-FOR.SPAM-AND-ABUSE.COM.

Go Daddy is one of the top five domain name registrars. NectarTECH manages over 200 Go Daddy domains (and 600 servers). The two companies have worked together since 1998, so this would appear to be an atypical event in their long relationship.

NectarTECH owner Nick Mariani says his nightmare began unfolding Monday morning, January 9, when he opened messages from both PayPal and Go Daddy warning that a PayPal phishing scam was running on the server of a NectarTECH co-location customer at http:// 69.50.229.44.ip.nectartech.com/ forum/ tcpsupport/ index.php (Phishing Scam 1).

Go Daddy - NectarTECH Messages

January 2006 Go Daddy! Message NectarTECH Response
Mon, 9 Notice of Phishing Scam 1 Response: Offending material removed.
Wed, 11 Notice of Phishing Scam 2 Offending material removed.
Thu, 12 Scam 1 stopped, but Scam 2 is still active Entire bulletin board removed.
Fri, 13 nectartech.com suspension notice Not a nectartech.com site, please restore.
Sat, 14 Options for restoring service Option #2, Pay $50 to transfer service.

That afternoon, Mariani says he checked the client’s server and discovered that it had indeed been commandeered through a vulnerability in an old version of the Open Source bulletin board package, phpBB. Mariani told Email Battles he cleaned up the system himself, then emailed confirmation of the fix to Go Daddy late in the day.

On Wednesday evening, Mariani says he received another phishing notice. The phishing scammer had apparently moved a couple of doors down the street: http:// 69.50.229.44.ip.nectartech.com/ forum/ forumphp/ tcpsupport/ primapagina.htm (Phishing Scam 2). The email transcripts provided to Email Battles by the ISP verify that he responded within four hours, stating that the compromised script had been removed, and the NectarTECH customer’s account had been suspended “due to repeated violations of our Terms of Service.”

At 5:17 PM on Thursday, the message from Go Daddy’s Spam and Abuse Department (GoDSAD) was terse. It confirmed that Phishing Scam 1 had been removed, but Phishing Scam 2 was still fully functional. GoDSAD growled:

Please remove the offending content off of the site reported on 1/11/2006 by the end of the business day tomorrow or your domain name will be suspended.

Obviously flustered, Mariani responded:

The entire bulletin board on this site has been removed by the customer, therefore no links will work for the phishing sites in question. If you are still able to access any links which are still valid, please indicate the exact links in question.

In any case, we do not believe there to be a reoccurrence of this activity since the offending content, as well as the exploitable bulletin board software was removed from the server.

According to Mariani, the next message he received from Go Daddy was the notice of account suspension on Friday.

“I was stunned,” Mariani said,”that they would wipe out my nameservers instead of simply blocking the domain in question.”

As Mariani tells it, his next shock came when he called Go Daddy tech support. He was told that only GoDSAD personnel could restore service, and GoDSAD was unavailable. His customer/friend/consultant Marc Perkel tried to make headway, but had similar results, though he recorded the tech support conversations. One Go Daddy rep told him,”Abuse is not here … No one can get your datacenter back up tonight. Nobody … The Abuse Department has control of your account … Nobody on the floor can fix it, sir.”

Perkel’s calls are instructive for both sides of the discussion: Late Friday / Saturday Morning.

Go Daddy’s public relations VP, Elizabeth Driscoll, told Email Battles, “The phone call was not up to our high standards and it’s being addressed internally. The Abuse Department is available 24/7, 365 days a year.”

She added that service was restored within an hour of GoDSAD’s receipt of Mariani’s email agreeing to its reinstatement terms. That was around noon on Saturday.

Driscoll’s version of the events that led to account suspension is much simpler:

Go Daddy warned nectartech.com on January 9, 2006.

After informing nectartech.com about dangerous phishing content on their site, they said they would remove the content.

They did not.

GoDaddy.com warned nectartech.com again on January 11, 2006.

The phishing content was still there.

We were told they would remove the phishing content - they did not.

Go Daddy took the site down when the content was still not removed on January 13, 2006.

It was not until the GoDaddy.com Abuse Department took action that NectarTECH followed through and removed the content on January 14, 2006. The issue was resolved that same morning, after NectarTECH eventually removed the phishing content - which they said they were going to do back on January 9, 2006.

She reiterated that GoSAD is staffed and open for business 24/365. Hopefully Driscoll will share that information with Go Daddy Tech Support.

As for NectarTECH… Did the ISP violate Go Daddy’s Terms of Service? “Probably… They worded it in such a way that anything can apply,” owner Nick Mariani responds, but “when you suspend a datacenter, you need to make sure there’s someone there to unsuspend it.”

Further, he advises Email Battles readers, “Make darned sure that you carefully read the Terms of Service.” (Are you listening, Qwest subscribers?)

All told, a lot of innocent domain owners like FreeBSD needlessly endured an 18-hour disruption in service. You can avoid this type of upstream-supplier interruption by maintaining (or contracting) domain servers with disparate registrars. For example, if yournameserver.com is registered with Go Daddy, you could register yournameserver.net with Tucows. That way, any disruptions with one service won’t affect your ability to perform Domain Name resolution.

In addition to spreading out registrars, Perkel told Email Battles, “You should make sure that the email address you list when you register a domain does not depend on that domain’s nameservers. If the nameservers get cut off, nobody can reach you to tell you how to turn them back on.”

Driscoll’s message is equally helpful: “GoDaddy.com takes all phishing attacks seriously and will continue to do so.”

Consider yourself warned.

So who’s right? Did Perkel help or hinder? Are you protected from a similar disaster?

Live It As It Happened:

Update 18 January 2006: NectarTECH owner Nick Mariani dropped us a line to let us know that Go Daddy senior management is talking to him. Although we profess no ownership of a crystal ball, we’re guessing these two old pals will ultimately stick together. We’ll keep you posted.

Email Battles Backgrounder: