{"id":293,"date":"2010-03-08T19:54:50","date_gmt":"2010-03-09T01:54:50","guid":{"rendered":"http:\/\/www.lordandrei.com\/blog\/?p=293"},"modified":"2010-03-08T19:54:50","modified_gmt":"2010-03-09T01:54:50","slug":"friday-night-i-crashed-your-subnet","status":"publish","type":"post","link":"http:\/\/www.lordandrei.com\/blog\/2010\/03\/08\/friday-night-i-crashed-your-subnet\/","title":{"rendered":"Friday Night I Crashed Your Subnet"},"content":{"rendered":"

(With all due respect. Please forgive the 15 second preceding commercial<\/a>)<\/p>\n

So, Friday started with me flooding the Internet<\/a> with unwanted spam. As of yet I have not worked up the cojones to attempt to turn on that faucet again.<\/p>\n

While I do offer Internet services this comes at a price. I maintain a business-calibre level of data throughput. My ISP has an average priced package for this. It is \u201cAverage\u201d in my opinion insofar as considering the cost of service compared with the speed of the connection compared with the quality of service and support. Actually, putting it down that way may make me have to reconsider my rating.<\/p>\n

Part of this package includes a strip of static IP addresses. Most people with Internet get dynamic addresses. What this means is that you have whatever address you are allocated and that\u2019s fine. The reason for Static is because you need a permanent address that people can get to. This is why you can get to Google or Yahoo, but people can\u2019t necessarily find you when you\u2019re on a chat client.<\/p>\n

One of the problems that I\u2019ve been suffering is slow download speed and intermittent download \u201cdrop-out.\u201d This sounds horrid for my customers, but their download is my upload. My upload has always worked as good if not better than advertised. This download issue has basically been presenting itself as getting 40-60% the download speed I\u2019m paying for.<\/p>\n

I had a tech come out early last week. He was running late. No one called to tell me. This was a bit of an annoyance. I do run my business out of my house; so it\u2019s not like my IT department is meeting with the CEO randomly. It does however mean that 3pm \u2013 5pm cuts into my toddler\u2019s dinner when you show up an hour late to start. (Without calling)<\/p>\n

45 minutes later. The line looks okay but the speed problem must be internal. \u201cShrug<\/strong>\u201d says the tech and shuffles off. On Friday another tech shows up to swap out my DSL modem. Normally this will only create about 10 minutes of downtime. He says it might be 15 if the tech in the home office in Kentucky doesn\u2019t do his job quickly.<\/p>\n

Actually, the tech in Kentucky does the job in 5 minutes. Of course now my internal network has to be reset. This of course is my bailiwick so I send the tech on his way. Now, the nifty part of this is that my publically served sites are on external static IP addresses. Anything on my internal network shouldn\u2019t affect them.<\/p>\n

This is about the time my wife tells me that the internal network has failed. I explain that I have to reset it. Thirty minutes later it\u2019s not working. Internally, I have a DNS server. The internal network is effectively one of my static addresses served via secure WiFi via NAT. For some reason, the WiFi Network has stopped talking to the DNS server altogether. I can ping the server. I can even do NSLOOKUP on the server. I just can\u2019t get the network to do the resolving.<\/p>\n

This situation also inconveniences my wife who\u2019s trying to get information and files from our server for her web site and for an art project she\u2019s trying to muster the energy to do for a colleague. The network is frustrating her and in turn really p*\u00a3&ing me off! I\u2019m getting short of patience because I just don\u2019t know why things aren\u2019t working.<\/p>\n

This disaster proceeds to burn the better part of 3 hours. I reset all the WiFi devices at least 10 times. I reboot the server (3 minute down times) at least 3 times. I finally change all the internal IP addresses and in doing so, make a bizarre mistake. I let the main WiFi unit choose it\u2019s own address. DNS starts moving. But there\u2019s a problem. Now the web server won\u2019t come back up. Ah, there\u2019s a conflict. The WiFi has decided it WANTS one of my served domains. (My wife\u2019s gallery and blog<\/a>)\u2026 Fine\u2026 I give her domain the old WiFi Address. Web Server is coming up. DNS is working. Another hour shot to hell.<\/p>\n

It is now about 3 pm. I still have no clue what went wrong. I put my wife\u2019s IP address into the server but that takes at least 30 minutes to propagate for me. I go to see if it\u2019s propagated and I can\u2019t hit the site. The network refuses to let me go to the server. (This is a situation we refer to as ITWTF) Then it hits me. I ping the domain\u2019s IP address (this is effectively a \u201cKnock knock\u2026anyone home?\u201d) The ping cannot go to the address. This address does not resolve.<\/p>\n

Oh for heaven\u2019s sake. I proceed to ping every one of my Static IPs. Of the n addresses I control. I can ping the first n\/2 \u2013 1. So, if I had for example 25, I can now reach 11. The mis-provisioned my account and dropped half of my addresses. I\u2019ve killed the past 4 hours diagnosing a problem that one idiot tech screwed up by transposing ONE DIGIT.<\/p>\n

The saving grace is that I prioritize my sites. The paying customers are on the lowest (smallest) IP cluster. As a result, those servers never even blinked at the networking mis-configuration. Some of my personal and favour-domains\u2026 those got knocked off for the better part of 4 hrs. I called up my ISP\u2019s \u2018elevated support center.\u2019<\/p>\n

I have access to the elevated support center because I have a problem that\u2019s been open for the better part of 2 months and I mouthed off on Twitter about the problems I\u2019d been having. When ESC (an ironic acronym) heard about the mistake and the amount of time both my wife and I lost\u2026 they were QUICK to fix it. They also offered me a token credit on the bill for the month.<\/p>\n

Since then I have put all the IP addresses back where they are supposed to be. My internal network is still \u2018renumbered\u2019 because that wasn\u2019t the problem and I just don\u2019t want to put it back again yet. (Maybe later in the week) Less-Than-Humourously, the original problem still exists.<\/p>\n

On the upside\u2026 on Saturday and Sunday despite this entire disaster happening, I picked up a new hosting customer<\/a> from GoDaddy who\u2019s been VERY happy with the level of service since joining my server.<\/p>\n

Hopefully, I can bring in more paying hosting clients while I\u2019m at it.<\/p>\n

Share this:<\/h3>