I get FIOS installed today. What's that wooshing sound? That's the sound my massive bandwidth makes! Heh.

The installation was certainly amusing. He asked to see where the line needed to go in the house. I showed him the server closet, and then another room full of spare parts. He stared for a little bit, then asked "Are you into computers or something?"

He later asked what I did. I mentioned something about Linux... he had never heard of it. I tried explaining that it was an OS just like Windows or a Mac with OS X, then told him I did Linux at work. At which point he said "So you work for Linux?"

Hah. Well, something like that.

After he got everything setup, we went out to my laptop to try it all out. I was able to connect to the DLink Router they setup with no problem, the next thing he wanted to do was configure the router. He told me to do it since I could probably do it faster than him anyway :-)

So, the first thing we did is get rid of the PPPoe, and go straight to "Dynamic". I'd of course prefer a static IP, but dynamic with dyndns (I use dnsexit, actually) isn't too far off.

I went with the 15Mb down / 2Mb up... and I seem to be able to get fairly close to that on the so called "bandwidth testing sites". Now for the real test... downloading an ISO. I tried a .edu site, and was seeing a little over 1.3MB/s down. Which is a bit closer to 10Mb, rather than 15... probably a limitation of the server I was pulling it from.

So is it worth it to get the full 15Mb, over the 5Mb service, for the additional $10/month? I'm not sure ATM, I'll think about it over the next month or so and see what all I download. I certainly see a difference in downloading files. Maybe I'll just do it to annoy the AT&T COO who said this:

"In the foreseeable future, having a 15 Mbps Internet capability is irrelevant because the backbone doesn't transport at those speeds," he told the conference attendees. Stephenson said that AT&T's field tests have shown "no discernable difference" between AT&T's 1.5 Mbps service and Comcast's 6 Mbps because the problem is not in the last mile but in the backbone."

I would just like to say that the above is untrue, as I believe I've just proven... and I reiterate my stance that he's simply jealous because he's stuck at lousy download speeds of DSL :-)

So, in a bold and brash move, someone actually mentioned Perl at the Perl Mongers Meeting last night. In fact, in addition to Patrick talking about Parrot, we actually wrote Perl code as well. It must be a first.

Jeff had an issue where the wonderful admins at his workplace weren't getting the DNS straight on a new DNS server. So, given several hundred names, and a list of the correct IP's, how does one show the admin's which names don't have correct IP's on the new DNS server? Why, you use Perl of course.

The simple, boring, 10 minute answer is this code:

#!/usr/bin/perl

use strict;
use warnings;

use Net::DNS;

my  =  or die "First argument should be the nameserver\n";
my        =  or die "Second argument should be the file\n";

open FH,  or die "Can't open ]\n";

my  = Net::DNS::Resolver->new( nameservers =>  );

while (my  = ) {

    my (, ) = split /,/, ;
    chomp ;
    check_dns(, );

}

sub check_dns {
    my (, ) = @_;

    my  = ->query(, "A");

    if () {
        my () = ->answer;
        if (->address ne ) {
            print " is off! Says , should be \n";
        }
    }
    else {
      warn "Query on  failed: ", ->errorstring, "\n";
    }
}

That takes a nameserver, and file, as arguments. The file has one name / ip per line. It then looks up the hostname at the specified nameserver... and if the IP the nameserver returns is not the same as the IP listen in the file, it displays an error.

The real fun was, how little code can one use to perform this task? Golf, as they say!

Todd used a bunch of shell commands, and had it in some 75 characters. Well, that doesn't really count, that's like being on a different golf course, but saying you finished with fewer strokes ;-)

At the meeting last night, I was able to get it down to 95 characters of pure Perl:

perl -MSocket -nle '($x,$y)=split/,/;$y;$z=inet_ntoa((gethostbyname($x)));print if$z ne$y' h

I've since shortened it to 81 characters by using a different method to perform the DNS lookup:

perl -MSocket -nle '($x,$y)=split/,/;$z=inet_ntoa inet_aton $x;print if$z ne$y' h

And then, 75 characters by dropping the need for $z, and getting rid of some spaces:

perl -MSocket -nle'($x,$y)=split/,/;print if(inet_ntoa inet_aton $x)ne$y' h

Is it possible to not assign variables in the split? I'm not sure, but I think that may be the only avenue left for shortening it more :-)

Update: Okay, we can in fact drop the variable assignment in split, and use the default array, @_. I was originally hoping split set $a and $b the way sort does... it doesn't seem to. But, using @_. we can get it down to 73 characters:

perl -MSocket -nle'split/,/;print if(inet_ntoa inet_aton $_)ne$_' h

Update 2: Aha! We can get rid of split, with a good 'ol regex, bringing us in at 69 characters:

perl -MSocket -nle'/(.*),(.*)/;print if(inet_ntoa inet_aton$1)ne$2' h

Update 3: Woo! Save a char by using -p instead of -n, 68 chars:

perl -MSocket -ple'/(.*),(.*)/;$_=""if(inet_ntoa inet_aton$1)eq$2' h

What exactly is inside a hard drive? Well, I let myself be talked into manually destroying some hard drives, rather than using the simple Derek's Book and Nuke.

So, we pop all the cases off, then do some terrible things to the platters which hopefully prevent most people from being interested in the Word documents we send around here at work.

The one thing I found interesting was that... in addition to the head, platters, and some other gizmos and gadgets, there's not one, but two rather powerful magnets inside. And they're attached to the base of the head, a few millimeters away from the platters.

I had never really known how careful one had to be with magnets around hard drives. Well, I don't think I own a single magnet anywhere near as strong as the ones I pulled out from inside of the drives.

They make great toys though, I have a chain of them now to dangle off various metal objects :-)

I've been using muttng over mutt for the last few months. Basically, the idea boils down to the fact that there's been quite a few really useful mutt patches available for some time, but fewer and fewer of them were being added into the main mutt tree.

Finally, some folks got fed up with this, forked mutt, and imported a bunch of patches. It's hard to say for sure, but at least 15-20 patches are in there now, on top of everything that mutt already does.

Of course, you could get much of this functionality in standard mutt by applying these patches yourself... go ahead and try though, trying to get 15-20 patches to apply and not clash is quite a task :-)

Some of the more interesting features that I make use of now are header caching -- which means opening a folder with several thousand emails no longer takes 20 seconds to open (my sent mail, some 6600 messages, took that long).

It offers a sidebar, not unlike the folder list found in Thunderbird/Evolution/Outlook/etc... you can show and hide it with a single key combo, and it allows you to flip through your various mailboxes in a hurry, without actually having to flip screens by hitting 'c', then '?', then browsing.

They also went through, and made a number of "sanity" changes to the config file syntax. For example, mutt seemed to flip between using 'pgp_*' and 'gpg_*' for various config options. Muttng renamed them to the more sane 'crypt_*' (oh the irony).

And just a tiny little thing that makes a world of difference to me is the menu context patch. If you're in a folder with 100's or 1000's of messages, and you find yourself scrolling up or down through a bunch of them for whatever reason, the messages themselves only begin scrolling once your cursor hits the top of bottom of the screen. So, it's really easy to flip past the one you want. Sure, right after it slides by you scroll back down, and open it with no problem. But what if it were even easier? The menu_context option says not to scroll messages when the cursor is at the top or bottom of the screen, but instead, when it's N messages from the top of bottom of the screen. I have mine set to '5'.... meaning there are *always* at least 5 messages above and below the cursor (unless you're at the beginning or end of the mailbox, of course)

Simple pleasures, perhaps. But, anyone who knocks it just hasn't had the opportunity to use a mail client that allows you to tweak absolutally every aspect of it :-) Indeed, that may be it's one primary fault as well, the fact that you can tweak every option :-) Once you take the time to sit down and do it though, it's really hard to beat.

Oh, and muttng can act as a news reader too, but I haven't yet had a chance to try this out.

Well, sort of.

As Don recently mentioned, I've recently grown interested in working on some gigs of my own. So, on one end, we have things like the webhosting that he mentioned, as well as hosted groupware and other goodies (if you're reading this, I'll give you a discount on hosting, just ask :-) (note to current customers: it's the deal you're currently all getting... and no, there's no miracle 'free' plan, sorry ;-)

However, I've also been getting involved in side sysadmin and programming projects.

One such project deals with a small business who actually manages to pull in some 15 million hits a day on just one of his sites, which is more than Slashdot. It's been fun working on a setup that can handle that sort of traffic, without actually using a load balancer :-)

A new project that's come up is to create a simple-to-use program which can do all the following:

• Get information about movies/streams of any type, from static mpeg movies to proprietary Microsoft MMS based streams
• Given a url, verify said file/stream is working
• Given a time (in seconds), get a screenshot of the video at that time

Really, there's tools out there that can do most of the above... I don't actually have to write a video parser. What doesn't exist is an easy means of doing the above.... one has to take the existing tools, come up with the right options, and put it all together.

Having never really spent much time dorking around in the video world, I hadn't realized the complexity of many video tools. Have you ever looked at the mplayer manpage? It has nearly 40,000 words across 9000 lines.

It was a lot of fun putting this together... when building it, I had my sights on developing a tool that would allow someone to easily do what Google Video's has going. That is, given a set of video url's, verify each works... and if so, display the length, format, set of screenshots, and so on, all the goodies that you can see in this Movie on Google video's.

I honestly don't even know what they're going to use it for, I was simply given a list of options it needed to have available, and recognized that hey, they could implement Google Video's if they wanted :-)

Apparently, you can't just take a working CentOS/RHEL system, and add RAID to it. You'd think that would work, but no. You'll boot, and get a message like this:

Unable to find volume group volgroup00

Followed by things like "LVM Exiting", "Kernel Panic", and other crazies.

The issue is with /boot/initrd. Your initrd image is created at installation time. The initrd loads all sorts of stuff *before* the filesystem is loaded.... so anything necessary to load the filesystem is in it. Including things like ext3 drivers, LVM, RAID, and so on.

Well, if you don't setup RAID during your inital installation, for better or worse the RAID drivers are not added to your initrd. And if your system doesn't have RAID drivers, it's not going to be seeing the LVM on top of the RAID.

I was lucky enought to have a initrd with RAID laying around, but I also suspect there's a way to re-create it. Good luck :-)

Just thought I'd mention what I'm sure each and every geek is wondering... on the new Hershey's Chocolate World ride, there are 9 Wireless AP's. Six don't broadcast the ESSID. Three do, and they are "CDA3". All nine are WEP encrypted. Enjoy the ride :-)

Last week, there were several sources such as Slashdot and Digg reporting on this Ars Technia article. In it, we get to hear about AT&T's COO, Randall Stephenson, tell us about how Verizon's new fiber rollout, and 15Mb Internet connections, are irrelevant. For example:

"In the foreseeable future, having a 15 Mbps Internet capability is irrelevant
because the backbone doesn't transport at those speeds," he told the conference
attendees. Stephenson said that AT&T's field tests have shown "no discernable
difference" between AT&T's 1.5 Mbps service and Comcast's 6 Mbps because the
problem is not in the last mile but in the backbone."

Now, it's an interesting point worth verifying... but easily proven false. As a Comcast customer, if his statement were true, I would not be capable of receiving downloads at full speed, I'd be getting no more than the 1.5Mbps he's discussing. In his defense, there are certainly sites out there where I don't see that sort of bandwidth. But, that's not even the norm.

I think I know why he bought the above falsehood up. It's not even because he doesn't want people using Verizon. Though losing customers would be a bit of a bummer for them, but I'm sure they'll have plenty who stick with AT&T DSL.

No, instead, I think the problem is pure jealousy. Imagine a COO of a huge company.... so large, that you're under constant scrutiny by all your competitors, each one eager to find a chink in the armor.

This COO is absolutally using their Internet service at home... and he's feeling screwed because the rest of the United States is going to be able to get a 15Mbps Net connection, and he's stuck on a measly 1.5Mbps DSL line.

I mean, I'd be upset to. Lets keep an eye out to see how long this guy sticks with AT&T :-)

I got the letter the other day. I was elated to learn that Verizon's FIOS, their fiber rollout, would be coming to my house. That means, for what I pay the cable company now for 4Mb down / 384Kb up, I can 15Mb down and 2Mb up from Verizon. Nice.

We were told they'd let us know before they start digging, and give us a phone number, so that we can let them know about any concerns we have -- including wires of our own that we may have installed, such as dog fences and, well, robomower fences.

So, I was a little less than thrilled when I drove to work yesterday, and saw them digging in our neighborhood. No notification or phone number, just a bunch of shovels digging a 4 foot hole for the cable conduit.

The only thing going through my head was that if they cut through my robomower's line, I'm gonna have to mow the lawn myself. That's certainly enough to get anyone worked up.

I threw the car in park, and ran out to talk to them. The first thing I discovered is that none of the workers spoke English. They looked at me funny, then eventually pointed at some guy with an ipod blaring. He appeared to be listening for a bit, but when I stopped, he mumbled somethign about coming back later when the supervisor was there.

I came back over lunch to see how we could work out sparing our robomower's electric fence... they were all eating lunch as I walked up. I got to practice my Spanish -- "Habla Englais?"

I still got the funny looks, but this time they pointed me to some guy sitting in a pickup, smoking a cigarette and talking on a cell phone. When he was done on the phone, he was very amicable to the idea that I had concerns about how things would work. Which is an excellent trait for someone taking out the front lawns of an entire neighborhood.

These guys weren't Verizon employees, they were workers contracted by Verizon to lay this conduit. Verizon would come later and lay the actual fiber.

This guy explained in general what all was going to happen, which was a bit too high level for the problem we might be dealing with. He realized I wasn't quite satisfied yet and offered to walk over to our house to take a look. On the way, he explained how they ran conduit underneath the road without digging up the road. Apparently, if you dig a deep hole... say, 6 feet on either side of the road, there's some contraption you can shoot through the dirt. He called it a missle, which you can follow up with the conduit. Interesting. In fact, that's what I said, "Interesting". He said "Right, spending all day digging holes and covering them up, it's real interesting." Heh. Ok.

Anyhow, when we got to my lawn, he pointed out the likely place they'd be digging, and I explained where the wire was and how it worked. We decided the best way to handle it would be to simply mark the wire, which could be done by putting spray paint on the grass.

I asked if the color mattered... he said if I did it with anything other than white or pink, the township would likely get up in arms and have someone out there trying to figure out what was going on... apparently, the township has some elaborate scheme of colors for what represents what. Red represents electrical lines, blue is water lines, orange is sewage, etc etc. Leaving white and pink. And, Verizon had already used white to mark off some other things.

So, I was going to be painting the yard pink. Lovely.

He also said that they were capable of splicing any wire they happened to cut, which works for me.

I was glad to have spoken with him, and raced off to buy a can of pink spray paint. After work, Shana and I worked to mark off the wire (which was hard, as it's really well covered at this point).

Well, the yard is now surrounded by pink. I saw the workers get started this morning, and they appeared to be moving the line that was in their path (which is probably a pain in the butt for them, as it's staked down every few inches). But, I was thrilled to see them moving it, there's no way they would have seen it without the paint we put on it.

With 15-20 minutes of work or so, we'll be able to put the moved cable back, rather than doing the entire area.

Which brings up another point, whenever I get the FIOS installed to my door, I think I'm finally going to have to upgrade, having 802.11b is no longer fast enough to keep up with the Internet connection :-)

So, I thought maybe the cops finally caught up with me. As I was pulled over shortly after 11pm while on the way home from my Monthly Perl Meeting, I wondered which of my crimes it was they wanted me for.

To my surprise, my only offense that bothered him was the fact that not one, but both of the lights by my license plate were out. Oops.

He didn't seem very concerned, but he proceeded to ask for my license and registration, which I dug up and gave him.

I didn't realize then, but after I arrived at home without incident, I looked at the registration I handed him, and discovered that it was expired. Heh.

So, I either gave it to him with enough confidence that he didn't notice the year, or he mumbled something about what an idiot I was, but let me go because he didn't feel like doing any paperwork. Hrm.

Update: For the record, I do actually have a current registration. I just handed him the wrong one :-)

Some days, I feel like I should be paying other people to be even allowed to log in to their server, let alone touch the configuration on anything that could be used in production.

Some notes for future reference --

First, DNS and BIND isn't magic. You can't just make stuff up and expect the computer to know what you mean (retardo). The DNS zone name in the named.conf file has to match the name of the domain it's authorative for. You can't just leave arbitrary parts of the domain off and hope it knows what you mean.

Second, if you spend half an hour making changes to a config file, and no matter what you do, it's just not having the expected effect on the service you're setting up, consider the idea that you're working on the wrong config file.

Also, if you're going to be so absent minded as to leave two copies of PHP on a server, both called 'php', both in your path, don't even bother to complain when it causes problems.

If you setup an anti-DoS module on your webserver... and when you benchmark your webserver, over half the requests fail, maybe it has something to do with the fact that it's doing what you configured it to do.

Okay, so this is just really cool.

The idea is that you want a snapshot of your filesystem at one moment in time. The larger your dataset is, the harder this becomes. If, while making a backup, something changes within your data, the snapshot is corrupt.... some data is newer than others, it's out of sync.

So, some would resort to shutting down all services while the backup takes place. The problem here is obvious, there are cases where shutting down service is undesirable.

LVM Snapshots to the rescue. We take an LVM partition... lets say it's 200GB, and we use the lvm tools to make a snapshot of it, onto a 1GB partition.

If we look at the new 1GB partition, we see all the data there that's available on the 200GB partition. Since our snapshot is created, if we create new data on the 200GB partition, it does *not* show up on the 1GB snapshot.

So, how in the world do we fit 200GB into 1GB? I'm tempted to simply say magic. But the real answer is that we don't have to.

Remember, it's a snapshot, a copy of a partition at one point in time. So really, we only need to make the 1GB partition look the way the 200GB partition did, when the snapshot was taken.

Basically, what that means is that any time there is a request to change data on the 200GB partition, we just copy the existing data over to the 1GB partition first, before it's changed. That is, the kernel blocks the update for just a moment while it makes the copy.

The only data stored on the 1GB partition is the data that changed since the snapshot was taken.

So, though it looks like the 1GB partition has 200GB of data, really, most of it still resides on the original partition. Now we can make a backup of the snapshot, then remove the partition. Very slick.

The caps lock key is annoying. It simply sits there to be accidently turned on in circumstances which we least expect...

In vim, when using the command mode, capitol letters are significantly different from lower case ones.

In mutt, the difference between 'd' and 'D' is whether you want to delete one message, or a whole bunch of messages.

And in Red Orchestra, 'f' lets you dive to the ground when people start shooting at you. 'F' does nothing at all :-)

So, no problem... Linux makes is easy to disable the caps lock key. I put this in my .xinitrc file:

xmodmap -e "remove lock = Caps_Lock"

All better now :-)

After months of working just fine, I turned off my laptop to bring it into work.... and when I turned it on, the wireless card drivers (ipw2200) stopped working. Ugh. I was getting this error:

   ipw2200: Intel(R) PRO/Wireless 2200/2915 Network Driver, 1.0.6
   ipw2200: Copyright(c) 2003-2004 Intel Corporation
   ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
   ipw2200: ipw-2.3-boot.fw load failed: Reason -2
   ipw2200: Unable to load firmware: 0xFFFFFFFE
   ipw2200: failed to register network device
   ipw2200: probe of 0000:01:03.0 failed with error -5

It turns out this is a problem with new versions of udev. The module firmware has to get loaded somehow.... and hotplug normally takes care of it. But, newer versions of udev register themselves as the default firmware handler. Compounding the matter is that it doesn't come with a mechanism for handling said firmware loading, it simply registers itself as the handler. Silly.

It's completely fixable, you just have to tell udev how to load firmware [edit: before following this solution, see the update below].

To solve it, you need to add a rule to udev. In the dir /etc/udev/rules.d, I created the file '999-firmware.rules' (it has to run last). In that file, I added this line:

ACTION=="add", SUBSYSTEM=="firmware", RUN+="/sbin/firmware_helper"

If your system doesn't have firmware_helper, you'll have to get it from theudev source. It's just a single .c file in the 'extras' dir. Compile udev with:

make EXTRAS=extras/firmware

And it'll compile everything for you... but all you need is firmware_helper. Just copy it to /sbin, or /usr/local/sbin, or where ever you like (making sure that the line you added to 999-firmware.rules matches), and you're set.

Now rmmod ipw2200 and the ieee80211* modules, reinsmod, and you're set (no need to reboot or restart anything).

Update (2/13/2006): I'm both excited and distressed that this fix has helped so many people, I'd like to think using udev with ipw2200 should be simpler. Hopefully, distros will take care of this as they start shipping the newer udev versions with their software.

Alas, I've installed some newer versions of udev, and ran into this problem again, even with the above fix. After some more exploring, I discovered that udev is now shipping with a program called '/sbin/udev_run_hotplugd'. I found that using this works wonderfully, and is simpler than compiling 'firmware_helper'.

So, instead of compiling 'firmware_helper' as I had initially suggested above (which doesn't seem to work anymore anyways), just create the file '/etc/udev/rules.d/999-firmware.rules', and add the following line in it:

ACTION=="add", SUBSYSTEM=="firmware", RUN+="/sbin/udev_run_hotplugd"

Then rmmod ipw2200 and the ieee80211* modules, insmod ipw2200, and you should be good to go.

Good luck!

Mostly to help Google catch this, the new error I was received was:

ipw2200: Intel(R) PRO/Wireless 2200/2915 Network Driver, 1.0.8
ipw2200: Copyright(c) 2003-2005 Intel Corporation
ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200: Unable to load ucode: -62
ipw2200: Unable to load firmware: -62
ipw2200: failed to register network device
ipw2200: probe of 0000:03:03.0 failed with error -5

And the error that showed up in the udev logs was:

firmware_helper[14006]: main: error loading '/lib/firmware/ipw-2.4-bss.fw'
for device '/class/firmware/0000:03:03.0' with driver 'ipw2200'

It's recently come up that there's things I can't do on Linux that work on Windows. Windows users don't really get Linux, but simply ask "If you insist on Linux, why don't you dual boot"?

It's a terribly difficult question to answer, especially to a Windows-only user. But, I think I may have come up with an analogy that works...

Dual booting is like having sex transformation surgery. There are inherit advantages and disadvantages to either sex, but really, someone who is truly comfortable with their current sex just really has no desire to change. You've learned to work with the advantages and disadvantages of who you currently are, and switching is more than just a little obnoxious. Especially if you know you'll want to go back later.

Linux, of course, is the male of the species. But, I think I'll stop the analogy there :-)