LinLog

What follows is the saga of a bug that took about 2 hours to pinpoint and 5 minutes to fix. It reminds me of one reason I some find dynamically typed languages annoying: implicit casting.

Here's the scenario: The manager tells me he noticed some weird output on one web page. Specifically, this one particular field, which should be empty, is showing the word "null." I own this field, so it's my job to fix it.

Naturally, I start by looking at the code. The site is written in PHP, so I figured that the field is null in the database and PHP is just printing out the word instead of the empty string, as I'd expect. But that's not it. There's actually already a check for null in the data retrieval code. So, just to check, I look at the relevant row in the database. And guess what - the field in question has the value "null." And just to be clear, I don't mean the field is null, I mean it contains "null" - as a string.

Now the plot thickens. This row is user-submitted content. This particular field has a default value of NULL (that's actual, database null) and users need special permissions in the application to add or edit it. The user in question does not have these. Another check of the database reveals that there are a number of similar records, submitted by users without the proper permissions, that have the string "null" in this field. However, it's not all of them, which means it's not related to the default field value. So something is weird.

Off the top of my head, I knew that there are only two places in the code where a user can set that field. One was the main submission form, the other was an AJAX-based update page. A quick check of the database revealed that the updated timestamps on the affected records were empty, so naturally I concentrated on the submission form. Of course, this turned out to be the wrong decision, as the submission form was correct. It turns out that the updated timestamps for user-submitted records never actually got updated.

So I checked out my other possibility - the AJAX update page. It turns out that the server-side code was correct. The null was actually coming from the JavaScript!

When I added the problem field to this page, I made one bad mistake. You see, since only privileged users can edit the field in question, I decided that the server-side code would simply not put it on the page for non-privileged users. That's fine. However, I failed to account for the dynamically-created edit boxes. You see, because this page is actually a listing page with an edit feature, we were actually dumping the database field values into DIVs and then replacing these DIVs with input boxes when the user clicked the "edit" link. We then took the values of those boxes and sent an HTTP request to the server.

Well, if you're paying attention, you may see what happened. I wasn't careful with my JavaScript. When I added the ability to edit this one field, I didn't check if it actually existed in the page. I just used our JavaScript framework to grab the value and then dump it into the POST data I was sending to the server. Well, if the DIV for that field didn't exist, then the value I got back was null. And when I put that in the POST data, it was type-cast to the string "null." So, when the server-side page got the data, it saw that string, not the empty object value, and dutifully stored it in the database.

The moral of the story: when working in a weakly-typed language, you still have to be careful about your variable types. In fact, you have to be more careful. Because the compiler/interpreter won't help you find problems and you can't always count on implicit conversions being done the way you mean. Especially in JavaScript.

As previously mentioned, I have had some file attribute problems with my new Sansa e200 series MP3 player. Basically, when I plugged it in to charge, the "hidden" attribute on various directories on the device's FAT32 filesystem were being changed.

Well, I have now automated the correction of that particular annoyance. With a simple script and udev rule, the attributes are now reset automatically whenever I plug in the device.

The script to change the attributes is as follows:

#!/bin/bash

DEV_PATH="/dev/$1"
export MTOOLS_SKIP_CHECK=1

mattrib -i "$DEV_PATH" +h ::\TMP
mattrib -i "$DEV_PATH" -h ::\MUSIC
mattrib -i "$DEV_PATH" -h ::\PLAYLISTS

Note that this script requires no changes to the mtools configuration. Instead, the -i parameter and the environment variable take care of the stuff we did with .mtoolsrc last time.

To fire this script when the device is plugged in, I added to following to my /etc/udev/rules.d/80-programs.rules file:

KERNEL=="sd[a-z][0-9]", ACTION=="add", ATTRS{model}=="Sansa e2[0-9]0*", RUN+="/home/pageer/bin/sansa-perm-fix.sh %k"

This runs when a new SCSI drive device is added and the model string matches the Sansa e200 series. It then runs the above script, passing it the name of the device, e.g. sdb1.

So, that fixes that. Now I only see the files and directories I want to see in the Rockbox file browser. I consider this a small victory for the picky among us.

As I alluded to in my last post, I finally broke down and bought a new MP3 player. In fact, it's the first MP3 player I've bought for myself. Up until now I've been using the MPIO FL100 my parents gave me as a gift 4 or 5 years ago. It's a nice player, but with 128MB of internal storage plus a 1GB SD card, it's been feeling a little cramped. Plus I cracked the screen a few weeks ago, so I fear its days may be numbered.

For my new player, I chose a SanDisk Sansa e280. I paid $130 for it at New Egg. I chose the Sansa because it received generally positive reviews (some even went so far as to call it an iPod killer), has a color screen, small form-factor, microSD expansion port, and, most importantly, it doesn't require special software to load music.

Hardware-wise, the Sansa e280 comes with 8GB of internal flash storage and a microSD port to allow for extra space. In addition to playing MP3's, it also has an FM radio receiver and a voice recorder. The screen is 1.8-inches and full color, running at 220x176 pixels. It has a mechanical scroll wheel with 4 buttons surrounding it. The controls are a little awkward at first, but you get used to it quickly. On the down side, the Sansa has a non-standard USB port, very similar to the iPod - but not close enough that the plugs are interchangeable.

The Sansa firmware has its ups and downs. Probably the nicest feature is that it includes a metadata database for browsing media. However, rather than requiring special client software to keep the database up-to-date, the Sansa simply auto-generates the database from the media file ID3 tags when it boots. You can just mount the Sansa as a USB mass storage device, copy your files anywhere under its music directory, and they will be detected. On the down side, this means that you may have to wait a minute or two after boot while the Sansa rebuilds the database - it does this in the foreground, so you can't listen to anything until it's done.

My main complains about SanDisk's firmware is that the Sansa's native data formats are fairly non-standard. This means that, unless you use the bundled Windows-only conversion software, it's a pain to use any of the media features other than generic MP3 playing. In particular, while the Sansa supports movies, pictures, and playlists, its native formats are Quicktime, 16 bit-per-pixel bitmap, PLP/PLA respectively. No MPEG or WMV, no JPEG or PNG, and no M3U.

All in all, the Sansa is a good MP3 player out of the box. I like it and I wouldn't hesitate to recommend it to anyone.

But on the other hand, if you're willing to do a little hacking....

One of the reviews on New Egg led me to a project called Rockbox. It's an open-source firmware replacement that runs on a number of MP3 players, including the iPod and the Sansa e200 series. Rockbox is still a work in progress, but the main features are all stable and it is more than usable for every-day purposes.

If you have or want a Sansa, I definitely recommend installing Rockbox on it. Rockbox takes the Sansa from "good" to "exciting." Just take a look at some of the key features. Rockbox offers more music navigation options than the SanDisk firmware, includes an MPEG viewer, a JPEG viewer, M3U playlists, theme support, and assorted games and applications. You can play DOOM on Rockbox, for crying out loud! They have solitaire, Pacman, a Gameboy emulator, a text editor, and a paint program, among others. Of course, the controls for all this are a little awkward, but the point is that using Rockbox over the SanDisk firmware buys you a lot of functionality compared to the little that you lose (mainly WMA support and control through the USB port).

While the Rockbox installation is isn't graphical, it's quite easy: run one interactive command to install the boot loader and then just extract a ZIP archive into the Sansa's root directory. That's it. To remove it, you just re-run to boot loader installer and tell it to uninstall. It's also worth mentioning that installing Rockbox is not an all-or-nothing deal. The boot loader includes a dual-boot feature. By holding the "prev" button during boot, you can boot back into the standard SanDisk firmware.

So if you're searching for good MP3 player, give the Sansa a look. And if you're looking for a great MP3 player, give Rockbox a look.

Here's a quick and useful tactic for dealing with file attributes on FAT32 drives. I got the idea from this post on the Ubuntu blog

My new MP3 player (which I'll be blogging about when I have more time) uses a FAT32 filesystem. I needed to change the attributes on some of file attributes so that it would show the media folders but hide the system folders. Why I needed to do that is another story. Anyway, the point is that there was no obvious way to do this from Linux and since charging the MP3 player seems to reset these attributes, I didn't want to have to rely on a Windows machine being handy.

After way more Googling than I thought necessary, I discovered that you can do this with good old mtools. The really old-school people in the audience will probably remember them from the days when floppy disks were still in common use. Well, it turns out that they can be used with USB mass storage devices too.

The first step, after installing mtools of course, is to set up a drive letter for your USB device in your ~/.mtoolsrc file. This can be done by adding something like the following:
drive s: file="/dev/sdb1"
mtools_skip_check=1
The first line associates the S: drive letter with the device file for my player. The mtools_skip_check line suppresses errors which, I believe, arise from the fact that this is a USB device, not an actual floppy disk. Either that, or there's something about the FAT that mtools doesn't like, but can still work with.

Once that's set up, I was able to simply use mattrib to change the file attributes and [cdoe]mdir[/code] to show the attribute-sensitive directory listing. The actual commands look something like this:
mdir S:
mattrib +h S:/TMP
mattrib -h S:/MUSIC
Note the use of the S: drive letter to prefix paths on the root of the device. The +h and -h flags turn the hidden attribute on and off respectively. Also note that you can have the device mounted while doing this - mtools doesn't need exclusive access as far as I know.

Eventually, I'll be scripting this so that I can (hopefully) run it automatically after charging my player. Ideally, that script would include some HAL or udev magic to detect the dynamically assigned device node and add that to the mtoolsrc file. When I get around to writing that, I'll post the result.

As I mentioned the other day, Kubuntu now ships with Strigi as the default desktop search engine. So, I decided to give it a try. I started the daemon a few days ago and left it to build an index.

My reaction, when I came back last nigh, was - and I quote - "What the %$@#?!?" (And yes, I actually said, "What the percent dollar at pound?!?")

You see, I had noticed that I now had only 500MB of free space left on my home partition. This was surprising because. I knew the partition was filling up, but I hadn't moved any substantial amounts of data around lately and I should have had at least 5GB left.

A quick du -sch `ls -A` revealed the culprit. My ~/.strigi directory had ballooned up to a whopping 7.5GB.

Let my repeat that. The Strigi index was up to 7.5 gigabytes. How? Why? What the hell was it doing? My beagle index was only about 950MB. Why the heck does Strigi need so much more space?

This will definitely merit a little research. I'm hoping this was some freak malfunction. But if not, then I guess I won't be using Strigi. I mean, 7.5 gigs? Come on!