LinLog

The other day, one of my co-workers posted a link to a semi-comprehensive list of everything that's wrong with PHP. This should be required reading for anyone who claims to like PHP.

The best thing about this article is that I actually learned a number of things from it. Apparently PHP has a lot of WTFs that I simply hadn't run into. Though it's relatively minor, my personal favorite is :

"gzgetss - Get line from gz-file pointer and strip HTML tags." I'm dying to know the
series of circumstances that led to this function's conception.

I would also love to know what motivated someone to include this function in the zlib extension. I mean, how many people have ever actually used that function? Like, three? It's ridiculously specialized and isn't part of the C API (I know because I checked) - there's no reason for it to exist. To me, that perfectly sums up the language's lack of any coherent design or sense of priority.

Of course, there were plenty of other nice little gems. For instance, the face that when $x is null, $x++ is 1, but $x-- is null. Or the notion that named parameters were rejected as a feature because they would "make for messier code". There's certainly lots of information and humor value there for anybody who is familiar with PHP.

Personally, after reading that, I think it's time for me to brush up on my C# and Python. I've been making my living writing PHP for five years now, and I fear it may be causing brain damage. After all, Edsger Dijkstra said that BASIC warps the mind, and PHP has been called "the Visual Basic of the web". And after reading this article, can there really be any doubt?

Author's note: This is another entry that's been sitting in my drafts folder for years and which, after three glasses of wine, I've now decided to publish. This one is from April 2007, about four months before I left the government IT world for the world of commercial product development. It's funny how my posts are much less humorous when I'm enjoying my job than they are when I'm frustrated. But, then again, working with non-technical end users generates a lot more funny stories than working with top-notch developers. Oh, well.

The corporate IT world can be frustrating at times. The IT department is often viewed as a cost center, a necessary liability, rather than a helpful resource. As a result, it often seems to the IT people like the entire organization is against them. This leads to frustration, apathy, hostility, and using a sniper rifle to pick off people walking in from the parking lot. Thus I present a few tips to keep your IT department on your good side.

1. Don't report a problem by just saying you're "getting error messages" on your computer. We're the IT department, not the Psychic Friends Network. You're going to have to be a little more specific if you want any help.
2. If you do call about "getting error messages," then at least have the decency to tell us what the messages say. I know fixing computer problems may seem like magic at times, but we do need something to go on. If you dismissed the error dialog, didn't take a screenshot, didn't write down the message, don't remember what it said, don't know what application raised it, and don't remember what you were doing when it came up, there's not a hell of a lot we can do for you.
3. Please realize that "computer people" don't automatically know everything there is to know about computers. I may work in the IT department, but that doesn't mean I can fix your computer myself. It also doesn't mean I can tell you how to do something in your CAD or GIS software. It's not like I practice drawing maps in my free time. We specialize too, you know.
4. When requesting custom software, don't expect me to read your mind. I understand that you might not think of some things until after you start using the software. I have no problem with adding things later. In fact, I expect it. However, I really don't appreciate getting attitude because the program doesn't do X when nobody ever mentioned to me that it needed to do X.
5. When you ask how to do something, pay attention to the answer. I don't mind showing you how to add a border to that cell in Excel. I do mind showing you how to do it once a week for a month. And the same goes for clearing out your deleted items in Outlook.

I ran into an interesting bug in OSX the other day. Well, I regard it as a bug, anyway - Apple may feel differently, as it's been there for a while.

I was using a PHP library called ZipStream. It basically lets you create a zip archive on the fly and stream it to the client as each files is compressed, as oppsed to generating the entire on the server and then sending it to the client. This is nice because the user doesn't have to have a (potentially) long delay before the download starts - you can start sending data right away.

Anyway, the library was working wonderfully...until I tested the archive on my MacBook. Turns out that OSX doesn't like the archives that ZipStream generates. Or, rather, the OSX Archive Utility doesn't like them. When you double-click the archive in Finder, rather than properly decompressing it, OSX extracts it into a .cpgz file. And if you double-click that file, it extracts into another archive, and so on ad infinitum.

By way of contrast, everything else seems to be able to extract the archive normally. The Windows zip archive handles manages it fine, as does WinRAR and 7-zip; on OSX, Safari's built-in zip handling transparently decompresses it without problems; even the OSX command-line "unzip" program handles it without problems. It's just the Archive Utility - which is, unfortunately, the default handler in Finder.

Luckily, the solution is pretty simple. It turns out that the OSX archive tool doesn't like the "version needed to extract" set by ZipStream. The value set for ZipStream's archives is 0x0603. If you change that to 0x000A, then the OSX Archive Utility will open the file normally, just like every other program. Of course, you have to modify ZipStream itself to get this to work, but that's not really a big deal - it's just a one-line change.

I'm not entirely sure why the OSX archiver doesn't like that version number. Perhaps that flag implies some other features that the Archive Utility doesn't support. Or maybe it requires additional metadata which wasn't set in the archive, and so it was technically out of spec. But to me, it really doesn't matter - either way, it's a bug in the OSX archiver. If the zip file was out of spec, they should just detect that and handle it, because everybody else does. And if the program doesn't support other features implied by that version number, then they should have either implemented those features (I mean, it's not like the ZIP format is new or anything) or they should have done the check based on the archive content rather than version number - if the file doesn't use any of the unsupported features, then there's no reason that the archiver shouldn't handle it correctly.

OK, I partially take back my comments from yesterday. I found another thing about my new Sansa Clip Zip that's annoying.

Yesterday I tried upgrading the Clip Zip firmware. It came with firmware version 01.01.12, but the current release is 01.01.18, so I figured I'd get up to date. The firmware upgrade process itself was actually totally painless. You can use Sandisk's upgrade utility or just copy the file into the root of your device and disconnect it from the PC to start the upgrade. Nice and simple.

The problem is with the new firmware. As far as I can tell, there were no major bug fixes or improvements that I care that much about, but there is one major bug. In the 01.01.12 firmware version, the media refresh will pick up M3U files and add them to the device's list of playlists. The file paths need to be relative, of course, or else you end up with an empty playlist, but it's actually a very nice feature.

Well guess what's missing in firmware version 01.01.18 - no playlists. They don't even show up. A thread on the official Sansa forum suggests that this problem is actually shared by all the newer firmware versions, not just the latest. So nice going, Sandisk! Way to snatch defeat from the jaws of victory.

On the up side, reverting to the 01.01.12 firmware was dead simple. Just download the appropriate archive and follow the normal upgrade instructions. Once I did that, my playlists were back...after the hideously long media refresh.

I got myself a little treat for the new year - a new MP3 player. I'd been thinking about it for a while, and a combination of my Christmas bonus and the fact that my old Sansa e280 would no long connect to my computer finally motivated me to take the plunge.

This time, I decided to go minimal. As I mentioned, my last MP3 player was a Sansa e280, which isn't exactly the Cadillac of MP3 players, but has a pretty decent feature set. It was certainly one heck of an improvement over the one I had previously - an old Digitalway MPIO with a whopping 256MB of internal storage and a massive 3-line monochrome display. The Sansa e280, on the other hand, has 8GB of internal storage, a decent sized screen, and can display pictures and videos. With the RockBox open-source firmware, it also has some simple games and apps, as well as microSDHC support (the Sandisk firmware only goes up to 2GB microSD cards).

The thing is...I never really used any of that. Granted, the microSD slot is a God-send, and I used to occasionally play games on it. But I never had more than a couple of pictures of videos on it and really only used it for listening to music and podcasts. And for videos or games, I now have a CAANOO. So why not get something a bit smaller that just does the one thing I really care about?

So this time I went with the Sansa Clip line. In particular, the new Clip Zip, successor to the Sansa Clip+, which I was originally considering.

The Clip Zip has a lot of things going for it. First, there's the form factor. It's tiny - only 2.25" by 1.42", about the same size as the pouch I use for my headphones. Second, it's cheap. The Clip Zip comes in both 4GB and 8GB models and starts at $50. I got a 4GB model on sale for $40, which is about what you'd pay for a 4GB no-name MP3 player. Third is the storage capacity. Of course, there's nothing impressive about the internal storage, but it doesn't matter because the Clip Zip has a microSDHC slot. So for another $30, I added a 32GB microSD card to my order, giving the Clip Zip a total capacity of 36GB. Not bad for a $70 investment.

So far, I really like the Clip Zip. I was initially disappointed that RockBox didn't have a stable port for it yet, but it turns out that the Sandisk firmware is actually pretty nice. The menu system is easy to use and looks nice, as does the "playing" screen with background cover art turned on. The firmware automatically sorts by artist, album, genre, etc., and separates podcasts and audio books into a separate menu item, which is kind of nice. It also includes the option to let you browse the actual folder structure on disk rather than using the database, which is something I always wanted in the Sandisk firmware on the e280.

The only thing I don't like about the firmware is the media refresh. As with the e280, the Clip Zip will do a "media refresh" every time you write data to the disk when it's connected to a computer or when you insert a microSD card (even if it's the same card you just removed and nothing has changed on it). This normally wouldn't be a huge deal, except that with a large media collection the refresh can take a very long time. I have a little over 20GB of audio on my microSD card, and the refresh was initially taking between 12 and 15 minutes. With some adjustments to the ID3 tags on my files, I was able to knock it down to about 8 minutes - still a long time, but not as bad. (For future reference, I used Mp3tag to tag everything as ID3v2.4 UTF-8, removing all other tag versions and blanking the comments field.)

As far as the hardware goes, I have no real complaints. I must confess that the Clip Zip does feel a bit flimsy compared to my e280, probably because the casing is all plastic as opposed to the metal backing on the e280. However, it seems to be reasonably well made. The controls are easy to use and I've found myself using the clip on the back far more than I ever thought I would. And I have to say the that the micro USB charging port is a welcome change from the proprietary port that the e280 used. With the Clip Zip, I don't even have to bring a dedicated cable with me when I travel - I can just use the same one I use for my cell phone or Kindle.

So I would definitely recommend the Clip Zip. It's simple, inexpensive, does its job well, and is surprisingly full-featured for the price and size.

Author's note: This is a piece I wrote back in 2006, but didn't publish because I was afraid people might find it offensive. I found it while looking through the backlog of unpublished/unfinished entries in my drafts folder and thought it was funny enough to share. It's a satire of "magic" frameworks like Ruby on Rails (which was the big thing at the time), which are sold as the solution to all your problems. Thinking about "Rails evangelist", my mind naturally jumped to the new testament evangelists, so this is sort of a merger of the two. No offense intended to any Christians in the audience - I just thought software and eternal salvation made for a funny juxtaposition. But maybe I just have a weird sense of humor. Enjoy!

That's it - the "programming problem" will soon be solved. Next week is the official first, last, and only ever release of the ultimate web development framework - Ruby on the Cross.

What is it

What is RC? It's the ultimate web development tool based on the Christian faith. Think of it as a cross between Ubuntu Christian Edition and Ruby on Rails, but infinitely more productive. It was inspired by the Book of Matthew, chapter 17, verse 20:

And Jesus said unto them, Because of your unbelief: for verily I say unto you, If ye have faith as a grain of mustard seed, ye shall say unto this mountain, Remove hence to yonder place; and it shall remove; and nothing shall be impossible unto you.

Using this principle, RC is able to reach untold levels of productivity.

What does it do?

As you probably already know, Ruby on Rails takes all the pain out of developing CRUD applications by taking your database schema and automatically generating models, controllers, and default views for it, giving you a working, if simple, web application just by running a few commands. RC takes this one step farther: it generates your domain logic for you.

That's right, you heard me: Ruby on the Cross is capable of doing your business analysis for you. In fact, not only can it generate your domain object, but it generates fully functional controllers; rich, AJAX-driven views (which are fully backward-compatible with Mosaic 1.0, by the way); and even writes your database schema. Best of all, it is 100% guaranteed to be future-proof and to be maximally performant. Thus there will never be any need to maintain the generated program!

How does it work?

At this point, you're probably thinking that this is too good to be true. How could anyone possibly promise those things?

Well, remember the quote from Mat.17:20? By drawing on the faith of the programmer, RC is able to miraculously harness the power of God to create genuinely perfect software. There are no algorithms to debug, no trade-offs or design compromises to make. Just the pure invokation of divine power, which not even Murphy's Law can resist.

Hard to believe? It shouldn't be. Just think about all the things people thank God for. If He can get someone a parking space or help the high school basketball team win their big game, why can't He write software?

How do you use it?

Using Ruby on the Cross is quite simple. You simply need to run a few shell commands. You start with:
ruby cross praise jesus
This command initializes Cross and prepares your system for an infilling of the Holy Spirit.

The next command is:
ruby cross supplicate <app_name>
This command forms a prayer, asking the Almighty to design your application. Note that no configuration, or even knowledge of the problem domain, is necessary for this step. Being omniscient, the Lord is able to determine exactly what your customer needs, and will need in the future, with perfect precision.

Lastly, you simply need to ask God to put the project files on your PC so that you can deploy them. To do this, you simply run:
ruby cross create ex nihilo <app_name>
This will cause the completed project source code to appear on your hard drive. Note that you do not need to specify a path on the filesystem - the Lord knows the most appropriate place and will put the code there. And don't worry about source control: God has already added the code to your RCS - not that you're ever going to need to change it.

You know, I've been making my a living writing software for over 10 years now. If my resume is any indication, I'm pretty good at it. But every now and then I still read something that gives me that. just for a minute, like maybe I've just been fooling myself all these years, like I'm actually completely inadequate as a software developer.

This Slashdot article gave me such a feeling. It links to a Google case study that involved porting MAME to the Chrome Native Client. The summary ends with this quote from the article: "The port of MAME was relatively challenging; combined with figuring out how to port SDL-based games and load resources in Native Client, the overall effort took us about 4 days to complete."

Now, I don't follow Chrome development at all, so when I read this, I had absolutely no idea what the Native Client actually was. My initial guess was that it was probably some API to access native libraries or hardware from JavaScript, or something like that. So it sounded like "porting MAME" would entail porting the code to a different programming language, or a different set of APIs for audio and video, or some similar.

That sounds like a pretty huge undertaking to me. And they did it in four days?!? Wow! I know Google is supposed to be all about hiring the best and the brightest, but that's just ridiculous. I mean, that would take me months to finish! And even if I was already familiar with the internal workings of MAME and Chrome, it would still be a matter of weeks, not days. How did they do it? Are they really that good? Am I just some third-rate hack by comparison?

Well...turns out they're not actually that good. This comment on the article sums it up nicely. Turns out that the Native Client isn't actually a totally different API, but presents a POSIX-like API with support for compilers and build tools. So really, this was less like porting MAME to something totally new and more like making it run on a different operating system. And they didn't even do a real production-quality port. Instead, they simly removed several problematic parts of the code and just sort of "made it work." Granted, that's still a pretty impressive amount to accomplish in only 4 days, but it's hardly the super-human feat it seemed a first.

This sort of story is one of the things that's always bothered me about the culture of software development - it's full of tall tales. Listening to the stories people tell, you'd think everyone was building big, impressive programs in a couple of days. It's not until you pry for details that you realize that the impressive sounding thing is actually little more than a prototype. Sure, Bob may have built a working C compiler over the weekend, but he doesn't mention that it can only reliably compile "Hello, world" so far.

It's almost a lie by omission - you report a truncated version of your accomplishment and rely on an implicit comparison to something much more involved to make you sound super-human. And I say "almost" because it's not just self-aggrandizers doing this. In many cases, the tale just grows in the telling. This case is an excellent example - Slashdot took an impressive sounding quote, stuck it in a brief summary, and made the whole thing sound bigger than it was.

I sometimes wonder what effect this sort of rhetoric has on beginning programmers. Do they find it inspirational? Does it make them want to become the "programming god" who accomplished this sounds-more-impressive-than-it-is feat? Or is it discouraging? Do they hear these stories and think, "I'd never be able to do something like that."

Looking back, I think that it was kind of a progression for me. When I was first learning to code, those stories sounded cool - I could be the guy telling them one day. Then, after a few years in the industry, they started to be discouraging. I mean, I'd been doing this for years and I still wasn't the guy in those stories. Now I'm over ten years in and I'm just skeptical. Whenever I hear one of those stories, my first thought is, "So what's the catch?" (Because, let's face it, there's always a catch.)

And the worst part is, I don't even know if that story should be inspiring or just sad.

You know what annoys me? People with crazy ideas. Especially when they pimp them like crazy.

That's why this tutorial on "code separation" from a web forum I occasionally visit annoys me so much. The author links to this thing like his life depends on it. Whenever somebody has the nerve to post a code snippet that has both HTML and PHP in it, he brings it up. Even if the code is just echoing a variable inside an HTML block. It's ridiculous.

Don't get the wrong idea - separation of concerns is obviously a good thing. If you're outputting HTML and querying the database in the same file, you're doing things wrong. But this guy takes it to absurd lengths and insists that you should never have any PHP code mixed in with your HTML. Not even echo statements.

The real kicker is the content of this "tutorial". It's basically a half-baked template system that does nothing but string replacement of pre-defined placeholders. At best, it grossly over simplifies the problem. I suppose it does demonstrate that it's possible to output a page without having HTML and PHP in the same file (as if anyone really doubted that), but that's about it.

The thing that really bothers me about this approach is that the author promotes it results in code that's easier to understand than code that has both PHP and HTML in it. Except that it's not. The guy apparently just has a pathological fear of having two different languages in the same file. it's completely irrational.

The problem with his approach is that it doesn't actually solve the problem, but just moves it. Sure, basic replacement like that it's fine for simple cases, but as soon as the requirements for your markup get more complicated, things blow up. For example, how do you do conditionals? Well, you have another template file and you do a check in your controller for which one to inject into the page. What about loops? Well, you have a template file for the loop content and you run the actual loop in your controller, build up the output, and inject that into the page.

The net result? What would normally be a fairly simple page consisting of one template with a loop and two conditionals is now spread across six template (one main template, one for the loop body, and two for each if statement) and pushes all the display logic into the controller. So instead of one "messy" template to sort through, you now have a seven-file maze that accomplishes the same thing.

I find it difficult to see how this is any sort of improvement. At best, it's just trading one type of complexity for another in the name of some abstract principle that mixing code and markup is evil. Of course, if you want to follow that principle, you could always go the sane route and just use something like Smarty instead. But let's be honest - that's just using PHP code with a slightly different syntax. It may be useful in some cases, but it's not really fundamentally different from just writing your template files in PHP.

Personally, I've come to be a believer in keeping things simple. PHP is a template system. It was originally designed as a template system. It's good at it. So there's no need to introduce additional layers of template abstraction - just stick with PHP. There may be cases where things like Smarty are useful, but they're far from necessary. And the half-baked templating systems like those advocated in that tutorial are just intellectual abortions. There's no need to reinvent a less functional version of the wheel when you can just use a working, tested wheel.

OK, that was a little confusing.

So after my initial attempt at a Mercurial hook to connect my public repository to my Mantis instance, I decided to rearrange things a little. Initially, I was using a "commit" hook on my local machine. However, for the public bug tracker with the public repository, it made more sense to put the hook on the server. So I switched it to an "incoming" hook in my hgweb.config.

Actually converting the script to work with my Linux based web host was pretty easy. Just one command, really:

#!/bin/bash

basedir='/path/to/base/hg/install/dir'
mantis='/path/to/mantis/wwwroot/scripts/checkin.php'
hg="$basedir/hg"

$hg log -vr $HG_NODE --style $basedir/mercurial/templates/multiline | php $mantis

Nice and simple, right?

The hard/annoying part was in the style template file. Seem I wanted my commit message to have multiple lines and indent the file list, like this:

Changeset 85:5f7504e02f1c by Peter Geer, Mon Jan 17 23:12:46 2011 -0500
Updated plugin for PHP5 and fixed blog root when taken from defined constant (fix bug 0000021).
lib/blog.php
plugins/menubar_breadcrumbs.php

Problem was, it just didn't work. I was using exactly the same template text as in the old hook script, but I was getting everything on the same line. By switching from "\n" to "\r\n" as my newline escape code, I was able to get newlines to work, but the indentation still didn't.

Guess what fixed it - switching from double to single quotes.

changeset = 'Changeset {branches}{rev}:{node|short} by {author|person}, {date|date}\n{desc}\n{files} '
file = ' {file}\n'

Seriously. That's it. No mention of that in the manual - they're supposed to be the same. In fact, the example of this in the manual uses double-quotes.

Oddly enough, using double-quotes worked just fine on the command line. I'm wondering if it has something to do with the fact that I'm using hgweb or something. Either way, that really sucked.

Well, it's been a while since I've posted anything here. That's because I've been busy with a new job.

The new job is doing LAMP development for a major online community site. True to form, I'm not (yet) going to name the company. That policy has served me well so far. However, I will share some details of my new position and some of the challenges I've faced with it, both expected and unexpected.

The first innovation with regard to this job is it's location - my house. Yes, it's a 100% telecommute, work-from-home position. In fact, it's a totally distributed team, so everybody on the development works from home. And while that's great, so far I've actually found it to be one of the biggest challenges with the job. The challenge isn't motivating myself to actually work, which my mother thought it would be (seriously, she was concerned about that). In fact, quite the opposite. Until recently, I've had more problems with spending too much time working.

There are a three main reasons for this. Part of it is because I was still learning our system and felt like I had to "catch up" to the rest of the team. Another part is simply schedule diversity. We communicate primarily via Skype chatrooms, so we can when people are online. And since we have developers literally from California in the west to Russia in the east, many of them work non-standard schedules, there's always somebody "on", which adds some subconscious pressure. And lastly, of course, part of it is probably emotional damage from my last job. At that (highly dysfunctional) company, we were allowed to work from home on occasion, but if we did it too often, the CEO would make needling comments to the VP of Engineering suggesting that "working form home" was a code-word for "taking the day off". So I think I've partly been stuck in the mindset of trying to prove that, "No, really, I am actually working!"

The other new feature of this position is the sheer size of the operation. And I mean that in pretty much every conceivable way - the amount of traffic, the infrastructure, the size of the code-base, the size of the development team. In fact, pretty much the only thing that isn't big (to me) is the size of the overall company. By way of comparison:

• The largest site I'd work on was when I was at ebaumsworld.com, where we had something like 30 servers in our environment and were ranked about 1200 in the world by Alexa. The current job has nearly 100 web servers alone and is ranked in the top 150 in the world by Alexa.
• The largest code-base I'd worked on was at my last job - a little over 200K total LOC when you include unit tests. The new job has over 500K lines of just PHP code (that's not counting CSS or JavaScript) in just the web portion of the tree.
• The largest group I'd worked in was when I was with the county IT department - about 30 people total, about 9 of them programmers, no collaborative projects to speak of. My last two teams have been very tight 4-man operations. At the new job, we have about 30 people on the tech team, all of them programmers and all of them very good.

So, needless to say, I've been very busy figuring things out in my new position. I've been there for about three months now and I think I'm finally getting comfortable. I'm still learning the code-base, of course (nobody understands the entire thing - it's just too big), but I'm getting more confident and feel like I'm making progress. So in other words, so far, so good.

And most importantly, I'm much happier than I was at my last job. It's nice to feel like you can trust your colleagues and that the work you're doing is helpful. Amazing how increased morale boosts productivity.