Powershell is not BASH and SVN pain

Note to self: just because Powershell defines aliases that mimic many of the standard UNIX commands does not mean they function the same way.

Last night, I was trying to migrate my company's Subversion repository to Mercurial - not for production use (yet), just as an experiment. After eventually getting the latest Mercurial installed on the Ubuntu 8.04 VM that hosts our Subversion repository, I tried running hg convert -s svn /path/to/svn/repo /path/to/hg/repo. As expected, the conversion process took some time, but chugged along nicely...for a while. Eventually, it hit an error and came back with:
svn: In file '/build/buildd/subversion-1.6.9dfsg/subversion/libsvn_ra/ra_loader.c' line 595: assertion failed (*path != '/')

I Googled around a bit, but still have no idea what that error message means or how to fix it. My best guess is that something is borked in our repo - not broken enough to break SVN, but maybeSo I tried a different tack - take the repository dump I had, import it into a fresh repository, and try again. That didn't go so well....

Since the SVN VM has a very small drive, I decided to load the dumpfile on my local Windows box. As you may know, svnadmin load reads streams, so you have to either pipe the dump file in or redirect standard input. Well, my first instinct was to do something like this:
svnadmin load newrepos < dumpfile.repo
One problem with that: the "<" character that you normally use for redirecting STDIN is reserved in Powershell. Drat! So I figured I'd just use a pipe instead:
cat dumpfile.repo | svnadmin load newrepos
So I ran that and waited. And shortly after I started waiting, I noticed my system slowing down. And then things started grinding to a halt - it was just barely responding. When I finally managed to get resmon up, I noticed that Powershell was eating nearly all of my system's RAM! And the command still hadn't produced a single line of output!

I'm not sure exactly what Powershell was doing, but it must have something to do with the Get-Content commandlet (for which "cat" is an alias) not liking the 1.4 GB dump file. Why it would use up more than twice the size of the file in memory, I'm not sure.

Anyway, I just switched to cmd.exe and did the input redirection method, which didn't eat huge amounts of memory. However, it didn't work either. The import died shortly after starting with an error about a bad transaction. Looks like the gods of revision control are not smiling on me today.

PHP uploads on IIS

I sometimes forget that UNIXy things aren't as easy on Windows.

Today I spent way too long trying to figure out why PHP file uploads weren't working on IIS 7. First, I was getting permission denied messages on "C:\Windows\Temp". OK, that's fine, we'll just change upload_tmp_dir to something else, say "C:\Inetpub\wwwroot\uploads".

So I change that in my php.ini and restarted IIS. The result? Same permission denied message on "C:\Windows\Temp". Hmm.... Apparently, according to the docs, if you don't have write permissions on the upload_tmp_dir, PHP falls back to the system default. So I checked the permissions - IIS_IUSERS has read/write permissions. Apparently that's not good enough. I ended up adding write permissions for the "Users" group, which didn't sit well, but worked. However, after I removed write permissions for that group, just to check if that was the fix, it still worked. (Edit: Apparently it actually didn't work after that. Man, I really must have been out of it last night.)

So now, uploads work, but I have no idea what's going on or what the proper ACLs are. It's probably just too late and I need to go to bed. But for future reference, make sure to change the permissions on Windows when working with uploads.

FeedDemon upgrade

Yesterday I upgraded FeedDemon to version 4. In the process, I did something new - I paid for a license.

For the last year or two that I've been using it, FeedDemon has been ad supported. The full feature set was available for free, but there was a little ad in the lower left-hand corner. If you wanted to get rid of that, you could buy a license key, but frankly, I never really cared that much.

Now, however, the business model has changed. There's now a free "Lite" version with some missing features and a paid "Pro" version with all the features and no ads. One of the annoying parts of this was that the features in the "pro" version are not all new - the for-pay features include some things that were previously free.

This bothered me a little for a couple of minutes. Then I read the FeedDemon author's article about why he changed the business model. Apparently, FeedDemon is no longer run by NewsGator. It's still branded for them, but they let the author, Nick Bradbury, go and returned control of the program to him. Apparently he's now doing the independent developer thing and making his living selling FeedDemon.

Well, that pretty much sealed the deal. I've tried a number of RSS readers - Linux, Windows, and web - and FeedDemon is the only one I've ever really liked. This is a program I use every day, I want some of the "pro" features, and buying it would help out a fellow developer. And to top it all off, he's only asking a lousy $20 for it. That's a no-brainer. So I bought a license and I'd encourage any other FeedDemon users to do the same.

Incidentally, some of the comments on Nick's article were really depressing. With users like some of those, I don't envy him trying to make his living off FeedDemon. Supporting a one-man project is tough enough when you release it as open-source and you don't owe anything to anyone. It must be hard trying to do right by people who paid you money when they feel your work is lacking - even (especially?) if they're clearly being unreasonable. Working 9 to 5 (well, officially 8 to 5 in my case, but you get the idea) might not be sexy, but at least it's stable and they pay someone else to deal with unhappy customers.

One thing I did notice in the new version that I found a little disappointing was the change in some of the newspaper themes. FeedDemon basically displays feeds as an HTML document and has a number of themes that control the look of the feed list and allows you to set themes on a per-feed basis. Changing the theme is now a "pro" feature, and I really wanted this, as I have several feeds that I have set to use the Outland theme, which is a nice, clean theme that displays full entries rather than just summaries. However, this theme is gone in FeedDemon 4.

The up side is that, while the theme format may have changed in FD4, it hasn't changed much. The "styles" are still just that - CSS and XSLT. In fact, I still had the Outland.fdxsl2 theme file sitting in my "C:\Program Files (x86)\FeedDemon\Data\Styles" directory, so I just tried renaming it to Outland.fdxsl4, restarting FD, and guess what? It worked! The only real problem was that the border on the feed was a little off, so I commented out the border on the HTML element (line 24), and it was just fine.

Mercurial hooks

Last time I mentioned that I'd set up Mantis with a Subversion hook. Well, I've been rethinking the Subversion part.

I was listened to a back episode of Hanselminutes last week in which Scott was interviewing SourceGear's interviewing Eric Sink about DVCS and it was a very interesting conversation. Source control is one of those things you don't usually think about as a developer. You learn how to use a source control system that's "good enough", and after that, just sort of take it for granted. You certainly don't spend a lot of time jumping back and forth between systems and trying out the possibilities.

I really liked the way Eric explained distributed version control. As he pointed out, most of the arguments for DVCS that you hear seem pretty pointless, such as "There's no central server," or "You can commit on an airplane." Well...so what? Why wouldn't I want a central server - how else do you determine the authoritative repository? And honestly, who does important work on an airplane?

Instead, Eric framed the discussion in terms of flexibility. The flip-side of the above stupid arguments is actually pretty compelling. With DVCS, there's no central server, but that just means that your central server is determined by convention, not by the software, and it's much easier to set up "slave" repositories for remote sites. Likewise, while being able to commit on a plane is pointless, there's definite value in not needing to have a connection to the server to do core version control operations. Or, to put it another way, I may not code on a plane, but I do often code in coffee shops with slow WiFi connections.

So, in light of that, I decided to give Mercurial a try. According to Eric, it's fairly user-friendly, especially for Subversion users, and it's fairly Windows-friendly (it even has a TortoiseHg client, for whatever that's worth). So, since I was thinking of resurrecting LnBlog, I decided to use that as my test-case.

That leads me back into the Subversion hooks tie-in. Since I went to all the trouble of setting up an issue tracker and hooking it to SVN, I figured I'd do the same with Mercurial. Fortunately, it wasn't too bad once I figured out what I needed to do. I was able to fairly easily adapt the Powershell script I wrote for Subversion to work with Mercurial. In fact, the Mercurial version was actually shorter.

Actually adding a commit hook in Mercurial is pretty simple. You just add a line to your .hgrc file:
[hooks]
commit = powershell \path\to\hg-commit-hook.ps1

It seems that Mercurial preserves the environment for hooks, so you don't seem to have to worry about absolute paths and such like you do in Subversion.

The changes to the script itself were fairly small. This wiki page had a few good examples that got me started. The two big things were the passing of data and generating the message to pass to Mantis. Mercurial actually passes data into hook scripts through environment variables rather than command-line parameters, which is nice in that you actually get meaningful variable names coming in. As for the message generation, Mercurial's log command allows you to specify a template for its output, including substitution variables and some simple formatting functions. The result is a nice, short script with only a couple of calls to hg:

[System.Reflection.Assembly]::LoadWithPartialName("System.Web") | Out-Null

$issue = hg log -vr $env:HG_NODE

# If we don't have an issue number, just exit.
if ($issue -match "\b(?:bug|issue)\s*[#]{0,1}(\d+)\b") {
   
   $style_file = @'
changeset = "Changeset {branches}{rev}:{node|short} by {author|person}, {date|date}\n{desc}\n{files}"
file = " {file}\n"
'@
   Write-Output $style_file | Out-File -Encoding ASCII -Width 1000 hgstyle

   $data = hg log -vr $env:HG_NODE --style hgstyle
   rm hgstyle

   # Keep the cast to string from clobbering line breaks.
   $data = [String]::Join("`n", $data)
   $postData = "secret=somePassword&message="
   $postData += [System.Web.HttpUtility]::UrlEncode($data)

   C:\Cygwin\bin\wget.exe -O req.out --post-data=$postData http://yourhost.com/path/to/mantis/scripts/do_checkin.php
   rm req.out
}

That's it - a measly two calls to Mercurial, including one to see if we even need to run the hook. By specifying the right template, we can get the entire message in one command. That template gives me something like this:
Changeset 70:cbdb625298ca by Peter Geer, Mon Dec 13 16:45:53 2010 -0500
Got rid of unneeded requires now that we have an autoloader (issue #0000005).
   lib/article.php
   lib/creators.php

Perhaps not quite as pretty as the Subversion version, but close enough for now. Perhaps if I feel like getting really fancy, I'll look at the Python API for Mercurial and try my hand at writing some in-process hook code.

Edit: Updated the script to put each of the listed file on a new, indented line.

Bug tracking and SVN hooks for Mantis

I came across an interesting article on Task Driven Development the other day. I came to it via the recent Linus on branching story on Slashdot.

One of the first things mentioned in that article is that you need an issue tracking system and it needs to be integrated with your source control system. After a moment's consideration, I said to myself, "That's a good idea." I've been thinking lately about getting back into doing some open-source stuff as well as some educational projects for myself, and something like that might be helpful. In the past, I usually didn't bother to track bugs and features in my personal projects, unless you count post-it notes and unorganized text files, because, well, they were personal projects and it just didn't seem important. However, now I'm used to living and dying by Jira and am fully aware of how crazy it is to think I'm just going to remember all the bugs I find and potential features I think up.

So, to that end, I installed a copy of Mantis on my site. I settled on Mantis, after a good 5 minutes of careful research, based largely on the facts that the article recommended it and that the requirements were low enough that I could run it on my cheap-ass shared hosting account. Fortunately, it seems nice enough so far - not radically different from Bugzilla or Jira, but OK.

Anyway, I came across a slight snag in setting up the Mantis integration with my Subversion repository. You see, Mantis supplies a script that can be called from Subversion's post-commit hook that will update any issues referenced in the commit message. However, the script is written on the assumption that Mantis and SVN live on the same box. In fact, that is not the case - SVN lives on my local Windows 7 box, while Mantis lives on my hosting provider's box, to which I do not have shell access (yes, I'm that cheap).

With a little Googling, I was able to turn up a few useful resources to help me get this set up. This article gave a nice overview of SVn hooks and Mantis integration, while this one provided a nice perspective for SVN on Windows. However, I had to cobble these together to make it work on my system.

My main problem with the documentation I had was my lack of SSH access to the box on which I'm running Mantis - all the articles I found assumed you had it. So the first order of business was a simple gateway PHP script to wrap the command-line Mantis script that would normally be run via SSH. Therefore I quickly banged out this do_checkin.php script and dropped it in a web-accessible directory.

<?php
# Very cheap security measure, simply so that just *anybody* can't post issues.
if ($_POST['secret'] !== 'somePasswordHere') {
   exit;
}

$cmd = 'php checkin.php <<< ' . escapeshellarg($_POST['message']);
system($cmd);

Next was the Subversion hook. I'm using SlikSVN, and it turns out that this will only execute hook scripts that are .exe or .bat files. I know this because I tried to write a Powershell hook script and it didn't work. But no worries - it's easy enough to write a wrapper batch post-commit.bat file such as this:

@ECHO OFF

SET REPOS=%1
SET REV=%2

CD %REPOS%\hooks
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe %REPOS%\hooks\post-commit.ps1 "%REPOS%" "%REV%"

For the Powershell script, it was easy enough to just port the BASH script posted in the previously linkd article. However, instead of calling the Mantis checkin.php script by SSH, I actually just used wget to send a POST to the do_checkin.php posted above.

[System.Reflection.Assembly]::LoadWithPartialName("System.Web") | Out-Null

$REPOS = $args[0]
$REV = $args[1]

"$REPOS $REV"

$svnlook = 'C:\Program Files\SlikSvn\bin\svnlook.exe'

$auth = & $svnlook author -r $REV $REPOS
$dt = & $svnlook date -r $REV $REPOS
$changed = & $svnlook changed -r $REV $REPOS
$changed = $changed -join "`n"
$log = & $svnlook log -r $REV $REPOS
$log = $log -join "`n"
$data = "Changeset [${REV}] by $auth, $dt
$log
$changed"

$postData = "secret=somePasswordHere&message="
$postData += [System.Web.HttpUtility]::UrlEncode($data)

C:\Cygwin\bin\wget.exe -O req.out --post-data=$postData http://yoursite.com/path/to/mantis/scripts/do_checkin.php
rm req.out

This seems to work fine in my testing. We'll see if that changes once I start adding in bugs and features for my projects and then writing the code to complete them. Hopefully that process will be somewhat easier for being more organized.

Ron Jeffries rocks

Last night I spent some time browsing Ron Jeffries web site. While I've seen a few of the items on Ron's site before, I never really stopped to look through - and boy was that a mistake! It turns out that Ron has a number of very interesting and entertaining articles on his site. I particularly recommend his classics section.

Included in the "classics" is an article entitled We Tried Baseball and It Didn't Work. Unless you're lucky enough to have only ever worked in high-quality organizations, which really, really had their act together, this article will probably ring true on some level. It also nicely sums up a number of Ron's other articles on the use and abuse of the term "Agile".

This kind of reminded me of when my current team first started to "do Scrum". For the first few months, the entire team agreed that it just wasn't working. Of course, we weren't exactly following the book in our application of Scrum: our "stand-up" meetings often lasted 30 to 60 minutes; we didn't always deliver "done-done" software at the end of a sprint; our sprints started at a month long, and when we failed to finish our tasks by then, we pushed the date. So "Scrum wasn't working" for us, if by Scrum you mean "some process that involves a daily meeting." But we had to do Scrum, or come up with some other "Agile" process, because if it's not Agile, then it's not good. I mean, how could all those industry pundits be wrong?

Fortunately, we've figured things out a bit better now. We're much closer to a real Agile process and it's working better for us. Of course, we're not really there yet, but we're making progress. And, really, that's what matters.