Powershell version of "which" command

As yet another note to myself (because I keep having to look this up), the equivalent of the UNIX which command in PowerShell is Get-Command.  That commandlet will let you pass in a name, with no extension, and return you the executable in your path that corresponds to that name.  This is really handy because it works with any executable, including native binaries and batch and PowerShell scripts.  So, for example:

PS pgeer> Get-Command rsync

CommandType     Name                 Version    Source
-----------     ----                 -------    ------
Application     rsync.cmd            0.0.0.0    C:\Users\pgeer\bin\rsync.cmd

The point of estimates

The other day I listened to a very interesting talk by Allen Holub on the #NoEstimates movement, which I've posted about before.  It's definitely worth a look if you have some time.  Mr. Holub makes the slightly controversial claim that all estimates are evil and we should all just stop doing them immediately.  His rationale for this is essentially that estimates are made up and harmful.  And, of course, he's not wrong.

I've always been a little skeptical of the #NoEstimates movement, but the thing it clearly get right is that estimates are frequently misused and misapplied.  Just about every developer has a story about estimates being interpreted as promises, or having to work late just to "meet your sprint commitments", or skimping on quality because the sprint is about to end and you need to close out your tasks.  And we won't even get into managers who try to use estimates as a performance evaluation tool - the dysfunction of this should be obvious to anyone with any experience.

Is it the estimates?

So estimates can be used for bad things.  That's not controversial.  But that's also not a problem with estimation per se.  I guess you could think of it as the software equivalent of "guns don't kill people, people kill people."  It's not that estimates are inherently bad, it's just that they're seldom used for good.  That doesn't mean that you should abolish estimates, but it does mean that you should use them with caution.

The thing about estimates is that, while they're often just guesses, they don't have to be.  In my study of the Personal Software Process (PSP), I learned about doing estimates based on historical data.  I've even seen it work - in my last job, I reached the point where my time estimates were within about 15% of actual task completion time, which I consider to be pretty good given that the estimates were in minutes.  Granted, it doesn't always work - sometimes you have tasks that are truly unprecedented or have too many unknowns - but it can be done.

The main problem with data-based estimation is that most organizations and developers are not equipped to do it.  For one thing, it requires data, and a lot of shops don't actually collect the kind of data that would be useful.  Most developers certainly don't collect the kind of data that the PSP estimation method uses.  It also takes more time than many people seem to want to devote to estimates.  It's not that much time, but still a lot longer than saying, "well, I guess that's about eight story points".

What's the point?

The real question here is: why do you want an estimate?  What are you trying to accomplish?

When I do PSP estimates, I'm using them to gauge my performance.  I'm looking at my historical data and using that to project out time, code volume, and defect counts.  I can compare these projections to my actual results, see if my performance is consistent, and if not, analyze the reasons and look for ways to improve in the future.

Yes, these are "estimates", but not in the same sense as most teams do them.  My manager never sees these "estimates"; I make no commitments based on them; nobody depends on their being accurate; in fact, nobody but me even knows what they are.  They're a part of my task planning, so they're short-term estimates - they go out two or three weeks at most, usually more like a few days.  I start with some requirements analysis, sketch out a conceptual design, and then do the estimate based on that.  So the process of coming up with the estimate actually helps me flesh out and validate my plan, which is valuable in and of itself.

The important thing to note in the above description is that my estimates are not targets.  If I'm working on a task and it takes 500% longer than my estimate, I'm the only one who knows and I don't care.  Sure, I'll try to figure out why that task took so much longer than I thought, but the fact that "I missed my estimates" has no relevance and no consequences.

But that's usually not how it works when your boss or your project manager (or even your scrum master) asks you for an estimate.  They want you to tell them when the task will be done.  And they'll be very disappointed if it's not done when you said it would be.  And that's where the dysfunction begins....  Because even in the best of circumstances, we don't really know exactly how long s development task will take.  And we seldom work in the best of circumstances.

So...no estimates?

The interesting thing about doing PSP estimates is that, until relatively recently, I had trouble not thinking of the estimates as targets.  Process Dashboard, my PSP support tool, has a progress bar that counts down the time remaining for your estimate and then turns red when you go over the estimate.  And I would actually get upset when that bar turned red!  Why?!?  There was no reason for that - there was literally nothing at stake!  I was just psychologically stuck in the wrong mindset.  And that's the same mindset that most people get stuck in when they talk about estimates.

So I'm starting to come around to this "no estimates" thing.  I still believe that estimates can be useful and productive, but they usually aren't.  And if your company is particularly enlightened and can do estimates in a way that's productive and avoids the pathological effects associated with them, then by all means, keep doing estimates.  But most people don't work in companies like that.  And even if your company is great, you still need to be careful of the psychological pressure created by the inherent biases and preconceptions of your developers.

In other words, estimate carefully.  Consider simpler, lower precision (but not necessarily lower accuracy) methods like Mr. Holub and the other #NoEstimates advocates describe.  Don't be fooled - those are still estimates, but they at least make an effort to work around the damaging side-effects associated with traditional estimation.  After all, it doesn't matter how good the estimates are if they destroy team morale.

TDD...it works?

For the last six months or so, I've spent a lot of time reading about the SOLID principles and listening to lectures about agile development practices by the likes of Bob Martin, Dave Thomas, and Martin Fowler.  The result has been a new and different understanding of what "agile" really means.

One of the agile practices I've really warmed up to is test-driven development. Uncle Bob advocates very strongly for it, and it was one of his talks that got me to finally try it again.  I'm too lazy to find the exact quote, but it was something like:

What if you had a button on your keyboard that you could press and it would tell you if your code was working?  How would that change your life?  Test driven development gives you that button.

Of course, that's a bit hyperbolic, but the point still stands.  The idea that you could have a tool that could tell you, "yup, this code works as intended" is really powerful.  Even if building such a tool is a lot of work, it seems like it would be worth it.

But the thing is, I've tried TDD before.  In fact, I've tried it for about a week or so every year for about the last five years.  And you know what?  It never really worked for me.  It was awkward, the tests were brittle and hard to read, and it just didn't seem to help me much at all.  I came away thinking that it was a stupid fad that didn't help and didn't really make sense. 

But things were different this time.  I'd spent months immersing myself in the SOLID principles, object-oriented design, and techniques for writing good tests.  This time I was ready.  It turns out that knowing how to structure your code and your tests makes the entire process infinitely smoother and simpler.  Heck, it's even kind of fun.

TDD Results

So how did it go this time?  Well, I'd say it was fairly successful.  I've been able to actually use TDD and stick with it without feeling like I was wasting my time.  That alone is a big improvement from my previous attempts.  But I can go deeper than that.  Thanks to the data I've collected using the PSP, I can actually do some analysis of just how well TDD is working for me.

About the Process

Before getting into the details, let's talk about what my process looked like before and what it looks like now.  That will help give some context to the numbers and make it more obvious why they look the way they do.

My process is based on the (now deprecated) PSP 3.  For each task, I start with a quick high-level design sketch, and then one or more development cycles, each of which consist of design, design review, code, code review, and then test.  The "design" phase consisted of conventional design and some skeleton coding (defining interfaces, defining UI components, outlining algorithms, etc.), the "code" phase consisted of fleshing out those outlines and implementing the rest of the design, and the "test" phase was writing unit tests followed by manual system-level testing.

The new TDD-based process changes that a bit.  The "design" phase is very similar, but I've been going into less detail in the algorithms involved.  The idea was to let the detailed design "emerge" in the process of writing tests, though I've started moving away from that a little.  The "code" phase is now a series of TDD cycles.  That means that it encompasses both actual coding and unit testing.  The "test" phase has become just manual testing.  I did this because, realistically, trying to track test vs. code time when I'm switching back and forth every few minutes was going to be tedious and error prone and I didn't want to do it. 

Unfortunately, this makes the analysis much harder because we're not comparing apples to apples.  Both processes did try to achieve comprehensive unit test coverage, so they're not completely incomparable, but the change in definition of the "testing" and "coding" phases does make certain comparisons moot.

About the Data

I've been collecting PSP data on the coding tasks I've had over the last six months at my current job.  Currently I have 25 data points - 15 using the old process and 10 using TDD.  The data is collected on a per-task basis, and the task sizes vary wildly, from about an hour and a couple dozen lines of code, to two weeks and a couple thousand lines of code.  The tasks are all webdev work and include a combination of back-end PHP, front-end functionality in JavaScript, and HTML/CSS display stuff in varying proportions.

Of course, the variety in the individual tasks undermines the data a bit, but that can't really be helped.  I'm just working with the tasks I'm given - I don't have the time to do a genuinely scientific comparison.I mean, I do actually need to get some work done at some point - I'm not getting paid to analyze my own performance.  

The Results - Summary

Overall, the TDD-based process seems to be working as well as, if not better than, what I'll call the "conventional" process.  Below is a summary table of overall metrics:

Summary Plan (TDD) Actual (TDD) Plan (Conv.) Actual (Conv.)
LOC/Hour 53.2 58.4 38.4 48.9
Total Time 73:18:00 107:41:00 84:32:00 102:30:00
Total New & Changed LOC 3903 6290 3249 5017
Test Defects/KLOC 21.3 15.3 14.4 26.7
Total Defects/KLOC 53.1 39.4 65.4 58.6
Yield %
55.90% 61.00% 59.00% 54.20%
% Appraisal COQ 16.00% 15.30% 16.40% 16.20%
% Failure COQ 32.30% 15.30% 34.00% 38.00%
PSP summary data for TDD vs. conventional process.
Definitions: Yield % = percentage of defects removed before testing, % Appraisal COQ = percentage of time spent on review activities, % Failure COQ = percentage of time spent in testing.

For the two data sets, the total time and code volume were substantial and in the same neighborhood, so the data sets are at least comparable.  We can see the TDD data shows higher productivity (in LOC/hour), fewer defects, better review yield, and lower cost of quality. 

So this means that TDD is definitively better, right?

Well...no.  The change in definition of the "code" phase means that I'm logging fewer defects in general.  Part of my defect standard is that I don't record a defect if it's introduced in the same phase where it's fixed.  So with TDD, little things like typos and obvious logic errors are caught by unit tests in the code phase, so they don't show up at all in these numbers.  In the conventional process, they wouldn't have been picked up until code review or testing.  The same reasoning applies to the failure cost-of-quality - since writing unit tests is now part of the code phase rather than the test phase, less time would naturally be spent in testing because there's simply less work that needs to be done in that phase now.

However, the productivity difference is still interesting, as is the improvement in percent yield. Those do suggest that there's some improvement from using TDD.

Drilling Down - Time

So since the change in phase definition makes defect numbers incomparable, let's look at time data instead.  This table shows the percentage of time spent in each phase.

Phase Actual % (TDD) Actual % (Conv.)
  Planning 3.08% 4.60%
  High-level Design 1.42% 1.59%
  High-level Design Review 0.87% 1.04%
  Detailed Design 8.05% 12.30%
  Detailed Design Review 4.27% 4.59%
  Code 51.90% 22.10%
  Code Review 10.20% 10.50%
  Test 15.30% 38.00%
  Postmortem 4.91% 5.35%
Percentage of time by phase.

 The results here are largely unsurprising.  The percentages are roughly equivalent for most of the phases, with the notable exceptions of code and test. 

However, it is interesting to note that there's still a discrepancy between the total code+test time for the two processes.  Theoretically, based on the phase definitions, that combination should encompass the same work for both processes.  But the TDD process spent 67.2% of the time on coding and testing, but it was only 50.1% for the conventional process.  It appears that the single largest chunk of that disparity comes out of design time.  This makes sense because there was less detailed design in the TDD process.

Note that the review times were roughly equivalent for each process - in fact, they were slightly lower with TDD.  Yet the percent yield for reviews was higher with TDD.  This gives us some evidence that the difference in percent yield can be attributed is likely due to the use of TDD.

It's not entirely clear why this would be the case.  The most obvious outcome would be lower yield, as the use of TDD means that fewer bugs escape to be found.  My working hypothesis, however, is that using TDD lowers the cognitive load of code review by removing most of the noise.  With TDD, you already know the code works.  You don't have to review for things like typos, syntax errors, or misused functions because you've already tested for those things.  That leaves you more time and energy to dig into more substantive issues of design, requirements coverage, and usability.

Back to Defects

Hmm....  Maybe we should take another look at the defect data after all.  We've already established that number of defects isn't going to be useful due to the change in phase definition.  But what about average fix times?  If TDD is weeding out the simpler bugs so that they don't escape the coding phase and don't get recorded, we'd expect the average fix time to be higher.

    Found in test (TDD) Found in other phases (TDD) Total defects found (TDD) Found in test (Conv.) Found in other phases (Conv.) Total defects found (Conv.)
Injected in HLD Tot. fix time - 28.4 28.4 - 18.2 18.2
Tot. defects - 9 9 - 13 13
Avg. fix time - 3.16 3.16 - 1.4 1.4
Injected in Design Tot. fix time 448.4 321 769.4 271.1 241.6 512.7
Tot. defects 48 57 105 51 63 114
Avg. fix time 9.34 5.63 7.33 5.32 3.83 4.5
Injected in Code Tot. fix time 223.2 293.3 516.5 341.6 230.4 572
Tot. defects 39 91 130 78 84 162
Avg. fix time 5.72 3.22 3.97 4.38 2.74 3.53
Total Injected Tot. fix time 724.9 647.1 1372 629.3 500.1 1129.4
Tot. defects 96 160 256 134 165 299
Avg. fix time 7.55 4.04 5.36 4.7 3.03 3.78
Defect count and fix time (in minutes) by injection phase.

The above table shows the break-down of defect count and fix time by injection phase.  So what does this tell us?  Well, my hypothesis that TDD would produce higher average fix times seems to be correct.  If we look at the fix times for defects injected in code and found in test, the average fix time is about 30% higher. 

However, it's worth noting that the TDD data has higher average fix times across the board.  It's not entirely clear why this should be.  One possible explanation is that the use of TDD means that unit tests are introduced earlier in the process, so defects fixed in code and code review would require changes to the test suite, whereas in the conventional process that would only happen for defects found in test.  That's something I'll have to watch in the future.

Conclusion

So far, my new TDD-based process seems to be working out pretty well.  The results are somewhat ambiguous, but the numbers suggest that TDD is resulting in higher LOC/hour productivity and more efficient defect removal. 

From a more human perspective, it has the benefit of making unit testing much less tedious and painful.  It gets you earlier feedback on whether your code is working and gives a nice sense of satisfaction as you watch that list of test cases grow.

But ore importantly, TDD is a great way to test your test cases.  When you're writing your unit tests after the fact, it's very easy to get stuck in a situation where a test is unexpectedly failing (or, worse, passing) and you can't tell whether the problem is with your test case or the code under test.  With TDD, you're changing things in very small increments, so it's easier to pinpoint the source of problems like that.

But the big lesson here is that TDD isn't something you can just jump into.  You have to understand the context and the design techniques first.  If you do, then it's pretty great.  If not, then you're going to have a hard time and end up wondering who came up with this hair-brained idea.

Making Windows Git Do SSH

Another quick note to my future self: setting up Git under Windows to use SSH key authentication is pretty easy...once you know what to do.

At work, we have some Composer and Bower packages that we fetch from our internal GitHub Enterprise server.  And, of course, all the source lines in the composer.json and bower.json files use SSH references.  I just use HTTP for my code checkouts, so I finally had to figure out how to make Git for Windows authenticate with my SSH key.

Turns out it's pretty easy.  I have a key pair I created with PuTTYgen.  All I had to do was export my private key to OpenSSH format and copy that file to C:\Users\<my_username>\.ssh\id_rsa.  Git picks it up with no further configuration.  Then I just added my key to GitHub and I was good to go.

JavaScript unit testing

Today I watched a very interesting talk on good and bad unit testing practices by Roy Osherove, entitled "Unit Testing Good Practices & Horrible Mistakes in JS".  It really was an exceptional talk, with some good, concrete advice and examples.  I highly recommend it.

I actually recognized a couple of the examples in the talk.  The big one was OpenLayers.  I still remember being horrified by their test suite when I was using that library at my last job.  We had a number of extensions to OpenLayers and I was thinking about adding some tests into their test suite.  Then I ran the tests and saw a bunch of failures.  Assuming our code must be bad, I tried a fresh copy of the OpenLayers release.  A bunch of tests still failed.  And that's when I gave up and decided to just pretend their tests didn't exist.

If nothing else, the talk provided some nice validation that my JavaScript unit testing practices aren't completely wrong and stupid.  I've been writing unit tests for my JavaScript as part of my standard process for the last six months or so, and I was actually watching the video waiting for the hammer to drop, for Roy to describe an anti-pattern that I'd been blissfully propagating.  I was pleased and somewhat surprised when that moment didn't come.

Using PuTTY inside ConEmu

This is yet another one of those "post this so I can refer back to it later" things.  So if you're not a Windows user, or if you don't use ConEmu, then I suggest you go get a cup of coffee or something.

So for a while now I've been using ConEmu as my Windows console app.  It supports multiple tabs, transparency (ooooh), customizable hotkeys, customizable sessions, Far manager integration and a whole bunch of other nifty stuff.  

A couple of months ago, I saw that it was possible to embed PuTTY, the popular Windows-based SSH client, directly in a ConEmu tab.  So I tried it out and found it to be pretty slick.  The only down side was some key binding weirdness.

First, there's the general putty issue that if you accidentally press ctrl+S - you know, the key combination that means "save" in just about every editor in existence - it effectively locks the terminal and it's not obvious how to get control back.  The second issue is that, when you embed an application like PuTTY in ConEmu, it steals most of your keyboard input, so the standard key bindings for switching between window tabs don't work.

Luckily, these problems are easily fixed.  The fixes are just hard to remember, which is why I'm writing it down.  For the ctrl+S iissue, you can just hit ctrl+Q to turn input back on.  For the tab-switching issue, you can use the global key bindings for ConEmu - namely Win+Q, Win+Shift+Q, and Win+<Number> to switch consoles, as well as Win+Z to toggle focus between ConEmu and PuTTY.

 

Fixing Lenovo IdeaPad function keys

Note to self: you actually can turn the function keys on your Lenovo IdeaPad back into actual function keys.  It's just a BIOS setting.

For anyone else who comes across this, Lenovo's IdeaPad series of laptops (at least the version I have) does weird things with the function keys.  You've probably seen those keyboards where the F1 through F12 keys have alternate functions.  They often have a "function lock" button that's equivalent to caps lock or scroll lock and you can turn it off to access the alternate functionality. 

What Lenovo does is similar, except that the alternate functionality is on by default and there's no function lock button.  There's just an "Fn" button that you have to press and hold to access the normal function button keystrokes.  So on my laptop, F11 and F12 turn the screen brightness up and down and I need to press Fn+F12 to do a regular F12 keystroke.

This is probably great for "regular" users, most of whom wouldn't miss the function keys if they went away completely.  On the other hand, for a developer, this adds a second keystroke to a row of what were previously absurdly convenient hotkeys.  Granted the volume and brightness keys are convenient, but when I'm coding I use F11 and F12 for my IDE keybindings much more often than I need to change the screen brightness.

But luckily, as I found in the link above, you can turn that off in a BIOS setting.

PSP Break-down, part 4: Results

Welcome to the end of my series of PSP posts.  In part one, I started with an overview of the Personal Software Process.  Part two covered the process of learning about the PSP and how to apply it.  Part three was the sales pitch of nice things that the PSP is supposed to enable you to do.  Now, in part four, we'll get down to brass tacks and talk about how well the PSP actually works for me.

I'll start with a quick overview of the good, bad, easy, and hard parts.  Then I'll dive into a deeper discussion of my experiences and the details of what did and didn't work.

PSP at a Glance

This table gives you a nice little overview of my experience.  It rates various parts of using the PSP on two axes - whether I judge them to be useful or not (good/bad) and how difficult they are to apply in practice (easy/hard).

 GoodBad
Easy Maintaining process
Defect tracking
Time tracking
Size tracking
Code reviews
PROBE estimates
Reviewing on paper
Setting up tool support
Test report template
Coding standards
Hard Planning process
Design reviews
Postmortem analysis
Defect data analysis
Creating relative size tables
Creating review checklists
Design templates
Design verification methods

As you can see, I find the benefits of the PSP to outweigh the drawbacks.  For me, it's useful enough that I plan to keep using it, in some form, for the foreseeable future. 

Using the PSP

Despite what you might read about how cumbersome the PSP is to use, I actually didn't find it that difficult at all.  Granted, it takes a little getting used to - every process change does.  But once I had the basics down, actually using and sticking to the process wasn't that hard.  The use of written scripts with well defined entry and exit criteria helps keep you honest and disciplined.

Likewise, with the help of Process Dashboard, I found the entire data collection process to be relatively painless.  There are a few pain-points, of course.  For me, the biggest one was simply properly configuring a line counting tool so that you can measure project size.  This is actually more annoying than you'd think.  I use the integrated line-counter in Process Dashboard to count change size and cloc to measure initial size for estimation purposes, mostly because I've integrated it into my IDE.  The Process Dashboard tool has some nice features, but will require you to write custom language definitions for pretty much anything that doesn't use C-style syntax.  It uses a fairly easy to follow XML format for configuration, but still....  Cloc has much better language support, but is harder to customize.  As a further annoyance, while Process Dashboard does have VCS diff support, it currently only supports Subversion.  So if you're using Git, Mercurial, or anything else reasonably modern, you'll have to set up two copies of your local repo for before and after comparison.

Code Review

Personal pre-commit code review is one of those things that every responsible developer does, but hardly anyone seems to talk about.  I know I've been doing an informal version of it for years.  It simply consisted of looking over the diff of my changes before hitting the "commit" button and making sure that I didn't make any obvious mistakes, that I'm not checking in any changes I don't mean to, etc.  It's something you quickly learn to do after making a few embarrassing mistakes.

Doing an organize code review with a checklist really takes this to the next level.  Instead of being a CYA thing, code review becomes a way to preemptively find problems in your code.  And at its best, it can be hugely effective.  As any experienced developer knows, there are classes of problem that are hard to find in testing, but stick out like a sore thumb when you actually stop and read the code.

The only hard thing about code review is actually customizing your review checklist.  I find that it's difficult for me to do that simply by looking at the defect categories in my data because those seldom tell you anything actionable.  There are a few defect categories that readily translate into checklist items, but there are many defects that are more subtle and are either difficult to categorization or difficult to generalize into checklist items.

The one thing I really didn't like about the PSP code review process was the recommendation to do it on paper.  Using paper does have the benefit that you can get more code in front of you at a time, and it is easier to make annotations.  However, it also bypasses the navigational and analysis power baked into modern IDEs.  For instance, Komodo let's me easily navigate between functions, access standard library documentation at a click, and search for uses of an identifier.  Those things are much more tedious to check on paper. 

But the big kicker for me was that trying to review a diff on paper is just painful.  It's hard enough on screen with color highlighting, but it really sucks on paper.  And on paper I don't have things like the Komodo diff viewer's feature to jump from a diff item to that location in the file to view the context.  It might work well to review new code on paper, but for changes to existing code it feels really clunky.

Design Reviews

While we're on the topic of reviews, let's talk about design reviews for a minute.  Again, this is a really good idea, for the same reasons that code review is a good idea.  And the PSP does offer some productive advice in the recommendation to adopt a design standard and a checklist for common errors.

However, the specific methods the PSP recommends just don't work for me.  The four standard design templates are a nice idea, but they feel very repetitive and clunky to work with.  And even if I switched to the UML equivalent, the recommended list of artifacts is just too much for a lot of the things I do.  And at the risk of having my CS degree rescinded, I have to admit that I have trouble constructing a state machine for many of the projects I do - at least, one that's even remotely enlightening.  In general, I just find them painful to work with and biased toward "new program" development rather than incremental enhancement.

And the design verification techniques are even worse!  They're time-consuming and tedious - you're basically executing your program on paper.  It's a nice idea, and might come in handy occasionally, but frankly, I have less confidence in my ability to perform those verification exercises correctly than I do in my code being correct in the first place.  And, again, they're just way too heavy for most of the projects I do.

I'm still working on finding a design and design review approach that's sustainable for me.  Since my last few shops have used agile methodologies, most of the "projects" I do are fairly small enhancements to existing code - usually just two or three days, seldom more than a week.  So a heavy-weight design process with lots of templates or UML diagrams just isn't going to work. 

My current approach is as follows (note that I'm using a variant on the PSP3 process in Process Dashboard):

  1. Sketch out a high-level design in a word processor.  This is a refinement of the conceptual design used for planning, usually in the form of plain prose and bullet lists.
  2. Review that primarily for feasibility, completeness, and requirements coverage.
  3. For each component I've broken out, do a more detailed "transient design" (I'll describe that in a moment).
  4. Review the transient design primarily for completeness, correctness, and requirements coverage.

I refer to the detailed designs as "transient designs" because I don't actually create separate documents for them.  I blend implementation and design and actually do the design right in the source code.  I generally stub out things like classes and methods and fill in the details either with actual code (for simpler items) or with "design annotations", which are just comments that use a special formatting to mark them as design artifacts.  Sometimes they're pseudo-code, other times they're just descriptions of what needs to be done, whatever seems appropriate.  Then, in the code phase, I simply replace those annotations with the actual implementation.  It's certainly not perfect, but it seems to be working well enough for me so far.  As a next step, I'm going to look at a TDD-like approach and try incorporating unit test definitions as one of the design artifacts.

Estimation

In my experience so far, PSP estimation using PROBE actually works remarkably well. In the data from my last job, I was eventually able to get to the point where my actual development times were generally within about 15% of my estimates.  I consider that to be pretty good, especially when you consider that the estimates were done in minutes and based on fairly sketchy user-stories.

Of course, estimation is a learned skill, and PROBE doesn't change that.  You still need to be able to accurately account for the possible changes when constructing the conceptual design.  And as with any data-driven approach, the results are going to be sensitive to the quality of your data.  So if your relative size tables are just made up rather than being based on your past work, then don't expect your estimates to be too accurate.

It's also important to note that there's a bias against project diversity here.  For example, line counts can differ wildly for different programming languages, different problem domains, etc.  So if you tend to work on projects that are generally very similar to each other, then PROBE will work much better than if all your projects are widely divergent.  My data from my last job is based largely on a single code-base, so while the purposes of the individual projects varied wildly, the technology stack was consistent.

The hardest part about estimation, at least for me, is coming up with those relative size tables.  It's one of those things that sounds easy, but actually isn't.  For one thing, I don't have a tool to automatically count lines and methods in classes - much less one that works across a diversity of languages.  For another, my data largely comes from work, which is a problem because I do mostly web development on a team, which means I need to extract my method and class size data from a code base written in five different languages by five people.  When you wrote a quarter of the methods in one class, half of a third of the methods in another class, etc., how do you count all that?  You can just forget that and count the files you have, but then it's not really your data, so it's not clear how useful it will be. 

I've also found it challenging to come up with useful categorizations for my relative size tables.  Perhaps it's just the products I've been working on, but I end up with a whole lot of database-related classes and a smattering of other categories.  That's fine for those products, it's hard to figure out how to extrapolate that to other kinds of projects.  My suspicion is that that's a result of sub-optimal system design.  I'm currently trying to adhere religiously to the SOLID principles, which should result in more and more targeted classes, which should solve that problem.

Other Bad Things

There are a few other annoying things about the PSP as Humphrey describes it.  While the general focus on templates and checklists is not bad in and of itself, their value tends to vary.  The test report template, for instance, is one that I've not found particularly valuable.  While it is useful to sketch out your testing strategy, or maybe make a quick checklist of your test cases, the test report template is more like something that you'd give to a QA team to do manual testing.  It has a bias towards verbosity that makes it seem like more effort than it's worth.

Likewise with the focus on coding standards.  We can all agree that having coding standards, and following them consistently, is a very good thing.  Everyone should do it.  However, I've been working as a software developer for a long time now and my "coding standard" is something I've long since internalized.  I don't need to spend time formalizing it or checking it in my code reviews.  Ditto the size counting standard.  You can get really fancy if you want to, but to I'm not convinced that anything much more complicated than counting physical lines is likely to be helpful.  I suspect that, at least for my purposes, any elaborate counting standard would just serve to complicate measurement.

And, of course, there's the simple fact of process overhead.  It's really not too bad when you use Process Dashboard, but it's still there.  For example, there's a non-trivial amount of work that goes into configuring Process Dashboard itself.  It's a useful and powerful tool, but its not always simple to use.  There's also the analysis time use to assess and correct your process.  This is unquestionably valuable, but it's still some additional time that you need to plan for.  And, of course, there's just the time to actually follow the process.  This isn't actually that much, but for very small tasks (e.g. one or two hours), your estimates might be thrown off by the fact that your standard phase break-down results in a phase that's one or two minutes, and it takes you longer than that just to type in the data you need.

And Some Good Things to End On

Last but not least, I wanted to highlight two more "good but hard" things from the table above: the planning and postmortem stages.  At first, these seemed like silly, pro forma phases to me, but they're actually quite valuable.  And the most valuable thing about them is something that doesn't show up in the script: they make you stop and reflect.

The planning phase forces you to think about what you're doing.  To construct an estimate, you have to think about the project you're trying to do, break it down, and define its scope.  Even if you don't believe there's value in the estimate itself, simply going through the process gives you lots of good insight into just what you're trying to do, which reduces the number of surprises later in the development cycle and makes everything go smoother in general.

Likewise, the postmortem stage prompts you to reflect on how you're doing and figure out how you can improve.  It's like a one-man sprint retrospective.  And like the sprint retrospective, it's actually the most important part of the process.  The simple task of looking at your statistics and filling out a Process Improvement Proposal forces you to stop and focus on your performance and what you can do to improve your work.  I find that simply looking at that PIP line in the postmortem exit criteria keeps me honest and makes me stop and think of something I could improve.  And if you can't thing of at least one thing you could do better, you're just lying to yourself.

Conclusion

So there you have it.  In many ways, the PSP is working well for me.  Some of Humphrey's suggestions work well out of the box, some don't, and some need tweaking.  I do find that my process is evolving away from the "standard" PSP, but that's neither bad nor unexpected.  The basic techniques and ideas are still useful to me, and that's what matters.

Upgrading Mercurial on shared hosting

Disclaimer: This is yet another "note to self" post.  If you're not me, feel free to ignore it.

After God alone knows how many years (at least six, since I have posts related to it from 2010), it's finally time up upgrade the version of Mercurial that I have installed on my shared web hosting account.  This is a shared hosting account with no shell access - nothing but web-based tools and FTP.  I also don't know what OS it's running - just that it's some form of Linux.  So I've been putting this off for obvious reasons.

Unfortunately for me, the defaults for repository creation in Mercurial 3.7 turn on general delta support by default.  That isn't supported by the old version I was running (1.7), so my choices are to either use the now non-standard, older, and less efficient format my repositories, or just bite the bullet and upgrade.  So I did the latter, since the version I had was pretty ancient and I was going to have to do it eventually anyway.

Fortunately, my hosting provider supports Python 2.7, which gets you most of Mercurial.  However, there are some C-based components to Mercurial.  Since I have no shell access to the hosting server, and there are probably no development tools installed even if I did, I had to try compiling on a VM.  I was able to do that by spinning up a Fedora 24 VM (on the assumption that they're running RHEL, or something close enough to it), and doing a local build.  The only caveat was that apparently my provider is running a 32-bit OS, because building on a 64-bit VM resulted in errors about the ELF format being incorrect.

Once the Fedora VM was up and running, I was able to do a build by running the following:
sudo dnf install python-devel
sudo dnf install redhat-rpm-config
cd /path/to/mercurial-3.8.x
make local

That's about it.  After I had a working build I was able to copy the Mercurial 3.8 folder to the server, right over top of the old version, and it just worked.  Upgrade accomplished!

Fixing Synaptics right-click button

Tonight I finally got around to fixing my trackpad.  Again.

So here's the thing: like most Windows-based laptops, my little Lenovo ultrabook has a trackpad with two "button" regions at the bottom.  They're not separate physical buttons, but they act as the left- and right-click buttons.  However, rather than force you to use the right-click pseudo-button, the Synaptics software the comes with the laptop lets you turn off the right-click button and use two-finger-click as the right-click, a la a MacBook Pro.  I find this preferable, in particular because my laptop centers the trackpad under the space bar, rather than in the actual physical center of the unit, which means that when you're right-handed, it's easy to accidentally hit the right-click button when you didn't mean to.

Prior to upgrading to Windows 10, I had this all set up and it was fine.  After the upgrade, not so much.  Sure, the same options were still there, but the problem was the that option to turn off the right-click button did just that - it turned it completely off!  So rather than clicking in that region doing a standard left-click, that part of the trackpad was just dead, which was even worse than the problem I was trying to fix.

Luckily, it turns out that the functionality is still there - the Synaptics people just need some better UX.  I found the solution in this forum thread on Lenovo's site.  It turns out you can just change the value of the registry key HKEY_CURRENT_USER\Software\Synaptics\SynTP\TouchPadPS2\ExButton4Action to 1, which is apparently the standard left-click action.  By default, it's 2, but when you turn off the "Enable Secondary Corner Click" (i.e. right-click to open context menu) feature in the Synaptics UI, that gets changed to 0, which is apparently the "do nothing" action.

Long story short: my mouse is much better now.