I discovered something enlightening and annoying this morning: PowerShell uses different execution policy settings for the 32-bit and 64-bit versions. This is something I did not know and did not expect.
I'd seen this problem before, but could never figure out what was happening. I'd try to run a command through PowerShell using another program, in this case Vim, and it would fail with the standard error about execution of my profile.ps1 being prohibited by the execution policy setting. Yet when I bring up Powershell on the command line, everything works and the execution policy looks correct:
PS pgeer> Get-ExecutionPolicy -List
Well, it turns out we're talking about two different versions of Powershell. When I run it from the terminal, I'm using the 64-bit version. But my Vim build is 32-bit, which apparently means that it's running the 32-bit version of Powershell. And that version uses its own execution policy setting. You can confirm that by explicitly opening "Windows Powershell (x86)" from the start menu and running
As for the fix, it's simple: just run
Set-ExecutionPolicy RemoteSigned as admin, but do it in both the x86 and x64 versions of Powershell.
As for why Microsoft chose to make the different builds use different execution policies, I couldn't say. It doesn't make much sense to me. Although, to be honest, I'm not sure why I even need both versions on the same system. Maybe to support interop between commandlets and unmanaged code? Who knows?
In my last post about KeePass, I mentioned that you can integrate your KeePass password database with your web browser. In this post, I'll tell you more about how to do that and why it's an extremely handy thing.
So why would you want to bother with integrating your browser with KeePass? I mean, most browsers have a feature to remember your passwords anyway, so why not just use that? Or if you want to use KeePass, why not just use that auto-type feature I talked about in the last post?
It's true, you could just use the password manager that's built into your browser. Pretty much all of them have one, these days. Most of them will even secure your data with a master password. They may even synchronize your passwords to the cloud, so you can access them on more than one device. Granted, that's pretty handy.
However, browser password managers generally just do passwords - they don't allow you to enter extra information or attach files like KeePass does. They also don't work for things outside the web browser, like for security software such as VPN clients. So they don't provide you with a single, secure location for all your important account information. But more importantly, they're generally tied to a single browser. Sure, Google Chrome can store and synchronize all my passwords, but what if I decide I don't like Chrome anymore? Maybe I just bought a Mac and decided I really like Safari. Is there an easy way to get my passwords out of one browser and into another? I don't know.
By using KeePass with a plugin for your browser, you can get the best of both worlds. KeePass itself gives you more power and features than browser password managers and allows keeps you from being tied to a single browser. Using a browser integration plugin adds on the ability to have the browser automatically fill in your username and password when you visit a website. It's not quite as convenient as the browser-integrated password managers, but it still pretty good. And it's definitely a lot easier than trying to use auto-type or copy-and-paste to fill in password forms.
What are my options?
In general, there are a lot of plugins available for KeePass. Just look at the list. Or maybe don't - you probably don't care about 90% of those plugins. The main thing you need to know about is which browsers have plugins available.
Short answer: Chrome, Firefox, and Safari.
Long answer: Chrome, Firefox, and Safari have proper browser plugins available. The Chrome plugin also works in Vivaldi and possibly other browsers that are based on Chrome. There are also form-filling plugins that work with Internet Explorer. To my knowledge, there is no plugin support available for Microsoft Edge.
For this entry, I'll just talk about setting up a plugin with Chrome. We're going to use a Chrome extension called ChromeIPass. It adds a KeePass button to the toolbar in Chrome and can automatically detect login forms on webpages you visit. It works with a KeePass plugin called KeePassHttp.
First, you need to install the KeePassHttp plugin. Start by going to the KeePassHttp website and clicking the "download" link, or just download it directly here. Sadly, KeePass doesn't have a nice way to install plugins - you just have to copy the plugin file to the KeePass plugins folder on your system. Inconvenient, but fortunately not something you need to do very often. On most computers, this will be
C:\Program Files (x86)\KeePass Password Safe 2\Plugins. So just copy the KeePassHttp.plgx file that you downloaded and paste it into that location. Since this is a system directory, you will probably be prompted to grant access. Click "continue" to copy the file. Note that if KeePass is running, you will need to close and restart it for it to detect the plugin.
Now that the KeePassHttp plugin is installed, KeePass will be able to communicate with Chrome. You just need to install the ChromeIPass extension. You can do that by going to the Chrome web store page here and clicking the "Add to Chrome" button.
OK, now that ChromeIPass is installed, what do you do with it? Well, not really much until it's time to log into a site. So pick a site that's in your KeePass database and go there - I'll use sourceforge.net for this example because it's a pretty standard login form.
The first time you try to log into a site using ChromeIPass, you'll need to connect it to your KeePass database. You should notice a KeePass icon is now in your toolbar. Make sure KeePass is running and click that button.
You should see a "Connect" button. Click that and KeePass will prompt you to add a new encryption key for the KeePassHttp plugin. This is a security mechanism - the KeePassHttp plugin encrypts its communication with your KeePass database and this is just the initial step where it sets that up. Don't worry about the details right now - just type in a unique name for the key, maybe based on your browser and computer, e.g. "Laptop - Chrome". You only have to do this the first time you connect a browser to your database - after that, the encryption is automatic.
Now that ChromeIPass is connected to your KeePass database, you can click the ChromeIPass button in your toolbar and click the "Redetect Credetials Fields" to fill in your username and password. Alternatively, you can just refresh the webpage and they should be auto-filled. You won't see anything in the browser yet, but KeePass itself ill prompt you to allow access to the password for this site. You can check the "Remember this decision" box to not be prompted to allow access the next time you visit this site.
(I should probably stop to acknowledge that this thing of having to grant a site access to your KeePass database before you can log in is kind of a drag. I agree, it is somewhat annoying. This is actually a security feature of KeePassHttp - that's the portion of this that runs inside KeePass itself and allows the ChromeIPass extension to talk to it. It actually has a lot of security-related settings. This is actually a good thing, though, because it essentially provides a way for other programs to read your KeePass database, and you want to make sure that malware or dodgy websites aren't able to do that. However, if you want to disable some of these settings, like prompting to allow access, you can do that by going into KeePass and selecting the "Tools > KeePassHttp Options" menu item. The KeePassHttp documentation has some more information on the available settings.)
The good news is that now you're done! After you allow access to KeePass, ChromeIPass will automatically fill in your username and password. If you selected the "remember" option when allowing access to the site, ChromeIPass will automatically fill in your login info the next time you visit the site, no action required. You will only have to allow access the first time you visit a new site of if you elect not to have KeePass remember the approval.
If you're so inclined, ChromeIPass has a number of other features, as detailed in the documentation. For instance, it can save or update entries automatically when you enter a password into a webpage; it has a built-in password generator that lets you create strong passwords right in the browser; it can customize the login fields for non-standard login forms; and it provides a handy right-click menu to fill in passwords and access other functionality.
Hopefully this will help get you started. Using a password manager is a must for keeping your accounts secure these days, and integrated browser support makes using one that much easier, which means you're more likely to keep using it.
You should be using a password manager. If you're a technical person, this is probably not news to you - you're very likely already using one.
This article is for the non-technical people. People like my wife (hi, honey!) and my mom. People who access a lot of websites and have a lot of passwords to remember.
So why is using a password manager a good idea?
Well, you may have seen guidelines for cyber security that tell you things like:
- Don't write down your passwords.
- Don't reuse passwords on different sites.
- Don't use short, easy to guess passwords.
- Don't use passwords that are easy to figure out from public data (like a birthday that anyone can get from your Facebook profile).
Such guidance raises the question: if I have to use long passwords that aren't related to anything in my life, and I can't reuse them or write them down, how the hell am I supposed to remember them?!?
This is a totally reasonable question. Yes, ideally we would all memorize a hundred different 32-character-long, randomly generated passwords. But in real life, nobody can actually do that. So a password manager is a good compromise.
What is a Password Manager
My mother has a little paper "password book" that she keeps in a drawer next to her computer. When she has to create a new account for some website, she writes down all the login information in that book so that she can look it up later.
A password manager is the digital equivalent of that password book. It's an application that lets you record your login information and them look it up later. Most password managers have lots of other handy-dandy features as well, but that's the core of what they do.
So how is this different from, say, writing down all your passwords in a Word document on your desktop? Well, a password manager encrypts all your data. It requires a "master password" to decrypt your information, so if some nasty hacker steals that file, they won't be able to actually read it.
Is this as secure as just memorizing all your passwords? No. But as we said, nobody can do that anyway, and this is one heck of a lot more secure than the alternatives, i.e. reused or weak passwords. With a password manager, you can still have strong, unique passwords for all your sites, but you're relieved of the burden of having to remember them all.
There are a number of password managers out there, but the one I'm going to talk about is KeePass. It's a free, open-source password management application that will run on Windows, Linux, and Mac, and has compatible apps available for iOS and Android. KeePass works offline (i.e. it requires no internet connection and doesn't depend on any online services), but it's possible to sync your KeePass passwords between devices using file sync tools like DropBox or OneDrive. So it provides you some flexibility, but you aren't beholden to a single company that can get hacked or go out of business.
KeePass creates files password files that end with ".kdbx". You can open those files from within KeePass or double-click on them in Window Explorer. When you try to open one, KeePass will prompt you for the master password to that file. Every KDBX file has its own master password. This allows you to do things like create a password file to share with the rest of your family, and have a different one for the accounts that are just yours. (That's a topic for a different post.)
One of the handy extra functions of KeePass is that each entry in your password save can have a bunch of extra data associated with it. For example, you can add custom fields and attach files to each entry, which are handy for things like account validation questions and activation files for software licenses. Basically, you can keep all the important information in one place. And since KeePass encrypts your entire password file, it will all be kept secure.
So how do you use KeePass? Let's walk through it.
Step 1 - Download
The first thing you need to do is to get yourself a copy of KeePass. You can go to this page and click the download link for the "professional edition". (There's not really anything "professional" about it - it's just a newer version with more features.) When that's done, you can double-click the file to install it like any other program.
You can also install KeePass through Ninite. If you're not familiar with Ninite, I strongly recommend you check it out. It's a great tool that makes it brain-dead simple to install and update a collection of programs with just a few clicks. You basically just select a bunch of applications you'd like to install from a list, click a button, and you get an installer program you can run to put everything you selected on your computer. And if you run that program again later, it will actually update any of those programs that have a newer version. It's very slick and a real time-saver.
Step 2 - Create a password safe
Next, open up KeePass and click "File > New". You will be prompted to choose where you want to save your new password database. Choose a name and folder that work for you. Remember - your password database is just a regular file, so you can always move or rename it later if you want.
After that, you should get a dialog that looks like this:
This shows several options for securing your password safe. But don't worry about that - the one you really want is the first one, "master password". So choose a password and type it in. If you click the three dots on the right, KeePass will display the password as you type, so that you don't have to re-enter it.
There are two important things to note when choosing a master password. First, since it's going to protect all your other passwords, you want to make it good. KeePass provides a password strength meter to help you judge, but the main things to bear in mind are that you want a range of different characters and you want it to be long. And no, ten letters does not qualify as "long" - it should be more of a passphrase than a password. One common technique is to use a full sentence, complete with capitalization and punctuation (an maybe some numbers, if you can work them in). That will generally give you a pretty strong password, but it will still be easy to remember.
The other important thing to remember is that the password to a KDBX file is your encription key for that file. That means that the only way to decrypt the file is with that password. If you forget your master password, your data is gone forever. So write down your master password and keep it in a safe place until you're certain you've committed it to memory. And if you want to change your master password later, make sure to make a backup copy of your KDBX file first.
After you've chosen a master password, you should see a screen that allows you to configure some of the settings for your password file. However, you don't really need to worry about this - those are all optional. You can safely click the "OK" button to just continue on.
Step 3 - Organize your passwords
Alright! You now have a password database set up. You should see a list of groups on the left and a list of password entries on the right, like in the image below. These are the sample groups and entries that KeePass creates by default. They're just to give you an idea of how to use your password database - you can safely delete them at any time.
You can click on each group at the left to see what entries it contains. The groups are basically like folders in Windows. There's a top-level folder, and it contains a bunch of sub-folders and each of those sub-folders can contain other folders. So in the screenshot, you can see that "NewDatabase" is highlighted in the group list. That's the top-level folder for my example database. You can see on the right that it contains two entries. You can move an entry into another folders by dragging it from the entry list on the right onto one of the folders on the left.
Step 4 - Create passwords
To add a password entry to your database, select "Edit > Add Entry" from the menu. That will bring up the entry edit screen. This is the same screen you'll see when you double-click on the title of an existing entry, except that it is mostly blank.
There are a lot of tabs and options on this screen, but you don't really need to worry about those. The main things are right in front of you: the entry title, user name, and password. You'll probably also want to fill in the URL field with the web address of the site this information is for. This will come in handy if you want to use a KeePass plugin for your web browser (which we'll cover in another post). When you're done entering your info, click the OK button to create the entry. You should then select "File > Save" from the menu or push the "save" button on the toolbar to save the changes to your password database.
You'll probably notice that there's already a password filled in. KeePass will generate a random password for new entries. You are free to change this yourself or use the button to the right of the "repeat" box to generate other random passwords using different rules. KeePass has a password generator that lets you specify the allowed characters and length for a random password, which is handy for those sites that insist on specific password length or complexity rules.
Step 5 - Getting your passwords out
Now let's back up and say you've just started up your computer, are logging in to some website, and want to get a password out of KeePass. The first thing you need to do is open up your password database. You can do this by double-clicking on it in Windows Explorer or by opening up KeePass then selecting your database from the "File > Open" menu. When you open the database, you'll be greeted by a screen asking you to enter your master password - you know, the one you came up with in step 2. (Hint: remember that you can click the button with the three dots to display the password as you type it.) After you enter your master password, the database will be decrypted and you'll find yourself and the entry browsing screen from step 3.
There are several ways to get your passwords out of KeePass. Here's the list in order of preference:
- Use a browser plugin to automatically fill in login forms. Since most of the passwords you end up creating are for websites, setting up your browser to fill in the forms from your KeePass database makes life much easier. I'll talk about how to do that in the next post. But don't worry - it's not all that hard.
- Use auto-type. This is a feature of KeePass where you to click a button in the KeePass window and it will automatically send the keystrokes for your username and password to the last window you used. So, for example, you would navigate to the login page of a site in your web browser, click in the username box, and then switch over to the KeePass window and click the auto-type button on the toolbar (the one that looks kind of like some keyboard keys - hover your cursor over the buttons to see the descriptions). By default, the auto-type feature will type your username, the "tab" key, your password, and then the "enter" key. This will work for probably 90% or more of login pages, but it's not universal, so be aware of that.
- Copy them to the clipboard. If all else fails, you can always just copy your passwords to the clipboard so that you can paste them into another window. KeePass makes this fairly easy. In the main password list that you saw in step 3, when you double-click on the actual username or password for an entry in the list, it will copy that to the clipboard. This saves you having to open up the entry edit screen and copy things there. You can then switch to another window and paste the data into a login screen.
- Just read it. Last, but not least, you can always go low-tech and just read the passwords out of the window. Just double-click the name of your entry, then click the "three dots" button to make the password visible. Clearly this is not great, but sometimes it's necessary. For example, you will need to do this when entering a password on a system that doesn't have KeePass installed, such as to login into your Amazon or Netflix account when setting up a Roku or other streaming media system.
With any luck, I've made this "password manager" thing sound good enough for you to try it out. You really should look into it. Password reuse has become increasingly dangerous, with hackers trying the usernames and passwords they harvested from one hack on other sites just to see if they work. Password cracking tools have also advanced a lot in recent years, including information gleaned from previous attacks, so relying on things like "133t 5p34k" passwords is no longer good enough. A decent password manager, if used consistently with randomly generated passwords, will provide you with a good trade-off between convenience and security.
As yet another note to myself (because I keep having to look this up), the equivalent of the UNIX
which command in PowerShell is
Get-Command. That commandlet will let you pass in a name, with no extension, and return you the executable in your path that corresponds to that name. This is really handy because it works with any executable, including native binaries and batch and PowerShell scripts. So, for example:
PS pgeer> Get-Command rsync
CommandType Name Version Source
----------- ---- ------- ------
Application rsync.cmd 0.0.0.0 C:\Users\pgeer\bin\rsync.cmd
The other day I listened to a very interesting talk by Allen Holub on the #NoEstimates movement, which I've posted about before. It's definitely worth a look if you have some time. Mr. Holub makes the slightly controversial claim that all estimates are evil and we should all just stop doing them immediately. His rationale for this is essentially that estimates are made up and harmful. And, of course, he's not wrong.
I've always been a little skeptical of the #NoEstimates movement, but the thing it clearly get right is that estimates are frequently misused and misapplied. Just about every developer has a story about estimates being interpreted as promises, or having to work late just to "meet your sprint commitments", or skimping on quality because the sprint is about to end and you need to close out your tasks. And we won't even get into managers who try to use estimates as a performance evaluation tool - the dysfunction of this should be obvious to anyone with any experience.
Is it the estimates?
So estimates can be used for bad things. That's not controversial. But that's also not a problem with estimation per se. I guess you could think of it as the software equivalent of "guns don't kill people, people kill people." It's not that estimates are inherently bad, it's just that they're seldom used for good. That doesn't mean that you should abolish estimates, but it does mean that you should use them with caution.
The thing about estimates is that, while they're often just guesses, they don't have to be. In my study of the Personal Software Process (PSP), I learned about doing estimates based on historical data. I've even seen it work - in my last job, I reached the point where my time estimates were within about 15% of actual task completion time, which I consider to be pretty good given that the estimates were in minutes. Granted, it doesn't always work - sometimes you have tasks that are truly unprecedented or have too many unknowns - but it can be done.
The main problem with data-based estimation is that most organizations and developers are not equipped to do it. For one thing, it requires data, and a lot of shops don't actually collect the kind of data that would be useful. Most developers certainly don't collect the kind of data that the PSP estimation method uses. It also takes more time than many people seem to want to devote to estimates. It's not that much time, but still a lot longer than saying, "well, I guess that's about eight story points".
What's the point?
The real question here is: why do you want an estimate? What are you trying to accomplish?
When I do PSP estimates, I'm using them to gauge my performance. I'm looking at my historical data and using that to project out time, code volume, and defect counts. I can compare these projections to my actual results, see if my performance is consistent, and if not, analyze the reasons and look for ways to improve in the future.
Yes, these are "estimates", but not in the same sense as most teams do them. My manager never sees these "estimates"; I make no commitments based on them; nobody depends on their being accurate; in fact, nobody but me even knows what they are. They're a part of my task planning, so they're short-term estimates - they go out two or three weeks at most, usually more like a few days. I start with some requirements analysis, sketch out a conceptual design, and then do the estimate based on that. So the process of coming up with the estimate actually helps me flesh out and validate my plan, which is valuable in and of itself.
The important thing to note in the above description is that my estimates are not targets. If I'm working on a task and it takes 500% longer than my estimate, I'm the only one who knows and I don't care. Sure, I'll try to figure out why that task took so much longer than I thought, but the fact that "I missed my estimates" has no relevance and no consequences.
But that's usually not how it works when your boss or your project manager (or even your scrum master) asks you for an estimate. They want you to tell them when the task will be done. And they'll be very disappointed if it's not done when you said it would be. And that's where the dysfunction begins.... Because even in the best of circumstances, we don't really know exactly how long s development task will take. And we seldom work in the best of circumstances.
The interesting thing about doing PSP estimates is that, until relatively recently, I had trouble not thinking of the estimates as targets. Process Dashboard, my PSP support tool, has a progress bar that counts down the time remaining for your estimate and then turns red when you go over the estimate. And I would actually get upset when that bar turned red! Why?!? There was no reason for that - there was literally nothing at stake! I was just psychologically stuck in the wrong mindset. And that's the same mindset that most people get stuck in when they talk about estimates.
So I'm starting to come around to this "no estimates" thing. I still believe that estimates can be useful and productive, but they usually aren't. And if your company is particularly enlightened and can do estimates in a way that's productive and avoids the pathological effects associated with them, then by all means, keep doing estimates. But most people don't work in companies like that. And even if your company is great, you still need to be careful of the psychological pressure created by the inherent biases and preconceptions of your developers.
In other words, estimate carefully. Consider simpler, lower precision (but not necessarily lower accuracy) methods like Mr. Holub and the other #NoEstimates advocates describe. Don't be fooled - those are still estimates, but they at least make an effort to work around the damaging side-effects associated with traditional estimation. After all, it doesn't matter how good the estimates are if they destroy team morale.
For the last six months or so, I've spent a lot of time reading about the SOLID principles and listening to lectures about agile development practices by the likes of Bob Martin, Dave Thomas, and Martin Fowler. The result has been a new and different understanding of what "agile" really means.
One of the agile practices I've really warmed up to is test-driven development. Uncle Bob advocates very strongly for it, and it was one of his talks that got me to finally try it again. I'm too lazy to find the exact quote, but it was something like:
What if you had a button on your keyboard that you could press and it would tell you if your code was working? How would that change your life? Test driven development gives you that button.
Of course, that's a bit hyperbolic, but the point still stands. The idea that you could have a tool that could tell you, "yup, this code works as intended" is really powerful. Even if building such a tool is a lot of work, it seems like it would be worth it.
But the thing is, I've tried TDD before. In fact, I've tried it for about a week or so every year for about the last five years. And you know what? It never really worked for me. It was awkward, the tests were brittle and hard to read, and it just didn't seem to help me much at all. I came away thinking that it was a stupid fad that didn't help and didn't really make sense.
But things were different this time. I'd spent months immersing myself in the SOLID principles, object-oriented design, and techniques for writing good tests. This time I was ready. It turns out that knowing how to structure your code and your tests makes the entire process infinitely smoother and simpler. Heck, it's even kind of fun.
So how did it go this time? Well, I'd say it was fairly successful. I've been able to actually use TDD and stick with it without feeling like I was wasting my time. That alone is a big improvement from my previous attempts. But I can go deeper than that. Thanks to the data I've collected using the PSP, I can actually do some analysis of just how well TDD is working for me.
About the Process
Before getting into the details, let's talk about what my process looked like before and what it looks like now. That will help give some context to the numbers and make it more obvious why they look the way they do.
My process is based on the (now deprecated) PSP 3. For each task, I start with a quick high-level design sketch, and then one or more development cycles, each of which consist of design, design review, code, code review, and then test. The "design" phase consisted of conventional design and some skeleton coding (defining interfaces, defining UI components, outlining algorithms, etc.), the "code" phase consisted of fleshing out those outlines and implementing the rest of the design, and the "test" phase was writing unit tests followed by manual system-level testing.
The new TDD-based process changes that a bit. The "design" phase is very similar, but I've been going into less detail in the algorithms involved. The idea was to let the detailed design "emerge" in the process of writing tests, though I've started moving away from that a little. The "code" phase is now a series of TDD cycles. That means that it encompasses both actual coding and unit testing. The "test" phase has become just manual testing. I did this because, realistically, trying to track test vs. code time when I'm switching back and forth every few minutes was going to be tedious and error prone and I didn't want to do it.
Unfortunately, this makes the analysis much harder because we're not comparing apples to apples. Both processes did try to achieve comprehensive unit test coverage, so they're not completely incomparable, but the change in definition of the "testing" and "coding" phases does make certain comparisons moot.
About the Data
Of course, the variety in the individual tasks undermines the data a bit, but that can't really be helped. I'm just working with the tasks I'm given - I don't have the time to do a genuinely scientific comparison.I mean, I do actually need to get some work done at some point - I'm not getting paid to analyze my own performance.
The Results - Summary
Overall, the TDD-based process seems to be working as well as, if not better than, what I'll call the "conventional" process. Below is a summary table of overall metrics:
|Summary||Plan (TDD)||Actual (TDD)||Plan (Conv.)||Actual (Conv.)|
|Total New & Changed LOC||3903||6290||3249||5017|
|% Appraisal COQ||16.00%||15.30%||16.40%||16.20%|
|% Failure COQ||32.30%||15.30%||34.00%||38.00%|
For the two data sets, the total time and code volume were substantial and in the same neighborhood, so the data sets are at least comparable. We can see the TDD data shows higher productivity (in LOC/hour), fewer defects, better review yield, and lower cost of quality.
So this means that TDD is definitively better, right?
Well...no. The change in definition of the "code" phase means that I'm logging fewer defects in general. Part of my defect standard is that I don't record a defect if it's introduced in the same phase where it's fixed. So with TDD, little things like typos and obvious logic errors are caught by unit tests in the code phase, so they don't show up at all in these numbers. In the conventional process, they wouldn't have been picked up until code review or testing. The same reasoning applies to the failure cost-of-quality - since writing unit tests is now part of the code phase rather than the test phase, less time would naturally be spent in testing because there's simply less work that needs to be done in that phase now.
However, the productivity difference is still interesting, as is the improvement in percent yield. Those do suggest that there's some improvement from using TDD.
Drilling Down - Time
So since the change in phase definition makes defect numbers incomparable, let's look at time data instead. This table shows the percentage of time spent in each phase.
|Phase||Actual % (TDD)||Actual % (Conv.)|
|High-level Design Review||0.87%||1.04%|
|Detailed Design Review||4.27%||4.59%|
The results here are largely unsurprising. The percentages are roughly equivalent for most of the phases, with the notable exceptions of code and test.
However, it is interesting to note that there's still a discrepancy between the total code+test time for the two processes. Theoretically, based on the phase definitions, that combination should encompass the same work for both processes. But the TDD process spent 67.2% of the time on coding and testing, but it was only 50.1% for the conventional process. It appears that the single largest chunk of that disparity comes out of design time. This makes sense because there was less detailed design in the TDD process.
Note that the review times were roughly equivalent for each process - in fact, they were slightly lower with TDD. Yet the percent yield for reviews was higher with TDD. This gives us some evidence that the difference in percent yield can be attributed is likely due to the use of TDD.
It's not entirely clear why this would be the case. The most obvious outcome would be lower yield, as the use of TDD means that fewer bugs escape to be found. My working hypothesis, however, is that using TDD lowers the cognitive load of code review by removing most of the noise. With TDD, you already know the code works. You don't have to review for things like typos, syntax errors, or misused functions because you've already tested for those things. That leaves you more time and energy to dig into more substantive issues of design, requirements coverage, and usability.
Back to Defects
Hmm.... Maybe we should take another look at the defect data after all. We've already established that number of defects isn't going to be useful due to the change in phase definition. But what about average fix times? If TDD is weeding out the simpler bugs so that they don't escape the coding phase and don't get recorded, we'd expect the average fix time to be higher.
|Found in test (TDD)||Found in other phases (TDD)||Total defects found (TDD)||Found in test (Conv.)||Found in other phases (Conv.)||Total defects found (Conv.)|
|Injected in HLD||Tot. fix time||-||28.4||28.4||-||18.2||18.2|
|Avg. fix time||-||3.16||3.16||-||1.4||1.4|
|Injected in Design||Tot. fix time||448.4||321||769.4||271.1||241.6||512.7|
|Avg. fix time||9.34||5.63||7.33||5.32||3.83||4.5|
|Injected in Code||Tot. fix time||223.2||293.3||516.5||341.6||230.4||572|
|Avg. fix time||5.72||3.22||3.97||4.38||2.74||3.53|
|Total Injected||Tot. fix time||724.9||647.1||1372||629.3||500.1||1129.4|
|Avg. fix time||7.55||4.04||5.36||4.7||3.03||3.78|
The above table shows the break-down of defect count and fix time by injection phase. So what does this tell us? Well, my hypothesis that TDD would produce higher average fix times seems to be correct. If we look at the fix times for defects injected in code and found in test, the average fix time is about 30% higher.
However, it's worth noting that the TDD data has higher average fix times across the board. It's not entirely clear why this should be. One possible explanation is that the use of TDD means that unit tests are introduced earlier in the process, so defects fixed in code and code review would require changes to the test suite, whereas in the conventional process that would only happen for defects found in test. That's something I'll have to watch in the future.
So far, my new TDD-based process seems to be working out pretty well. The results are somewhat ambiguous, but the numbers suggest that TDD is resulting in higher LOC/hour productivity and more efficient defect removal.
From a more human perspective, it has the benefit of making unit testing much less tedious and painful. It gets you earlier feedback on whether your code is working and gives a nice sense of satisfaction as you watch that list of test cases grow.
But ore importantly, TDD is a great way to test your test cases. When you're writing your unit tests after the fact, it's very easy to get stuck in a situation where a test is unexpectedly failing (or, worse, passing) and you can't tell whether the problem is with your test case or the code under test. With TDD, you're changing things in very small increments, so it's easier to pinpoint the source of problems like that.
But the big lesson here is that TDD isn't something you can just jump into. You have to understand the context and the design techniques first. If you do, then it's pretty great. If not, then you're going to have a hard time and end up wondering who came up with this hair-brained idea.
Another quick note to my future self: setting up Git under Windows to use SSH key authentication is pretty easy...once you know what to do.
At work, we have some Composer and Bower packages that we fetch from our internal GitHub Enterprise server. And, of course, all the source lines in the
bower.json files use SSH references. I just use HTTP for my code checkouts, so I finally had to figure out how to make Git for Windows authenticate with my SSH key.
Turns out it's pretty easy. I have a key pair I created with PuTTYgen. All I had to do was export my private key to OpenSSH format and copy that file to
C:\Users\<my_username>\.ssh\id_rsa. Git picks it up with no further configuration. Then I just added my key to GitHub and I was good to go.
Today I watched a very interesting talk on good and bad unit testing practices by Roy Osherove, entitled "Unit Testing Good Practices & Horrible Mistakes in JS". It really was an exceptional talk, with some good, concrete advice and examples. I highly recommend it.
I actually recognized a couple of the examples in the talk. The big one was OpenLayers. I still remember being horrified by their test suite when I was using that library at my last job. We had a number of extensions to OpenLayers and I was thinking about adding some tests into their test suite. Then I ran the tests and saw a bunch of failures. Assuming our code must be bad, I tried a fresh copy of the OpenLayers release. A bunch of tests still failed. And that's when I gave up and decided to just pretend their tests didn't exist.
This is yet another one of those "post this so I can refer back to it later" things. So if you're not a Windows user, or if you don't use ConEmu, then I suggest you go get a cup of coffee or something.
So for a while now I've been using ConEmu as my Windows console app. It supports multiple tabs, transparency (ooooh), customizable hotkeys, customizable sessions, Far manager integration and a whole bunch of other nifty stuff.
A couple of months ago, I saw that it was possible to embed PuTTY, the popular Windows-based SSH client, directly in a ConEmu tab. So I tried it out and found it to be pretty slick. The only down side was some key binding weirdness.
First, there's the general putty issue that if you accidentally press ctrl+S - you know, the key combination that means "save" in just about every editor in existence - it effectively locks the terminal and it's not obvious how to get control back. The second issue is that, when you embed an application like PuTTY in ConEmu, it steals most of your keyboard input, so the standard key bindings for switching between window tabs don't work.
Luckily, these problems are easily fixed. The fixes are just hard to remember, which is why I'm writing it down. For the ctrl+S iissue, you can just hit ctrl+Q to turn input back on. For the tab-switching issue, you can use the global key bindings for ConEmu - namely Win+Q, Win+Shift+Q, and Win+<Number> to switch consoles, as well as Win+Z to toggle focus between ConEmu and PuTTY.
Note to self: you actually can turn the function keys on your Lenovo IdeaPad back into actual function keys. It's just a BIOS setting.
For anyone else who comes across this, Lenovo's IdeaPad series of laptops (at least the version I have) does weird things with the function keys. You've probably seen those keyboards where the F1 through F12 keys have alternate functions. They often have a "function lock" button that's equivalent to caps lock or scroll lock and you can turn it off to access the alternate functionality.
What Lenovo does is similar, except that the alternate functionality is on by default and there's no function lock button. There's just an "Fn" button that you have to press and hold to access the normal function button keystrokes. So on my laptop, F11 and F12 turn the screen brightness up and down and I need to press Fn+F12 to do a regular F12 keystroke.
This is probably great for "regular" users, most of whom wouldn't miss the function keys if they went away completely. On the other hand, for a developer, this adds a second keystroke to a row of what were previously absurdly convenient hotkeys. Granted the volume and brightness keys are convenient, but when I'm coding I use F11 and F12 for my IDE keybindings much more often than I need to change the screen brightness.
But luckily, as I found in the link above, you can turn that off in a BIOS setting.