Apparently CrashPlan eats inotify watchers

As a brief follow-up to my last post, I had a problem trying to remote into my newly installed Ubuntu box today.  As I mentioned in that post, I'm using xRDP for my remote GUI access and I'm tunneling it over SSH.  When I connected to xRDP, I logged in and was greeted by the Ubuntu lock screen for my running session.  So I entered my password, hit "enter", and waited.  And waited.  And waited some more.  My session just never unlocked.

OK, so apparently xRPD is freaking out.  I already have a an SSH session open, so I'll just restart the service and maybe that will clear up the problem.  So I ran service xrdp restart and it told me:

Failed to add /run/systemd/ask-password to directory watch: No space left on device

That didn't seem right.  I had plenty of disk space and plenty of RAM - what could possibly be out of space?  So naturally I asked the internet, which led me to this Stack Exchange post.  Turns out that CrashPlan has a habit of eating up inotify watchers, which is what I was actually out of.  Stopping the CrashPlan service fixed the problem.  Fortunately, I don't need CrashPlan anymore, so that shouldn't be a problem again.

Of backups and reinstalled

I finally decided to do it - reinstall my home media server.  I switched it from Windows 8.1 to Ubuntu 17.10.

The reinstall

Believe it or not, this is actually a big thing for me.  Fifteen years ago, it would have been par for the course.  In those days, I didn't have a kid or a house, so rebuilding my primary workstation from scratch every few months was just good fun.  But these days...not so much.  Now I have a house and a kid and career focus.  Installing the Linux distribution of the week isn't fun anymore - it's just extra time I could be spending on more important and enjoyable things.

But, in a fit of optimism, I decided to give it a shot.  After all, modern Linux distributions are pretty reliable, right?  I'll just boot up and Ubuntu DVD, it'll run the installer for 15 minutes or so, and I'll have a working system.  Right?

Well...not quite.  Turns out Ubuntu didn't like my disk setup.  Things went fine until it tried to install Grub and failed miserably.  I'm not sure why - the error message the installer put up was uninformative.  I assume it had something to do with the fact that I was installing to /dev/sdb and /dev/sda had an old Windows install on it.  After a couple of tries, I decided to just crack open the case and swap the SATA cables, making my target drive /dev/sda, and call it a day.  That did the trick and Ubuntu installed cleanly.  I did have to update my BIOS settings before it would boot (apparently the BIOS didn't automatically detect the change of drives), but it worked fine.

The only real problem I had was that apparently Wayland doesn't work properly on my system.  I tried several times to log into the defaut GNOME session and after a minute or two, system load spiked to the point that the GUI and even SSH sessions became unresponsive and eventually the GUI just died.  And I mean died - it didn't even kick me back to the GDM login screen.

I suspect the problem is my system's archaic video card - an integrated Intel card from the 2010 era which I have absolutely no intention of ever upgrading.  I mean, it apparently wasn't good enough to run Windows 10, so I wouldn't be surprised if Wayland had problems with it.  But in any case, Ubuntu still supports X.org and switching to the GNOME X.org session worked just fine.  I don't really intend to use the GUI on this system anyway, so it's not a big deal.

The restore

Once I got Ubuntu installed, it was on to step two of the process: getting my data drive set up.  It's a 1TB drive that used to serve as the Windows install drive.  It was divided into a couple of partitions, both formatted as NTFS.  Since I'm switching to Linux and really just wanted one big data drive, this was a sub-optimal setup.  Therefore I decided to just blow away the partition table and restore from a backup

I currently use CrashPlan to back up this computer.  It's set to back up to both the cloud and a local USB hard drive.  So my plan was to repartition the disk, install CrashPlan, and restore from the local hard drive.

This was fairly easy.  Installing the CrashPlan client was the first task.  There's no .deb package, but rather a .tgz that contains an installer script.  It was actually pretty painless - just kick off the script and wait.  It even installs its own copy of the JRE that doesn't conflict with the system version.  Nice!

Next was actually restoring the data.  Fortunately, CrashPlan has their process for restoring from a USB drive well documented, so there wasn't much to figure out.  The process of connecting a particular backup dataset on the drive to the CrashPlan client was slightly confusing because the backup interface (as far as I can remember) doesn't really make it obvious that a USB drive can have more than one dataset.  But it just boils down to picking the right directory, which is easy when you only have one choice.

The only surprise I ran into was that running the restore took a really long time (essentially all day) and created some duplicate data.  My data drive contained several symlinks to other directories on the same drive and CrashPlan apparently doesn't handle that well - I ended up with two directories that had identical content.  I'm not sure whether this was a result of having symlinks at all, or if it was just moving from Windows symlinks to Linux symlinks.  In any case, it's slightly inconvenient, but not a big deal.

Other services

While the restore ran, I started setting up the system.  Really, there were only a handful of packages cared about installing.  Unfortunately for me, most of them are proprietary and therefore not in the APT repository.  But the good news is that most of them were pretty easy to set up.

The first order of business, since this box is primarily a media server, was setting up Plex.  This turned out to be as simple as installing the .deb package and firing up the UI to do the configuration.  From there, I moved on to installing Subsonic.  This was only marginally more difficult, as I also had to install the OpenJDK JRE to get it to run.  I also followed the instructions here to make the service run as a user other than root, which seemed like not such a great idea.

The only thing I wanted to install that I couldn't get to work was TeamViewer.  But this wasn't a deal-breaker, though, because the only reason I was using TeamViewer under Windows was because I was running the Windows 8.1 Home and was too cheap to pay for the upgrade to Professional just to get the RDP server.  But since this is Ubuntu, there are other options.  Obviously SSH is the tool of choice for command-line access and file transfers.  For remote GUI access, I tried several variations on VNC, but it eventually became clear that xRDP was the best solution for me.  It's not quite as straight-forward to get working, but this guide provides a nice step-by-step walk through.  There are also a few caveats to using it, like the fact that the same user can't be logged in both locally and remotely, but those weren't big issues for my use case.

Results

For the most part, the transition has gone pretty smoothly.  The setup didn't go quite as smoothly as it could have been, but it wasn't too bad.  In any event, what I really cared about was getting Plex and Subsonic up and running and that was pretty painless.  So I'm back to where I was originally, but on an up-to-date operating system, which is all I really wanted.

LBlog Refactoring Step 1: Publishing

The first part of my LnBlog refactoring is kind of a large one: changing the publication logic.  I'll start by giving an overview of the old design, discussing the new design, and then looking at some of the actual project data.

History

LnBlog is a very old system.  I started it back in 2005 and did most of the work on it over the next two years.  It was very much an educational project - it was my first PHP project (in PHP 4, no less) and only my third web-based project.  So I really didn't know what I was doing.

As you might expect, the original design was very simple.  In the first version, "publishing an entry" really just meant creating a directory and writing a few files to the disk.  As the system grew more features, more and more steps were added to that process.  The result was a tangle of code where all of the logic lived in the "controller" (originally it was a server-page) with some of the actual persistence logic encapsulated in the domain objects.  So, roughly speaking, the BlogEntry object knew how to write it's metadata file to the appropriate location, but it didn't know how to properly handle uploaded files, notifications, or really anything else.  

Originally, LnBlog used a server-page model, with each URL being a separate file and code being shared by including common configuration files and function libraries all over the place.  Shortly prior to starting this project, I had consolidated the logic from all those pages into a single, massive WebPages class, with one public method for each endpoint - a monolithic controller class, for all intents and purposes.  This still isn't great, but it gave me a common entry point to funnel requests through, which means I can better control common setup code, handle routing more easily, and generally not have the pain of dealing with a dozen or more endpoints.

Anyway, this WebPages class served up and edited entries by directly manipulating domain objects such as BlogEntry, Blog, BlogComment, etc.  The domain objects knew how to save themselves to disk, and a few other things, but the majority of the logic was right in the WebPages class.  This worked fine for the main use-case of creating an entry from the "edit entry" page, but it made things very awkward for the less-used locations, such as the Metaweblog API endpoint or scheduled publication of drafts.

Furthermore, the publication logic in the WebPages class was very hairy.  All types of publication flowed through a single endpoint and used a common code path.  So there were flags and conditions all over the place, and it was very difficult to tell what needed to be updated when making changes.  Bugs were rampant and since there was no test automation to speak of, testing any changes was extremely laborious and error prone.

The New Design

There were two main goals for this refactoring:

  1. Create a new Publisher class to encapsulate the publication logic.  The idea was to have a single entity that is responsible for managing the publication state of blog entries.  Given a BlogEntry object, it would know how to publish it as a regular entry or a static article, unpublish it, handle updating or deleting it, etc.  This would give us a single entity that could own all the steps involved in publishing.
  2. Create unit tests around the publication process.  The logic around the entire process was more complicated than you'd think and the old code was poorly structured, so it broke with disturbing regularity.  Hence I wanted some automated checking to reduce the risk of bugs.

So the design is actually pretty straight-forward: create a "publisher" class, give it operations for each of the things we do with blog entries (create, update, delete, etc.), copy the existing logic for those cases into the corresponding methods, update the endpoints to use this new class, and call it a day.  

So it was mostly just a reorganization - there wasn't any significant new logic that needed to be written.  Simple, right?  What could go wrong?

Results

While I was happy with the result, this project turned out to be a much larger undertaking than I'd assumed.  I knew it was going to be a relatively large task, but I was off by a factor of more than two.  Below is a table summarizing the project statistics and comparing them to the original estimates (from PSP data I captured using Process Dashboard). 

Planned and actual project statistics
  Planed Actual
Total Hours  29:01 64:00
LOC added and modified  946  3138
LOC/Hour  32.6 49.0
Total Defects  82.1  69.0
Defects/KLOC 86.8 21.7

When I originally did the conceptual design and estimate, I had planned for relatively modest changes to the Article, BlogEntry, and WebPages classes and the creators.php file.  I also planned for new Publisher and BlogUpdater classes, as well as associated tests and some tests for the WebPages class.  This came to 29 hours and 946 new or changed lines of code across nine source files.  Definitely a big job when you consider I'm working in increments of two hours or less per day, whenever I get around to it.

In actuality, the changes were much larger in scope.  I ended up changing 27 additional files I hadn't considered, and ended up creating two other new utility classes (although I did ditch the BlogUpdater class - I no longer even remember what it was supposed to do).  The resulting 3138 lines of code took me 64 hours spread over five months.

Effects of Testing

I did test-driven development when working on this project.  I've found TDD to be useful in the past and it was very helpful to me here.  It was also very effective in meeting my second goal of building a test suite around the publication logic.  PHPUnit reports statement coverage for the Publisher tests at 97.52% and close to 100% coverage for the tested methods in the WebPages class (I only wrote tests for the endpoint that handles creating and publishing entries).

More importantly, using TDD also helped me to untangle the logic of the publication process.  And it turns out there was actually a lot to it.  I ended up generating about 2000 lines of test code over the course of this project.  It turns out that the design and unit test phase occupied 65% of the total project time - about 41 hours.  Having a comprehensive test suite was immensely helpful when I was rebuilding the publication logic across weeks and months.  It allowed me to have an easy check on my changes without having to load all of the code I worked on three weeks ago back into my brain.

Sadly, the code was not such that I could easily write tests against the existing code.  In fact, many of the additional changes came from having to break dependencies in the existing code to make it possible to unit test.  Luckily, most of them were not hard to break, e.g. using an existing file system abstraction layer, but the work still adds up.  It would have been very nice to have an existing test suite to prevent regressions in the rewrite.  Unfortunately, even integration tests would have been awkward, and even if I could have written them, it would have been very hard to get the level f coverage I'd need to be confident in the refactor.

Conclusion

In terms of the results, this project worked out pretty well.  It didn't really go according to plan, but I got what I was looking for in the end - a better publication design and a good test suite.  However, it was a long, slow slog.  Maybe that was too big a slice of work to do all at once.  Perhaps a more iterative approach could have kept things moving at a reasonable pace.  I'll have to try that on the next big project.

Fluent-ish interfaces

This evening I was reading a post on Jason Gorman's blog about fluent assertions in unit tests and it made me smile.  Jason was updating some slides for a workshop and decided to illustrate fluent assertions by putting the "classical" assertion right next to the fluent version.  After doing that, he started to doubt the text on the slide that claimed the fluent version is easier to read than the classical one.

That made me feel good, because I've often wondered the same thing.  I've heard the the assertion that fluent assertions are easier to read many times - for instance, they said that in a Certified Scrum Developer class I took last year.  Apparently that's just "the direction the industry is going."  I've followed Jason's blog for a while and he seems like a pretty smart guy, so seeing him doubt the received wisdom gives me a little validation that it's not just me.  

Fluent assertions are...sort of fine, I guess.  I mean, they work.  They're not terrible.  I never really thought they were any easier to read than the old-fashioned assertions, though.  

From what I can tell, the claim is generally that fluent interfaces are easier to read because they naturally read more like a sentence.  And that's definitely true - they undeniably read more like a sentence.  Just look at some of Jason's examples (classical above, fluent below):

Assert.AreEqual(50, 50);
Assert.That(50, Is.EqualTo(50));

Assert.IsTrue(true);
Assert.That(true);

Assert.AreSame(payer, payee);
Assert.That(payer, Is.Not.SameAs(payee));

Assert.Contains(1, new ArrayList() {1, 2, 3});
Assert.That(new ArrayList() {1, 2, 3}, Contains.Item(1));

It's undeniable that "assert that 50 is equal to 50" is much closer to a real sentence than "assert are equal of 50 and 50" (or however you would read the classical version).  However, it would behoove us to remember that we're reading code here, not English prose.  We can't just assume that making your code read like an English sentence makes it more readable.  They're different things.

My theory is that we tend to think differently when we're looking at code than we do when we're reading text.  The fluent version might look closer to a sentence, but that's really just on the surface.  It's only more natural if you mentally strip out all the punctuation.  But that's not how we read code, because in code you can't just strip out the punctuation.  The punctuation is important.  Likewise, the order of things is important, and not necessarily the same as in English.

When I look at the fluent assertion, I don't just see "assert that 50 is equal to 50", I see all the pieces.  I mentally separate the "assert" from the "that", because "that" is a method, which usually means it contains the more immediately applicable information.  Likewise with "is equal to".  And, of course, I recognize Is.EqualTo(50) as a method call, which means that I have to mentally substitute the result of that into the second parameter.  But wait a minute - equal to what?  There's only one parameter to EqualTo, which doesn't make any sense.  Oh, wait, that's NUnit magic.  So we pass 50 and the result of EqualTo to That...what the heck does "that" mean anyway?  Wait - magic again.  We have to read that backwards.  So the actual assertion comes at the end and the "that" just ties things together.  OK, I've got it now.  So...what were we testing again?

OK, maybe that's a little silly, but you get the idea.  The point is that code is written in a certain way, and it's not the way English is written.  When you're reading code, you get into a certain mindset and you can't easily just switch to a different one because the next line just happens to call a fluent interface.  The old-fashioned Assert.AreEqual(50, 50) might not be sexy or natural looking, but it's short, simple, and perfectly clear to any developer worth his salt.  Why switch to an interface that's wordier and less aligned with how the rest of our code is written?  If it ain't broke, don't fix it.  

LnBlog: Blogging the redesign

Today, we're going to talk a little about design and refactoring.  As a case-study, we're going to use a little blogging application called LnBlog.  You probably haven't heard of it - it's not very popular.  However, you have used it, at least peripherally, because it's running this site.  And you also have at least a passing familiarity with the author of that application, because it's me. 

Motivation

Software is an "interesting" field.  The cool new technologies, frameworks, and languages get all the press and they're what everybody wants to work with.  But let's be honest: it's generally not what makes the money.  I mean, how could it be?  It just came out last week!

No, if you have the good fortune to work on a grown-up, profitable product, it's almost certainly going to be the "old and busted" tech.  It might not be COBOL or Fortran, but it's almost certainly "legacy code".  It might be C++ or Java or, in our case, PHP, but it's probably old, poorly organized, lacking unit tests, and generally hard to work with.

I work on such a product for my day job.  It's a 10-year-old PHP codebase, written in an old-fashioned procedural style.  There are no unit tests for the old code, and you couldn't really write them even if you wanted to.  Sure, there's a newer part with proper design, tests, etc., but the old code is the kind of stuff that "works for the most part", but everybody is afraid to touch it because it's so brittle and tightly coupled that God alone knows what will break when you make a change.

This also applies to LnBlog.  It was my very first PHP application.  I started it way back in early 2005, in the bad old days of PHP 4.  Over the next two or three years, I managed to turn it into something that was relatively functional and full-featured.  And for the last ten years or so, I've managed to keep it working.

Of course, it hasn't gotten a whole lot of love in that time.  I've been busy and, for the most part, it worked and was "good enough".  However, I occasionally need to fix bugs or want to add features, and doing that is a truly painful process.  So I would very much like to 

The Issue

Let me be honest: I didn't really know what I was doing when I wrote LnBlog.  I was about four years out of school and had only been coding for about six or seven years total.  And I was working mostly in Visual Basic 6 at the time, which just barely counts.  It was also only my third web-based project, and the first two were written in classic ASP and VBScript, which also just barely counts.

As a result, it contains a lot of questionable design decisions and overly-complicated algorithms.  The code is largely procedural, kind of buggy, and makes poor use of abstraction.  So, in short, it's not great.

But, in fairness to myself, I've seen worse.  In fact, I've seen a lot worse.  It does have a class hierarchy for the domain objects (though it's a naive design), an abstraction layer for data access (though it's inconsistently used), and a templating system for separating markup from domain logic (though the templates are an ungodly mess).  And it's not like I had a role model or mentor to guide me through this - I was figuring out what worked on my own.  So while it's not great, I think it's actually surprisingly good given the circumstances under which it was built.

The Goal - Make the Code "Good"

So I want to make LnBlog better.  I've thought about rewriting it, but decided that I wouldn't be doing myself any favors by going that route.  I also hold no illusions of a grand re-architecture that will fix all the problems and be a shining beacon of design perfection.  Rather, I have a relatively modest list of new features and bug fixes, and I just want to make the code good enough that I can make changes easily when I need to and be reasonably confident that I'm not breaking things.  In other words, I want to do a true refactoring.

If you haven't read Martin Fowler's book, the word "refactoring" is not a synonym for "changing code" or "rewriting code".  Rather, it has a very specific meaning: improving the internal design of code without changing the external behavior.  In other words, all you do is make the code easier to work with - you don't change what it does in any way.  This is why people like Bob Martin tell you that "refactor X" should never be an item in your Scrum backlog.  It is purely a design and "code cleanliness" activity, not a "feature" you can deliver.

So my goal with LnBlog is to gradually reshape it into what it should have been in the first place.  This is partially to make changing it easier in the future.  But more importantly, it's a professional development goal, an exercise in code craftsmanship.  As I mentioned above, I've worked professionally on many systems that are even more messed up than LnBlog.  So this is a study in how to approach refactoring a system.  

And So It Begins...

My intention is to write a number of articles describing this process.  I've already completed the first step, which is rewriting some of the persistence and publication logic.  I'm using the PSP to track my planned and actual performance, so I'll have some actual data to use in my discussion of that process.  Hint: so far, the two are very different.

With any luck, this project will enable me to draw some useful or interesting conclusions about good and bad ways to approach reworking legacy systems.  Maybe it will also enlighten some other people along the way.  And if nothing else, I should at least get a better codebase out of it.

And the current editor is - Vim?

So, as I mentioned before, I'm looking for a new go-to text editor/IDE.  So far, I've taken cursory looks at a few of the options.  These include:

  1. PHPStorm.  As I mentioned in the last post, I use PHPStorm at work.  And while it's really good in a lot of ways, I'm not in love with it.  On the up side, it's got great code intelligence and the VI-mode plugin is actually quite good.  On the down side, it's a single-language IDE (well, not quite, but it's still got a limited set of supported languages).  I guess I could just buy the entire JetBrains IDE suite, but I'm not crazy about switching back and forth.  Also, PHPStorm is kinda heavy - like, Eclipse heavy - both in terms of memory footprint and, more important, conceptual weight of the UI.  So while I don't dislike it, I don't really have any enthusiasm for it.
  2. Visual Studio Code.  I initially liked the look of VS Code.  It's got the cool Visual Studio intellisense, which is nice, and it seems to have a lot of extensions available.  The Vim-emulation plugin seemed fairly good, but not great.  The most popular PHP plugin, however, didn't seem to work out of the box at all.  I'm not sure why, though it could be its wonky install process/implementation (apparently it's written in PHP).  At any rate, I didn't care enough to look into it, though it might be worth taking a closer look at VS Code at some point.
  3. Atom.  I liked the look of Atom.  I read a little about the philosophy behind it and I really wanted to like it.  But then I fired it up and tried opening one of my project directories and it didn't work.  And by "didn't work", I mean that Atom actually crashed, consistently, on this particular directory.  So that's a no-go.  A quick Google revealed that it might be a problem with the Git library, which could possibly be fixed by changing the index version on the repo, but frankly I don't care.  If I can't trust my editor to just open a directory, then I can't trust it at all.  I mean, I don't even care if my editor has Git support at all, so I'm certainly not going to accept crashing if it sees a repo it doesn't like.  
  4. Sublime Text.  I've heard good things about Sublime.  I used to work with several people who really liked it.  Then I fired it up and immediately said, "What the heck is this?"  The UI is pathologically minimal, except for a gajillion menu items and the stupid friggin' mini-map (which I didn't like in Komodo either).  Configuration is done by editing a JSON file, but what the heck is with the weird out-of-box-experience?  The UI is extremely minimal and customization is done by editing a JSON file, which is weird (to be fair, VS Code and Atom do that too, but it's more forgivable because they're free), and the plugin manager was immediately confusing.  Seemed like getting used to Sublime might be a steep learning curve.
  5. Vim.  Yes, you read that right - Vim.  I was surprised too.  Let me explain.

After trying out Sublime, my initial reaction was, "Geez, if I want something that complicated, why don't I just use Vim?"  And then I stopped.  And I said to myself, "Actually...why don't I use Vim?"  Good Vim emulation is one of my must-haves, and no plugin is ever gonna beat the real thing.  It's free, open-source, hugely customizable, has lots of available plugins, and is extremely well established.

The thing is, I knew Vim could be customized into a pseudo-IDE, but I'd never really thought of myself as a hard-core Vim user so I'd never tried it.  But the truth is that I've been a Vim user for a very long time, and for the last few years I've been actively trying to pick up more Vim tricks for use in Vim emulation plugins.  So while I didn't know an awful lot about customizing Vim, I'm very comfortable actually editing code in it.

And it turns out that actually customizing Vim isn't really that bad.  Heck, there are even package managers for Vim now!  There are also IDE-like configuration/plugin bundles you can install, such as spf13, but I quickly determined that those were too big and overwhelming.  However, they are good as a source of ideas and settings to copy into your own custom ~/.vimrc file.  That's actually part of the beauty of Vim - despite the fact that Vim script is a little weird and the configuration is in no way intuitive, there's enough information already out there that it doesn't really matter.

So over the course of a week or so, I pulled out some of the more interesting settings from the spf13 config, found a selection of plugins that I liked, and got myself a nice, functional Vim setup.  I even set up some symlinks and file syncing so that I can have my setup synchronized between home and work.  

Is it perfect?  No, but nothing ever is.  But so far, it's working pretty well.  And what's more, I actually enjoy using it.  It might not have the power of a specialized IDE when it comes to the more advanced features, but it's got a heck of a lot of power in general.  And the amount and types of customization you can do are amazing.  With a little effort, you can really shape the editor to your workflow, which is a thing of beauty.

Looking for a new editor

A couple of weeks ago I started looking into new code editors.  I've been using Komodo for eight or nine years now, counting both KomodoIDE and KomodoEdit, but I'm growing dissatisfied with it.  In fact, I'm becoming unsatisfied enough that I think the time has come to move on.  However, I'm not sure I see an obvious candidate for a replacement.  So in this post, I'll talk about some of the things I find lacking in Komodo and what I'm looking for in an editor.

Why the change?

Let me start by saying that I'm not trying to bad-mouth Komodo here.  I have used it for a long time and it has served me well.  The people at ActiveState are nice and the do good work.  But the direction it seems to be heading in doesn't mesh well with my needs and I'm not sure the cost/benefit analysis of sticking with Komodo makes sense anymore.  Maybe this can provide some insight to the Komodo development team that can help them improve the IDE.

My dissatisfaction started a few months ago, when my team transitioned to a different product.  Unlike the last product we worked on, this one doesn't belong to just us.  This one is core to the company's business, and has several other development teams working on it, not to mention all the other groups that come in contact with it.  As such, there's an existing process and structure that we needed to adhere to, and it quickly became apparent that adhering to that was much easier if you were running the standard development setup, which is PHPStorm on Ubuntu.  I was running KomodoIDE on Windows, so this was a problem.  I was able to use WSL to get around the Linux part, but KomodoIDE just didn't offer the same features that I needed from PHPStorm.

So let's use that as a starting point.  What does PHPStorm offer that Komodo can't compete with at present?  Well, for purposes of the product I'm working on, here's the list:

  1. Code intelligence.  This is by far the biggest factor.  I'm not just talking about intellisense here.  I mean finding and navigating the actual places in the code where a class, function, or method is used or defined.  And not just text search, but actually knowing about the class and its inheritance hierarchy.  PHPStorm is actually pretty awesome at that.  Komodo's code intel., at least least for PHP and JavaScript, is buggy at best and totally broken at worst.  The quality of the experience also seems to vary hugely depending on the codebase you're working with.  It's nice that they have something for code intel., but you can't really rely on it.
  2. Validations.  PHPStorm integrates with phpcs, which is nice because we need to actually adhere to PSR-2.  Komodo doesn't have that built in.  This might seem like a trivial thing because, after all, I can always just run phpcs from the command line.  However, having the check done in the editor is actually hugely useful, because we have a lot of legacy code that doesn't adhere to PSR-2, which makes selectively running the checks for only the code you changed awkward.  Seeing violations in your editor gives you a nice, simple way to catch mistakes you made before they get to code review.
  3. Symfony support.  While it's not built in, PHPStorm has a pretty good quality plugin for Symfony support.  We use Symfony, so this is really nice.  Komodo doesn't have that.

Those things are important, but they are specific to the new product I'm working on.  If these were my only complaints, I would happily continue using Komodo for the foreseeable future and just use PHPStorm for that one project at work.  But they're not.  The list of annoyances and things I don't like has been slowly growing over the years.  This includes actual problems and bugs, missing functionality, and road-map issues (i.e. disagreement with the direction I see Komodo going).  Here's a summary:

  1. Kinda buggy.  I hate to say it, but I've seen a decent amount of weird errors, crashes, and just plain strange behavior in Komodo.  Not a huge number - it certainly hasn't been buggy enough to get switch editors - but it's still a non-trivial issue.  I'll give just a few examples here to give you the flavor of my complaints.
    1. Maybe it's just my perception, but it's rare for the error log or console to not have something in it.
    2. Sometimes when I'm doing things in the "places" panel, some keypress will trigger a change to the next pane of that panel.  I'm not sure what it is and I can't seem to do it on purpose.
    3. I'm constantly empty getting "find" panes accumulating in my bottom panel.  Again, I'm not 100% sure what causes them and can't seem to reproduce intentionally.
    4. I've tried numerous times on numerous versions of Komodo to set up the keybindings for "focus X pane", but they just don't seem to work at all.
    5. There are various places in the settings UI where one setting determines another, but it's not clearly indicated.  So you can change a setting, click OK, and your change will just be ignored because it's invalid, but there's no indication of that.  A good example is the new unit test runner.  For options other than "custom", the testing framework determines which parser is used, but you can still change the parser in the UI.  It just doesn't do anything.
    6. The syntax checking preferences for JavaScript allow you to set a custom JSHint version to use, probably because the integrated one is kind of old.  I've tried this numerous times and have never been able to get it to work.  Oh, and speaking of JSHint, I'm still kinda miffed that Komodo 10 removed the graphical JSHint configuration UI.  Now there's just a text area to enter your settings, so you have to go to the JSHint site and look up the options rather than just being able to pick from a list.
  2. Pretty over functional.  In the last few releases, some of the tools have been revamped in such a way that makes them prettier, but in my opinion doesn't actually make them more useful.  The two big examples that spring to mind are version control and unit testing.
    1. In older versions of Komodo, my only complaint about the VCS integration was that there wasn't an easy way to do a commit of all pending changes in the project - I fixed that myself with a simple macro.  In the new version, they fixed that problem.  But at the same time, the VCS widget no longer displays changes for added files, which is a big pain.  I typically use my VCS diff for pre-commit code review, so it's kind of inconvenient to have to switch to another view to do that.
    2. The new unit test runner in Komodo 10.2 looks very pretty.  However, they removed the key bindings I was using to run the current test suite.  And it didn't detect my old test suites, so I had to recreate them.  They also changed the name-humanizing algorithm, so that test names that used to be rendered nicely in the runner aren't anymore.
  3. Features I don't care about.  It feels like there have been a few of these added lately.  Some of them seem like things that are really just gimmicks that look good in marketing material but either don't provide any value.  Others do provide genuine value, but seem tacked on to pad the feature set, i.e. they're useful, but I don't see the point of having them right in the IDE.  Some examples include:
    1. Collaboration.  You can do real-time editor sharing with someone else using KomodoIDE.  Cool!  And maybe it would be useful if I was doing remote pair programming with someone else who uses Komodo.  But I'm not, so I've never actually used it.  I suppose this could substitute for some of those "social coding" web editors, but this doesn't feel like a general enough use-case to want it integrated into my IDE.
    2. Sharing to kopy.io.  Upload code snippets directly to a code-sharing website.  Nice!  But again, that's something that's not hard to do without IDE support and that I seldom or never do it anyway.  And even if I did, I'd just create  gist on GitHub, not use a new site.
    3. Slack sharing.  You can share code in a Slack channel right from inside Komodo.  Great!  But again, I don't do this often and it's not clear how this is easier than just copy-and-pasting the code into Slack.
    4. Minimap.  A 10,000-foot overview of how your code looks that replaces the traditional scroll bar.  I think Sublime Text had this first.  Sure, it looks really cool, but does anyone actually use these things?  I don't spend a lot of time looking for code segments based on the shape of the text, so it's never been clear to me what useful information this provides.
    5. HTTP inspector.  This is actually an older feature - an HTTP proxy that allows you to inspect traffic.  And it's a genuinely valuable thing for a developer to have.  But you still have to set it up like any other HTTP proxy, so it's not clear how having this baked into the IDE is better than just using a stand-alone proxy app.  And again, this isn't something I need on a regular basis so I've never used it for actual work.
  4. Features that are strange or hard to use.  There are also features that I use, or would like to use, that either don't make sense or don't quite do what I need.  In other words, they could be really good and useful, but the fall short.  For instance:
    1. Keyboard navigation.  Komodo actually has a pretty nice configuration interface for setting up keyboard shortcuts.  Unfortunately, a lot of the actual bindings you might want don't exist (or don't work, as previously noted).  But my big gripe is that navigating around the UI using the keyboard is difficult, particularly in the side/bottom panels.  Trying to navigate between fields within a pane often either doesn't work, gets stuck, switches you to the main editor, or otherwise fails in strange ways.  And as I mentioned earlier, trying to use the keyboard to navigate between different panes seems to just generally not not work.
    2. Regex toolkit.  This seems like a really useful tool.  But I'll be darned if I can figure out how it's supposed to work.  Every now and then I try it and I always spend more time trying to figure out how it works than it would take so just write a one-off test script to test the regex.
    3. Publishing.  Komodo has a publishing tool that lets you push your code up to a remote system for testing.  That's a very nice and useful thing.  Except that it's actually a "synchronizer," by which I mean it only does two-way synchronization of files with a remote server.  Why?  What if I don't care what's on the server and just want to clobber it every time?  That's not such an uncommon occurrence with test servers.  In fact, for me, wanting to pull in changes from the remote servers is distinctly an edge-case, not something I'd want to happen by default.

I could probably go on, but I think I've made my point.  It's not that Komodo is a bad IDE - far from it.  But there are a number of rough edges and niggling little issues that are increasingly starting to bother me.  Choice of editor can be a very personal and subjective thing, and for me it just feels like it's time for a change.

What do I want?

So that leaves the question: what do I want in an editor or IDE?  Well, I'm not completely sure.  I've been using Komodo for a long time, and I do like a lot of things about it.  So let's start with a list of some of those things.  That should at least work as a jumping off point.

  1. Good VI emulation.  I've been using VI emulation in my editors ever since I worked for deviantART.  I got a MacBook Pro when I started there and I found the weird keyboard layout completely infuriating, particularly when I switched back to my regular PC keyboard so I decided to just switch to VI key bindings since they're always the same.  Since then, I've gotten progressively more friendly with VI mode, to the point where coding without it feels strange.  Komodo includes fairly decent VI emulation out of the box, and it's relatively easy to add customizations, so any editor I pick up will need comparable VI emulation support.
  2. Code formatters.  One of the really handy features of Komodo is built-in code formatters.  My most common use-case for this is copying some JSON out of the bowser network monitor, pasting it into Komodo, and formatting it so I can analyze it.  It would be really nice for a new editor to support something like that.
  3. Light-weight.  A concept of "projects" is nice in an IDE, but sometimes I just want to quickly open a file.  So I'd rather not use an IDE that takes ages to start up and insists that everything be part of a project (I'm looking at you, Eclipse).  Komodo is pretty good in that regard - it can just start up and open a file without it being too much of an ordeal.
  4. Extensibility.  No editor or IDE is ever perfect out of the box, so it's important to have a decent extension ecosystem around your editor.  Komodo is pretty good in this regard, with a community site offering a nice assortment of extensions.
  5. Scriptability.  In addition to extensions, one of the things Komodo gets really right is giving you the ability to easily write simple scripts to automate things.  It lets you write user scripts in JavaScript that can be fairly easily hooked into the UI.  This is huge.  There are a lot of "small wins" you can achieve with this kind of scripting that will really improve your workflow.  Supporting extensions is great, but it's hard to justify integrating a feature into your IDE if you need, e.g., an entire Java build environment to write an extension for something you could do in five lines of shell.
  6. Multi-language.  This is a big one.  If you regularly code in more than one language, having to learn a different IDE for each one is a real drag.  In the best-case scenario, you have to configure the same things for each editor.  In the worst-case scenario, you have to learn two completely different tools with completely different features.  Most of my professional work is in PHP and JavaScript these days, but I also do some Python, SQL, BASH and PowerShell scripting, and I'm trying to learn Haskell.  So given the choice, I'd rather learn one editor inside and out than have a bunch of language-specific editors that I use as "notepad with some extra features".
  7. Cross-platform.  Right now I use Windows both at home and at work, but that's not set in stone.  I actually use Windows at work by choice (yeah, yeah, I know) - there are a few people with MacBooks, but most of the other developers use Ubuntu.  /In the past, I've gone back and forth between platforms, so running on Windows, Linux, and MacOS is a hard requirement for me.  I don't want to be tied to one platform or have to learn a different IDE for each if I decide to switch.
  8. The right feel.  This one is highly subjective, but it's important to me that my IDE feel smooth to use.  I want it to work with me, rather than feeling like I have to adapt to it.  This has always been my problem with Eclipse - I feel like the UI doesn't really adapt smoothly to what I want.  Something about coding in it just feels a little "off" to me.

 Time to experiment

So it's time to start doing some experimentation.  It took me a long time to finally settle on Komodo, so I'll probably go back and forth a few times.  

I've got lots of choices, to be sure.  Back when I settled on Komodo Edit, I was looking primarily at free and open-source editors.  My use of Komodo IDE grew out of that.  These days, I'm not as price-sensitive, so commercial IDEs are definitely on the table, provided they have reasonable non-Enterprise pricing (i.e. I'm not gonna spend $2000 on one).

Last time I was IDE shopping I looked at a bunch of Linux-only editors, but those are out.  I used Eclipse for a while, but I'm not inclined to revisit that.  I also used jEdit for a while, and while it was fairly nice, it doesn't appear to be getting much active development these days, and I'm not sure it fits my current requirements anyway.  But now we have things like Sublime Text, Atom, Visual Studio Code and the entire suite of Jet Brains IDEs.  So I've got lots of things to look at.  If nothing else, the process will be good for a few blog posts.

Execution policy weirdness

I discovered something enlightening and annoying this morning: PowerShell uses different execution policy settings for the 32-bit and 64-bit versions.  This is something I did not know and did not expect.

I'd seen this problem before, but could never figure out what was happening.  I'd try to run a command through PowerShell using another program, in this case Vim, and it would fail with the standard error about execution of my profile.ps1 being prohibited by the execution policy setting.  Yet when I bring up Powershell on the command line, everything works and the execution policy looks correct:

PS pgeer> Get-ExecutionPolicy -List Scope ExecutionPolicy ----- --------------- MachinePolicy Undefined UserPolicy Undefined Process Undefined CurrentUser Undefined LocalMachine RemoteSigned

Well, it turns out we're talking about two different versions of Powershell.  When I run it from the terminal, I'm using the 64-bit version.  But my Vim build is 32-bit, which apparently means that it's running the 32-bit version of Powershell.  And that version uses its own execution policy setting.  You can confirm that by explicitly opening "Windows Powershell (x86)" from the start menu and running Get-ExecutionPolicy.  

As for the fix, it's simple: just run Set-ExecutionPolicy RemoteSigned as admin, but do it in both the x86 and x64 versions of Powershell.

As for why Microsoft chose to make the different builds use different execution policies, I couldn't say.  It doesn't make much sense to me.  Although, to be honest, I'm not sure why I even need both versions on the same system.  Maybe to support interop between commandlets and unmanaged code?  Who knows?

KeePass browser plugins

In my last post about KeePass, I mentioned that you can integrate your KeePass password database with your web browser.  In this post, I'll tell you more about how to do that and why it's an extremely handy thing.

Why bother?

So why would you want to bother with integrating your browser with KeePass?  I mean, most browsers have a feature to remember your passwords anyway, so why not just use that?  Or if you want to use KeePass, why not just use that auto-type feature I talked about in the last post?

It's true, you could just use the password manager that's built into your browser.  Pretty much all of them have one, these days.  Most of them will even secure your data with a master password.  They may even synchronize your passwords to the cloud, so you can access them on more than one device.  Granted, that's pretty handy.

However, browser password managers generally just do passwords - they don't allow you to enter extra information or attach files like KeePass does.  They also don't work for things outside the web browser, like for security software such as VPN clients.  So they don't provide you with a single, secure location for all your important account information.  But more importantly, they're generally tied to a single browser.  Sure, Google Chrome can store and synchronize all my passwords, but what if I decide I don't like Chrome anymore?  Maybe I just bought a Mac and decided I really like Safari.  Is there an easy way to get my passwords out of one browser and into another?  I don't know.  

By using KeePass with a plugin for your browser, you can get the best of both worlds.  KeePass itself gives you more power and features than browser password managers and allows keeps you from being tied to a single browser.  Using a browser integration plugin adds on the ability to have the browser automatically fill in your username and password when you visit a website.  It's not quite as convenient as the browser-integrated password managers, but it still pretty good.  And it's definitely a lot easier than trying to use auto-type or copy-and-paste to fill in password forms.

What are my options?

In general, there are a lot of plugins available for KeePass.  Just look at the list.  Or maybe don't - you probably don't care about 90% of those plugins.  The main thing you need to know about is which browsers have plugins available. 

Short answer: Chrome, Firefox, and Safari.

Long answer: Chrome, Firefox, and Safari have proper browser plugins available.  The Chrome plugin also works in Vivaldi and possibly other browsers that are based on Chrome.  There are also form-filling plugins that work with Internet Explorer.  To my knowledge, there is no plugin support available for Microsoft Edge.

For this entry, I'll just talk about setting up a plugin with Chrome.  We're going to use a Chrome extension called ChromeIPass.  It adds a KeePass button to the toolbar in Chrome and can automatically detect login forms on webpages you visit.  It works with a KeePass plugin called KeePassHttp.

First, you need to install the KeePassHttp plugin.  Start by going to the KeePassHttp website and clicking the "download" link, or just download it directly here.  Sadly, KeePass doesn't have a nice way to install plugins - you just have to copy the plugin file to the KeePass plugins folder on your system.  Inconvenient, but fortunately not something you need to do very often.  On most computers, this will be C:\Program Files (x86)\KeePass Password Safe 2\Plugins.  So just copy the KeePassHttp.plgx file that you downloaded and paste it into that location.  Since this is a system directory, you will probably be prompted to grant access.  Click "continue" to copy the file.  Note that if KeePass is running, you will need to close and restart it for it to detect the plugin.

Click "continue" when prompted to allow access to copy the plugin.

Now that the KeePassHttp plugin is installed, KeePass will be able to communicate with Chrome.  You just need to install the ChromeIPass extension.  You can do that by going to the Chrome web store page here and clicking the "Add to Chrome" button.  

So...now what?

OK, now that ChromeIPass is installed, what do you do with it?  Well, not really much until it's time to log into a site.  So pick a site that's in your KeePass database and go there - I'll use sourceforge.net for this example because it's a pretty standard login form.

The first time you try to log into a site using ChromeIPass, you'll need to connect it to your KeePass database.  You should notice a KeePass icon is now in your toolbar.  Make sure KeePass is running and click that button.

You should see a "Connect" button.  Click that and KeePass will prompt you to add a new encryption key for the KeePassHttp plugin.  This is a security mechanism - the KeePassHttp plugin encrypts its communication with your KeePass database and this is just the initial step where it sets that up.  Don't worry about the details right now - just type in a unique name for the key, maybe based on your browser and computer, e.g. "Laptop - Chrome".  You only have to do this the first time you connect a browser to your database - after that, the encryption is automatic.

Now that ChromeIPass is connected to your KeePass database, you can click the ChromeIPass button in your toolbar and click the "Redetect Credetials Fields" to fill in your username and password.  Alternatively, you can just refresh the webpage and they should be auto-filled.  You won't see anything in the browser yet, but KeePass itself ill prompt you to allow access to the password for this site.  You can check the "Remember this decision" box to not be prompted to allow access the next time you visit this site.

(I should probably stop to acknowledge that this thing of having to grant a site access to your KeePass database before you can log in is kind of a drag.  I agree, it is somewhat annoying.  This is actually a security feature of KeePassHttp - that's the portion of this that runs inside KeePass itself and allows the ChromeIPass extension to talk to it.  It actually has a lot of security-related settings.  This is actually a good thing, though, because it essentially provides a way for other programs to read your KeePass database, and you want to make sure that malware or dodgy websites aren't able to do that.  However, if you want to disable some of these settings, like prompting to allow access, you can do that by going into KeePass and selecting the "Tools > KeePassHttp Options" menu item.  The KeePassHttp documentation has some more information on the available settings.)

The good news is that now you're done!  After you allow access to KeePass, ChromeIPass will automatically fill in your username and password.  If you selected the "remember" option when allowing access to the site, ChromeIPass will automatically fill in your login info the next time you visit the site, no action required.  You will only have to allow access the first time you visit a new site of if you elect not to have KeePass remember the approval.

If you're so inclined, ChromeIPass has a number of other features, as detailed in the documentation.  For instance, it can save or update entries automatically when you enter a password into a webpage; it has a built-in password generator that lets you create strong passwords right in the browser; it can customize the login fields for non-standard login forms; and it provides a handy right-click menu to fill in passwords and access other functionality.  

Hopefully this will help get you started.  Using a password manager is a must for keeping your accounts secure these days, and integrated browser support makes using one that much easier, which means you're more likely to keep using it.

Using KeePass

You should be using a password manager.  If you're a technical person, this is probably not news to you - you're very likely already using one.  

This article is for the non-technical people.  People like my wife (hi, honey!) and my mom.  People who access a lot of websites and have a lot of passwords to remember.

Security 101

So why is using a password manager a good idea?

Well, you may have seen guidelines for cyber security that tell you things like:

  1. Don't write down your passwords.
  2. Don't reuse passwords on different sites.
  3. Don't use short, easy to guess passwords.
  4. Don't use passwords that are easy to figure out from public data (like a birthday that anyone can get from your Facebook profile).

Such guidance raises the question: if I have to use long passwords that aren't related to anything in my life, and I can't reuse them or write them down, how the hell am I supposed to remember them?!?

This is a totally reasonable question.  Yes, ideally we would all memorize a hundred different 32-character-long, randomly generated passwords.  But in real life, nobody can actually do that.  So a password manager is a good compromise.

What is a Password Manager

My mother has a little paper "password book" that she keeps in a drawer next to her computer.  When she has to create a new account for some website, she writes down all the login information in that book so that she can look it up later.

A password manager is the digital equivalent of that password book.  It's an application that lets you record your login information and them look it up later.  Most password managers have lots of other handy-dandy features as well, but that's the core of what they do.

So how is this different from, say, writing down all your passwords in a Word document on your desktop?  Well, a password manager encrypts all your data.  It requires a "master password" to decrypt your information, so if some nasty hacker steals that file, they won't be able to actually read it.  

Is this as secure as just memorizing all your passwords?  No.  But as we said, nobody can do that anyway, and this is one heck of a lot more secure than the alternatives, i.e. reused or weak passwords.  With a password manager, you can still have strong, unique passwords for all your sites, but you're relieved of the burden of having to remember them all.

About KeePass

There are a number of password managers out there, but the one I'm going to talk about is KeePass.  It's a free, open-source password management application that will run on Windows, Linux, and Mac, and has compatible apps available for iOS and Android.  KeePass works offline (i.e. it requires no internet connection and doesn't depend on any online services), but it's possible to sync your KeePass passwords between devices using file sync tools like DropBox or OneDrive.  So it provides you some flexibility, but you aren't beholden to a single company that can get hacked or go out of business.

KeePass creates files password files that end with ".kdbx".  You can open those files from within KeePass or double-click on them in Window Explorer.  When you try to open one, KeePass will prompt you for the master password to that file.  Every KDBX file has its own master password.  This allows you to do things like create a password file to share with the rest of your family, and have a different one for the accounts that are just yours.  (That's a topic for a different post.)

One of the handy extra functions of KeePass is that each entry in your password save can have a bunch of extra data associated with it.  For example, you can add custom fields and attach files to each entry, which are handy for things like account validation questions and activation files for software licenses.  Basically, you can keep all the important information in one place.  And since KeePass encrypts your entire password file, it will all be kept secure.

Using KeePass

So how do you use KeePass?  Let's walk through it.

Step 1 - Download

The first thing you need to do is to get yourself a copy of KeePass.  You can go to this page and click the download link for the "professional edition".  (There's not really anything "professional" about it - it's just a newer version with more features.)  When that's done, you can double-click the file to install it like any other program.

You can also install KeePass through Ninite.  If you're not familiar with Ninite, I strongly recommend you check it out.  It's a great tool that makes it brain-dead simple to install and update a collection of programs with just a few clicks.  You basically just select a bunch of applications you'd like to install from a list, click a button, and you get an installer program you can run to put everything you selected on your computer.  And if you run that program again later, it will actually update any of those programs that have a newer version.  It's very slick and a real time-saver.

Step 2 - Create a password safe

 Next, open up KeePass and click "File > New".  You will be prompted to choose where you want to save your new password database.  Choose a name and folder that work for you.  Remember - your password database is just a regular file, so you can always move or rename it later if you want.

After that, you should get a dialog that looks like this:

This shows several options for securing your password safe.  But don't worry about that - the one you really want is the first one, "master password".  So choose a password and type it in.  If you click the three dots on the right, KeePass will display the password as you type, so that you don't have to re-enter it.

There are two important things to note when choosing a master password.  First, since it's going to protect all your other passwords, you want to make it good.  KeePass provides a password strength meter to help you judge, but the main things to bear in mind are that you want a range of different characters and you want it to be long.  And no, ten letters does not qualify as "long" - it should be more of a passphrase than a password.  One common technique is to use a full sentence, complete with capitalization and punctuation (an maybe some numbers, if you can work them in).  That will generally give you a pretty strong password, but it will still be easy to remember.

The other important thing to remember is that the password to a KDBX file is your encription key for that file.  That means that the only way to decrypt the file is with that password.  If you forget your master password, your data is gone forever.  So write down your master password and keep it in a safe place until you're certain you've committed it to memory.  And if you want to change your master password later, make sure to make a backup copy of your KDBX file first.

After you've chosen a master password, you should see a screen that allows you to configure some of the settings for your password file.  However, you don't really need to worry about this - those are all optional.  You can safely click the "OK" button to just continue on.

Step 3 - Organize your passwords

Alright!  You now have a password database set up.  You should see a list of groups on the left and a list of password entries on the right, like in the image below.  These are the sample groups and entries that KeePass creates by default.  They're just to give you an idea of how to use your password database - you can safely delete them at any time.

You can click on each group at the left to see what entries it contains.  The groups are basically like folders in Windows.  There's a top-level folder, and it contains a bunch of sub-folders and each of those sub-folders can contain other folders.  So in the screenshot, you can see that "NewDatabase" is highlighted in the group list.  That's the top-level folder for my example database.  You can see on the right that it contains two entries.  You can move an entry into another folders by dragging it from the entry list on the right onto one of the folders on the left.

Step 4 - Create passwords

To add a password entry to your database, select "Edit > Add Entry" from the menu.  That will bring up the entry edit screen.  This is the same screen you'll see when you double-click on the title of an existing entry, except that it is mostly blank.

There are a lot of tabs and options on this screen, but you don't really need to worry about those.  The main things are right in front of you: the entry title, user name, and password.  You'll probably also want to fill in the URL field with the web address of the site this information is for.  This will come in handy if you want to use a KeePass plugin for your web browser (which we'll cover in another post).  When you're done entering your info, click the OK button to create the entry.  You should then select "File > Save" from the menu or push the "save" button on the toolbar to save the changes to your password database.

You'll probably notice that there's already a password filled in.  KeePass will generate a random password for new entries.  You are free to change this yourself or use the button to the right of the "repeat" box to generate other random passwords using different rules.  KeePass has a password generator that lets you specify the allowed characters and length for a random password, which is handy for those sites that insist on specific password length or complexity rules.

Step 5 - Getting your passwords out

Now let's back up and say you've just started up your computer, are logging in to some website, and want to get a password out of KeePass.  The first thing you need to do is open up your password database.  You can do this by double-clicking on it in Windows Explorer or by opening up KeePass then selecting your database from the "File > Open" menu.  When you open the database, you'll be greeted by a screen asking you to enter your master password - you know, the one you came up with in step 2.  (Hint: remember that you can click the button with the three dots to display the password as you type it.)  After you enter your master password, the database will be decrypted and you'll find yourself and the entry browsing screen from step 3.

There are several ways to get your passwords out of KeePass.  Here's the list in order of preference:

  1. Use a browser plugin to automatically fill in login forms.  Since most of the passwords you end up creating are for websites, setting up your browser to fill in the forms from your KeePass database makes life much easier.  I'll talk about how to do that in the next post.  But don't worry - it's not all that hard.
  2. Use auto-type.  This is a feature of KeePass where you to click a button in the KeePass window and it will automatically send the keystrokes for your username and password to the last window you used.  So, for example, you would navigate to the login page of a site in your web browser, click in the username box, and then switch over to the KeePass window and click the auto-type button on the toolbar (the one that looks kind of like some keyboard keys - hover your cursor over the buttons to see the descriptions).  By default, the auto-type feature will type your username, the "tab" key, your password, and then the "enter" key.  This will work for probably 90% or more of login pages, but it's not universal, so be aware of that.
  3. Copy them to the clipboard.  If all else fails, you can always just copy your passwords to the clipboard so that you can paste them into another window.  KeePass makes this fairly easy.  In the main password list that you saw in step 3, when you double-click on the actual username or password for an entry in the list, it will copy that to the clipboard.  This saves you having to open up the entry edit screen and copy things there.  You can then switch to another window and paste the data into a login screen.
  4. Just read it.  Last, but not least, you can always go low-tech and just read the passwords out of the window.  Just double-click the name of your entry, then click the "three dots" button to make the password visible.  Clearly this is not great, but sometimes it's necessary.  For example, you will need to do this when entering a password on a system that doesn't have KeePass installed, such as to login into your Amazon or Netflix account when setting up a Roku or other streaming media system.

Conclusion

With any luck, I've made this "password manager" thing sound good enough for you to try it out.  You really should look into it.  Password reuse has become increasingly dangerous, with hackers trying the usernames and passwords they harvested from one hack on other sites just to see if they work.  Password cracking tools have also advanced a lot in recent years, including information gleaned from previous attacks, so relying on things like "133t 5p34k" passwords is no longer good enough.  A decent password manager, if used consistently with randomly generated passwords, will provide you with a good trade-off between convenience and security.