Photo organization

Yesterday I started part one of 723 in what will be my ongoing attempt to get my photo collection under control.  This is a task that I've been thinking about and putting off for literally years because it's so daunting.  But when one of our cats died yesterday (rest in peace, Loki) and I was looking for pictures of him, I realized that it's time to actually do it.

My problem is simply that I have an absolutely huge collection of pictures and videos.  Ever since we got our first digital camera, I've been very liberal about taking pictures on vacations and such.  I mean, they're free, so why not take a bunch so that at least one comes out nice?  And, of course, ever since we got smart phones, I've been very liberal about taking pictures all the freaking time.  I have the OneDrive app on my phone configured to automatically sync them to the cloud and then down to my laptop, so I have multiple copies and they get included in my normal laptop backups.

A small sample of my picture collection

All that works great, but the issue is that I now have well over a decade worth of pictures and videos - over 12,000 files taking up about 60GB.  How the heck do you find anything in that many files?  My current organizational "system" (if you can call it that) is just to manually sort pictures from my camera roll into "albums", i.e. folders.  That's fine as far as it goes, but it's not very helpful if I want to find anything at a more granular level, e.g. pictures of a specific cat.

Step 1 of 723: Make a Plan

Ultimately, I want to be able to have some nice photo albums and still be able to search for individual photos based on date and content, e.g. pictures of my son from when he was two years old.  I  also want to be able to able to easily create and share photo albums, both with family and with myself through other devices, e.g. tablets, TVs, etc.

And, by the way, I really don't want to be locked into one application or service.  I've been collecting these photos for over 15 years and based on my experience with the software industry it's not unlikely that my collection will outlast any vendor I happen to pick.

Digikam main interface

So the plan is to start by adding tags to all my pictures.  Yes, this will take along time, but the idea is that once I have all the pictures tagged with the people and places that are in them, I can more easily search them, which will allow me to refine my tags as well as pick out meaningful groupings for albums.

To do the tagging and management, I decided on Digikam.  I picked this for several reasons:

  1. It's open-source and cross-platform (it runs well on Windows, despite being a KDE project), so it avoids lock-in on that level.
  2. It can write the tags I'm setting directly to the picture metadata.  So while Digikam has and uses its own database, I'm not tied to that.  The canonical source for metadata is the images themselves, which avoids a different level of product lock-in.
  3. It's pretty fast and makes it easy to navigate images.  The hierarchical tag structure Digikam supports is actually pretty nice for navigation.
  4. It's extremely powerful and gives you a wide range of editing capabilities and publishing/export options.

So far, I'm still working my way through adding all the takes.  Obviously that's going to take a while.  The process is tedious, but Digikam is working pretty well for adding the tags.  The interface takes a little getting used to, if only because there are so many options and tools, but once I got the hang of it, the tagging was pretty easy and went pretty fast.

The up side of the tedious tag adding is that I have an excuse to go back through many years worth of memories.  There are a lot of pictures in there that I haven't looked at in years and it's nice to be reminded.  This is also a good opportunity to clean up duplicate pictures and ones that are just junk, e.g. out of focus, too dark to see, or contain mostly my finger.

Text-based UML

Recently I discovered a new tool that I never knew I needed - PlantUML.

If you're like me, you probably want to do more UML.  I mean, I'm interested in software design and architecture.  I read books and articles about it.  I even wrote my thesis on formal modeling.  So I'd love to do more UML modeling.

The thing is...I don't like UML modelers.  I mean, it's not that the tools are bad - in fact, some of them are pretty good.  It's just that creating a UML model feels so heavy.  And while the actual modeling features that many tools have are really cool and useful in some circumstances, I find that 90% of the time all I really need is a simple diagram.  And while any UML tool can help you make a diagram, I feel like I usually end up getting bogged down in the mechanics of putting it together.  You know, you've got to select the right type of box, select the right type of relationship, then the tool renders the connections in a weird way by default so you have to fix it, etc.  Before you know it, you've spent 20 minutes on a diagram that would have taken two minutes if you'd done it on paper.

Enter PlantUML.  It bills itself as a "drawing tool" for UML, but the upshot is that it's a way to define your models in plain text.  You just write your models in your favorite text editor (and yes, there's a Vim syntax file available), run the tool, and it will spit out a rendered UML diagram.  Here's an example:

,

And here's the text that generated that: @startuml class Link { name : string description : string url : string } class Tag { name : string } class Folder { name : string } class User { username : string password : string setPassword(password : string) } class Comment { body : string } Link "1" -- "*" Tag : has > Link "1" -- "*" Comment : < belongs to Folder "1" -- "*" Link : contains Folder "1" -- "*" Folder : contains User "1" -- "*" Link : owns > @enduml

As you can see, the syntax is fairly straight-forward and pretty compact.  All of the standard UML diagram types are supported and the syntax allows you to provide minimal detail and still produce something meaningful.  In addition to the GUI shown above, it can also run from the command line and just create PNG images (or whatever format you like) of your diagrams, so you could easily work it into your build pipeline.  And the installation is simple - just download and run the JAR file.

The thing I really like, though, is that this text-based format makes it easy to store and source-control UML alongside your code.  Yes, you technically can do that with other formats, but it's awkward.  XMI files are huge and ugly and I don't even want to think about the project files for Eclipse-based tools.  But with PlantUML you can just have a directory with some "modelname.pu" files in it that are small, simple, and produce diffs that are easy to read when you change them.

I haven't tried it out yet, but I'm also interested in how feasible it would be to put the models right in the code, e.g. put the text in comments.  Seems like it might help with the whole "keeping code and models in sync" thing.  But maybe that's a bit much.

I recommend checking it out.  If you want a quick and easy method, there's an online version that you can test.

My UHK has arrived

My Ultimate Hacking Keyboard (UHK) finally arrived the other week.  It's only about a year and a half over-due, which I guess isn't really that bad for a crowd-funded product.  I was in love with this keyboard as soon as I saw the promotional video, so I've really been looking forward to getting my hands on one. 

If you haven't heard of the UHK, I recommend taking a look at it.  It's an extremely cool piece of hardware, even if you're like me and are neither a "gadget person" nor a keyboard aficionado.  It's a fully programmable mechanical keyboard that can control the mouse, splits down the middle, and has support for plug-in modules (not yet available).

Initial Impressions

Just taking the UHK out of the shipping box, it looks very nice.  I'm not sure what I was expecting, but I was pleasantly surprised.  The packaging was very slick and professional - far more so than conventional keyboards I've purchased.  It came with a nice "thank you" card and minimal instructions that just point to the URL for their online tutorial (which I highly recommend new users try out).

The very nice UHK and palm rest boxes.

I purchased both the keyboard and the palm rests.  At first glance, both look exactly as nice as they do on the marketing site.  The palm rests are a beautiful, smooth wood mounted on extremely solid metal plates.  The keyboard itself has a solid-feeling plastic case.  The seam where the halves separate is magnetic and has all metal contacts - no visible wires, circuit boards, or weird connectors.  The bottom has some thick no-skid nubs to stand on and metal mounting points for the palm rest.  Overall, it feels much sturdier and higher quality than any single-piece conventional keyboard I've used.

The opened UHK box.

The open UHK palm rest box.

Setup

Setting up the UHK was a bit of a mixed bag.  In the most basic setup, you can plug one end of the USB cable into the keyboard and the other end into your computer and it "just works" - no additional software or configuration required.  And that's great.  But if you have the palm rests and want to set up something more ergonomic, it's a different story.

My configuration of choice was to separate the keyboard and use a "tented" configuration with the palm rests, so that the center part of the keyboard is elevated.  This is similar to the setup of the Microsoft Ergomonic keyboard I had been using.  And once I got it set up, I found it to be very comfortable.

Comparison of the UHK with my old Microsoft Ergonomic keyboard.

The palm rest and tilting setup was the only aspect of the UHK that I'm not crazy about.  The setup process was not especially difficult, and there were clear instructions for all the standard palm rest configurations, but you can't do it without a screwdriver.  Installing the feet for tilting was the most painful part.  The feet are a thick plastic, which is good for durability, but makes it harder to bend them enough to fit into the mounting brackets.  And you can't really get the three screws for the mounting brackets in or out with the feet in them.  So it's not really feasible to quickly switch between configurations - at least not if you're using the "tented" setup.  I found that a little disappointing, but I can live with it.  The up side is that the final setup is surprisingly solid.  I had been worried that the palm rest would wobble or that the feet would have some give, but that's not the case at all.

The Agent

One of the really cool things about the UHK is that you can configure everything, but don't have to install any special software to use it.  The configuration is stored on the keyboard itself, so as soon as you plug it in, all your settings are already there.  You do, however, need a special program to do the configuration.

Screenshot of the UHK agent software.

The "agent" software itself is pretty intuitive.  It's cross-platform (looks like an Electron app - there's a web-based live demo here) and consists of a few settings panes and a visual representation of the keyboard that you can use to remap keys.  It allows you to remap literally any key on the keyboard, including modifier keys and layer-switching keys.  You can even map different functions for different modifier keys

The agent also has some support for running programs or doing other system functions, such as controlling volume.  Initially, these seemed to be a little dodgy, but that seems to have been resolved when I upgraded the keyboard's firmware.  That upgrade also gave me support for keyboard macros, which weren't yet implemented in the pre-installed firmware version.  I haven't actually had occasion to try out the macro feature yet, but it seems like a really cool idea.

Adapting to the UHK

The biggest challenge with the UHK is adapting to using it.  When you first look at the keyboard, the most remarkable thing is how small it is.  As one of my teammates put it, "Looks like you're missing some keys there."  And that's because, compared to most standard keyboards, it is missing a lot of keys.  Instead of having a lot of dedicated keys, the UHK has a concept of "layers".

My UHK setup.

If "layers" doesn't ring a bell, think of the "numlock" key on a conventional keyboard.  When you turn it on, the numeric keypad types numbers.  When you turn it off, the numeric keypad keys function as arrow keys, "delete", page up/down, etc.  That's two "layers" - the "number" layer and the "navigation" layer, if you will.  With the UHK, the entire keyboard is like that, but with four possible layers instead of two.  They are:

  1. The "base" layer, where you do normal typing.  This is the "no layers selected" layer.
  2. The "mod" layer, which gives you access to arrow keys, page up/down, home/end, F1 - F12 keys, and a bunch of other things that would have a dedicated key on a conventional keyboard.
  3. The "function" layer, which gives you access to the kind of things associated with the "function" key on a laptop - media control, volume control, etc.  Note that this is not for the function keys as in "F1", which was initially slightly confusing.  Instead, the number keys on this layer are pre-configured to change the key map (e.g. form QWERTY to Dvorak).
  4. The "mouse" layer, which is used to control the mouse, including left/right/middle clicking, movement, and scrolling.

I'll be honest - this setup takes a little getting used to.  You can't just open the box and immediately be productive using this.  However, it's really not as bad as I feared it might be.  The key arrangement is standard, so you can type text on the base layer with little to no adjustment.  It's just the layer switching that is an issue.

For me, the first week was spent mostly getting used to switching layers and getting the different key combinations into my muscle memory.  The second week was spent on customization and figuring out what did and didn't work for me.  For the most part, the default key mapping is pretty good, but there were a few things that didn't work for me.  For instance, I had to swap the left "Fn" and "Alt" keys because I was used to "Alt" being right next to the space bar and kept accidentally hitting the wrong key.  I also converted the right "Fn" key into a secondary "Mouse" key because, frankly, I never use the function layer and it seemed more useful to be able to control the mouse entirely with my right hand.  After the second week or so, I found that I pretty much had the hang of the layer switching.  My control started to become much faster and more natural.  After about a month, I found that when I used my laptop keyboard, I would instinctively reach for the non-existent "mod" key because it was more natural than moving my entire hand to find the arrow keys.\

Conclusion

It's been a little over a month and I LOVE my UHK.  If it wasn't so expensive, I'd consider buying a second one to use at home.  (Also I spend most of my time at home on a laptop, which doesn't lend itself to an external keyboard.)  It's a physically solid device with lots of features and it's just really comfortable to use.  I'm really enjoying the whole "not having to move off of the home row" thing.  It's not cheap, but I would definitely recommend it to any code who is willing to invest $300 or so in a keyboard.  I have no regrets and am actually looking forward to giving them more money when the modules come out.

Electronic voting

This XKCD comic pretty much sums up my reaction every time someone mentions electronic voting.  I usually explain it with an analogy involving Scotch tape and bubble gum, but same idea.

As a side-note, there's an interesting response to this here.  The criticism, essentially, is that the comic is comparing apples to oranges.  For aircraft and elevators we're mostly concerned about accidental failures, whereas for voting machines the issue is protecting against intentional attacks.  So planes and elevators are only considered "safe" because we're not counting "being blown up" as a valid scenario that they need to defend against.

That's a fair criticism, but that's not really the point. When I hear regular, non-techie people talking about "computerize voting", they're not interested in the electronic replacements for old mechanical voting machines.  They're interested in voting online, like you'd vote in poll on Facebook.  That's a very different problem than securing a computerized voting machine, and much harder to solve.

On the state of my Vim-as-IDE

Last year I posted about adopting Vim as my default editor.  I've been leveling up my Vim knowledge since then, so I thought I'd write a summary of the current state of my ad hoc Vim IDE configuration.

First, let's start with some sources of information and inspiration.  I found Ben McCormick's Learning Vim in 2014 series to be very useful.  The Vim as Language post in particular offers an eye-opening perspective on the beauty of using Vim.  I've also been working my way through Steve Losh's Learn Vimscript the Hard Way, which is a great source for information on customizing your Vim configuration.  And if you want a little inspiration to give Vim a try, here is Roy Osherove's "Vim for Victory" talk from GOTO 2013.

So how has my Vim adoption gone?  Pretty well, actually.  When I look back at my original post on looking for a new editor, it's pretty clear that a sufficiently customized Vim meets all my criteria.  However, to be fair, it did take a while to get it sufficiently customized.  The customization I've done wasn't actually that hard, but it took some time to figure out what I needed, what I could do, and then Google how to do it.  But paradoxically, that's one of the strengths of Vim - it's been around long enough that pretty much everything that you might want to do either has a plugin available or has been documented someplace on the web, so you rarely need to write any original code.

My Plugins

These days there are actually quite a few plugin managers available for Vim.  The nice thing about this is that they all support the same plugin format, i.e. GitHub repositories laid out in the standard ~/.vim directory format.  I'm currently using Plug because it provides an easy mechanism for selective or deferred plugin loading (in the case where you have a plugin that's not always needed and slows down Vim startup).

Here are some of the goodies my Plug plugin list currently contains:

  • scrooloose/nerdtree - A nice file explorer plugin that provides some enhancements to Vim's native file browser.  Pretty much a must-have for every Vim setup.
  • ctrlpvim/ctrlp.vim - A fuzzy file finder that works on buffers and other stuff too.  Just press ctrl+p and start typing the name you want.
  • jlanzarotta/bufexplorer - A handy plugin to list and switch between the current buffers.  Think of it as like the tab strip at the top of other editors, but easier to deal with from the keyboard.
  • tpope/vim-dispatch - A nice plugin for running external programs asynchronously.  By default, external command execution blocks the rest of the UI until the command is done.  This is fine sometimes, but not others.  Dispatch integrates with other plugins and provides a way to run things in the background and get their output back into Vim.
  • tpope/vim-surround - Provides a Vim movement that helps you manipulate surrounding entities.  Good for things like changing quotes, HTML tags, etc.
  • Chiel92/vim-autoformat - Provides an interface to various code formatters.  I use it as a replacement for the JSON beautifying feature that I loved so much in Komodo.
  • mileszs/ack.vim - A quick and easy integration of the ack! text search tool.  Like the built-in grep, but better.
  • joonty/vim-sauce - Sauce is a handy little plugin for managing multiple configuration files.  It's also useful for adding the concept of a "project" to Vim.  I use it to create project-specific configurations that handle all the customization that would be done in the project file of a regular IDE.
  • janko-m/vim-test - A unit test runner plugin that handles many different tools.
  • vim-airline/vim-airline - An enhanced status line that's both pretty and displays some useful information.
  • w0rp/ale - The Asynchronous Lint Engine, this offers syntax and style checking with inline error notifications, just like in a regular IDE.
  • majutsushi/tagbar - A tool for displaying the tags in the current file, similar to the structure browsers found in IDEs.

Needless to say, I also made a number of other customizations to my Vim configuration.  My full work-in-progress Vim configuration is in this repo if you're interested.  I do not hold this up as a great example of how to configure Vim, but it's working for me so far and, as previously noted, it actually wasn't all that much work.

The IDE Functionality

So what functionality do I have with this setup?  Well, it turns out I actually get most of what I previously had with Komodo.  Of course, I need to integrate with a few external packages for this, the key ones being Exuberant CTags, which indexes identifiers in the code, and ack for text search.  I also need various external formatters and linters, though the specific programs depend on what language I'm coding in.  Nothing fancy, though - they're pretty much all command-line executables that you can just drop someplace in your path.

So here's what I get for my trouble:

  • Insanely powerful key bindings.  I mean sersiously powerful - I don't think there's anything you can do in Vim that can't be bound to a keystroke.  And it's usually pretty easy.  Just the other week I defined a couple of ad hoc key bindings to help me add translation keys to a web template.  It's really a beautiful thing.
  • Inline syntax and style checking.  Using ALE in conjunction with the appropriate external linters, I get the same kind of inline checking I can get in PHPStorm.
  • Navigating identifiers.  Using Vim's ctag support, it's possible to navigate uses and definitions of a particular, for example, much like the code browsing abilities of PHPStorm.  Of course, it's not perfect because ctags lack knowledge of the context of the identifier, but it's not bad.  (And to be fair, I've seen the code navigation in PHPStorm and Komodo fall over on more than one occasion.)
  • Searching.  Between CtrlP and Ack, I have some nice facilities for searching for or within files.  Again, very similar to what I had in Komodo or PHPStorm.
  • Project management.  Between NERDTree and Sauce, I have some decent support for the concept of project.  They give me a nice file tree navigation panel and the ability to define project-specific configuration.

Conclusion

The short version is that this Vim experiment is going pretty well.  Granted, it is somewhat more work than figuring out a traditional IDE.  But on the other hand, it's not that bad and actually isn't as much work as I thought it would be.

In terms of functionality, I find that I haven't actually given up very much.  In fact, if you're talking about multi-language IDEs, I'm not even sure I've given up anything I care about.  It turns out that Vim is remarkably capable and the plugin ecosystem is very large and deep.

Would I recommend this to someone who's never touched Vim before?  Probably not.  But if you're familiar with Vim and interested in trying a new way of working, it might be worth a shot.  At worst, you'll improve your Vim skills and enjoy using a nice, fast editor that doesn't eat up half a gigabyte of RAM when it's just sitting there doing nothing.