LinLog

This is part two of my evaluation of the Personal Software Process.  This time I'll be talking about the actual process of learning the PSP.  In case you haven't figured it out yet, it's not as simple as just reading the book.

Learning the PSP

The PSP is not the easiest thing to learn on your own.  It's a very different mindset than learning a new programming language or framework.  It's more of a "meta" thing.  It's about understanding your process: focusing not on what you're doing when you work, but rather how you're doing it.  It's also significantly less cut-and-dried than purely technical topics - not because the material is unclear, but simply because what constitutes the "best" process is inherently relative.

Given that, I suspect the best way to learn the PSP is though the SEI's two-part training course.  However, I did not do that.  Why not?  Because that two-part course takes two weeks and costs $6000.  If you can get your employer to give you the time and shell out the fee, then that would probably be great.  But in my case, that just wasn't gonna happen and the SAF (spouse acceptance factor) on that was too low to be viable.

Instead, I went the "teach yourself" route using the self-study materials from the SEI's TSP/PSP page.  You have to fill out a form with your contact information, but it's otherwise free.  These are essentially the materials from the SEI course - lecture slides, assignment kits, and other supplementary information.  It's designed to go along with the book, PSP: A Self-Improvement Process for Software Engineers, which serves as the basis for the course.  Thus my learning process was simply to read through the book and do the exercises as I went.

(As a side-note, remember that this is basically a college text-book, which means that it's not cheap.  Even the Kindle version is over $40.  I recommend just getting a used hardcover copy through Amazon.  Good quality ones can be had for around $25.)

Fair warning: the PSP course requires a meaningful investment of time.  There are a total of ten exercises - eight programming projects and two written reports (yes, I did those too, even though nobody else read them).  Apparently the course is designed around each day being half lecture, half lab, so you can count on the exercises taking in the area of four hours a piece, possibly much more.  So right there you're looking at a full work-week worth of exercises in addition to the time spent reading the book.

Personally, I spent a grand total of 55 hours on the exercises: 40 on the programming ones and 15 on the two reports (for the final report I analyzed not only my PSP project data, but data for some work projects I had done using the PSP).  While the earlier exercises were fairly straight-forward, I went catastrophically over my estimates on a couple of the later ones.  This was largely due partly to my misunderstanding of the assignment, and partly to confusion regarding parts of the process, both of which could easily have been averted if I'd had access to an instructor to answer questions.

Tools

 As I mentioned in the last post, you'll almost certainly want a support tool, even when you're just learning the PSP.  Again, there's a lot of information to track, and trying to do it on spreadsheets or (God forbid) paper is going to be very tedious.  Halfway decent tool support makes it manageable.

I ended up using Process Dashboard, because that seems to be the main (only?) open-source option. In fact, I don't think I even came across any other free options in my searches.  I understand other tools exist, but apparently they're not public, no longer supported, or just plain unpopular. 

One of the nice things that Process Dashboard offers is the ability to import canned scripts based on the PSP levels in the book.  To use that, you have to register with the SEI, just like when you download the PSP materials, which is annoying but not a big deal.  (The author got permission from the SEI to use their copyrighted PSP material and apparently that was their price.)  This is really handy for the learning process because it puts all the scripts right at your fingertips and handles all of the calculations for you at each of the different levels.

In terms of capabilities, Process Dashboard is actually pretty powerful.  The UI is extremely minimal - essentially just a status bar with some buttons to access scripts, log defects, pause the timer, and change phases.  Much of the interesting stuff happens in a web browser.  Process Dashboard runs Jetty, which means that it includes a number of web-based forms and reports that do most of the heavy lifting.  This includes generating estimates, entering size data, and displaying stock and ad hoc reports.

Process Dashboard has fairly extensive customization support, which is good, because one of the basic premises of the PSP is that you're going to need to customize it.  You can customize all the process scripts and web pages, the project plan summary report, the built-in line counter settings, the defect categories, etc.  And that's all great.  The one down side is that the configuration is usually done by editing XML files rather than using a graphical tool. 

Since this is an open-source developer tool, I guess that's sort of fine.  And at least the documentation is good.  But it's important to realize that you will need to spend some time reading the documentation.  It's a powerful tool and probably has just about everything you need, but it's not always easy to use.  On the up side, it's the sort of thing that you can do once (or occasionally) and not have to worry about again.  Just don't expect that you'll be able to fire up Process Dashboard from scratch and have all the grunt work just done for you.  There's still some learning curve and some work to do.  But if you end up using the PSP, it's worth it.

As I mentioned in a couple of previous posts, I've been studying the PSP.  As promised, this is my first report on it.  After several failed attempts to write a single summary of the process, I've decided to divide my assessment up into a series of posts.  This first entry will be a basic overview of what the PSP actually is and how it's intended to work.

What is the PSP?

PSP stands for Personal Software Process.  It was developed by Watts Humphrey of the Software Engineering Institute (SEI) at Carnegie-Mellon University (which you may know as the home of CERT). Humphrey is also the guy who came up with the Capability Maturity Model, which is the CMM in CMMI.  His intent was to take the industrial-strength process methodologies he worked on for the CMM and scale them down to something that could be useful to an individual developer.  The result is the PSP.

Despite the name, at its core the PSP isn't actually a development process itself.  Rather, it is a process improvement process.  To put it another way, the goal is not to tell you how to develop software, but rather to give you the intellectual tools to figure out the development process that works best for you and help you improve that process to make yourself more effective.

Now, to be clear, Humphrey actually does describe a particular development process in his PSP books.  And yes, that process is kind of "waterfally" and extremely heavy.  And if you look for PSP information online, you'll no doubt find a lot of people who take issue with that process.  And they're right to.

But try not to get hung up on that.

As Humphrey puts it, the process he describes is his process.  That doesn't mean it should be your process.  The process he describes is for pedagogical purposes as much as anything else.  It is intended to demonstrate the kinds of methods that apply to large-scale industrial systems, so that you have those tools in your toolbox if you need them.  If this process doesn't really work for you (and it almost certainly won't), then you should modify it or replace it with something that will work for you.

This is something that bears dwelling on.  Although pretty much all of the PSP focuses on Humphrey's process, he notes several times that it is important to pick a process that's sustainable for you.  To put it another way, it's important to be realistic.  Sure, his process might be objectively "better" by some measure than whatever you're doing, but that doesn't matter if you aren't going to follow his process.  Maybe his process conflicts with your company's process; maybe it's not effective for the kind of project you usually do; maybe you just plain don't have the will power to stick to it.  It doesn't matter.  The process you define needs to be in line with what you actually do, otherwise it's not going to help you.  So while you can learn from Humphrey's process, don't take it as gospel and don't think "this is what I need to be doing."

For me, the real take-away of studying the PSP is to be mindful of your development process and constantly try to improve it.  The goal is simple: quality and consistency.  Monitor your process and tune it such that you can consistently build a quality product in a predictable time-frame.  Ground yourself in reality and be guided by the data, not the latest fads.

The PSP Approach

The PSP is all about data-driven process changes.  As you work, you collect data on what you produce. The general idea is that you analyze that data and make changes to your process based on that analysis.  You then repeat this process, evaluating the results of your changes and making further tweaks as necessary.  So it's basically a plan-do-check-act cycle.  This is the same sort of thing that agile methods advocate (e.g. Scrum retrospectives).

Project Structure

The PSP structures a task or project into phases.  The exact phases will obviously depend on your particular process.  The "standard" (i.e. from Humphrey's process) phases are Planning, Development, and Postmortem.  In the Planning phase, you define what is to be built and construct estimates of the effort involved.  The Development phase is where you actually construct the software.  In the standard PSP process, this is divided into a number of sub-phases, depending on which level of PSP you're using.  Lastly, the Postmortem phase is where you record your final process measurements and document any problems and potential process improvements that came up in the course of the project.

The PSP levels I mentioned above are essentially maturity phases.  In the process of learning the PSP, you go through six of them: PSP0, PSP0.1, PSP1, PSP1.1, PSP2, and PSP2.1.  You start with PSP0, which is basically just "whatever you do now, but with measurements" and progress up to PSP2.1, which includes five sub-phases: Design (with standard design templates), Design Review, Code, Code Review, and Test.  I'll analyze this structure and the contents of the phases in a later post. 

Data Collection

Data collection in the PSP happens at a very fine-grained level.  The idea is that you do light-weight data capture as you work in order to minimize inaccuracies.  Data is collected on three axes: how much time is spent in each phase of the project, size of the product, and what defects were found.  In the standard process, you would track how many minutes you spend in each phase or sub-phase, how many lines of code you add or modify, and the number and type of bugs you find in in each phase.

The general idea behind this is simply that the more detailed the data, the more opportunities you have to use it later on.  The PSP suggests a number of ways to use this data to drive and measure process change.  Two of the most visible are the use of historical size and time data to drive the estimation process and the use of defect phase, type, and time data to measure the efficiency of defect removal activities such as code review and testing.

Interestingly, when Humphrey started developing and teaching the PSP, students tracked this stuff on paper.  He later introduced spreadsheets to track it, which is better, but still a little clumsy.  These days, there are PSP support tools you can use.  The main open-source one seems to be Process Dashboard, which I used from day one.  It will help you track PSP data and do most of the calculations you need for data analysis.  Frankly, I can't imagine trying to track this by hand or with Excel spreadsheets - it just seems like it would be painfully tedious.  But with decent tool support its really not that bad.

Quality Management

One of the interesting things to note about the standard PSP process is that it explicitly takes a position on quality management.  This is something you don't tend to hear a lot about - in fact, it might not even have occurred to you that quality is something you can manage.

The position Humphrey takes is simple: the right way is always the fastest way.  And the right way is to get things right the first time to the maximum extent feasible.  That means finding defects as early as possible and taking steps to stop them from being injected in the first place.

I'll get into the details of this in a later post.  I mention it now simply Humphrey devotes a lot of ink to it.  In fact, much of the new material in the higher PSP levels is devoted to quality control.  Quality management is one of the big uses of the defect data that you track as part of the PSP, so it's important to realize at the outset that this is a core part of the process.

Should I Care?

I think you should.  You may not want to actually use the PSP, but it's still an interesting thing to learn about.  If you're like me, you probably never thought about how you build software in terms of hard data, so it's interesting to see how it can be done.  It's a very different and empowering way of understanding how you work - it makes programming feel more like a science and less like a black art.  So even if you don't want to go all the way and actually try to use the PSP, it's enlightening to read about the various statistical analysis techniques and quality improvement recommendations.  If nothing else, it broadens your perspective and gives you more intellectual tools to draw on.

In part two, I'll talk a little about what I did to learn about the PSP, including the resources and tools that I used.