Don’t Clog The Production Pipeline

Here’s a situation that you don’t want your team to be in:

  1. A low or medium priority (but still significant) bug gets reported in your production environment.
  2. A fix for the bug is checked into the branch of source code that represents production.
  3. The fix is not tracked well, and it languishes in a test environment for a while.
  4. A really hot production bug (undoubtedly written by someone else, not you!) gets reported and must be fixed and placed into production right away.  The new bug is in the same general area of the software as the first bug.



The problem, if you haven’t seen it, is that you can’t easily release the critical fix (bug #2) until you know that the less-critical fix (bug #1) is good.  Has it been tested yet?  Is the problem fixed?  Have regression tests shown that it didn’t break anything?  If not, then what do you do?

The bug fix that sat around in the pipeline to production has become like sludge, potentially blocking the way for something more important.  Either bug #2 has to wait until bug #1 is ready for release, or bug #1 has to be backed out.  Or worse yet, perhaps in the rush no one remembers bug #1, so it gets released in an unknown state—pushing the sludge through the pipeline and right into the machinery.  You may find out too late what you’ve broken.

Keep it moving

Make sure that someone in your organization is monitoring the bug list closely.  Make sure they understand the importance of keeping things moving.  If a fix can’t be put into release fairly soon, once checked into the code branch it should still be worked on until it is fully ready for release.  If there are not resources to get it production-ready quickly, it should not be checked into the production code branch.  Maybe it should be assigned to the next general release instead.

But don’t assume that someone else is responsible for this.  I currently work in a pretty small shop.  Besides the developers we have a handful of Quality Assurance people.  They’re an excellent group, and I mistakenly thought that we wouldn’t have this problem.  But we did, despite good people and good tracking software.

So, as the “keeper” of the source code repository, I instituted a very simple policy that has made a contribution toward keeping the pipeline moving:  I lock down permissions on a branch once it gets close to production, and I don’t open it up for anyone until a member of the Quality Assurance group makes the request (usually just verbally).  That way I know that the developer and a member of QA are talking with one another, and QA understands that when the branch is opened, they’re responsible for staying on top of the fix until it is ready to go out the door.

Such a policy is not for everyone, and wouldn’t scale well to a big team.  But it adds one more opportunity to ensure the we keep the pipeline clear.  The point is to make your own contribution.  What are you doing?


Are You Writing Fossilized T-SQL Code Every Day?

There’s a long list of SQL Server features that are deprecated; i.e., will not appear in some future version—such as perhaps the next version.  See for this list.

Ignoring the list is one of those things that can lock an organization into staying with an old, and eventually unsupported, version of SQL Server—or into retiring a system earlier than they would like, or into spending precious resources rewriting code.

Many items on the list will hit someone, forcing them to revise existing code.  But there are a few features that are so widely used that when they are removed the impact will be enormous.  I think Microsoft will be pressured not to remove them, but in order for the product to move forward sometimes it needs to be done. 

The Short List

Here are the ones from the list that I think will cause the most wide-spread problems:

  • Use of the SQL 2000 and earlier system tables—sysobjects, sysindexes, etc.  Microsoft always warned against using these but I’ve never worked in an organization that didn’t use them somewhere in their code.
  • And now the big one:  not ending T-SQL statements with a semicolon.  Think of all the lines of SQL code you have written, and that your colleagues have written, that are still in use.  What percentage have a semicolon at the end of each statement?

Coding habits die slowly.  I’ve found that even when presented with this warning from Microsoft, I have had great difficulty in getting programming groups to even seriously consider adding a semicolon at the end of each statement.  It’s not hard; it could probably become a habit within a few weeks for people who write a lot of T-SQL.  (That’s about how long it took for me to start typing it automatically).  And I think it actually makes the code easier to read.  But most people don’t want to fight the inertia.

How about you?