Tidbits on software development, technology, and other geeky stuff

Complexity Creep

“Controlling complexity is the essence of computer programming.” — Brian Kernighan

Complex Code The longer I have been developing software, the more I am convinced this quote by Kernighan is true. And when I am thinking about complexity in software, I like to think about the idea of “fighting” it rather than just controlling it. Why? I think it takes real initiative to keep things simple. The tendency for things to start out simple and to progressively get more unnecessarily complicated seems to be inevitable (unless you work against it). To be sure, some things are inherently complex to solve and demand more than a simplistic solution. The tendency toward unnecessary complexity, however, is what I like to call “complexity creep”.

It’s a Bad Thing

Why is complexity creep such a bad thing? Why is simplicity worth fighting for? When complexity increases, software becomes:

All of these things mean delivering and supporting your software is going to take longer and cost more money.

Think about it. Let’s take a simple example. Let’s say you need to write a C# program that logs the number of milliseconds it takes to send a request to a web application and get the entire response back. Easy right? Something like this…

static void Main() {
  WebClient client = new WebClient();

  DateTime startTime = DateTime.Now;
  string reply = client.DownloadString("http://www.YourAwesomeDomain.com");
  DateTime endTime = DateTime.Now;

  int millisecondElapsed = (int)((endTime - startTime).TotalMilliseconds);
  System.IO.File.WriteAllText(@"log.txt", string.Format("{0}: {1}",
    DateTime.Now.ToString(), millisecondElapsed.ToString()));
}

Now, let’s say this application goes through several revisions, keeping up with some changing requirements. And, along the way you pick up some nifty new design patterns and try to keep it testable and flexible. Now, it looks something like this:

const int WORK_QUEUE_BATCH_SIZE = 3;

static void Main() {
  IRequestProvider provider = GetRequestProvider();
  IWorkQueue queue = GetWorkQueue();
  ILogger logger = BuildLoggerFromConfig();

  WorkItem item = new WorkItem();
  item.Protocol = "http";
  item.Verb = "GET";
  item.Url = "http://www.YourAwesomeDomain.com";
  queue.AddWorkItem(item);

  QueueWorker worker = new QueueWorker(provider, queue, logger);

  bool moreWork = true;
  while (moreWork) {
    IWorkQueueStatus status = worker.DoWork(WORK_QUEUE_BATCH_SIZE);
    moreWork = status.CurrentQueueSize();
  }
}

...

IWorkQueueStatus DoWork(int batchSize) {
  for (int i = 1; i <= batchSize; i++) {
    WorkItem nextItem = m_queue.GetNextWorkItem();
    while (nextItem != null) {
      WorkItem currentItem = nextItem;

      m_logger.Debug(string.Format("About to PerformRequest on ID: {0}", currentItem.ID));
      currentItem.PerformRequest(provider);
      m_logger.Debug(string.Format("Done with PerformRequest on ID: {0}", currentItem.ID));

      nextItem = m_queue.GetNextWorkItem();
    }
  }
}

This thing can now handle more request types, logs more about what it is doing and supports operating in batches. But, at the end of the day this thing is supposed to log the number of milliseconds it takes to request a web page. Is this complexity necessary? I would strongly argue it is not. This is a hypothetical example but I have seen something simple turn into something overly complex time and time again.

Let’s take another (humorous) example that I think does a good job illustrating the point that something that starts simple can get crazy complex. The Fizz Buzz Test is algorithm prompt used to ensure prospective programmers can write basic code:

For numbers 1 through 100: if the number is divisible by 3 print Fizz; if the number is divisible by 5 print Buzz; if the number is divisible by 3 and 5 (15) print FizzBuzz; else, print the number

Some clever developer took this prompt to an extreme and came up with Fizz Buzz Enterprise Edition. It has 3,913 lines of code to implement an algorithm that should take more like 15 lines of code. It’s funny but demonstrates complexity creep. To be clear, complexity has nothing to do with how much code is used to implement a solution. An overly complex solution could be extremely verbose or it could be surprisingly terse.

How it Happens

Why do things naturally drift towards complexity? I think there are lots of reasons but the ones I have experienced first hand are:

It’s important to note that complexity creep can happen in many different areas of the software development process; not just when you are slinging out some new code or making a modification to a an existing component. It can happen during planning, such as when you are deciding the approach to take in solving a problem (or whether the problem actually needs to be solved). It can creep into the development process of your team, such as the steps developers must go through to get code developed and released. Some of it can be veiled in good intentions like “security”, “quality control”, “risk mitigation” or “best practice”; these are all good things but processes around them can become unnecessarily complex. It can show up anywhere!

Fight It

I hope you agree with me that complexity creep is a real thing and that it is a problem. Here are some strategies I’ve find that really work to fight the creep.

Don’t Be A Jerk

It is worth noting, you may have some opposition in your effort to avoid complexity creep. Your team might disagree with your approach and you might ruffle some feathers. However, I’ve learned that if you communicate with your team about why you are doing what you are doing they will usually be supportive (and if they are always fighting you on this then maybe you are not on the right team). It’s important to remind each other that you are on the same team and and the end game is shipping quality software in a timely manner. Sometimes your team will disagree and the consensus will not be in your favor. Pick your battles and don’t be a jerk.

Examples of Killing Complexity

The one-off report that was going to take 6 weeks

I was once on a team that received a “high priority” request to create a few custom reports for a single client. These reports couldn’t be generated within the existing reporting structure so the product owner and a developer starting going down the path of making major and fundamental changes to our reporting system to support these new one off custom reports for the single client. And these changes were going to make our reporting system significantly more complex. It was a large task and was going to take an absurd amount of time. During one of our stand-ups, I had an idea of how to avoid the complexity creep. Why not create a separate stand-alone reporting application that could generate the reports for the client and be done with it? It ended up taking 3 days to do this and the client was happy. Funny thing, that client went bankrupt 6 months later.

The application where only 15% of the code was in use

A few years back I was tasked with taking over a mission critical application that was old (10+ years) and “a mess”. It didn’t take long for me to realize that there was tons of code that wasn’t being used anymore and if I was going to reign this application in, I needed to identify and remove this code. I found that any time I needed to make a change, I had to wade through a huge amount of code, much of which was never even used. The goal was to reduce the size and complexity of the system. The problem was that no single person knew what was used and what was not. So, I implemented analytics to track usage of features so I could determine what was used and what was not used. Then, I started removing code. Not commenting it out, removing it. Source control had the history so I could get it back if I really needed it. There were a handful of times I removed a feature/page that was used twice a year that I had to restore but the end result was a much leaner application that was so much easier to maintain.

You know, when I was looking for a good “complexity” quote for this post I found a very large number of quotes on complexity. That’s why I have so many quotes in this post, because I liked a lot of them! I was actually surprised by all the quotes out there, especially how many of them were from famous people like da Vinci and Einstein. Anyway, I drew my own conclusion that it is not a coincidence; a lot of successful people have found that fighting complexity is a key to success. I’ll leave you with a few more good ones.

“Simplicity is the ultimate sophistication.” — Leonardo da Vinci

“Everything should be as simple as possible, but not simpler.” - Albert Einstein

“Complexity is your enemy. Any fool can make something complicated. It is hard to make something simple.” - Richard Branson

“Any intelligent fool can make things bigger and more complex… It takes a touch of genius — and a lot of courage to move in the opposite direction.” - Albert Einstein

Discuss on Twitter