How Hard Can It Be?

Footnote: This is a revised version of a post with exactly the same point and the same opening from 2009. Also, it is very closely related to the standards problem, here explained by xkcd.

Of all the reasons that people get sucked into rewriting software, the one that drives me craziest goes like this: "Well, there are a bunch of versions out there, yes, but they all suck. So we decided to write our own. How hard can it be?"

Well, I dunno, it can be hard enough that a bunch of other people tried it, and their versions all suck. It's true, the world is full of mediocre programmers, but take a look at those versions. 

One commercial version might have been written by 2 bozos in a corner and then shamelessly exploited by a money-grabbing company trying to fill a vacuum. More than one? Longstanding products? Somebody who can do more than grunt was involved at some point. 

One open source version might have been written by a crackpot and released in a forlorn attempt at self-aggrandizement. More than 5? Some of which are longstanding projects with multiple contributors? Once again, a reasonable amount of brainpower has been unleashed here. 

If the result sucks, the problem is hard. And unless you have a brilliant insight they didn't have, your version is also going to suck. (If you do have such an insight, you do not ask "How hard can it be?" You ask "How come nobody else ever tried [fill in the blank]?" and let me warn you that this is not a rhetorical question, because sometimes there are very good reasons.)

Why do some problems -- time card systems, bug tracking systems, trouble ticket systems, monitoring software, to name some of my more frequent nemeses -- suck people into believing they cannot be hard when manifestly they are? In general, some combination of the following features:

1. Every environment has some customization that is, in itself, relatively trivial, but is sufficiently environment-specific that it is hard to implement in a program meant for other environments. When you contemplate the programming difficult, you calculate the difficulty of adding that customization, not the problem as a whole, forgetting all the difficult problems that the existing system has already solved.
2. Small and specific versions of the problem are easy to solve; scaling and customization are the hard parts. You can solve your organization's whole current problem easily and will not discover the lurking traps until later (the timecard programs works great until employees start moving between time zones, the bug tracking system works fine until a million bugs, the trouble ticket system works fine until the second group of people need to use it, the monitoring system deals with the network devices fine but stumbles when you need to manage systems).
3. The underlying parts of the problem are relatively tractable, but putting a user interface onto it that  isn't disgusting and hard to use for large portions of the user base is difficult. (Do user interface designer design beautiful interfaces for things that then nobody can program the backend to adequately? Probably.) In this case, even if your program works right, the users will think it sucks. See, for instance, every time card system I have ever been forced to suffer through, most of which seemed quite competent at dealing with my more normal time card needs, after I had learned to perform the arcane rituals needed to actually enter the data. Nevertheless, once your program has made me cry, I do not care that it was just a few hundred UI issues, not a fundamental algorithmic flaw. 

If you actually have a novel insight into one of these problems, more power to you, please do build your new system (although I have to warn you that at best a significant minority of users will still hate it). If it just seems logical to you that there must be a better way, please do not unleash your version onto the universe until you figure out exactly what that better way is. 

And please resist the temptation to believe that it will be cheaper to build yours than to try to beat somebody else's into submission until it is bearable. It may be more personally rewarding, and that may be a perfectly good reason to do it. But it will almost certainly be just as expensive.