[/caption]I've been working, both on the job and as a freelancer, right now my 9-5 is starting to enter that ever so exciting phase where one project enters maintenance mode and another project starts to spin up. The new project starting to spin up is to rewrite the CMS system that powers our various publication websites. Now for those of you not in the know, a CMS system is crazy complicated. I'm writing this current blogpost in a very specialized CMS system called WordPress, just for blogging. This software probably has had somewhere upwards of eleventy-billion man hours go into it (once you factor in themes and plugins and all that jazz).
Of course whenever you are faced with a complex problem that's been solved to death your best bet is to look around and see if anyone has done the heavy lifting for you already. So I started looking around, WordPress, Joomla, Drupal, all mature PHP CMS systems, none that I'm familiar with even a little. I started reading, out of the three I ended up liking Drupal the best, it seems well engineered, flexible, and very stable. I read and read and read about nodes and the way Drupal thinks about content and I started to think, ok maybe we can use this. The problem is that management is used to a level a flexibility that I won't have with Drupal for months, maybe even years. They want this flexibility from the beginning and that was making using Drupal seem less and less likely. Then some requirements filtered in and out and it looks like a custom solution is going to be best... c'est la vie.
Now this is in no way a slight against Drupal or Joomla or any other CMS system out there, I'm sure in the hands of Drupal Ninja or Joomla Gurus this would be easy as pie. For me though, building a solid system on my time tested and well worn toolbox of PHP, SQL, and Flourish will probably be the best way to deal with the constantly changing features and requirements that we need to support. Now I'm going to start building, start sketching out how things will work, how data will be thought of in the system, user models, documents, revisions, all kinds of fun stuff. It's a bit intimidating, but as a Software Developer this is really the stuff I live for, big gnarly problem, blank TextMate buffer, well known tools, get cracking.
The big problem I'm fighting right now is the dreaded Second-system effect. Watch out for this my dear friends, it's not a nice trap to get caught in. It works a little something like this.
- Build a system, this is the First-system, it works, celebrate and have a shot of Jack.
- That stupid "real world" comes in and messes up your data model, you add a little hack to compensate.
- Oh, it has to be able to do what? No, you said it didn't have to do that!! Fine, hack hack hack.
- Maybe some cleanup, probably some more hacking and bending.
- You've tried your best to keep First-system shiny and new, but the shine's off the apple.
- Crazy requirement, requires rewriting huge parts of the system, oh noes!
- Alright, now that we know so much more let's build Second-system.
Herein lies the danger. You get the go ahead to build Second-system, and you have all this experience from First-system, you really think you understand the problem domain a lot better, so far so good. Here's where things get hairy though, you start thinking about how you would build the system so that you don't run into the same problem First-system had, you start abstracting and abstracting, it gets more and more complicated, but you are now a master in this domain, you can handle it. Users who have been frustrated for months or years using First-system start throwing in their advice, things that have always bugged them, their nice-to-haves, and hell, you haven't built anything yet, chuck that automated-frog-boiler in your system plan.
Before you know it you've got a beautiful masterpiece, it handles everything, all the problems encountered in the First-system, all the nice-to-haves, everything, it is a theoretical thing of beauty. Now all you need is 6 maybe 7 years to build it.... oh shit. But there's something even more wrong about the Second-system, it has buried in it a dirty lie, that you understand the problem domain. You probably have a pretty good understanding of the problem domain, as it stands right now. Having already tackled this problem once does not make you clairvoyant, there will be changes to the system and your system will have to adapt to those new realities. This compounds the Second-system effect, because your Second-system is almost inevitably more complicated than the First-system and therefore harder to change and bend and make work in that pesky "real-world."
Right now I have to fight the urge to over-engineer, to outwit this problem, because I'm just not smart enough to solve this problem forever and ever into the future. So I'm going to focus on building a solid platform that let's me build out modules and snap them off when they aren't needed anymore. Something robust but simpler than the current system. Doing this will mean trading things off, it will mean making hard decisions upfront about architecture to meet our current needs but also look to the future, and this is what a good Software Developer is supposed to do. And once my current project is safely resting in maintenance mode, quietly humming along making my company money, I will begin building a Second-system and hopefully avoid Second-system effect.
