usable in any place a human can be used



[caption id="attachment_515" align="alignright" width="293" caption="simplicity is sometimes surprisingly complex"]it's so simple[/caption]

I love abstractions, most people do, they allow us to take complex things and wrap them in simple things and everyone is happy. The thing to realize is that there are degrees of complexity and different goals of abstractions.

Programming is one of those things where more often than not our abstractions are really helpful. The problem becomes one of perceived complexity vs. actual complexity. A good example is network protocols.

Network protocols are these complex beasts that let us talk across networks with some degree of certainty. Its really quite the marvel of engineering prowess, man has taken the various blinkenlights and cables and a big ole' pile of Cisco and now you can watch a video of a cat playing keyboard whenever you like. Network protocols were designed to simplify the process of two programs on physically distant machines sharing information.

When you first try to learn a complex abstraction it can seem daunting, UDP, TCP/IP, sockets and fifos and netcats oh my! Sometimes it can seem overwhelming, but take a step back and realize that no matter how complicated a piece of software is, its just moving around bytes.

Perl has a philosophy that at the end of the day, everything is strings. That's why its the glue that holds together the interwebs, because it's just a really good string manipulation language. At the end of the day that's all the internet is, strings floating around, normally between angled brackets.

Like most of the things I write I'm almost certain I had a point... oh there it is. When I started writing prosper it seemed like an impossible mountain to climb. I wanted to write a library that would bridge the gaps between all supported database backends, to have it be a sort of universal data access object.

[caption id="attachment_516" align="alignleft" width="354" caption="This was much easier to think about than the actual problem"]prosper simplified[/caption]

I had a few false starts on my project until I took a step back, I simplified my problem domain, and created an architecture that worked. Instead of focusing on the end result of prosper I thought about it in a more abstract way, prosper was a mapper from function calls to string snippets. This isn't actually what prosper does today, prosper is in fact much more complex now than I would have ever imagined when I first started out.

The obstacle that stood in my way the most when starting out on prosper was being overwhelmed and unsure. The problem domain I was trying to tackle was too big, even when I tried solving it i ended up painting myself into a corner by focusing too much on how one specific backend acted. I needed a simpler problem that was more easily solvable, but one that would in the end help me reach my actual goal.

As much as I wish I could take credit for this idea, it's not mine, it's something called Minimum Viable Product. I wanted something that at its core would produce cross-platform SQL, so the first iteration was to make something that did exactly that. If you go on GitHub and pull one of the earlier commits you will see that that's exactly what I built. Over time I have added the ability to execute those strings, connection and transaction management, standardized results, prepared statement support, and much more.

I couldn't have accomplished any of this though if I hadn't taken a step back from the problem domain and looked at the simpler problem at the core of what I was trying to do. I stumbled in the beginning, I've refactored again and again, over the break my good friend Ian took a look at the library and found a whole slew of problems, and I thank him for it. Prosper isn't perfect yet, but it is moving toward being something really worthwhile, and to begin moving it required that I have something, anything, to start with.

So here is my suggestion for you for the new year. If you have some project that you've wanted to do but couldn't figure out how to start, step back from the problem domain and try to find the core of the problem and solve it with the simplest thing that could possibly work. Don't worry about the first iteration, or the first 20 iterations, you have to just get started. Once it is started you can iterate and refactor and maybe throw everything away and start again. No matter what the outcome of the first iteration it will be worth it, the best case scenario is that you can use the first iteration as a core moving forward, the worst case is that you learn a bunch about the problem domain and have to try again.

It's the start of a new year, resolve to start your dream project, because that's the hardest part. Once you get rolling momentum has a fantastic way of carrying you through.


  1. [...] software often requires a high degree of focus on the problem at hand, I wrote the other day about one approach to problem solving, simplifying the problem. This strategy can be coupled with [...]

  2. [...] talked about this before in my post about simplifying things. This simple question, “who cares?” (you can be more diplomatic about it if you like) [...]