climbing a mountain

I've been done with my last big project for about a month now, waiting on the customer to make up their minds on some enhancements. It's one of those positions that seem enviable until you're a month deep in it, sitting around trying to find something to do. So I've filled my work time tackling whatever tasks have come by, writing up documentation, working on and completing two smaller projects, and reading up on technology. It passes the days but sometimes can be a bit of a bore, and the real challenge is trying to figure out how not to waste my company's time, trying to make value without a project can be hard.

[caption id="attachment_67" align="alignright" width="300" caption="Stratego... similar to strategic"][/caption]

Then my boss told me that I would be working on a new project. Huzzah! I'm leaving behind .Net back to the land of Java, thus keeping my streak alive of never working in the same language 2 projects in a row. The project position has been described to me as a strategic technical challenge. Let's break that down real quick.

Strategic - That's a fun word, what does it mean, dictionary.com defines it (after a great deal of military definitions) as

Highly important to an intended objective

So this is an important position, one which is necessary for intended objectives. From what I've gathered of the work I'll be doing, this is an apt definition

Technical - Nothing touchy feely here, no user interaction, no pretty style sheets, no fancy jQuery magic, just straight CS4xx type algorithms and number crunching. I'm have a feeling I'm going to have to dust off that Math degree sitting in the corner.

Challenge - This is the word I love, the thing I live for, a good challenge. I've been working on enough quick one week, one off, hit-it and quit-it projects as of late, I've missed the thrill of a good challenge.

I don't know how much I'm at liberty to share about the project which is why this post so far has sounded a little bit like a riddle. The one point of information I'll share is that the task I'm going to be taking on was sent out for quotes and the number that came back was... 10,000+ man-hours. You aren't misreading, and I triple checked that I wasn't mistyping. In case you aren't familiar with what 10,000 man-hours means I'll take a quick detour to explain it.

[caption id="attachment_70" align="alignleft" width="196" caption="The Mythical Man-Month"][/caption]

In one day, one programmer puts in 8 hours of work, that is 8 man-hours. One week is 40 man-hours. Assuming you get two weeks vacation, one year of development is 2,000 man-hours. That means that 10,000 man-hours of work is 5 straight years of development, half a decade of work. Of course, throwing some more resources at it you could hit 10000 man-hours with a 5 man team in just 1 year, more or less. The problem though is that this reduction in time is non-linear, you can't just throw 10,000 developers at the problem for an hour, this is pretty well discussed in The Mythical Man-Month.

In less technical terms that means that there is approximately a metric shit ton of work to do. And not grunt work mind you, complex problem solving work. This is a mountain of work. There are a few things that you can do when a mountain blocks your path.

1. You can turn around and go home


2. You can rent a helicopter


3. You can ready your climbing gear

Going home isn't an option in this case, the task can't be overlooked and other things can't happen until we are up and over the mountain. The outside quote was a helicopter, it costs some cash but is easy and relatively painless for you. Well we can't wait around a year to get this thing done. So that only leaves us with one option, so I'm gathering up some rope and pitons. I'm dusting off the part of my brain that remembers how Java works, remembering that strings can only be compared with .equals(), and fondly remembering the joy that is Eclipse after the ghetto of Visual Studio (holy war can begin now, be sure to insult my mother).

I'm preparing myself for what will be the biggest technical challenge of my career, and it fills me with happiness. It reminds me that I love this profession, that I'm stupid lucky to get paid to do it, and that I'm going to go in there and climb that mountain. From the little information I have, I'm sure the climb is going to be arduous, but I'm determined to plant my flag at the top, and I know that that will be amazing.

I want to challenge you to look around for your own personal mountains, its easy to avoid them, to walk away, go home, or rent a copter. You should instead commit yourself to the climb, because when you are at the top of a mountain, all the other problems are pretty small.

duck typing works

[caption id="attachment_62" align="alignleft" width="226" caption="Design Patterns in Ruby"][/caption]

I recently had the joy of picking up the book Design Patterns in Ruby which is a great book, made my Christmas wish list. If you have been living on the moon or are just one of those weird people that haven't heard of ruby yet, it is a dynamically typed interpreted language made famous by the web framework, Ruby on Rails.

Do you see what I just did right there? I gave you some background information, information that you more than likely don't need, you are reading my blog, you know what ruby is. Russ Olsen (author of Design Patterns in Ruby) does the same thing with duck typing. What is duck typing you ask? (again you probably didn't, you know what it is, or you are familiar with this thing called Google). Well it comes from the term, "If it walks like a duck, quacks like a duck, then its a duck" and according to the authority on all human knowledge it can be defined as the following

In computer programming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface.

One of the properties this gets you is the ability to pass any kind of variable anywhere and as long as it acts like the object the function is expecting it will work, no matter what class it is. In practice this allows for some great code reuse and as described in the book the evaporation of some class GoF design patterns. Olsen goes into a bit of depth about duck typing and the classic fear of someone coming from a strongly typed language.

So there is no compile time or static checks, what stops you from passing a database connection into a function that accepts an int or calling the give_big_fat_bonus method on a lowly worker instance instead of an executive instance, or of course the ceo instance which is a derived class of executive.
Anonymous Object Oriented Programmer

The answer is a resounding, nothing! (Dovetails nicely with my last post) Nothing stops you from doing this, go ahead see what happens. Depends on the language, in ruby you get a NoMethodError, and you get something like that in most duck typed languages.

The question then is how do people write any code that works, the answer is by not being full-on stupid. Ruby, and other duck typed languages, have taken to heart the sentiment of my post i can't stop you from being stupid (The shameless self promotion is getting a little rampant at this point). The language can't stop you from treating an int like a database connection, and really why should it. What in the hell are you doing mixing up an int and a database connection, you deserve to get punched in the face, a NoMethodError is the least of your worries.

But, accidents happen, so your db variable holding the database connection and your dv variable holding the direct value modifier amount are dangerously close on the keyboard. Are you going to just pray that you are 100% correct in your typing all the time, who do you think you are, me?! (I'll wait patiently as grammar-nazis and typoistas tear apart this post). This is why unit testing is so huge in duck typed languages, if you are using sweet delicious lisp then repl is your friend, ruby also has repl (irb) but its not nearly as integral to the development process as repl is to lisp. Your tests let you know that you aren't doing something stupid, and performs all the static and dynamic checking one could want.

So go out and give a duck typed language a shot, pretty soon you will enjoy the added speed, and you will realize that the static syntax checking is not necessary if you know what the hell you are doing.

the power of nothing

Sometimes you get a bright idea, or you are given a difficult task, and you want to start working on it. It's intoxicatingly easy to dream up feature upon feature for a new project, or build a giant architecture to solve a problem, all of these things look powerful. The other problem that we run into is that with software anything is possible with enough code, and the people that write code like to work on interesting complex problems. This leads to conversations like this.

• First Dev: I want this new project to seamlessly integrate with eBay, digg, and amazon


• Second Dev: Those seem arbitrary, maybe we should define some sort of data import protocol


• First Dev: Yea, then we just need a scraper piece that uses the adapter pattern to translate various websites into the standard import protocol and feed the project


• Second Dev: I'll start defining a dtd and you can start writing up some xslt for eBay and digg and amazon as a proof-of-concept


• First Dev: Great, then we just need to get the exchange support and the poor man's cron jobs running, to send out the periodic updates


• Second Dev: Should we integrate with Twitter?


• First Dev: I don't see why not


• Third Dev (interrupting): Wait, what are we developing?


• First Dev: Todo list


• Third Dev: Maybe we should get it to store and delete todo list items first


• Second Dev: That's the easy part, the multitouch integration is going to be the hard part


• Third Dev: ...

[caption id="attachment_57" align="alignright" width="300" caption="the dream"][/caption]

Before we get the product to do something simple, something any moron could code up, we want to start slapping on features and bells and whistles and whizbangs. This is all fun, because whizbangs are fun things to think about and code and its all well and good until you realize you have coded up a great feed reader, but one hell of a lousy todo list. Focusing on least viable product in the beginning gives you a concrete base to start applying whizbangs, based off of real world feedback and grounded in reality. Incrementally building up value off of an existing product is easier and makes more sense than building a giant all-in-one application that can do everything but butter your toast that no one wants to use.

So this leads me to the best way forward in my estimation and experience, nothing. Nothing is incredibly powerful, tremendously liberating, and allows you to get something up and running in least viability mode or to solve a complex issue.

I can hear it already, what the hell are you talking about you mentally unstable person?! Let's look at some tools that exemplify nothing and how you can go from nothing to something quickly and incrementally, but only if you start with nothing.

Mock-ups. What is a mock-up, nothing really, just a fancy picture of what you want an interface to look like. There are many great tools for making mock-ups, a new one that just hit beta is mockingbird, a more seasoned veteran in this game is the excellent Balsamiq. These shouldn't help, we all know what a todo-list looks like, why would mocking one up help us? Because, even though we are capable of abstract thinking it is much easier if we have a concrete model to wrap our heads around. Like I said in tool roundup

[talking about a sweet visualization I made] ...but it is a much better reference when working with this particular part of the system than looking at the database table and trying to assemble parts of it in my brain as the situation demands.

If we don't have a concrete model we force our brains to construct one on the fly, taking us out of our work and into a conceptualization, having a concrete model frees up precious processor cycles in our brain. A mock-up is also great because it allows us to take nothing, a simple functionless mockup, and quickly turn it into an actual interface. What does that button do? Nothing! What about that one? Nothing!!! Then we can slowly but surely add functionality to our interface until we have a functional least viable product. The way we got there was by understanding that we start with nothing and work our way gradually to something. Sure we could have sat down at the keyboard and pounded away code for a day or two and then run the thing, but the faster we can get it on the screen and start iterating over it, the better.

The second way that nothing can be useful is when tackling a huge problem, lets look at the contrived feed reader problem from up above. How in the world do we go about solving that?! Well we employ are good friend nothing, and we build from there. What's would the simplest possible solution be?

[php]
$feed = FeedReader::read("http://www.digg.com");
[/php]

We can throw that in our program, now we need to deal with the FeedReader class

[php]
class FeedReader {
static function read($url) {
return '';
}
}
[/php]

Nothing to the rescue, we just defined our super simple API, but we aren't quite to functional yet, but we have a foothold now to iterate from.

[php]
class FeedReader {
static $readers = array('http://www.ebay.com' => new EbayReader(),
'http://www.digg.com' => new DiggReader(),
'http://www.amazon.com' => new AmazonReader));

static function read($url) {
$reader = self::get_reader($url);
if($reader) {
return $reader->process();
}
}

static function get_reader($url) {
return self::$readers[$url];
}
}
[/php]

Wow look we are almost all the way to functionality, amazing!!! Thank you nothing (that's what EbayReader, DiggReader, and AmazonReader do). Now its the simple task of implementing those, which each by themselves is a somewhat complicated task and outside the scope of this blog post.

This sort of top down design though can be very helpful, you need to solve a problem, just pretend that you have an API that already solves it and start calling it, stub out those functions (stub is a fancy computer science term for "make it do nothing"), and watch the power of nothing first hand. Then you fill in the blanks, allowing yourself to have more nothing functions, you successfully break up your huge monster task into simple solvable tasks.

When we combine the first technique with the second we can get truly awesome results, we normally call this Rapid Application Development (or the much cooler 1980's name, RAD). RAD can get us to something faster than almost anything, using the power of nothing.

i can't stop you from being stupid

I've been working on a skunkworks side project that is nearing release, I'm down to the last 5%, which is the most difficult part. Suffice it to say that this project is a library meant to be used by other developers (and myself) to develop neat and nifty things. One of the most complicated things is trying to define an API that makes doing the right thing easy and doing the wrong thing really hard or impossible. The problem is that the as I pressed on I realized that almost half my code was checking for some stupid thing and protecting against it, attempting to build up state machines and syntax trees in the hope of reporting to you that you had done something stupid, and then it dawned on me, I can't stop you from being stupid.

The other thing that dawned on me is that I shouldn't try to, there are already built in errors that will point out what you did wrong, I don't need to add another layer on top. So I came up with some simple guidelines, applied it to my project, and watched unnecessary code melt away.

• Pick sane defaults


• Assume the developer knows what they are doing


• Make it easy to do the right things


• Make it hard to do the wrong things

The beautiful part of all of it is that my code became easier to read and understand, which means when there is a problem and you need to drop into my code (since it is open source), you will be able to understand what's going on. Let's take a look at what I mean with a contrived example, an HTML building library with 4 functions for clarity.

[php]
class HTML {
static function open_html() {
echo "<html>";
}

static function open_body() {
echo "<body>";
}

static function close_body() {
echo "</body>";
}

static function close_html() {
echo "</html>";
}
}
[/php]

The way I was programming did a bunch of handholding and sanity checks, which were nice but unnecessary. The class became bloated with so much state and sanity checking that it was becoming unwieldy, let's take a look at an overly protective incarnation of the above code.

[php]
class HTML {
private static $html_open = false;
private static $body_open = false;

static function open_html() {
if(self::$html_open) {
self::handle_error("html is already open");
}
echo "<html>";
self::$html_open = true;
}

static function open_body() {
if(self::$body_open) {
self::handle_error("body is already open");
} else if (!self::$html_open) {
self::handle_error("body must be contained in html");
}
echo "<body>";
self::$body_open = true;
}

static function close_body() {
if(!self::$body_open) {
self::handle_error("cannot close unopened body");
} else if(!self::$html_open) {
self::handle_error("body must be contained in html");
}
echo "</body>";
self::$body_open = false;
}

static function close_html() {
if(self::$body_open) {
self::handle_error("cannot close html, unclosed body exists");
} else if(!self::$html_open) {
self::handle_error("cannot close unopened html");
}
echo "</html>";
self::$html_open = false;
}

static function handle_error($message) {
//What is the correct behavior, should we attempt to fix the error, report it, who knows.
//We'll just stop execution
die($message);
}
}
[/php]

Those sure are some nice error messages and it keeps you from creating malformed html, but what's the point of it all. The browser is more than happy to tell you that your markup is invalid, or run the output through a lint checker or W3C validator, why should this class have some buggy half-implemented validator inside of it. The answer is, it shouldn't.

If you write stupid code you should get stupid results, Garbage In - Garbage Out. Library code shouldn't hold your hand making absolutely sure that you never make a mistake, what's the point of it. If you write the following code

[php]
HTML::open_body();
HTML::close_html();
HTML::open_html();
[/php]

You are clearly missing something about how markup works, it's not the library's job to hold your hand and guide you through this crisis, the browser will slap you in the face and you will have to learn something. Now there is a caveat, here we are generating HTML, there are great utilities for finding errors in HTML, so we don't need to reinvent the wheel. If you are writing a library or application code where if something goes wrong the error displayed is fine, don't reinvent the wheel. If the error is unacceptable you may need to do some error reporting.

At the end of the day you can't keep the user of your creation from being stupid, they are going to do stupid things and stub their toes and curse your name and you need to make sure that they had to go far off the beaten trail to do so, overlooking obvious better ways to do it, so that it's their fault and not yours. Make it easy to do the right thing and hard to do the wrong thing, but if the user wants to open the body tag before the html tag, let them, they have to learn sooner or later.

workflows and rough edges

I've got this new blog now, and I have to say that I'm a huge fan of WordPress, they definitely have created a great piece of software here. I'm going to split this blog post in two, half for my programming diatribe about workflows and the other half for some rough edges commentary about my new blog.

workflows

The idea of a workflow is central to software development, on a simple project your workflow might be

1. Type up some code


2. Compile or run the interpretter

This can get more complicated, here is a standard workflow with some sort of version control software in the middle

1. Type up some code


2. Build


3. If broken go to step 1, else go to step 4


4. Check-in changes


5. If necessary merge changes with server

It can get more complicated even still with test driven development.

1. Write up a test, make sure it fails


2. Write code until test passes


3. ...same as above

And when office bureaucracy get involved.

1. Write up test, make sure it fails


2. Write code until test passes


3. Perform necessary QA testing and UA testing


4. Submit changes for Code Review and Architecture Board Approval


5. Deploy changes to production environments


6. Monitor health of production


7. Pray nothing goes wrong, if it does, prepare for more reviews and meetings

[caption id="attachment_35" align="alignright" width="300" caption="flow"][/caption]
The job of a programmer is to program, the "Write Code" part of all the lists above, the job of a software developer is to be able to do that and navigate the lists of bureaucracies and procedures necessary to get their code into the user's hands. Now I'm not a bash the process person, source control, unit tests, and peer-review are all great ideas and serve necessary purposes. The great thing is that we are humans and our brains are amazing at internalizing these kinds of tasks and making them second nature.

There have been many times when I have been asked to document a process I've performed hundreds of times, only to sit there blankly looking at my empty word document, silently thinking to myself, "How do I deploy to production?" Normally I then do a dry run through the task, observing the stranger that takes over my body and knows how to do a production deploy. He works quickly clicking this or that, as I furiously document the procedure, he knows all of my passwords, even knows when to stop to ask me a pertinent question or two, but more or less he effortlessly autopilots his way through. I'll look at the documentation I've taken of the steps I just performed and sometimes the list is staggering, 23 steps, wow! In my head it is just 1 thing, do the deployment.

I don't currently have a major project that I'm assigned to, so I've been relegated to working on several smaller projects, none to complicated, the only problem is that I don't have the workflows down yet. This cause a major problem because there is a psychological theory of flow that states

Flow is the mental state of operation in which the person is fully immersed in what he or she is doing by a feeling of energized focus, full involvement, and success in the process of the activity.

None of the new tasks I have are difficult, on the whole they are very easy, but the fact that I can't get into a flow state means that I'm less productive than I normally am, and it is a little bit frustrating. This is why we dislike changing gears, this is why when someone distracts you even for a moment it can be dreadfully annoying, because getting back into that state of focus can be incredibly difficult. The new tasks I'm on are constantly pulling me out of the flow, I have to stop and think about how to get my code into this repository or that, what was the password again, where is the remote test environment, ugh.

When starting a new project or when documenting an existing project one of the most important first things to do is to set up your workflow. I'm going to have my source code right here (points at directory), and I'm going to edit it with this program right here (points at program), and when I've made a change I'm going to see if that worked by doing xyz, and when it works I'm going to submit my changes by doing abc. It seems like a stupid thing to do, you can easily figure out from one step to the next what you should do, but explicitly defining it in your mind will help you stay in the flow.

rough edges

New blog means spending some time making it feel like home. Here are some rough edges you will notice for a while until I get everything set up nicely again.

• In article links may link to the old ihumanable.blogspot.com version of an article instead of the new ihumanable.com version


• Syntax Highlighting is broken in many posts


• JavaScript is being escaped and so some posts that rely on JavaScript don't function properly


• The import process changed all my tags into categories, which feels wrong


• The css is the shine theme default, it needs to be tweaked to my liking


• The navigation on the blog isn't as nice as it used to be


• ihumanable.com redirects to ihumanable.com/blog/ which takes a noticeable second, I'm planning on putting a home page there that will also have links to software projects and other stuff I think is worthwhile


• Still need to learn the ins and outs and workflow (see above) of WordPress

The move was fairly painless but there might be some wonkiness for the next week or so, I hope to get some time over the next few days to clean up links, fix the scripts, and restyle some stuff, so bear with me.

If you have any suggestions or comments about the new blog now is your time to act, since I'll be doing work already, your suggestions might actually get implemented instead of just thrown down the memory hole like I normally do.

new site

ihumanable.com

That is my new domain, blogger has been great for getting started but now that I know I will continue blogging in a regular fashion I've decided to move to my own domain. This blog will remain up, but I will not be updating it, for updates come on over to ihumanable.com.

new site

I've finally gotten around to registering a domain and setting up my own website, you are currently looking at it. There are some rough edges that need to be smoothed over, but WordPress made the process incredibly simple and painless.

I will be working on this new blog's theme and content over the coming week or two to get it pretty and personal. So if you enjoyed my daily rants and ramblings over at ihumanable.blogspot.com all you have to do now is drop the blogspot, ihumanable has gone legit.