spinning wheels

[caption id="attachment_295" align="alignleft" width="251" caption="keep pushing that boulder"][/caption]

Back from Thanksgiving, nice break, hope everyone had a good holiday. With the long weekend came some time to eat turkey, spend time with my family, and reflect on life. I've been thinking about what I want to ultimately accomplish in life and how to get there. The end goal is to work on complex technical problems and produce something useful for humanity.

I spend a good amount of time on Hacker News which is a great place to find out about new technology. It's a fantastic community where entrepreneurs and programming geeks get together and talk about technology and startups. It's the kind of place that incubates ideas that become the next twitter or youtube. Spending time there has made me think that I can do more, do better, it was part of the drive to make prosper.

The problem is that life doesn't happen in a bubble. You have to buy groceries, pay the rent, and keep the lights on. So I've found myself at an odd cross road, I'd rather not continue doing business programming, but I can't stop. What is a person to do? How does one pursue their dreams while maintaining financial security, is it even possible?

For a long while I've believed that you can work a 9-5 to keep the lights on and then work on a side project in the evening. The problem that I'm now finding is that any project that you can hang your hat on and make a living with is going to require more work than what you can give it in the 4 or 5 hours after you get home from work and before you go to sleep to get up for work the next day. Once you throw in trying to have some semblance of a life, it becomes nearly impossible to give a side project enough love and attention to make it work.

I've still been working full bore on prosper and its sister project that has yet to be released. The problem that I face now is this ever creeping feeling that I am merely spinning my wheels, and the encroaching danger of professional burnout. Prosper has yet to see any real adoption, which is not surprising its not ready for anything concrete yet, it is unproven, and it is still in a huge state of flux. The 0.5 release and the current bleeding edge on GitHub are fairly different, with the bleeding edge being a fair bit better.

There is no use moping about all this, so I need a plan to move forward with. Can I quit my day job to focus on the side projects, no. Should I give up on my dreams, no. What is the way forward then? Normally I would double down, work harder and push through. The problem is that I think I slipped from the first stage of professional burnout (Physical, Mental and Emotional Exhaustion) to the second stage (Shame and Doubt), and pushing harder, doubling down, may only accelerate this process.

I still believe that I can change the face of PHP database access, that my library can provide a hugely needed service to hundreds or thousands of PHP developers. I still believe in my ability to create useful technologies and solve difficult problems. So I'm creating a list of things to accomplish in the hopes of defeating my professional burnout.

• Re-institute a workout and diet regimen - Long hours at work and on projects have caused me to spend too many hours sitting around and running to McDonald's or Wendy's for a quick bite to eat.


• Create concrete project goals for prosper - So far development on prosper has been haphazard, I'll get some idea and then work implementing it. The problem being that this leads to an almost manic-depressive like development schedule. When there is a new feature to implement I could spend 6 hours working on it, then have days and days of nothing until a new idea springs up.


• Make time to relax - I find that I spend most of my off-time working on prosper or its sister project. The other "down-time" that I have is spent reading technical articles and ebooks about programming. This stems from my deep love of programming, I do enjoy reading about lisp for hours, but I need to rekindle some of my love for other subject matters, too much of anything is a bad thing.


• Reflect on the good - Too often I spend time thinking about the negative. There are many great things that I take for granted.

Like GI Joe said, knowing is half the battle. This post is more about coming to terms with the difficulty that I'm currently facing, but I hope that it can help others in a similar position. No one is going to hand you your dreams, you have to fight and scrape and kick and scream to make it. It is difficult grueling often thankless work. It's not easy, but if you want more out of life than a paycheck and a cold grave you must fight for it. The struggle takes it toll on everyone, but by being cognizant of the dangers, and recognizing that you are not impervious to the grind, you can make it through, and hopefully make something unique and beautiful.

refactor

[caption id="attachment_292" align="alignleft" width="225" caption="google thinks this means refactor"][/caption]

I've been working a lot on prosper lately (this means that if you want to play around with it, get the bleeding edge on GitHub). I recently added a lazy loading feature, mostly to facilitate unit testing, and pretty soon I will be adding some unit testing. I was able to take care of some things recently that have bugged me for a while, and its because of ruthless refactoring.

Refactoring is not black magic, it just means taking some code and reworking it, rename a function here, encapsulate some functionality there, done. The sticky part is that refactoring is a compounding problem. Things that should be refactored build up on each other, so if you put off refactoring for too long you will have a real mess on your hands. Refactoring can be painful but should never be put off, you see a function named poorly but know that it is used in 20 places, get it now, don't put it off, because by the time you get around to renaming it, it will be in more places.

A good IDE will help you in your refactoring process, the one that I love is (surprise, surprise) Eclipse. Eclipse is brilliant at refactoring, probably because it keeps semantic knowledge with symbol names. Rename the function foo to bar and you don't have to worry about Eclipse destorying all those variables named foo (ps. if you have variables named foo you have refactoring to do!). Eclipse (and other IDEs) are great at all kinds of refactoring, renaming, extracting and encapsulating, pull up, push down, etc. Get to know these tools, get comfortable with using them, and refactor without fear.

Wait, change big chunks of code without fear, how!? Well you should have tests, be they unit or functional. I'm introducing unit tests into prosper, but for the time being I have a local version of todo list case study. The todo list application is pretty good because it exercises all the important bits, if I change something and everything in there keeps working, I haven't broken anything too badly.

The reason I want to introduce unit tests though is that I've introduced regressions even with the todo list humming along, some dark corner of prosper has started shooting out unicorns, but I haven't noticed, until I see some bit of code that I know can't be doing the right thing. Test coverage allows you to refactor without fear, and if you want to have a living project that people might someday use, you need to refactor without fear to keep things current.

---

That's it for the technical part of this blog post, I would like to announce a few things though.

• The move to GitHub has been completed and I'm starting to get into the workflow of using it now.


• The new skunkworks project is nearing a beta release, so keep an eye out.


• Development on the new project has spurred improvements in prosper, there will be a point release (0.6) soon


• There probably won't be a post tomorrow as I'm traveling with Heather, my beautiful girlfriend, up to see my family in Cleveland.

That's all for now, I will have something up Black Friday more than likely, until then, Happy Turkey Day everyone, safe travels, see you after I put another couple of pounds on.

discomfort

[caption id="attachment_284" align="alignright" width="300" caption="was the collander really that hard?"][/caption]

I am an American, which means that my life is constant bombardment of people offering to make it simpler, faster, easier, and more comfortable. Watch an hour of tv and count the number of times people are trying to make your life easier and more comfortable, its quite a few. The problems they solve aren't even real problems, they are imagined problems that we've been convinced are real. I used to think these problems were real too, I didn't have enough time, I didn't have enough skill, I needed the pre-packaged solution that was quick and ready in 5 minutes.

Then I woke up one day and realized that I had been swindled by a bunch of bullshit.

Who has 15 minutes to make pasta for dinner?! [Show woman dumping pot of scolding hot water and noodles all over herself]
There's just not enough hours in the day!! Buy our microwave-licious pasta bowls!!!

You know who has 15 minutes to make pasta, everybody does. Want to write a comment saying there is no way you could free up such time, well fuck you Mr. I-have-time-to-read-a-blog-and-post-a-snarky-comment-but-somehow-can't-find-15-minutes-with-both-hands-and-a-flashlight. At this point though, we are all thinking, am I trying to make some sort of point about pasta or is this going to get around to programming at some point?

Well thanks for going on that side-rant with me, let's look at what in the world I'm trying to get to. Being a programmer I like to be comfortable, I like to work with the languages I like, and the tools I know, and the frameworks I understand. It makes sense, people are more productive when they are working with the tools we know, that's why we have holy wars about what language is better and which framework to use, we invest some time getting our nice comfy language set-up just the way we like it and we look at people sitting on uncomfortable languages and wonder what the hell is wrong with them.

The problem with being comfortable is that its hard to learn and grow when you are comfy. You know the language, you might learn some edge cases here or there or uncover a new feature or bug every once in a while (and this is important too, its important to be a master of something), but new learning is rarely done this way. Its very hard to learn new things by doing the thing you already know how to do day in and day out. Yesterday I decided to do something new, and it was uncomfortable, but I've started learning and hope to learn a bunch more.

What was this horrible discomfort that I suffered, moving my code for prosper from Google Code to GitHub. You see, I've been using Subversion since college, when one of the smartest software developers I know, Ian Potter, introduced me to it. I had heard some stuff about svn before and compared to the source control system we were using (a folder called test and a folder called production) had determined svn would be a step up. He set up svn and patiently answered my questions of which code was on my machine and which code was on the server over and over again until my brain finally wired it up.

Moving to svn was much more complicated than the tried and true 2 folder methods, but the safety, history, and concurrency that svn offered was more than worth the week or two of learning the new system. In my career so far I've gotten to use cvs, svn, and vss, they are all similar concepts, there is a central repo and you can commit, update, branch, merge, blame, etc. This is how source control works in my brain, and I am completely comfortable with how this all works.

Well svn is no longer the new hotness, git is in big time, and a good portion of its popularity can be attributed to GitHub. GitHub is the facebook of source hosting, its a really great site to host your project on, most interesting new projects are hosted there, and the social aspect allows people to easily interact with you and your project. I've had a GitHub account for a while, and I tried it out before, only to go back to the comfortable embrace of Google Code and svn.

[caption id="attachment_285" align="alignleft" width="300" caption="who could resist octocat?"][/caption]

So why move to git, just because its shiny? Well, GitHub is almost enough of a reason, but the distributed nature of git, the increased popularity of git, and the ability to quickly make new branches and merge them back into trunk, make it worth learning. If I want to continue being active in a community that more and more relies on git, its probably best to learn it.

I've taken the first step, I've moved my stuff over to GitHub and I've got git up and running on my laptop, I've been able to push some changes upstream and over the next few days I will be reading the excellent help that GitHub provides you. It will not be comfortable, I will probably be scratching my head for a while, wishing to go back to the centralized repo that I know and love. I will more than likely treat git like svn for a while until I "get it." But I've decided to stick it out, to live with and own my discomfort and to learn this new thing.

Here is my challenge to you: maybe you've read a few things about a new technology and its interested you, maybe you've even taken a few trembling steps into the new technology, and you've walked away or put it aside. Go back to it, jump in head first, don't look back and don't give up. The great thing about technology is there is usually a manual to read, a community to turn to, and our old friend Google. Make yourself uncomfortable, you might just learn something.

division of labor

[caption id="attachment_280" align="alignright" width="216" caption="cartoon labor"][/caption]

When building or analyzing any system of significant complexity the most important thing to determine is a proper division of labor. The entire idea of n-tier architecture is built on dividing labor up into the appropriate tiers and keeping them cleanly separated. For anyone that's ever worked on a real world system half of the difficulty is caused by one layer leaking into another and the other half is getting around the separation caused by separation.

Despite my last sentence, division of labor is a necessary and good part of design. When done right it allows you to know where functions, variables, and constants should live, how things should interact, and who is the system of record. This is why it is imperative to decide beforehand or shortly into a project what the lines of demarcation are and who is responsible for what.

Prosper is one of a group of libraries, with well defined divisions of labor. This allows the libraries to be decoupled, and also has the added benefit of letting me know when I'm done. When I found myself adding functionality to Prosper the other day that I knew belongs in the next library, I realized that it was time to officially start the next library. So a new skunkworks project is moving ahead full steam, it contains 2 parts that work together, and so again requires a well defined division of labor.

Just because work has begun on a new library doesn't mean that Prosper is left out in the cold, yesterday I added support for the deprecated mysql library, standardized construction, and added better support for native functionality. If you are checking out Prosper, you can follow the latest development by pulling from the svn repo.

Back on topic though, figuring out the division of labor is the most important part of any analysis or design work. We recently at work had a 2 hour long discussion about order numbers. It went something along the lines of this:

BA: This order number comes from the Mainframe
Expert: No the Application creates that number
Dev: So, what does the SOA layer need to do?
BA: Generate the number
Expert: No, do nothing, the Application will automatically do it
Dev: Wait so who is responsible for creating the order number, and if its not the Mainframe why is it making one?
Expert: The Mainframe always made that number and the Application has always ignored it.
BA: So we will make it in the SOA Layer?
Dev: Who is in charge of making the unique identifier order number?!
Expert: [shrug]?
BA: [shrug]?
Dev: Let's decide that first

Explicit division of labor requires you to pull the unspoken assumptions everyone has about the system out of the silence of our own minds and puts it into the blinding daylight. Suddenly communication is smoother, arguments come to an end, because there is some record to consult to help guide decisions.

Draw out your explicit divisions of labor, don't carve it in stone though, if your original design ends up being too inflexible, allow it to change, but again, bring all the unspoken assumptions back out into the open. This allows you to cleanly define what piece is in charge of what function, keep layers from leaking into each other, and make for smoother development and analysis.

solve it forever

[caption id="attachment_277" align="alignright" width="200" caption="having a bully day"][/caption]

This is a very simple rule that I have for solving problems at work, solve them forever. Its not always practical and sometimes you need to bend the rule, but its a great guideline. When I'm presented with a problem at work, I take it as a personal insult, someone could have solved this before and made my life easier but they chose not to and burdened me with it, and I refuse to do that to anyone else.

This has some downsides, you have to put in a lot of effort, and sometimes little things take a lot longer than the "man" thinks they should. There are some upsides though, you avoid technical debt. A lot of coverage has been given recently to the idea of technical debt, how you define it, how you avoid it, should you avoid it and so on. It is one of those nebulous concepts that everyone has a different definition for, we all know what it feels like to run up against some technical debt, its that situation where something that should be simple is needlessly difficult.

Technical debt is not complexity, sometimes doing seemingly simple things within a complex system can be cumbersome, but that's the price you pay for some other benefit (dynamic forms, automatic mapping, etc.) You know that you've hit some technical debt when something is complicated and difficult, but there is no upside.

Boss: Change the text on that button
Dev: Ok, I'll just update the html
...several hours later...
Boss: Why'd that take so long?
Dev: You see that text is pulled from an i18l file, which is populated dynamically by a database table, that table is autorouted by the orm layer, so I had to change a bunch of configuration entries, then I had to run an update across several tables, some tables were using the name as a foreign key relationship, even though they didn't declare that formally to the rdbms, so I had to go track down what tables were doing that, and once all that was through I had accidently made a typo and had to go through the whole damn process again.

Debt like this is incurred when you solve a problem for now, not thinking about the future. Solving a problem forever means that as long as the problem domain remains the same that you don't have to do anything. The issue is that many people misinterpret solving a problem forever as solving all problems forever, and that is not what I wish to promote.

There is an important difference, you should solve the problem domain thoroughly, but your solution should be specific to that domain, don't try to do too much. There are innumerable problems that someone will take the time to solve for themselves, but give no thought to what happens to the next guy. Coming onto a new project I've come face to face with an age old developer problem, rolling on.

Rolling onto a new project is often an exercise in navigating technology to set up a development environment, navigating the corporation to integrate yourself into it, and navigating the team to find a place. All this navigation seems to be on the shoulders of the new person, there are people you need to talk to to get mainframe access. Which people? Where are they? When do I talk to them? How? There are often answers locked up inside someone's brain, and if you ask the right people you can, like a detective slowly unraveling a mystery, find the answers and get your email and passwords and id badges.

This is a problem that should be solved forever, once the first guy joined the team he should have documented what he did, when, how, why, and that document should be part of team cannon, available as a helpful guide to the next new guy. This rarely happens though, it is a shame, but understandable, new people are struggling to get up to speed on a project they don't have time for the meta-project of documenting their up-to-speed-getting project.

That is just an example, the next time you are guts deep in a problem that you've encountered before think about your fellow developer, and think about how you can solve it forever. The more problems you permanently put to bed, the smoother development can go and the faster you can get your project done.

clean slate

[caption id="attachment_270" align="alignleft" width="300" caption="new desktop"][/caption]

Today is my first day at my new assignment. My company gave me a new(er) corporate laptop with a plain vanilla install of Windows XP Pro to use at my client site. Last night I had the joy of looking upon that fresh computer and thinking of the wonders that could be. I got ready to install the base programs that take a windows machine from factory to awesome. Here is my essential install list.

• Firefox - The first and only time I interact with Internet Explorer is to go download Firefox




• Heaven.cube theme


• Delicious Bookmarks extension


• Echofon extension


• Firebug extension


• FireGestures extension


• Flash plugin




• Programming Languages




• Haskell Platform 2009.2.0.2


• Java EE 5 SDK Update 8 (JDK 6 U17)


• CLISP LispBox


• Python 3.1.1


• Ruby 1.8.6 (I would have rather gone with 1.9.1, but the one-click windows installer is lagging behind and I one-click was all I wanted to do)




• Development Tools




• Dia - a drawing program


• ClipX - best clipboard manager


• Eclipse IDE for Java EE Developers


• PSPad - awesome free editor




• Tools Windows should come with




• 7-zip - small, fast, free


• Process Explorer - The task manager Microsoft should have built, but didn't, so they just bought it.


• Foxit Reader - small, fast, free (again)


• Pidgin - Take that MSN Messenger


• Paint.NET


• Royale Noir XP Theme

Then I spent plenty of time monkeying with the filesystem, the wallpaper, things that don't particularly matter, but make the laptop feel like home to me. I made my icons and text smaller, since I'm used to a giant monitor and the 1280x800 resolution felt a little cluttered to me. Now though I feel right at home with this machine, and feel like this is the best possible time of laptop ownership, right after you've got it set up just the way you like it, right before you add stupid things that pollute the registry and mess up your beautiful filesystem.

The other clean slate, the less fun one, is my brain. I have all kinds of knowledge in my brain about my last projects, but today I find myself swimming in a see of paths and acronyms and weird buildings, and a chair that isn't quite high enough to be comfortable. I'm starting to wrap my mind around the acronyms and getting a grip on what exactly we are doing here, but I fear it will take more time to feel as comfortable with this new position as I do with my new laptop.

Suffice it to say it could be much worse, I can clearly still push to my blog, so that's sweet. The people I am working with seem cool and fun, the project seems like a lot of work but nothing that we can't handle. I will probably have a whole different viewpoint in a week when I truly realize the scope of the project and settle into a nice little routine.

So far, so good, I'm off to read another IBM pdf that is half marketing / half documentation. Wish me luck.

baggage

[caption id="attachment_265" align="alignleft" width="300" caption="i know how you feel"][/caption]

Finally! I am finally about to start working at my new project after all those squishy human beings got done drawing pretty charts and signing dead pieces of trees. I now can put myself into the beautiful cold clutches of the machine again and get back to what I do best, code stuff. I'll be moving from .Net web development to Java middle ware development, and I'm excited to get back to programming.

That seemed like quite the dig at .Net, I don't intend it to, it's just been my personal experience. The problem with the .Net projects I've worked on is that they've been amalgams of third party libraries, and I've been in charge of gluing them together. This isn't a weakness, some would say its a great strength, drop some cash and boom, big problem solved by a nice clean little dll (or 4). I will say that it greatly reduces time to market which is a Good Thing© and that it lets you get down to the customizations that clients spend the big bucks on.

The problem is that I'm a programmer, not a customizer. I like to write code, not tweak little settings here or there (although as a nerd I'm a fan of that as well). The .Net environment always seemed like a bunch of black boxes that I was just trying to lash together in a pretty enough package to call it done. Any actual code I wrote was always just taking business rules and turning them into something a computer can understand. That's ok, in fact companies make a lot of money doing this type of work, I just don't as a programmer feel fulfilled by it.

Now though I'm heading into a project where there is a problem without a solution, something that actually needs to be designed, implemented, tested, deployed, iterated, and other fun nerd words. I'm excited to be back to programming, but I'm worried about technological baggage. What is technological baggage you ask, let me find you an example (anonymized, of course).

[javascript]
function Recalculate(...)
{
if (isNaN(CleanNumber(ctrl.value)) ||
CleanNumber(ctrl.value) == "")
{
ctrl.value = "0";
}

var newValue = parseFloat(CleanNumber(ctrl.value));
var oldValue = parseFloat(CleanNumber(prevCtrl.value));
var diff = newValue - oldValue;

ctrl.value = newValue;
prevCtrl.value = parseFloat(CleanNumber(changedPrevControl.value))
+ diff;

//Snip some similar code
FormatMoneyWithCommas(ctrl, 0, 0, 0);
}
[/javascript]

There is nothing wrong with the above javascript it works just fine. But look at it, that's .Net wrapped in script tags if I've ever seen it. There are many non-javascript things going on in this code, anyone familiar with javascript will see them straight away. My intent is not to shame the author of the code or to call it out as bad javascript, it is merely to show that when steeped in a technology it can be very hard to pull yourself out. I picked on .Net but I've see plenty of javascript written like java, in fact I've gotten actual java code thrown between script tags back from offshore "resources" before.

There is a lot to keep jumbled up in our heads when we are programming, to lessen this burden we adopt naming conventions, and calling conventions, and all kinds of conventions and that is a good thing. The bad thing is carrying your languages conventions into other languages, its a kind of linguistic egocentricity that makes the natives upset.

Now I am trying to purge the .Net from my fingers and get them ready for Java again. I'm sure I won't remember all the little gotchas, but the important part is to be aware. There are some basic strategies for carrying over as little baggage as possible.

• Google - Go and google Language X for Language Y developers, Language X and Langauge Y can be almost any combination of languages and there is a good chance someone wrote an article, feature matrix, common pitfalls type document for it.


• When in Rome - If you are going into an established project, adopt its idioms.


• Tutorials - Even if you know the language you are heading into, go read a few tutorials, brush up on the basics

I remember the painful period between Java and .Net, being frustrated trying to figure out what the hell a HashMap was called (Dictionary) or wanting to declare what my functions throw. I'm sure I will have the reverse, but the one thing I'm looking forward to is getting back to Eclipse, I love Eclipse, and I will not miss Visual Studio at all.

dogfood, terror, and tweets

[caption id="attachment_251" align="alignright" width="300" caption="too literal"][/caption]

Yesterday I released my new library Prosper (in case you hadn't noticed the new menu entry at the top of this blog). It was a ton of work, way more work than I ever expected, but it was worth it. Prosper is now out the door and ready for human consumption, I've been using it all day on a fun little project, mostly playing around with the twitter API (which I've done before but never from php).

I was talking the other day with the esteemed Rick Kierner about the upcoming CodeMash convention. Since I work for an amazing company that is sponsoring the event, I get to go for just about free. I plan on going to the precompiler sessions and that day comes out of my pocket, but its a small price to pay. Rick is taking care of the nitty gritty details but in the end I will have to give him some cash money to pay for the precompiler and the first night's hotel room, he decided to use BillMonk to record the debt and keep everything straight.

After perusing BillMonk for a while I decided that I really liked the site, its a cool idea, and I thought to myself, "hey self, now that tipjoy is defunct could there be a space for me to make my mark." (Side note, I often think hyperlinks at myself). Rick and I bounced some ideas off each other and at the end of the day decided that there really wasn't a good business in what we wanted to do, so we abandoned the thought and moved on, c'est la vie.

Today though the idea kept bouncing around in my head, really its been bouncing since we first started talking. Today I finished my proof of concept for my new minimalist bill tracking system, currently dubbed cha-tweet (a portmanteau of cha-ching and tweet). I'll outline quickly the concept behind it, a rough approximation of its implementation, its status, and what the title has to do with anything so far in the post.

The idea is basically this, you can tweet a specially formed tweet and cha-tweet will aggregate those into a nice little summary of who owes you cash and who you owe cash. The tweets look like this

The debt cha-tweet

@I_owe_this_guy_money > $5 lunch: delicious sandwich!

The give cha-tweet

@I_gave_this_guy_money < $5 lunch: mcdelicious

The basic idea is that money comes out of or goes into the username to or from you, respectively. It's a simple syntax that lets you sort by debtor/debtee and by type of debt. Since it starts off with the @username syntax it only gets displayed (by default) to people that know both of you and would maybe care, and its easy to find by the service because it is treated by twitter as a reply. The system works fairly well, and I'm happy enough with the concept.

The problem was that to create an accurate summary it had to go get and process the entire tweet stream, which can take a long while, and assuming that most of the tweets in a users tweet stream will not be cha-tweets, there is no need to do all this processing everytime (and in steps sexy miss rdbms). So, throw a database at the problem, store the seen cha-tweets and operate on them, this has now been implemented, it took one hour, and I used prosper to do it.

So how was it? Really easy, I stole the configuration from the todo list project, switched around the credentials and the schema name, and was off and running. Everything was working well until I tried to update something and Prosper spit up a bunch of non-sense at me, and my heart sank. Did I just release some library full of horrible bugs and will be mocked endlessly, the cries of n00b and pwned reverberating so loudly as to drive me mad?! No, I just don't know how to type so instead of writing this
[php]
Prosper\Query::update('user')
->set(array('since' => $since))
->where('id = :id', $_POST)
->execute();
[/php]
I wrote
[php]
Prosper\Query::update('user')
->set(array('since' , $since)) //WHOA!
->where('id = :id', $_POST)
->execute();
[/php]
Changing that double arrow to a comma makes a big difference, and causes prosper to correctly try to set `0` to 'since' and `1` to 'some_huge_twitter_id_number' which is what I told it to do.

Crisis averted, my library still functions properly, hip hooray!

• Is it easy to use? Yes, I found that it cut down on my development time considerably.


• Is that because I wrote it and have no learning curve? Quite possibly, I'm not going to rule that out.


• Is the code easy to read? Yes, its fairly straightforward crud stuff, made much simpler by prosper.


• Did I learn anything eating my own dogfood? The verbose function is awesome! I wrote the verbose function in the beginning as a simple debug / sanity check of sorts to make sure that things were being translated and serialized correctly (pro tip: the magic __toString function will print the sql statement that will be executed). During development though it was incredibly useful to be able to switch the call to execute for a call to verbose and immediately see at a fairly granular level what was going on. This got me to thinking how much prosper needs some official documentation, and that will be my project for the next week or two.

In the end I have a nifty little twitter gadget, I will probably be hosting it somewhere soon so keep an eye out for that, but more over I have some more experience as an end-user with prosper and a renewed energy to write the official documentation.

---

I'm also adding some stuff to the blog, there are now social media icons, the rss feed link got a new friend mr. twitter link, and there are some backend changes for speed and stability. This is another round of blog tweaks and that is the best time to have your opinion incorporated into the blog's overall look and feel, so comment away!

announcing prosper

[caption id="attachment_248" align="alignright" width="128" caption="prosper"][/caption]

I've been talking and teasing and hinting about my side project. Well today is the day I can finally announce my project, prosper.

Prosper has been in the works for a few months in my spare time, it is a database abstraction layer written in php. For anyone who has ever written a web application in php you know that they are written for one database and one database only. It is a MySQL application or an Oracle application or an X application. If the application gets popular enough, the developer might get around to adding some support for other backends, and when they do it is always some homegrown solution. With prosper you can write your database code once and have it work across 18 different backend databases.

Prosper came about when I was looking for a quality ORM layer, the one thing that struck me as odd is that everyone was rolling their own database abstraction layer. Doctrine has DQL, Flourish has F-SQL, php.activerecord has an adapter architecture, and so each library has its own quirks and compatibility.

Following the Unix philosophy, prosper does one thing and tries to do it well. Here are some things that prosper is not. Prosper is not an ORM. Prosper is not an Active Record implementation. Prosper is not an MVC framework. Prosper will not pluralize names, will not handle sequences for Oracle, will not slice and dice your onions. Prosper is one thing and one thing only, a library that turns function calls into sql.

Official documentation is coming soon, but for now you can view the case study, learn why you should use proper, and get the code

I read the documentation, I wrote the code, now its your turn. Give it a try, submit issues to the Google Code Page, and let me know what a great / terrible job I did.

Note that prosper is not ready for primetime yet, this is version 0.5, lots of the backends should work but only the MySQL one has gotten any kind of real testing. The code is open source currently under an MIT License, although the license may change for the 1.0 release, it will always be free to use and open source though. Here is the list of things that still need to happen.

• Thoroughly test the backends


• Add any backends that I missed


• Write up official documentation


• Move code to github to facilitate collaboration


• Forge partnerships with other open source projects


• Shamelessly promote

Go get the code and let me know what else I need to get to work on.

leaps and bounds

I keep talking about (more like won't stfu about), the sideproject I've been working on. Today I added several draft pages to this blog containing documentation and release information, I'm expecting to have the actual rollout sometime next week. I've made a flowchart to show you how my mental process worked so far on this project, and where I'm at now.

[caption id="attachment_131" align="aligncenter" width="331" caption="how my brain works"][/caption]

Most of the time I get stopped in the "will it ever be any good?" decision, it returns no, and I scrap my code, and wait to have a worthwhile idea. This time though, somehow, I've managed to move past that one, and for the last 2 weeks or so have been spinning around the "is it any good?" -> "no" -> "refactor" -> "is it any good?" -> ... loop. I've been tweaking this or that and in my head there was still so much to do before releasing.

Today though I've found myself writing up release information and documentation, and I came to realize that the ever present list of things to add or get working or refactor had been brought down to nil. I'm out of code work to do for the time being on this project, the best thing I can do is release. There seems to be this huge leap though from hacking to releasing, for me at least. I can sit all day adding features and refining APIs, but I realize now how frightening it is to put your work into the wild and try to promote it.

I feel this anxiety doubly compounded because I didn't write an application where the code, even if open sourced, is neatly tucked away behind the scenes. I wrote a library, with an interface and code that if it gets any kind of adoption, people will probably look at, curse at, laugh at. Really though, there is nothing to do but put it out there, track the bugs, iterate, and make it something I can be proud of.

Next week I will be officially releasing this side project, my hope is Monday morning. I want to write up some great documentation, explanation, set up bug and feature tracking, set up a forum, and move the project over to github, because github is the official hotness. So this is a head's up, watch out next week because big things are coming.

teaching

[caption id="attachment_113" align="alignright" width="226" caption="typed \'classroom discipline\' into google"][/caption]

For those of you unaware of my background it may surprise you to find out that I have two degrees, one is in Computer Science, the other is in Theoretical Mathematics, the kind you can't use for anything ;). During my day to day, 9 to 5, actually getting paid to do stuff job, I work all day using the Computer Science part of my brain, programming various things. This has left the Math part of my brain atrophied and depressed, I barely visit, hardly ever call, and never send it flowers anymore. But, for this post I will be dusting off the Math part of my brain, don't worry nothing too in depth. For those that want to skip to the point click here

My senior year I took a 400 level course in the History of Mathematics, it was probably my favorite and most interesting math course. We discussed the evolution of number systems and mathematical theory from the Greek Ionian Numerals to Modern Abstract Algebra, it was a fantastic journey through human knowledge. I will always remember the lesson on Euclidean Geometry and most importantly why it was taught.

Euclidean Geometry has 5 basic principles (called axioms) upon which all other theorems, corollaries, and other fancy math talk are built off of. The important thing about Euclidean Geometry is not the subject that it examines, moreover it is the way in which it examines it. When Euclid penned Elements it was the first major examination based off of a logical system and comprehensive deduction.

The whole idea was that within this beautiful system, no matter how big and magnificent the structure became, you could always walk a theorem through the structure and down to the axioms without making any leaps of logic. This used to be the point of teaching Euclidean Geometry, it is a concrete example of a deductive and logical system, working on objects simple enough that children can understand it.

Sadly, we no longer teach the most important part of Euclidean Geometry to our children in America. The important part, deductive reasoning, is very difficult to teach, it requires magnitudes more work than what we currently teach, mechanical geometry. We have children memorize rules and tricks and mnemonic devices so that they can perform mechanical operations on lines and angles and shapes. They learn nothing from this, they will forget these machinations, and they will have never learned the deductive reasoning skills that seem so lacking these days.

What am I writing about though, why does this belong on a programming blog, have I gone mad? To answer these question in order: This is an interesting story you just got for free (ingrate) and will dovetail nicely with the next section, its reason for existing will become apparent in a second, and you can't go somewhere you already are. The point of the above is that in teaching a concrete something (in the above case Geometry) you might actually be trying to teach an abstract something else (in the above case Deductive Reasoning and Logic).

As a Mathematician I think that we should go back to teaching Euclidean Geometry the right way, but as a Computer Scientist I like flashy fun interactive keyboardy things, so let's talk about those. You see there is another way to trick kids into learning about structured logic and deductive reasoning, programming. We can if we try a bit even make it fun. Let's take a look at past and present attempts to do just that.

Logo

I remember you fondly from my childhood, Mr. Turtle. Logo was a simple language built at MIT on top of LISP, possibly the seed of my love for LISP was planted by this simple turtle. The idea was to give the child a simple interface with a little turtle that they could command around using a simple syntax.
[caption id="attachment_105" align="alignnone" width="488" caption="mit logo guide"][/caption]
[sql]
FORWARD 50
RIGHT 90
FORWARD 50
RIGHT 90
FORWARD 50
RIGHT 90
FORWARD 50
RIGHT 90
[/sql]
That code would draw a nice simple square, 50 pixels by 50 pixels, but logo didn't stop there, it had functions and loops and all many of list processing. The above code could be simplified to
[sql]
REPEAT 4 [ FORWARD 50 LEFT 90 ]
[/sql]
and suddenly the child would learn loops, it was great fun and is still used today to teach programming to young children. There are still active versions, like FSMLogo for Windows or ACSLogo for Mac

Alice

Alice is a new attempt to make programming easy and fun, it allows you to start making 3D scenes almost immediately. It is free from Carnegie Mellon and they do a pretty good job summing up what it is.

Alice is an innovative 3D programming environment that makes it easy to create an animation for telling a story, playing an interactive game, or a video to share on the web. Alice is a freely available teaching tool designed to be a student's first exposure to object-oriented programming. It allows students to learn fundamental programming concepts in the context of creating animated movies and simple video games. In Alice, 3-D objects (e.g., people, animals, and vehicles) populate a virtual world and students create a program to animate the objects.

Here is a video introducing Alice

Some of the innovative features are the fully 3D environment, the drag-and-drop source editing in which code constructs are represented by physical objects, and the integration with Java and the JVM.

Scratch

Another effort from MIT. Scratch's motto is "imagine, program, share" and they live up to it. Like Alice they utilize a drag-and-drop source editing system, but there is a much greater emphasis on the social aspect of it all. In a web 2.0 nod, they encourage people to upload their projects and let other mix and match projects in fun and interesting ways. Its easy for kids to jump right in because there are just under 600,000 projects being shared on the Scratch website, so there is plenty of examples. Here is the Scratch video.

Scratch from Karen Brennan on Vimeo.

Hackety Hack

[caption id="attachment_112" align="alignright" width="150" caption="HacketyHack Logo"][/caption]

I didn't know whether or not to include this one, the creator _why decided to leave the internet behind a while ago and as such abandoned this project. It has been picked up and rescued by Steve Klabnik. It is based off of ruby and allows kids to get interesting programs up and running quickly because of the powerful and intuitive libraries included. It is an integrated IDE with tutorial, editor, and runtime all rolled into one. The best part of HacketyHack is that so much of what you learn can be transferred directly to the mature ruby language. You can start your kids off with HacketyHack and when they become jaded nerdy teens that can start their own website in RoR to talk about Twilight or whatever the hell it is teenagers do.

Lego Mindstorms

This last one comes from Lego and costs some cash unlike the previous offerings, but the money gets you a badass programmable robot. It has a drag-and-drop development environment similar to Alice and Scratch. Kids (and adults) can get really into it, making lots of cool things and holding sweet lego competitions. It's one thing to see "Hello World" on a monitor, its an entirely different feeling to have a robot do your bidding.

---

So that's the educational round-up, with these tools we can find something to appeal to the youth today and hopefully teach them some sweet, sweet logic. Give it a try with your kids, or try to start a program up at a local school or social club. Spreading the joy of programming will enrich their lives and your own.

leaving

[caption id="attachment_92" align="alignright" width="300" caption="leaving for somewhere"][/caption]

I've been working on a project for the last 5 months, but finished active development about a month ago. I finished in the best possible way, the product shipped, the customer likes it, and there hasn't been much to do besides the minor bug or feature here and there. It's a good place to be in, since I'm a consultant it's also not economically viable. If I'm just sitting here writing blog posts and drinking smooth delicious Diet Dr. Pepper all day, I'm not making money for my company, so they went and found me another project to work on, which I talked about here.

I'm looking forward to starting my new project, but as Semisonic taught us, "every new beginning comes from some other beginning's end." (Have to try to pull this post out of the tailspin caused by quoting Semisonic lyrics now). The problem is that when you start a new project, the glitz and glamor of a new project you haven't learned to hate yet can blind you from the responsibilities of being the guy leaving.

Leaving a project comes bundled up with a bunch of responsibilities. The cliched, "what if you got hit by a bus tomorrow?" question that managers love to ask (because they are too afraid to say, "what if you quit" or "what if I fire you" but oddly enough have no qualms with theoretically knocking you off) actually has some value. The "bus" of leaving this project is barreling down at me, and I have to make sure I dump enough of my brain out that people can still maintain the code I wrote.

If you read my rant / guide to project documentation, here's a chance for me to prove myself. Throughout the project I've dutifully created and more or less kept documentation up-to-date about everything from server credentials to step-by-step deployment procedures. I cannot recommend this approach enough, after reading through some of my wiki pages on these topics it is clear that trying to compile this information after the fact would be difficult if not impossible. So since, ostensibly, everything is done already, what am I left to do?

1. Read through documentation - Projects move fast, things that made sense a month into a project probably won't make sense 5 months into a project, be sure everything is up to date. Read the documentation with the mindset that you are trying to start working on a project, is it clear, could you set up a development environment, fix a bug, and push out your changes in a day? If not, figure out why not, and fix it.


2. Bug your coworkers - Someone will be around to work on the project, start bugging them to read the documentation, have them go through a dry run of some complex task you normally do, and let them mock your ability to document things properly. It is much easier to have a minute or two of verbal in-person explanation then trying to remember how things work and having a 20 email long thread of questions and answer a few weeks into your new project.


3. Clean house - Those tasks you were putting off doing, get those done, and for fsm's sake, do it right! There is nothing worse than being knee-deep in a new project to have a past project rear its ugly head, forcing you to dredge up long forgotten knowledge to fix some minor issue.


4. Cut ties - Clients and other developers get used to coming to you with issues, if you are moving on to a new project, let them know about it. Inform them that you will no longer be the contact for the garthok narfler and that they can talk to developer x about it from now on. For an extra touch of class let them know that it was nice working with them.

The way you leave a project is almost as important as the work you did on the project. Rest assured no matter how good a job you think you did, a few weeks after you leave, other developers will be cursing your name. Your job as you leave is to make sure that those angry expletives are few and far between and that you can rub it in their face when they complain about this problem or that obstacle with the fact that you left them very clear documentation on how to handle such a situation.

convention vs convenience

[caption id="attachment_84" align="alignright" width="300" caption="convenience, american style"][/caption]

I'm a huge believer in Convention over Configuration because it makes life easier and makes me more productive. I use to program in Java, and unless there is some seismic shake-up, I will soon be going back to Java. I like Java well enough, it's no lisp or ruby, but it has its place in the business world. I have many gripes with Java, verbosity, complexity, etc, but when you are in the enterprise trying to work with a bunch of third party pieces cobbled together into a hulking software nightmare the worst, by far, is configuration.

It makes some business sense, if a customer won't use your product because they want to change the tooltip on the help page and they can't without hiring a Java programmer, or at all because you distribute closed source .jar files, the simplest solution is to toss a config file at them. Just change this or that setting in the config file and, look at that, the whole application is christmas colored and in sanskrit. There are a some huge problems with Java configuration though.

1. The unfortunate pairing with XML - XML hit its high water mark around the same time as Java, maybe there was some reciprocal love there, and it can be maddening.
   [xml]
   <env-entry>
   <env-entry-name>maxExemptions</param-name>
   <env-entry-value>10</env-entry-value>
   <env-entry-type>java.lang.Integer</env-entry-type>
   </env-entry>
   [/xml]
   Oh fuck me, are you serious?! I took that from the official Tomcat Documentation. It just sets maxExemptions = 10, but it takes 5 lines, 2 layers of nesting, 4 open tags, 4 close tags, and as you can see, this is a straight copy and paste from the official documentation and it has a pretty clear error. Clear to me because my eyes have been trained to read xml like a champion, the env-entry-name tag is closed by a param-name tag, that isn't right.


2. Undocumented DSL - Every configuration is basically an undocumented Domain Specific Language wrapped up in XML's ugly ass clothing. There is little transfer of knowledge between Java and a Java Configuration file, or even between XML and a Java Configuration file.


3. Undiscoverable - Maybe it would be more accurate to put hard to discover. Where do you go to figure out what belongs in your config, or what nodes configure what, your best bet is to hope the developer wrote up (and subsequently kept up to date) some documentation. Little to no help from your favorite IDE's code completion and the constrained nature of the problem domain makes web searches less likely to yield helpful information.


4. Twiddling - Like in field of dreams, if you make a configuration, they will come. Sure there is no good reason for X to be configurable, but I don't like having hardcoded values in my code, so every hardcoded value is now configurable, and I'm on the slippery slope of softcoding now. This allows the end user too much power to twiddle around and configure things that don't ever need to be played with, just because we can doesn't mean we should.

This isn't just Java's problem, they are just the easiest to pick on because it seems like its everywhere. Then Ruby on Rails came onto the scene and made popular this idea, convention over configuration. I want to put my models in the fliggity directory, that's too bad, they go in the model directory. I would like to name my table that stores user data 'tc_people_datastore', yea well I would like a billion dollars, you are going to call it users. This means that if tomorrow I'm told to go work on a RoR project, having never seen it, I would have a good idea how the project is laid out, where the data lives, and how everything is hooked together. This convention eases the mental load I have to carry around, replacing it with simple, sane rules.

Convention over configuration, especially the rails way, has been called opinionated software. The software, in this case rails, has an opinion about how things should be laid out, what your tables should be called, etc. I'm in the midst of writing my own software and API and I've decided that my software should have an opinion about stuff, but more importantly that things should follow certain conventions.

[caption id="attachment_86" align="alignleft" width="205" caption="conventional breakfast"][/caption]

As a corollary to the conventions is a strive for consistency. Maybe a better title for this post would have been consistency vs convenience, but I'm already 700 words in, no going back now. I'd like a consistent API for a few reasons. Consistency makes it easy to remember, does this function go ($needle, $haystack) or ($haystack, $needle)? They all go ($needle, $haystack) calm down. Consistency makes wrong code look wrong, after seeing the same type of thing over and over again, the pattern gets burned in your brain, any deviation is obvious. I'm a little OCD, and making things consistent feels better.

The problem is that convention taken too far leads to an ugly little town called boilerplate, and no one wants to live there. This is where convenience comes in, it allows you to more-or-less follow convention but allow yourself an out to skip over the obvious parts. The problem is trying to strike an appropriate balance. I have a story of failure and redemption that I will quickly share with you.

My new side project is written in phpand makes use of the ubiquitous associative array when it makes sense to. I love me some php, but I hate the associative array literal syntax, and it's not going to change anytime soon For those of you unfamiliar here is how you would make the same associative arrays, in json and php.

[javascript]
var example = {'a': '1', 'b': '2', 'c': '3'};
[/javascript]
[php]
$example = array('a' => 1, 'b' => 2, 'c' => 3);
[/php]

Doesn't look too bad or different, but if you have to have nested arrays or use an array in a function signature (my case) it gets a ugly pretty quickly. I have also been reading up on lisp a lot recently and they have an idea that successive arguments can act as a pair. I thought this was a pretty nifty idea, so I set about creating an alternative calling convention.

[php]
$foo->bar(array('name' => 'Matt', 'age' => 23));
[/php]

Would be identical to

[php]
$foo->bar('name', 'Matt', 'age', 23);
[/php]

I thought that this looked much nicer, and it is fairly trivial to implement

[php]
class Foo {
function bar($values) {
if(func_num_args() > 1) {
$values = self::associate(func_get_args());
}
...
}

static function associate($args) {
$count = count($args);
if($count % 2 == 1) {
$args[] = null;
++$count;
}
for($i = 0; $i < $count; %i += 2) {
$result[$args[$i]] = $args[$i + 1];
}
return $result;
}
}
[/php]

I had done it, bam, lisp style associative arguments in php. The problem though is that, well, wtf? That is going to be the reaction to any php programmer unfamiliar with the lisp convention. I failed to follow the conventions of the language, so this morning I tore this code out. It added a secondary way to call a function, and it also introduces several edge cases, the benefit is also dubious. I was scratching my language implementer itch, but not in an appropriate fashion. This time convenience had to be sacrificed for convention's sake.

api design

[caption id="attachment_76" align="alignright" width="197" caption="blinkenlight interface"][/caption]

I'm still working on my skunkworks side project, over the weekend I had the joy of integrating several third party php libraries. I got to spend a good amount of time on php.net reading over APIs and figuring out how to fit them into my project. Some of them were sublime, as though the author had read my mind and knew my exact mental model. Some of them were abominations, fighting me all the way. This got me to thinking about the design of a good API

What makes an API good? There are a few things that make an API really nice to work with.

1. Similarity of behavior - Writing an API that does searching through a b-tree? Look at how searching is implemented for arrays or strings, and then copy the crap out of that API. This allows the developer to use all that knowledge they've built up about searching, so if I know [php]array_first($needle, $haystack)[/php] returns the first instance of $needle in $haystack or FALSE on failure to find $needle, then [php]btree_first($needle, $haystack)[/php] should work the same way.


2. Readability - Your API should make code that is readable, the function names should be descriptive (without being overly verbose), and code written with it should flow nicely. Avoid using difficult to pronounce function names like strcspn.


3. Minimalism - You're writing an API because you are doing something non-trivial, something complex enough that you want a simple looking API to interact with, so do exactly that, make it simple. I'm sure that reflangulating the zyffer is a complex process that involves juxtaposing the allibaster and repeppering the kilgore while making sure not to narfle the garthok, but the reason you are writing an API is to hide that complexity away, don't write a Leaky Abstraction. Allow me to write code like this
   [php]
   $zyffer = new Zyffer();
   $zyffer->reflangulate();
   [/php]
   Not like this
   [php]
   $zyffer = new Zyffer();
   $zyffer->juxtapose(Zyffer::allibaster);
   $zyffer->denarfle(Zyffer::garthok);
   $zyffer->repepper(Zyffer::kilgore);
   $zyffer->reflangulate();
   [/php]
   

In the course of writing the API for my side project I've found it useful to put myself in the shoes of a new programmer trying to use my API. How long would it take them to figure out X? How often would they curse my name? What is the WTFs/min ratio looking like? It has been a helpful tool to adopt that mindset and ask myself, why am I requiring this parameter, why do they have to call this function before that function, and would that be apparent, why am I making their life so difficult. It has helped my slim down my API considerably, this combined with Convention over Configuration has led to an API approaching "not terrible."

Then it hit me, I have stumbled upon a big important rule, that I've implicitly been following for years, but now my brain is aware of it.

You should write everything like it will one day be a public API

Of course, like any rule that is written in absolutes, there are sure to be exceptions. But I think on the whole, it will serve you well for a few reasons. Public APIs are written to be simple to work with, which means that after you've encapsulated all the hard complicated stuff you can interact with a nice clean API. This will make your life nice when working with your API, but the best part is maintenance. Remember that new programmer we imagined to help write the API, that will be you, intrepid API writer, just a few short months after moving on from the API. You will inevitably be called back to add a feature or fix a bug, and if your API is simple to pick up, then you will remember more of it, and relearn the parts you forgot that much quicker.

Another advantage is that a well-written API is much more likely to encapsulate well-written code. When your API is separated nicely and parsed up cleanly between well defined units of work, the code powering them will probably have well defined separation of labor and understandable flow. The API itself becomes the documentation for how a process is accomplished, the various actors nicely laid out as well defined classes and the behaviors as fancy-pants interfaces. I'm certain you can create a clean API over a horrible pile of spaghetti code, just as surely as you can create a crap API over a beautiful collection of clean OOP, but a good API encourages good code.

At then end of the day the API is the face of your code. You can write up all the pretty documentation and how-to's and promotional websites with sweet web 2.0 reflection and fun oversized graphics, but when it gets down to brass tacks the developer is going to be instantiating your objects and calling your functions. The fact that there is a pretty floating cloud icon isn't going to make a developer feel any better that she just spent 4 hours figuring out that she forgot to juxtapose the allibaster and that made shit hit the fan when she called the promulgate function on the zyffer's subclown. Make sure that your code has a pretty face, even if its only you using it, the benefits will far outweigh the minor upfront costs

climbing a mountain

I've been done with my last big project for about a month now, waiting on the customer to make up their minds on some enhancements. It's one of those positions that seem enviable until you're a month deep in it, sitting around trying to find something to do. So I've filled my work time tackling whatever tasks have come by, writing up documentation, working on and completing two smaller projects, and reading up on technology. It passes the days but sometimes can be a bit of a bore, and the real challenge is trying to figure out how not to waste my company's time, trying to make value without a project can be hard.

[caption id="attachment_67" align="alignright" width="300" caption="Stratego... similar to strategic"][/caption]

Then my boss told me that I would be working on a new project. Huzzah! I'm leaving behind .Net back to the land of Java, thus keeping my streak alive of never working in the same language 2 projects in a row. The project position has been described to me as a strategic technical challenge. Let's break that down real quick.

Strategic - That's a fun word, what does it mean, dictionary.com defines it (after a great deal of military definitions) as

Highly important to an intended objective

So this is an important position, one which is necessary for intended objectives. From what I've gathered of the work I'll be doing, this is an apt definition

Technical - Nothing touchy feely here, no user interaction, no pretty style sheets, no fancy jQuery magic, just straight CS4xx type algorithms and number crunching. I'm have a feeling I'm going to have to dust off that Math degree sitting in the corner.

Challenge - This is the word I love, the thing I live for, a good challenge. I've been working on enough quick one week, one off, hit-it and quit-it projects as of late, I've missed the thrill of a good challenge.

I don't know how much I'm at liberty to share about the project which is why this post so far has sounded a little bit like a riddle. The one point of information I'll share is that the task I'm going to be taking on was sent out for quotes and the number that came back was... 10,000+ man-hours. You aren't misreading, and I triple checked that I wasn't mistyping. In case you aren't familiar with what 10,000 man-hours means I'll take a quick detour to explain it.

[caption id="attachment_70" align="alignleft" width="196" caption="The Mythical Man-Month"][/caption]

In one day, one programmer puts in 8 hours of work, that is 8 man-hours. One week is 40 man-hours. Assuming you get two weeks vacation, one year of development is 2,000 man-hours. That means that 10,000 man-hours of work is 5 straight years of development, half a decade of work. Of course, throwing some more resources at it you could hit 10000 man-hours with a 5 man team in just 1 year, more or less. The problem though is that this reduction in time is non-linear, you can't just throw 10,000 developers at the problem for an hour, this is pretty well discussed in The Mythical Man-Month.

In less technical terms that means that there is approximately a metric shit ton of work to do. And not grunt work mind you, complex problem solving work. This is a mountain of work. There are a few things that you can do when a mountain blocks your path.

1. You can turn around and go home


2. You can rent a helicopter


3. You can ready your climbing gear

Going home isn't an option in this case, the task can't be overlooked and other things can't happen until we are up and over the mountain. The outside quote was a helicopter, it costs some cash but is easy and relatively painless for you. Well we can't wait around a year to get this thing done. So that only leaves us with one option, so I'm gathering up some rope and pitons. I'm dusting off the part of my brain that remembers how Java works, remembering that strings can only be compared with .equals(), and fondly remembering the joy that is Eclipse after the ghetto of Visual Studio (holy war can begin now, be sure to insult my mother).

I'm preparing myself for what will be the biggest technical challenge of my career, and it fills me with happiness. It reminds me that I love this profession, that I'm stupid lucky to get paid to do it, and that I'm going to go in there and climb that mountain. From the little information I have, I'm sure the climb is going to be arduous, but I'm determined to plant my flag at the top, and I know that that will be amazing.

I want to challenge you to look around for your own personal mountains, its easy to avoid them, to walk away, go home, or rent a copter. You should instead commit yourself to the climb, because when you are at the top of a mountain, all the other problems are pretty small.

duck typing works

[caption id="attachment_62" align="alignleft" width="226" caption="Design Patterns in Ruby"][/caption]

I recently had the joy of picking up the book Design Patterns in Ruby which is a great book, made my Christmas wish list. If you have been living on the moon or are just one of those weird people that haven't heard of ruby yet, it is a dynamically typed interpreted language made famous by the web framework, Ruby on Rails.

Do you see what I just did right there? I gave you some background information, information that you more than likely don't need, you are reading my blog, you know what ruby is. Russ Olsen (author of Design Patterns in Ruby) does the same thing with duck typing. What is duck typing you ask? (again you probably didn't, you know what it is, or you are familiar with this thing called Google). Well it comes from the term, "If it walks like a duck, quacks like a duck, then its a duck" and according to the authority on all human knowledge it can be defined as the following

In computer programming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface.

One of the properties this gets you is the ability to pass any kind of variable anywhere and as long as it acts like the object the function is expecting it will work, no matter what class it is. In practice this allows for some great code reuse and as described in the book the evaporation of some class GoF design patterns. Olsen goes into a bit of depth about duck typing and the classic fear of someone coming from a strongly typed language.

So there is no compile time or static checks, what stops you from passing a database connection into a function that accepts an int or calling the give_big_fat_bonus method on a lowly worker instance instead of an executive instance, or of course the ceo instance which is a derived class of executive.
Anonymous Object Oriented Programmer

The answer is a resounding, nothing! (Dovetails nicely with my last post) Nothing stops you from doing this, go ahead see what happens. Depends on the language, in ruby you get a NoMethodError, and you get something like that in most duck typed languages.

The question then is how do people write any code that works, the answer is by not being full-on stupid. Ruby, and other duck typed languages, have taken to heart the sentiment of my post i can't stop you from being stupid (The shameless self promotion is getting a little rampant at this point). The language can't stop you from treating an int like a database connection, and really why should it. What in the hell are you doing mixing up an int and a database connection, you deserve to get punched in the face, a NoMethodError is the least of your worries.

But, accidents happen, so your db variable holding the database connection and your dv variable holding the direct value modifier amount are dangerously close on the keyboard. Are you going to just pray that you are 100% correct in your typing all the time, who do you think you are, me?! (I'll wait patiently as grammar-nazis and typoistas tear apart this post). This is why unit testing is so huge in duck typed languages, if you are using sweet delicious lisp then repl is your friend, ruby also has repl (irb) but its not nearly as integral to the development process as repl is to lisp. Your tests let you know that you aren't doing something stupid, and performs all the static and dynamic checking one could want.

So go out and give a duck typed language a shot, pretty soon you will enjoy the added speed, and you will realize that the static syntax checking is not necessary if you know what the hell you are doing.

the power of nothing

Sometimes you get a bright idea, or you are given a difficult task, and you want to start working on it. It's intoxicatingly easy to dream up feature upon feature for a new project, or build a giant architecture to solve a problem, all of these things look powerful. The other problem that we run into is that with software anything is possible with enough code, and the people that write code like to work on interesting complex problems. This leads to conversations like this.

• First Dev: I want this new project to seamlessly integrate with eBay, digg, and amazon


• Second Dev: Those seem arbitrary, maybe we should define some sort of data import protocol


• First Dev: Yea, then we just need a scraper piece that uses the adapter pattern to translate various websites into the standard import protocol and feed the project


• Second Dev: I'll start defining a dtd and you can start writing up some xslt for eBay and digg and amazon as a proof-of-concept


• First Dev: Great, then we just need to get the exchange support and the poor man's cron jobs running, to send out the periodic updates


• Second Dev: Should we integrate with Twitter?


• First Dev: I don't see why not


• Third Dev (interrupting): Wait, what are we developing?


• First Dev: Todo list


• Third Dev: Maybe we should get it to store and delete todo list items first


• Second Dev: That's the easy part, the multitouch integration is going to be the hard part


• Third Dev: ...

[caption id="attachment_57" align="alignright" width="300" caption="the dream"][/caption]

Before we get the product to do something simple, something any moron could code up, we want to start slapping on features and bells and whistles and whizbangs. This is all fun, because whizbangs are fun things to think about and code and its all well and good until you realize you have coded up a great feed reader, but one hell of a lousy todo list. Focusing on least viable product in the beginning gives you a concrete base to start applying whizbangs, based off of real world feedback and grounded in reality. Incrementally building up value off of an existing product is easier and makes more sense than building a giant all-in-one application that can do everything but butter your toast that no one wants to use.

So this leads me to the best way forward in my estimation and experience, nothing. Nothing is incredibly powerful, tremendously liberating, and allows you to get something up and running in least viability mode or to solve a complex issue.

I can hear it already, what the hell are you talking about you mentally unstable person?! Let's look at some tools that exemplify nothing and how you can go from nothing to something quickly and incrementally, but only if you start with nothing.

Mock-ups. What is a mock-up, nothing really, just a fancy picture of what you want an interface to look like. There are many great tools for making mock-ups, a new one that just hit beta is mockingbird, a more seasoned veteran in this game is the excellent Balsamiq. These shouldn't help, we all know what a todo-list looks like, why would mocking one up help us? Because, even though we are capable of abstract thinking it is much easier if we have a concrete model to wrap our heads around. Like I said in tool roundup

[talking about a sweet visualization I made] ...but it is a much better reference when working with this particular part of the system than looking at the database table and trying to assemble parts of it in my brain as the situation demands.

If we don't have a concrete model we force our brains to construct one on the fly, taking us out of our work and into a conceptualization, having a concrete model frees up precious processor cycles in our brain. A mock-up is also great because it allows us to take nothing, a simple functionless mockup, and quickly turn it into an actual interface. What does that button do? Nothing! What about that one? Nothing!!! Then we can slowly but surely add functionality to our interface until we have a functional least viable product. The way we got there was by understanding that we start with nothing and work our way gradually to something. Sure we could have sat down at the keyboard and pounded away code for a day or two and then run the thing, but the faster we can get it on the screen and start iterating over it, the better.

The second way that nothing can be useful is when tackling a huge problem, lets look at the contrived feed reader problem from up above. How in the world do we go about solving that?! Well we employ are good friend nothing, and we build from there. What's would the simplest possible solution be?

[php]
$feed = FeedReader::read("http://www.digg.com");
[/php]

We can throw that in our program, now we need to deal with the FeedReader class

[php]
class FeedReader {
static function read($url) {
return '';
}
}
[/php]

Nothing to the rescue, we just defined our super simple API, but we aren't quite to functional yet, but we have a foothold now to iterate from.

[php]
class FeedReader {
static $readers = array('http://www.ebay.com' => new EbayReader(),
'http://www.digg.com' => new DiggReader(),
'http://www.amazon.com' => new AmazonReader));

static function read($url) {
$reader = self::get_reader($url);
if($reader) {
return $reader->process();
}
}

static function get_reader($url) {
return self::$readers[$url];
}
}
[/php]

Wow look we are almost all the way to functionality, amazing!!! Thank you nothing (that's what EbayReader, DiggReader, and AmazonReader do). Now its the simple task of implementing those, which each by themselves is a somewhat complicated task and outside the scope of this blog post.

This sort of top down design though can be very helpful, you need to solve a problem, just pretend that you have an API that already solves it and start calling it, stub out those functions (stub is a fancy computer science term for "make it do nothing"), and watch the power of nothing first hand. Then you fill in the blanks, allowing yourself to have more nothing functions, you successfully break up your huge monster task into simple solvable tasks.

When we combine the first technique with the second we can get truly awesome results, we normally call this Rapid Application Development (or the much cooler 1980's name, RAD). RAD can get us to something faster than almost anything, using the power of nothing.

i can't stop you from being stupid

I've been working on a skunkworks side project that is nearing release, I'm down to the last 5%, which is the most difficult part. Suffice it to say that this project is a library meant to be used by other developers (and myself) to develop neat and nifty things. One of the most complicated things is trying to define an API that makes doing the right thing easy and doing the wrong thing really hard or impossible. The problem is that the as I pressed on I realized that almost half my code was checking for some stupid thing and protecting against it, attempting to build up state machines and syntax trees in the hope of reporting to you that you had done something stupid, and then it dawned on me, I can't stop you from being stupid.

The other thing that dawned on me is that I shouldn't try to, there are already built in errors that will point out what you did wrong, I don't need to add another layer on top. So I came up with some simple guidelines, applied it to my project, and watched unnecessary code melt away.

• Pick sane defaults


• Assume the developer knows what they are doing


• Make it easy to do the right things


• Make it hard to do the wrong things

The beautiful part of all of it is that my code became easier to read and understand, which means when there is a problem and you need to drop into my code (since it is open source), you will be able to understand what's going on. Let's take a look at what I mean with a contrived example, an HTML building library with 4 functions for clarity.

[php]
class HTML {
static function open_html() {
echo "<html>";
}

static function open_body() {
echo "<body>";
}

static function close_body() {
echo "</body>";
}

static function close_html() {
echo "</html>";
}
}
[/php]

The way I was programming did a bunch of handholding and sanity checks, which were nice but unnecessary. The class became bloated with so much state and sanity checking that it was becoming unwieldy, let's take a look at an overly protective incarnation of the above code.

[php]
class HTML {
private static $html_open = false;
private static $body_open = false;

static function open_html() {
if(self::$html_open) {
self::handle_error("html is already open");
}
echo "<html>";
self::$html_open = true;
}

static function open_body() {
if(self::$body_open) {
self::handle_error("body is already open");
} else if (!self::$html_open) {
self::handle_error("body must be contained in html");
}
echo "<body>";
self::$body_open = true;
}

static function close_body() {
if(!self::$body_open) {
self::handle_error("cannot close unopened body");
} else if(!self::$html_open) {
self::handle_error("body must be contained in html");
}
echo "</body>";
self::$body_open = false;
}

static function close_html() {
if(self::$body_open) {
self::handle_error("cannot close html, unclosed body exists");
} else if(!self::$html_open) {
self::handle_error("cannot close unopened html");
}
echo "</html>";
self::$html_open = false;
}

static function handle_error($message) {
//What is the correct behavior, should we attempt to fix the error, report it, who knows.
//We'll just stop execution
die($message);
}
}
[/php]

Those sure are some nice error messages and it keeps you from creating malformed html, but what's the point of it all. The browser is more than happy to tell you that your markup is invalid, or run the output through a lint checker or W3C validator, why should this class have some buggy half-implemented validator inside of it. The answer is, it shouldn't.

If you write stupid code you should get stupid results, Garbage In - Garbage Out. Library code shouldn't hold your hand making absolutely sure that you never make a mistake, what's the point of it. If you write the following code

[php]
HTML::open_body();
HTML::close_html();
HTML::open_html();
[/php]

You are clearly missing something about how markup works, it's not the library's job to hold your hand and guide you through this crisis, the browser will slap you in the face and you will have to learn something. Now there is a caveat, here we are generating HTML, there are great utilities for finding errors in HTML, so we don't need to reinvent the wheel. If you are writing a library or application code where if something goes wrong the error displayed is fine, don't reinvent the wheel. If the error is unacceptable you may need to do some error reporting.

At the end of the day you can't keep the user of your creation from being stupid, they are going to do stupid things and stub their toes and curse your name and you need to make sure that they had to go far off the beaten trail to do so, overlooking obvious better ways to do it, so that it's their fault and not yours. Make it easy to do the right thing and hard to do the wrong thing, but if the user wants to open the body tag before the html tag, let them, they have to learn sooner or later.

workflows and rough edges

I've got this new blog now, and I have to say that I'm a huge fan of WordPress, they definitely have created a great piece of software here. I'm going to split this blog post in two, half for my programming diatribe about workflows and the other half for some rough edges commentary about my new blog.

workflows

The idea of a workflow is central to software development, on a simple project your workflow might be

1. Type up some code


2. Compile or run the interpretter

This can get more complicated, here is a standard workflow with some sort of version control software in the middle

1. Type up some code


2. Build


3. If broken go to step 1, else go to step 4


4. Check-in changes


5. If necessary merge changes with server

It can get more complicated even still with test driven development.

1. Write up a test, make sure it fails


2. Write code until test passes


3. ...same as above

And when office bureaucracy get involved.

1. Write up test, make sure it fails


2. Write code until test passes


3. Perform necessary QA testing and UA testing


4. Submit changes for Code Review and Architecture Board Approval


5. Deploy changes to production environments


6. Monitor health of production


7. Pray nothing goes wrong, if it does, prepare for more reviews and meetings

[caption id="attachment_35" align="alignright" width="300" caption="flow"][/caption]
The job of a programmer is to program, the "Write Code" part of all the lists above, the job of a software developer is to be able to do that and navigate the lists of bureaucracies and procedures necessary to get their code into the user's hands. Now I'm not a bash the process person, source control, unit tests, and peer-review are all great ideas and serve necessary purposes. The great thing is that we are humans and our brains are amazing at internalizing these kinds of tasks and making them second nature.

There have been many times when I have been asked to document a process I've performed hundreds of times, only to sit there blankly looking at my empty word document, silently thinking to myself, "How do I deploy to production?" Normally I then do a dry run through the task, observing the stranger that takes over my body and knows how to do a production deploy. He works quickly clicking this or that, as I furiously document the procedure, he knows all of my passwords, even knows when to stop to ask me a pertinent question or two, but more or less he effortlessly autopilots his way through. I'll look at the documentation I've taken of the steps I just performed and sometimes the list is staggering, 23 steps, wow! In my head it is just 1 thing, do the deployment.

I don't currently have a major project that I'm assigned to, so I've been relegated to working on several smaller projects, none to complicated, the only problem is that I don't have the workflows down yet. This cause a major problem because there is a psychological theory of flow that states

Flow is the mental state of operation in which the person is fully immersed in what he or she is doing by a feeling of energized focus, full involvement, and success in the process of the activity.

None of the new tasks I have are difficult, on the whole they are very easy, but the fact that I can't get into a flow state means that I'm less productive than I normally am, and it is a little bit frustrating. This is why we dislike changing gears, this is why when someone distracts you even for a moment it can be dreadfully annoying, because getting back into that state of focus can be incredibly difficult. The new tasks I'm on are constantly pulling me out of the flow, I have to stop and think about how to get my code into this repository or that, what was the password again, where is the remote test environment, ugh.

When starting a new project or when documenting an existing project one of the most important first things to do is to set up your workflow. I'm going to have my source code right here (points at directory), and I'm going to edit it with this program right here (points at program), and when I've made a change I'm going to see if that worked by doing xyz, and when it works I'm going to submit my changes by doing abc. It seems like a stupid thing to do, you can easily figure out from one step to the next what you should do, but explicitly defining it in your mind will help you stay in the flow.

rough edges

New blog means spending some time making it feel like home. Here are some rough edges you will notice for a while until I get everything set up nicely again.

• In article links may link to the old ihumanable.blogspot.com version of an article instead of the new ihumanable.com version


• Syntax Highlighting is broken in many posts


• JavaScript is being escaped and so some posts that rely on JavaScript don't function properly


• The import process changed all my tags into categories, which feels wrong


• The css is the shine theme default, it needs to be tweaked to my liking


• The navigation on the blog isn't as nice as it used to be


• ihumanable.com redirects to ihumanable.com/blog/ which takes a noticeable second, I'm planning on putting a home page there that will also have links to software projects and other stuff I think is worthwhile


• Still need to learn the ins and outs and workflow (see above) of WordPress

The move was fairly painless but there might be some wonkiness for the next week or so, I hope to get some time over the next few days to clean up links, fix the scripts, and restyle some stuff, so bear with me.

If you have any suggestions or comments about the new blog now is your time to act, since I'll be doing work already, your suggestions might actually get implemented instead of just thrown down the memory hole like I normally do.