Lipsum Lives

[caption id="attachment_903" align="alignright" width="300" caption="This is how I feel right now"][/caption]

I've been hard at work for the last 7 weeks, sprinting from start to finish on a custom CMS system that will manage our company's various online publications. It has been a grueling experience. When I started my friend, Jeremiah, told me, you are crazy, don't do that, I'm going to hit you in the face. And yet, undeterred, I soldiered on.

I was not so foolish or prideful to think that other CMS systems were inferior or stupid or ugly. I use WordPress to power this blog, and let me tell you, WordPress is friggin sweet. Drupal 7 just came out, it looks to be an amazing piece of software as well. I looked at various PHP CMS systems, but at the end of the day I decided to build my own for the following reasons.

• I have no experience with developing for WordPress / Drupal / Insert favorite CMS here


• Time was at a premium


• Flexibility is a must

Now I'm sure there are Drupal Ninjas and WordPress Wizards out there, thinking, if you need to get a flexible website up fast, just use [the CMS system I know and love] and it will be easy peasy. I'm not against learning, and I try hard to fight NIH syndrome. The long story short version of this was I was told, "5 weeks we need A, B, C, D, E, F, G and it needs to work in X, Y, Z, (several other alphabets worth of letters) ways." I knew that I didn't have the time to climb up the learning curve of a CMS and become a WordPress Guru or Drupal High Priest. I did have time to write software, and since I know PHP and Flourish like a fat kid knows cake (aka, like I know cake), I decided that the route with the highest possible success was to write something.

The next most important point is that I work for an amazing company. We are growing fast and the future is anything but certain. Ideas for the publication sites come fast and furious, sometimes radical, sometimes insane, sometimes brilliant, having total control of the software is a must. Getting something up was important, but building a solid foundation that would allow us to build new engaging features in the future was even more important. With our custom framework we will be able to pivot and experiment in a way that we simply couldn't with off-the-shelf CMS software given the engineering skills at hand.

Here's what I learned building a "successful so far" CMS system. CMS systems are hard. They are really really really friggin hard. They are hard in stupid ways and hard in smart ways. Getting it just right is impossible, getting it half-way good is a tremendous challenge. Every feature is 10x more complicated than it looks on the surface and you won't know how until you talk to the person that cares about that feature. Every feature, no matter how much you talk to people while building it and no matter how much you model and poke and prod, is going to be wrong in some way. You have to simultaneously be so cocksure that you look at this mountain of a task and say, I can take this on, and humble enough to listen to bug reports, user frustrations, and endless visual tweaks so that it's actually worth using.

I have a new found respect for the tremendous task software like WordPress and Drupal and Joomla and so many others try to tackle. I want to go and learn one, Drupal 7 looks nifty, just to see how the pros do it. I have also been reminded that even when you are Inventing it Here, you don't need to go NIH crazy. Building this software was an exercise in standing on the shoulders of giants. Our avatars are powered by Gravatar which easily has the most straightforward and easy API ever. Our comments are protected by ReCaptcha cutting edge captcha technology that also helps read unOCRable books. Our email campaigns are being handled by the amazing MailChimp. And of course the whole project is built on my favorite unframework, Flourish.

So now it's live, you can go view the fruits of my labor at DIG Baton Rouge. It's the first of four sites that will be moved over to the new Lipsum CMS engine. I will spend the following weeks, tweaking, adding new features, fine tuning and bug smashing. I'm incredibly proud of what our small team was able to accomplish in such a short time. Lipsum represents a solid technical foundation on which we will build the future of our publication sites. It was not easy, but it was necessary, and I for one am glad we took the road less traveled. As Joel Spolsky famously wrote, "If it's a core business function -- do it yourself, no matter what."

freelance

[caption id="attachment_875" align="alignright" width="300" caption="I\'m not sure what this has to do with Freelancing, thanks Google!"][/caption]

I've been hard at work making sure that Faveroo keeps humming along, pushing out new features here and there and making sure everyone is happy. The growth we've seen has been steady and we are continuing to push forward. Work has been a little hectic lately, if anyone ever tells you that growing a product and a brand is easy, you just punch them right in the face, I don't care that it's your Grandma, do what I say, I'm a blogger. Anyways this blog post isn't about my 9-5 it's about my 6-10.

I do a lot of cool side projects, some of them like Prosper are just for fun, they scratch an itch I have and I think I might be able to use them later. Some never see the light of day, they lose momentum or turn into something that I can't figure out how to make any money or help anyone with so they just languish on my hard drive. Lately I've been trying to break into the freelancing stuff, I thought, how hard could this be, I have years of experience, I'm willing to work for a reasonable price, I write high quality code really fast, and I tend to think that I'm a fairly nice person to work with. Well its been an uphill battle so far, but I want to share my thoughts with you.

Online freelance boards just suck. They do, I hate you, I hate them, hate is really the only emotion I can feel about these boards. 90% of the posts are for ridiculously ill-defined or impossible jobs, 99.999% of the replies are someone in some third world country willing to write the new facebook for a shiny new nickel, and there is no way to get the point across to anyone that you get what you pay for. I've yet to find any useful paying work from these sites.

Craigslist is a mixed bag so far. When I started on this journey I got some help from a friend who basically just spammed me with every blog post they could find on the subject of freelance consulting. A nice thread on Hacker News, here's a Two part series by Ryan Waggoner on freelancing. One of the pieces of advice was to use Craigslist, so I tried that, there is a ridiculous volume of stuff in the Computer Gigs section, and since you aren't constrained by geography you can apply to a huge amount of them. I tried this for a few weeks but after sending out tens of emails, most custom written specifically for the project and only receiving one reply, I decided to try another avenue. I'm not giving up on the Craigslist approach yet, if for no other reason then the absolute mass of postings there.

Friends of friends. Right now this is where I've seen my only success. I'm currently working on a project for a friend of a friend and so far things are looking great. I'm enjoying the work, sweet lady PHP with my favorite unframework, Flourish, and I'm making great progress. It's been a good project to work on so far, the client has been great to work with, and the code has been flowing. I'm sure there's a downside to to this course, friends of friends will want you to do extra work for them for free or change things up as a solid, but so far so good.

Future plans. This blog post is part of it, but basically some self-promotion is in order. I'm going to be putting together an online portfolio of stuff I've done, Prosper is a nice large code base for someone to look at to see the quality of my PHP code. I realized I just need to get my name out there as an option, the current project I got was because I happened to tell my friend I'm trying to start consulting on the side, bam, first paying gig. I'm going to keep fighting the good fight on Craigslist, maybe come up with a more efficient system for submitting my name into consideration for postings.

If you need a good PHP programmer drop me a line in the comments or send an email to ihumanable@gmail.com and maybe I can write some awesome code for you.

saying nothing

[caption id="attachment_838" align="alignright" width="326" caption="Thank you, very helpful"][/caption]

As a software developer I have the great joy of making things that piss people off. Nothing makes someone angrier than after an hour of meticulously entering financial data, you hit the submit button and get back the error pictured to the right. In actuality, good software developers go to great pains to make their systems as resilient as possible. A large amount of the code written for any complex piece of software will be dealing with error conditions, recovering from the recoverable, and gracefully reporting fatal errors in a way that indicates exactly what happened and why and what to do about it.

One of the best features an error message can have is Google-ability. If the error has a prominent and distinct feature, then a Google (insert your favorite search engine here) Search will be able to deliver relevant results with a much higher degree of accuracy.

Good Error Message Example:

Error ERS1708A - Invalid Character
The field "Zip Code" should only contain numbers, "47CD101" is invalid.
Examples of valid input: 44130, 90210, 43220

Bad Error Message Example:

Form Invalid!!

The first error might be a little overkill, but the first part meets are Google-ability test. Typing ERS1708A into Google will pull up solutions for this problem. The error goes on to report exactly what went wrong in a user friendly way, it even goes so far as to give some samples of what correct input looks like. The second example gives you a vague idea that something bad has happened, something dealing with a form, and if that form happens to have 100 fields on it, well good luck figuring it out (maybe it's even one of those awesome programs that will clear your entire form on error). The problem is that you are equally likely to get either of these error messages from the same input depending on how much effort the programmer put into error recovery and reporting.

We all know that we should strive to help out the end-user and give them good error messages (even better if we can have robust recovery so we don't have to report errors). The problem is that programmers often have to be toolmakers as well. Sometimes when making a tool for other programmers to use we let our error reporting skills slip, these aren't mouth breathing users, these are 1337h4x0rz and will find your undocumented command line tool completely natural. I ran into this mentality today while troubleshooting a Windows Service installation problem. The Service works great on my development machine, works fine on other peoples laptops, but on the security locked down Mobile Data Terminals that the Service actually has to run on, no dice. The error that was being reported during installation was...

The system cannot execute the specified program

This fails in every way imaginable. Completely un-Google-able, go ahead and try it, you will find everything from people installing things on WinPE, to game development, web controls, Server installation.... a deluge of crap. The error message is amazingly useless, the system can't do something, what can't it do, I don't know, it doesn't even tell me. Can it not execute the install.bat file, maybe the executable called by the install.bat, maybe the executable that executable invokes, who knows?!

Well being a clever programmer and systematic debugger I was able to isolate each piece working towards the fact that what the message actually should have said was, "Although I have all the permissions necessary to access and execute all the programs you want me too, one of the programs requires that the system have the Microsoft Visual C++ 2008 SP1 Redistributable Package (x86) installed on the machine before you try to execute it. If you install that, then I can find the dependencies I'm missing and everything will be cool, kthxbye!"

Error messages are one of the few times that we as programmers get to actually directly communicate to the user. Don't waste this opportunity by effectively saying nothing to the user. The error message is a traumatic thing for most users, happily working along until it pops up, keep it short, sweet, descriptive and helpful.

sitting

[caption id="attachment_827" align="alignright" width="300" caption="In case you didn\'t read it: None of Us is as dumb as All of Us."][/caption]

<rant>

There is a lie we like to tell ourselves that goes something like this, "I am in control of my life." You are not, it's a cultural lie though, one of those things that we've all agreed makes us feel better so we politely nod our heads and lie to each other and all get through the day a little easier. It's similar to that radical 90's style lie I grew up with during school that went something along the lines of, "Everyone is special and magical and made out of ponies." Everyone gets a trophy, even if you didn't win, because you are special just for trying. It's a lie that everyone knows is a lie, but it makes people feel better, so what's the harm.

Well over the last few months it's become apparent to me that these things are lies, that I'm no more in control of what goes on in my life then my dog is of his own. At any moment someone could walk through the door and kill us all just for laughs, or someone could hand me a million dollars. The uncertainty of it all is unsettling and exciting. The one thing that has slowly crept into my life recently is the subject of my post, and that is the hallowed institute of meetings. Somehow I went from a code ninja hacking down mountains of whatever was thrown in front of me to a table jockey. I am now scheduled to attend no fewer than 8 meetings per work week.

That works out to 1.6 meetings/day, and rest assured there is at least 1 meeting everyday. I know people that do nothing all day but meetings, it boggles the mind. I would say that I "hate" meetings, but that really doesn't convey the depth of my loathing for them. Meetings are the bane of my existence, I could stomach maybe 1 a week, a short sweet status meeting, but what I can't handle is the bread and butter corporate meeting, and I'm going to detail why.

1. Marketspeak: I don't know what the fuck happens to people when they get in meetings but everything, everything gets some fancy new moniker. Sure I could say that I'm working on some stuff, but wouldn't it be better to say that goal-forwarding has been achieved on several Tier-1 action items. There are no people anymore, just assets and resources, dehumanized by the cruel tongue of Marketspeak. Every concept, no matter how simple, is wrapped in shiny new clothes, as though you might confuse the dog turd covered in whipped cream as a delicious fudge sundae.


2. Teleconferencing: There is always some person (sorry, meant to say "stakeholder touchpoint") who thinks its appropriate to call into the meeting on a 1992 Nokia Celfone while in the middle of a monsoon while driving through a tunnel. Bonus points if this person has a thick accent that makes even high-fidelity communication less than understandable. The other issue is having 10 people on one side of the conference line and 10 on the other, and the fun of each side trying to figure out who is talking.


3. Pointless Status: Managing people is difficult, so management is obsessed with the idea of status. The problem is that status is normally not quantitative but qualitative, too bad that doesn't fit into the Holy Spreadsheet. People are forced to turn things into percentages, I'm 80% done reflanging the spline, even if that doesn't make any sense. Status reports are a proxy for involvement, they are a way to feel connected to a process you don't have any part in.


4. Workturbation: Meetings are insidious, if you have 8 hours of meetings you will go home mentally exhausted, more than likely frustrated, and with a feeling that you've really put in a hard day's work. The problem is that when you try to figure out what you've actually accomplished the list is either non-existent or incredibly paltry. Meetings feel like work but rarely accomplish anything of actual significance. Meetings also have the incredible ability of spawning meta-work just for the sake of the meeting itself. Before the meeting you have to prepare whatever pointless agenda you have for your meeting, secure a place to have the meeting, work around everyone's schedules. Afterward someone prepares the minutes and it gets emailed to the companyname-all mailing list where inevitably the tiny list of things that were actually decided are rehashed again and again. Personal bikesheds are floated around, and more than likely all the chatter on the email list will require ANOTHER FUCKING MEETING to sort out.


5. Punting: Punting is the amazing technique where something interferes with the meeting so it is moved down the road to another meeting. Punting is sometimes necessary when someone wants to discuss in detail some trivial point, but the best and most frequent kind of punting is the following. Design meeting is convened, difficult non-trivial thing is brought up that requires actual thinking and problem solving, meeting participants get frightened by the idea of actually having to dust off the critical thinking part of their brain instead of blithely parroting made up statuses of underlings, issue is tabled to be dealt with in the future, a piece of me dies inside. Keeping the meeting moving becomes MORE important than getting anything decided or accomplished.

The list is in no way exhaustive or universal. At the end of the day though, a meeting is taking up time you could be doing something, they should be looked at that way. All that time you are sitting in meetings talking about all the stuff that could be getting done if everyone wasn't stuck in meetings all the time... my head hurts. What we need are people with their hands on the problem empowered to make decisions and move things forward. Then we can meet once a week to talk about how things are going, instead of sitting around all week wondering why nothing is getting done.

</rant>

macgyver

[caption id="attachment_781" align="alignright" width="216" caption="All I need are these everyday objects — a toothpick, some liquor, a gun with no bullets, bullets, and three of my MacGyver writers. "][/caption]

So it's come to this, a blog meme post. Every so often in the hallowed world of writing content for free to be distributed to people that you don't know for no particular reason other than hearing your fingers clickity-clack, you meet someone else doing the same thing and they challenge you. Considering that this is a hobby built upon the sturdy foundation of honor, glory, and red bull, you must rise to meet this challenge, and write a themed blog post. David Stein started this and it continued to Brent Ozar who has called out my name in the darkness, to answer the call to write about: My MacGyver Moment.

Now I've written about my favorite MacGyver Moment before in a previous post toolmaker, the time that I wrote an automated PL/SQL to T-SQL translation program. In that post though the MacGyver moment played a small roll and didn't get the attention it deserved, so I will go into its history and explain it in greater detail.

The seed is planted

[caption id="attachment_782" align="alignleft" width="300" caption="This is how MacGyver would solve most of his problems in the future."][/caption]

It was my last semester in college, I had finished up all the required course work and just needed to take some 400-level electives to finish off my degree. I looked through the catalog and found a course that looked interesting, "Language Design and Implementation." The course seemed exciting for a few reasons, the professor had worked as a professional compiler writer for years and was a decided brilliant and fun teacher, the course work included building a compiler, and compiling source code was a part of the development chain that I had a theoretical understanding of but was still somewhat of a blackbox. I signed up for the course, it was fun, it was hard, I built a compiler (for a made up language called LITTLE targeting the JVM), and I learned a lot. I learned about parsers, tokenizers, compiler optimizations, bootstrapping, and a hundred other interesting concepts. The value that I took away from the course was a litany of problem solving techniques and an appreciation for what runs under the gcc hood.

The problem

My company sold some software. Doesn't seem like a problem at first, in fact, it's how the company does all those little things like pay my salary, so it's actually a good thing. The problem was that the software was written to run on an Oracle backend and we sold it to an organization that only had a SQL Server backend. There we sit looking at a mountain of PL/SQL code that does everything from execute a simple search to closing out and actualizing a fiscal year. 60,000 LoC representing tens of man-years of effort. Take this and put it in an extremely aggressive timeframe for porting and limited resources... suddenly the problem is easy to understand.

Bubblegum, paperclips, and string functions

60,000 lines of PL/SQL needed to become T-SQL, looks like compiling code to me. I knew my source language and my target language, time to write up a parser, tokenizer, emitter, etc. Then I noticed something while scanning through the PL/SQL, the problem domain was constrained, although PL/SQL can be constructed in a number of permutations all syntactically different, this was not the case. This PL/SQL was very uniform in construction, I didn't need all the pieces parts of an actual compiler. I didn't need to tokenizer and parse into abstract syntax trees for emission into the target language, I could just cheat, there were only 3-4 syntactic structures present in the source file.

I began writing the translator, after quieting that part of my brain that was telling me I was wasting time, just trying to avoid the unpleasant task of translating this by hand. Hackity hack hack, code is complete, hack up some nifty progress bars, run run run... translation complete! I looked at it, not quite perfect but I would estimate that the success rate on translation was around 99%.

The lesson

Sometimes hacking up a kludge to transmogrify data is the best course of action. Shutting up the part of your brain that doubts your ability, or the part telling you to do something the "right way" is the best way forward. Learning something for no other reason than that it seems interesting can provide you with the tools necessary to solve thorny problems down the road. Sometimes you really can defuse the bomb with a rubber band, #2 pencil, and thumbtack.

blackbox

[caption id="attachment_751" align="alignright" width="241" caption="Something happens in there... with gnomes I think."][/caption]

The Blackbox, cornerstone of computer science, thing of mystery and beauty. It is the fundamental concept that allows us to build bigger and better abstractions, making life easier for everyone. A blackbox is something that has well defined inputs and outputs but the manner in which it functions in unknown. The classic example of this is a library function, sqrt(9) takes the number 9 and returns the square root 3. There is a well defined input (in this case 9) and a well defined output (the square root, 3). How does the function produce this result, who cares?

As programmers we can focus on what we are good at, if someone loves programming math algorithms they can make a kickass math library, if they love web programming, here comes Rails. Then when I'm interested in writing a web app that needs to do some math, I don't have to worry about how to find the square root of some number, there is a handy blackbox all ready to go. This is a huge boon to productivity, I'm happy, math programmer man is happy, and the user is happy because they don't have to use my buggy homegrown math libs that say they owe a $47 tip on a $9 check. (And my math degree weeps at me)

This is all well and good, the problem comes when you need to know what's going on in a blackbox. There are any number of scenarios in which you may need to shine light into the depths of a blackbox. You have some strict performance or memory requirements, the blackbox's "well-defined" outputs are failing (what do you do when you call out to sqrt(9) and get 7.2 back) or the use of a blackbox is having some negative side-effect. In these situations the whole damn thing comes crumbling back down to earth, you now need to roll up your sleeves and get it done. Here is how you should approach the problem.

• RTFM (Read the Fucking Manual) - This is the first step, if a blackbox is eating up too much memory see if there is a way to force a limit, if it is throwing an exception, check why that could be the case. Basically before you do anything else you should try to make sure that the blackbox is actually broken, because chances are that any sufficiently popular blackbox is going to have much more real-world use and rigorous testing than your new client code. This boils down to, don't blame the compiler for your shitty code.


• Ask someone familiar with the blackbox - After you've actually RTFM you can move on to this stage, seek help from someone more familiar. If there are colleagues that have used this particular blackbox ask them first, they may have already gone through this entire list and can shortcut you to the answer. If you have no colleagues to turn to, try turning to the community or the creator, if that fails try throwing it up on a more generic community like, stack overflow.


• Go to the source - If you are working with an open source blackbox then it isn't really a blackbox, just go read the code and see what it's doing. This is much easier said than done, so make sure you have exhausted the first two steps before diving into this one. If you are using a closed source blackbox you can either attempt to get the source through some sort of development channel (although this is unlikely) or you can decompile and look at that. Tools like RedGate's Reflector or the Java Decompiler Project can help to get at that closed source.


• Workaround - Sometimes no matter how dogged your determination, you can't seem to figure out why the blackbox isn't working right, at this point you have exhausted the detective route, time to pull a MacGyver. If you can figure out someway to use the blackbox and workaround the deficiency then go ahead and do that. The important thing is to take this workaround and record it properly. Create an adapter, thin wrapper, or function. Make your own lightweight blackbox that is 1 part the old blackbox and 1 part workaround aggregated together.


• Roll your own - Sometimes you get seriously desperate, you have researched your fingers to the bone and tried to work around the problem. You are at the end of your rope, the deadline is looming and the only thing standing between you and your project compiling is some pos third party closed source blackbox. Sure it would be nice to have a sqrt function nicely wrapped up for you, but hell you can dust off that part of your brain that remembers what a square root is and write your own function to find it. The important thing is that this should be the very last thing you do, after all other options are exhausted. Many a programmer will do this first or second, and that is Seriously Fucking Wrong©.

The power of a working blackbox is only matched by the frustration of a wonky blackbox. The good news is that a lot of blackboxes work really well sitting there quietly trimming our strings and rooting our squares. There is that every once and a while though when despite our best efforts we have to dive headlong into a blackbox and shine some light around.

telecommuting

[caption id="attachment_739" align="alignright" width="300" caption="Just let it warm up for a minute, you will be fine."][/caption]

I am not in Indiana today, the trip has been postponed until the Snowpocolypse is over. I awoke this morning to find another 6-9 inches of snow had fallen on top of the unmelted foot and a half of snow already on the ground. I groggily turned on the TV, Columbus is under a Level 2 Snow Emergency and just about everything is closed. I check with the Franklin County Sheriff Office and see that for a Level 2 Snow Emergency the situation is as follows: "Roadways are hazardous with blowing and drifting snow. Only those who feel it is necessary to drive should be out on the roadways. Contact your employer to see if you should report to work."

With the threat of hazardous roadways and the fact that my job in no way, shape, or form requires my physical presence at any location, I shoot off an email, following the sheriff's advice to "Contact your employer to see if you should report to work." The sad news, of course, is to put on your business casual, bundle up, clear off the car, fight throw the blowing snow, and carry your laptop with you so you can plug in and get to work. The alternative of course would be to stay nice and warm, not endanger my life and property, and plug in my laptop at my desk at home. This alternative is clearly unacceptable, against the advice of the Sheriff's office and against all common sense I physical transported myself and my laptop several miles down the snow covered roads to plug in here at work today.

Here I am doing the same work I could be doing from home, as far as I know, the SOA book that I'm studying and the Message Broker Toolkit examples I'm working through are unaware of my physical location. I can easily access my corporate email, sametime account, and all relevant files from home, I can even participate in the conference call meeting that we have scheduled for 1 o'clock. The question is why, in spite of the rest of the city shutting down, was I required to come into work today? The rest of this post will present the Case For Telecommuting, the Case Against Telecommuting, and an Analysis. All of these sections are written with programming or some other suitable telecommuting profession in mind.

The Case For Telecommuting

Technology has long been on the march for decentralized development. Being able to reach resources from any location is more and more possible with web based solutions and DVCS. In addition to having resources available from any physical location our ability to communicate and collaborate despite physical distance has been steadily improving. Tools like Google Wave, Instant Messaging, and Collaborative editing have matured and become rich and expressive mediums. Phone conferences are now common place when working with outsourced resources and with physically distant resources.

For the particular case when travel is dangerous telecommuting has a distinct advantage over commuting. Commuting is another part of life that is ever growing.

In fact, so many people now commute more than an hour-and-a-half to work, the Census Bureau has given them a new name: Extreme commuters. ... It's growing five times as fast as the general growth in commuting — about three million, 3½ million [commute] more than 90 minutes a day, about ten million more than 60 minutes.

Commuting also has a negative effect on your physical, mental, and emotional health. The impact of Voluntary Solitary Confinement (VSC) are not pretty.

Long hours of commuting, especially if you’re driving, is associated with high blood pressure, musculoskeletal disorders, increased anger and resentment at work... Long commutes can also increase the risk of heart attacks, flu, depression etc.

- Effects of Long Commutes to Work

The Washington Post has an article from 2007 discussing the problems with long commutes.

As a consequence, more drivers will probably suffer the health effects of a commuter lifestyle, researchers and doctors said. "You tell someone they need to exercise or go to physical therapy, but how can they? They leave at 5 a.m. and get home at 7 or 8 p.m. at night," said Robert G. Squillante, an orthopedic surgeon in Fredericksburg who has treated patients for back pain and other commuting-related issues.
...
"It's just tiring," Hutson said of her daily drill. Someone who was never much for caffeine, she now bolsters herself with coffee in the morning and soda for the evening rush. But by midweek, "I'm running on fumes. That's the biggest toll. It's not enough sleep."

No one pays for the commute except the worker, and as we saw during the gas price spike up to $4/gal (even now its not much better, only because we have that high water mark to compare against does ~$2.50/gal seem reasonable) the cost of commuting can become a rather high "work tax." The Washington Post article hits on it briefly, but if you have to get up early to commute and get home late because of a commute you are also paying the price in terms of your time. You may only be getting paid from 9 to 5 but you are effectively giving your job all the time from the moment your commute begins in the morning to the moment it ends and you are home. If you have to adjust your sleep schedule to support this then any time you could be awake, doing something you enjoy, actually living your life, is forfeit to the needs of your employer and not you.

The Case Against Telecommuting

If telecommuting gives the working man more free time, more money in his pocket, and better health, why not let everyone telecommute? The arguments against telecommuting are many and we will examine a few that I have encountered.

The problem of split attention. When you are face to face with someone you know that they are giving you their full attention. With an IM or phone call the employee could easily be working on something else at the same time. They could be doing something work related, reading documentation, writing some code, discussing a problem with another colleague, but whatever it is they are splitting their attention and multi-tasking, and sometimes that is unacceptable.

The problem of accountability. This is a corollary to the problem of split attention. Split attention isn't so bad when the attention is split between listening to a meeting and writing code, it is pretty awful though if the attention is split between listening to a meeting and playing World of Warcraft. Keeping people accountable for the time they claim to be working is more difficult if they are unsupervised and physically distant.

The need for humanity. Often times phone conferences and IMs are more than enough to meet our communication needs, but this is not always the case. Sometimes there is no substitute for "pressing the flesh." Important meetings demand a level of attention that necessitates being physically present.

The problem of ineffective communication. As good as the tools have become there is still a gap between talking with someone, being able to read their facial expressions and body language, to fully understand what they are communicating. Telepresence and other techniques are getting us closer, but a face-to-face conversation conveys a wealth of unspoken communication.

The problem of bureaucracy. I am no fan of bureaucracy and there is a certain liberating power to having a team physically assembled. Difficult technical problems can be jotted out, ideas bounced around more easily. Communication tools are great, but still don't beat a whiteboard that everyone can see or sitting down with another programmer and sketching something out on a piece of paper. These impromptu mini-meetings are often incredibly productive, and they can be difficult to have when physically separated.

The problem of facility. Some of us have great work space set up for working from home. Fast internet connections, powerful machines, quiet space. But this isn't universal, some people can't work from home for one reason or another. A lack of connectivity or acceptable development machines, an environment that is not conducive to productivity, or some other issue. Some people require a more structured working environment in which to be productive. The lure of facebook and Judge Judy prove too much and workers that could be productive in an office environment fail when working from home.

Analysis

Telecommuting is increasing and is getting better. For now it is an option but societal and economic pressures and an increasingly agile work environment will make it more and more necessary to effectively optimize one's workforce. Many of the concerns and arguments against telecommuting boil down to a deficit in trust. The alternative though, longer and longer commutes coupled with increasing transportation costs is untenable.

As with an paradigm shift, new etiquette will have to be formed, the way we think about work and home will have to change, and tools will have to be built and improved to rise to the challenge. Although we shifted to the information age and a knowledge based economy in many ways, we continued to hold over traditions from a bygone era.

Telecommuting will take some time to get right, for anyone interested in adopting it I think a blended approach is the way to go. Start off small by offering anyone who would like to try it to telecommute one day a week or month. The important part will be follow up, getting feedback on what worked and what didn't. You can use this feedback to fine tune the experience and come up with a manageable system. Physical presence will always have a place in the workplace but as we move into a future with increasingly global concerns and clientele, increasing transportation costs, and a better understanding of the physical and emotional ramifications of chronic voluntary solitary confinement, telecommuting will become less science fiction and more day to day fact.

leaving on a jetplane

[caption id="attachment_735" align="alignright" width="300" caption="Oh captain poopy-pants is cranky, he didn\'t get his FAA regulated maditory nap-time."][/caption]

Well as Central Ohio prepares for another 4 to 8 inches of snow today I just printed my boarding pass for tomorrow. I'm off to the most exciting place on earth, Indiana. I will be meeting up with the newly assembled SOA team, hopefully hitting the ground running making all kinds of fun message flows and translation logic. I'm excited because when I wrote climbing a mountain on November 6th, 2009 this was the mountain I was talking about. The road to get started has been mired in difficulties so far, we've accomplished a lot just none of the programming yet. This week will hopefully be when the tools get handed over and I get the keys to dad's car, so to speak.

What this means for me is the joys of post-9/11 nudity body scanners to take a 1 hour flight to Chicago to drive 45 minutes to a small town in Indiana. What this means for you is that between the jet-lag, long hours, corporate face time, synergy backflow, and spotty hotel internet there may not be any blog posts for a while. I know that this is a sad announcement, whatever will you do without me ranting everyday about technology?! Well this is my 91st post, and if you haven't been a reader from the beginning I suggest you dive into the archives, there are a ton of fun and interesting posts back there. If you don't want to dig around in the guts of this site like a blind man searching for a nickel, then look at the top of the right-hand sidebar, those are the most popular posts by hits. Other people, sentient human beings read those posts a lot, had nice conversations about them in the comments, and in general found them to be entertaining or informative.

Unlike other blogs comments never close at ihumanable.com and I haven't become jaded enough to stop reading the comments. Every time someone has a witty thing to add to the conversation I get an email about it, I read that email, sometimes I reply in the comment thread, if you are extra special lucky you may even get an email reply.

I hope to be able to post at least once in my absence, maybe while waiting for the snow to get cleared so that we can leave, but it may not happen. Fret not though, there is a wealth to read and enjoy, most of the stuff I write isn't really time sensitive (although some of the bad jokes might be). Thank you for reading, once I take care of this pesky job that allows me to pay for the hosting that provides you with free entertainment, I will be back to write more.

baby steps

[caption id="attachment_714" align="alignright" width="200" caption="Today I conquer steps, tomorrow France!"][/caption]

Last night I had the unadulterated fun of updating a client's blog. After handing off the reigns everything was going smoothly until some unknown combination of plug-ins was causing the mobile version of the site to be served to actual browsers. I was called in to triage the situation, nothing too dire, just two plug-ins with some configuration problems, once I told them to play nice, all was fine. It was then that I noticed there were 15 plug-in updates waiting and the whole WordPress install needed to be updated. So I took the time to take a backup and update everything, nothing too serious (for anyone that doesn't use WordPress, updating is a one-click affair now) just the day to day stuff that needs to be done to keep the lights on.

This got me thinking about upgrading and Google's decision to discontinue support for IE6. Some people are crying foul, saying that IE6 should still be supported, I'm decidedly of the I-hope-IE6-dies-in-a-fire camp. Lest this become a let's bash on IE6 rant, which it so easily could, fucking broken box-model CSS1 piece of shit, let's get back on track. The people that are upset about this are not technical luddites clinging to the familiar. Digg performed a user survey to figure out what they should do about IE6 and part of that was asking why people used IE6. Of IE6 users 70% responded that they either lacked administrative rights needed to upgrade IE6 or were told by someone at work that they could not upgrade. For anyone that has ever worked in Corporate America this should be no surprise to you. The nagging question though is why?

Why would the IT department, home of nerds always hell bent on trying out the latest and greatest, demand that you use a 9 year old piece of software? There are a number of reasons people point to.

1. Mission Critical application is only compatible with IE6


2. IE6 comes installed on all the machines


3. IE6 is maintained by Microsoft, less of a headache for us


4. Licensing / Partnership / Legal-mumbo-jumbo

If you fall into #4, I pity you. #3 is laughable because although the updates might be bundled with Windows Updates you aren't saving yourself any administrative costs by putting the most targeted web browser for malware on all your machines. #2 is true, but if you are big enough for an IT department you probably have some way to push software to machines (or you could, and this is a shocking idea, trust your employees to choose a browser that fits their needs). I really, really, really want to talk about #1 though, because that is the point of this whole post.

I've worked at a number of places that used some sort of Web Application that was deemed critical that only worked right in IE6. This was then used as a justification for never moving past IE6, the timecard application won't work or the spline flanger won't have those cool IE6 exclusive filters. I understand that no one wants to expend resources on a "working" application pulling it into the future. But what is the longterm idea here, are you just going to use IE6 forever? This prevailing consensus seems to be ultimately self-defeating. Google recognizes this and as they try to push what can be done with the web they find themselves expending far too much time and energy on a nearly decade old piece of technology.

The nice part about fixing an application on IE6 is that you avoid some cost in updating it to make it work with other browsers like IE7. You could have spent a week or two adding in some CSS fixes and some markup changes and bingo bango IE7 now works like a charm. Instead you decide to use Corporate Fiat to force the continued use of IE6. Well now IE9 is looming around the corner and instead of paying a week here or a week there you now have a mountain of technical debt to pay down to get it to work.

Although IE6 is a convenient example, we can see this happen in all parts of the software development world. This program compiles when linked to an older version of the library but not this new one, I don't have the time to figure it out, well just link in the old one. This is all well and fine to meet the deadline but you have just accrued some technical debt. Later when that library gets some great new speed-up or killer new feature, there you will be plodding along with version 0.91 because you can't get it to work with the new hawtness. The problem is that these debts are not additive, they compound on one another and make what would be a simple conversion to 0.92 a nearly impossible conversion to 3.76.

Keep your tools up to date, keep an eye on what's up and coming, you don't have to be the first to jump on the latest and greatest but when a new version has proven itself and become the new standard move to it. Technology doesn't stand still, if you don't keep up with the latest stable releases you will find yourself in a tight spot trying to do an overnight re-engineer of your product. Take the easy baby steps from version to version so you don't find yourself having to take a technological leap down the road.

persist

[caption id="attachment_696" align="alignright" width="300" caption="measure twice, turn once"][/caption]

I just witnessed an interesting even unfold in front of my current client's building. Today in beautiful Columbus, Ohio we are experiencing one hell of a winter storm, lots of fun. A semi truck decided that our parking lot was a great place to turn around in, he just apparently forgot to measure. CRASH BOOM and the lamppost has changed from its normal 90° to a more precarious 78ish°. After striking the lamppost and ramming a concrete barrier the semi was fairly well wedged, but the driver continued revving and the engine struggled and moaned. This persistence paid off and after about 10 minutes of maneuvering he was free. Now to call the insurance companies.

This got me thinking about the nature of difficult situations (self-imposed or otherwise). I also had a breakthrough largely due to persistence today, so happy coincidence. After months of working on documentation, being told that I would never be allowed to program the sacred system, the tides have turned (thanks to an excessively high quote) and the coach has called me up, and I'm ready to play under the bright lights.

I've just completed printing out 400 pages of documentation (my weekend is going to be so much fun!) and I will prepare myself for the difficult task ahead. My persistence has paid off, but the bigger realization is that persistence in general pays off.

To borrow one of my favorite quotes from Scrubs, "People are bastard-coated bastards with bastard filling." People don't ever want to think that you can succeed, and they are going to tell you no over and over again. There are two reasons for people to shut down your dreams:

1. They have tried and failed, the task is impossible, your success would undermine them. If you succeed where they have failed then it isn't an impossible task, they just weren't good enough to do it.


2. The have achieved it, but only by their virtue of their amazing talents and pure force of will. You don't have what it takes!

This is why it's vitally important to ask people for advice, sage wisdom, realistic assessments, but always keep in mind that very few people actually want you to succeed, especially if its in something they can't or already have. Don't worry though, these aren't bad people, it's human nature, and when you succeed you will act the same way. Success through hardship and against the odds plays a fun trick on the human brain, it makes you believe you pulled off the impossible. This feels amazing and so you will do anything in to hold on to it, and you don't want some dirty unwashed masses pulling off the same feat and diminishing it.

The big lesson here though is persistence, the most successful people in the world are those that were willing to have the door slammed in their faces 1000 times before making that sale. This is what you need to do, steel yourself for the struggle, prepare to be emotionally drained, and realize that this is what living is. Life is a stream of people telling you no, with the occasional yes. What separates the successful from the mediocre is that the successful are willing to wade through all the noes to get to the sweet succulent yes.

I've been paying my dues on this project for 3 months, and finally I get to go back to what I love the most, coding. There are a hundred stories of people trying and failing and trying and failing and trying and failing to finally succeed. It is definitely the path less taken, but it will make all the difference.

learn autoit

[caption id="attachment_655" align="alignright" width="300" caption="Bezels and shadows and reflections, oh my"][/caption]

If you use windows at work or at play this post is for you, macheads and linux people I'm sorry, this post probably won't help you much. I have had the joy of doing documentation work for the last 2-3 months at work. Gnarly, ugly documentation that serves little purpose other than making someone feel happy that there are reams and reams of documentation. At some point, probably around the 5th document, I made the realization that out of 20 pages it was mostly boilerplate, with about 8 things peppered throughout that changed. I decided that it would be easiest to make a template document and then just do a find and replace on those eight symbols [[DOCNUM]] [[DOCNAME]] [[DOCTRANSACTION]] etc.

Productivity shot up, I could now stub out a document in a few minutes instead of the hour or so that it used to take to manually hunt through and change things. All was well and after a day or two of find / replacing I had stubbed out the 80 or so documents. Then came the meeting where it was decided that 2 new sections would be needed and that 8 sections could be removed as being redundant. This victory for common sense was also a giant headache, as I didn't really look forward to restubbing 80 documents. There had to be an easier way, a better way, I remembered a screen driver tool I had some exposure to at a past engagement, AutoIt.

After reading through the documentation and goofing around with it for a while yesterday I now have a fully functional script that will automatically stub out all of the documentation with the click of a button. A task that used to be an error-prone, manually intensive process now requires no intervention. We can restub on demand now, so changing the template is no problem.

The Good

• Saves a ton of time


• Removes the human aspect from an existing process


• Centralizes specific knowledge


• Easy to write and test

The new script saves a ton of time, I can regenerate all documentation in about 10 minutes, so I click a button and go walk around the office bugging people. AutoIt simply takes a known working process and steps in to be the person driving it, I didn't have to take time dreaming up a brand new workflow, I could focus on just getting it done. The script is now the system of record for the specific things that change from document to document, which is nice for trying to determine at a glance what makes an HCR1 different from a ARC6 command, without digging through 40 pages of documentation. AutoIt also utilizes a stripped down version of SciTE with auto-completion and built in support for compiling and running, which makes writing and testing the script a breeze.

The Bad

• Ugly syntax (like VBScript and PHP had a bastard child)


• Oddly organized documentation and variably helpful community


• Inherently brittle scripts


• Still slower than I'd like


• Foreground processing

AutoIt has an ugly syntax (think VBScript), but it has all the pieces parts to make a nice script, functions and variables. The documentation takes a little getting used to, there is plenty in there, but it could be organized better. AutoIt depends on things like the window title and absolute paths, so it is inherently brittle, I doubt this script would run unaltered on someone else's machine. This could just be me being a n00b at AutoIt but I followed the practices laid out in the tutorials and the documentation. The other bad part about AutoIt is that it drives your screen, it simulates you pressing buttons and mousing about, so the script is slow and you can't interact with the machine while its running or you will probably mess everything up.

Alternatives

After proclaiming my victory over the documentation monster, I got some replies from colleagues asking why I didn't just make a powershell or ruby program or java program or something. I could have cracked open OpenWriter or something and attempted to build some massive program that could create .docx files, but that would have taken a ton of time. The AutoIt solution was incredibly quick to produce, took about 2 hours of playing around with it. There were a bunch of side benefits that the alternatives wouldn't have had. The template file is just a plain ordinary .docx file with all kinds of crazy formatting and images, instead of trying to figure out how to reproduce this in code, I can use Word to do the heavy lifting for me. This allows business users to change the template and we can rapidly restub and get back up and running.

Conclusion

You should go and learn AutoIt or AppleScript or whatever GUI driver program is available for your platform. It is not the tool for every job, but it's perfect for some jobs. Being the person on your team that can take some boring, error-prone, laborious task and turn it into a turn-crank task is a great thing. You can save your team time, get back to doing actual work more quickly, and make a name for yourself as a problem solver.

objective measure

[caption id="attachment_642" align="alignright" width="300" caption="this ruler is deep and spiritual, that\'s why it was photographed in black and white"][/caption]

I've found myself in an odd situation as of late regarding my work, I am a programmer by trade and education but haven't done any professional programming for 2-3 months now. It's not that I'm being lazy and don't want to program, I would jump at the chance to fire up Eclipse right now and start writing mountains of verbose Java code, it's that I'm not allowed to program. "Not allowed?" I hear the shock and dismay in your voice, so let me explain.

My role on this project is to design middle-ware but because of the way the business is set up I'm not actually allowed to program, the programming is all outsourced. Instead of actual programming I get to engage in some weird form of meta-programming where I turn a program into long flowing sentences in a Word document that take about 3 times the effort to write than the program itself so that it can be shipped off and someone else can program it. It is an odd situation indeed, but I am no business man, this project is important, and so I shall continue in this weird meta-programming.

This is not my first dance at the outsourcing-prom and with the economy in shambles I'm certain it won't be my last. I have yet to work on a project where the supposed benefits of outsourcing have actually materialized. I did some research (typed something into Google) to attempt to find out how many outsourced projects fail and saw figures between 25-40%. These seem fairly reasonable to me, and with some statistical hand-waving I will use these as "good enough" figures for the rest of this post. This leads to an interesting question though, if this strategy fails so often why do companies continue to outsource?

I would like to posit that a major reason for the continued use of outsourcing is the gap between subjective and objective measure. Subjective measures come from within, they are personal opinions, things like "Frank is a great worker!" or "Jim is so lazy" are subjective measures. Objective measures are fact-based measures things like "Frank worked 48 hours this week" or "Jim only produced 1 class file this week."

Often, like in the examples above, our objective measures inform our subjective measures. The problem with software development is that there are no good objective measures. As much as we like to think of what we do as a science or even an industry, it is still a craft. In the above scenario we are led to believe that Frank is a "great worker!" because he worked 48 hours this week and that Jim is "so lazy" because he only produced 1 class file. If Frank worked 48 hours doing simple things and just racked up the billable hours is he really "working" harder than Jim who wrote a full web server in one week (monolithic in design so it was only one class). Getting to the heart of the value that someone produces is a difficult process and is by its nature subjective.

Here in lies the problem, software development can really only be measured subjectively, which is hard. Objective measures are much easier though, and so those are often used as a proxy for doing the difficult work of subjective measures. There have been many attempts to make sure people are performing up to par, lines of code, commits, bugs, hours worked, etc. None of these are really ideal, together they can begin to form a picture of the subjective nature of the situation, but they are not nearly enough.

Objective measures are also very seductive because you can perform calculations on them, they are easy to compare, they are easy to play out what if scenarios with. No one balks at answering the question which child is taller, but ask them which one they love more and watch them squirm. Objective measures are great because you can throw them in excel and analyze them, this is also a weakness. It's easy to draw up a spreadsheet with your in-house assets pitted against outsourced assets and see that you can save a bajillion dollars by outsourcing, but that misses the bigger picture. What are the subjective downsides?

These subjective downsides end up failing some 40% of outsourced projects, communication, quality control, and responsiveness are hard things to chart and graph, but can easily cause a project to run off the rails. What is needed is either some way to make subjective measure easier or some fantastic new unit of objective measure. I would propose ihumanables per hour be the unit if anyone can come up with some really nifty objective system, but I don't see that happening anytime soon, so all that's left is making subjective measure easier, a tall order indeed.

At the heart of making subjective measure easier is really what a lot of the agile methodology is about, stand up meetings, scrums, kanban boards, they are all meant to produce a tight feedback loop so that the subjective nature of the situation is constantly being polled. I'm not going to sit here and tell you all to start running agile projects, although they can be a good bit of fun, but if you are managing people you need to get in the dirt with them. Progress reports and weekly status meetings are meant to supplement not substitute actually working with and understanding what the people under you are doing.

Remember the next time you are making a decision based off of objective measure, if you are "going by the numbers" that there are very real subjective side-effects that you should account for and be aware of. And if someone develops a really killer objective measure for software development, remember ihumanables per hour ;)

codemash review

[caption id="attachment_577" align="alignright" width="300" caption="Colonel Whiskers and I have both missed you dearly"][/caption]

You may have noticed that I have not posted anything since last Tuesday, gasp! Fear not, although various shadow organizations are still after me, I was not captured. I had the awesome opportunity to go to CodeMash, and spent most of last week in the beautiful Kalahari Resort in Sandusky, Ohio.

CodeMash, for those of you not in the know, is a pretty cool event. Instead of focusing on one technology (.Net or Ruby on Rails for instance), there are tons of sessions spanning the technology spectrum. Want to learn about ruby, there is something for you, F#, got it covered, Clojure, for goodness sake they even did a lisp. The great thing about this style of conference is that you can pick and choose whatever interests you, nibbling on concepts here and there to get a bit of everything, or really chowing down on one topic.

The other cool thing is that the company I work for HMB is forward thinking enough to not only sponsor CodeMash, but to send me there for free! Then the big topper came last month when they asked me to present on prosper.

Well now it is all over and I can give you a rundown of how things went.

The Good

• Kalahari - The Resort is beautiful, the rooms are huge, the staff is excellent. It is a really cool place to spend some time in, they served us some great food, and the facilities were excellent in their range, rooms small enough for a handful of people to collaborate in, all the way up to grand dining halls to house the 800 or so attendees.


• Keynotes - Mary Poppendieck gave a great talk on Lean Software Development, Hank Janssen talked about how Microsoft is moving into the Open Source world, and Andy Hunt, co-founder of Pragmatic Bookshelf and author of the Pragmatic Programmer, gave an amusing talk on the Mother of all Bugs, our brains.


• Prosper Talk - After weeks of preparing and fretting over my presentation I think it went well, there were some technical difficulties, but nothing that could stop the incredible power of Prosper!


• Session Leaders - The people leading the sessions don't get paid, they do it out of love of software development, and it shows.

The Bad

• Superficiality - Sessions are only an hour long, this means that no matter how great the sessions are they can't really get into the guts of anything in any kind of depth.


• Beginner Overlap - I went to a lot of Ruby sessions, as that is my current love, and even as a n00b it became tiring to hear the 5th explanation of duck-typing for the day.


• Kalahari is way too big - I'm just kidding about this one, but it was honestly about a 2 mile hike from the convention floor to my room.

What's Hot

• Ruby - Ruby is blowing up, not just rails anymore. Rails did a great job of introducing the world to the quirky little language that could. Now innovations are being made inside the Ruby community and people are starting to port that out to the rest of the software world. (See the next point)


• TDD and BDD - Test Driven Development and even moreso Behavior Driven Development are becoming huge and are here to stay for the Agile practitioners among us. A big driving force for BDD is Cucumber.


• Cloud and NoSQL - I'm lumping these two together because they are both technologies at scale. Virtual infrastructure is becoming a big business, be it Amazon's EC2, Microsoft's Azure, or Rail's Heroku. Applications running at huge scales are also turning to NoSQL storage solutions.

The Point

CodeMash is a chance to meet really cool people how are excited about technology. It's a chance to get exposed to a ton of cool technology. It is a chance to try a bunch of cool stuff and see what you like. Sessions are only an hour long so at worst you end up wasting 60 minutes, at best you get fired up on something wonderful. You get to meet a lot of cool people, here some great talks, and best of all you are at a gorgeous indoor water park that is 84 degrees when its freezing out in the harsh wasteland of Ohio.

who cares?

[caption id="attachment_573" align="alignright" width="300" caption="I shot who in the what now?"][/caption]

It's one of my favorite questions to ask during a technical discussion. Who cares? Some others I like to throw out our, why should I care? why does it matter? and I don't care! Am I some sort of raging lunatic completely out of my mind and trying to pick a fight with everyone I meet? Yes, but that's not why I ask these questions. I ask these questions because as great as the big picture is we sometimes have to drop complexities to get stuff done.

When you have a big flow diagram or giant UML Class Hierarchy it can be fun to think about the abstract way that things interact and how you can pass state around and whatnot. This definitely is useful for getting a feel for a new system, but then there comes the worst part of any software developer's job, actually getting things done. You see that's what we get paid for, actually putting our fingers on keys, typing stuff out, compiling it, and shipping a product. Part of this is doing the upfront design work and getting all of our t's crossed and i's dotted, but just as important is actually getting something working.

I talked about this before in my post about simplifying things. This simple question, "who cares?" (you can be more diplomatic about it if you like) is a great tool for getting to the heart of a problem. I like the simplicity and straight-forwardness of the question, it normally catches people off guard and challenges their assumptions.

Frank: So we need to pass this data into the layer
Bob: Who cares?
Frank (taken aback): I care, I care deeply about this data
Bob: But why do you care about it being in that layer, its just going to hand it off to this layer, and that layer doesn't care at all about it
Frank: I guess you are right, we don't need it there

BAM!!! Problem simplified! For anyone that's ever taken any education classes you may recognize this as a rather blunt application of Socratic Method. The point is confrontation, but not between the two people discussing something, between each person and their preconceived notions about a problem or situation.

In High School I took a chemistry class, it is one of the few classes that I really remember in any detail from so long ago, the teacher taught the entire semester in Socratic Method. He would question us, we would experiment, we would reason out why we observed what we observed, he prodded us, and we cherished our knowledge. Anyone can tell you water is composed of 2 hydrogen atoms and 1 oxygen atom, but when you had to work for it, when you had to defend your knowledge of it from scrutiny, it became concrete, it became your discovery.

Applying this to software development is useful, we can get locked into our preconceived notions, and direct, blunt, sometimes even rude questions can spur you to really consider what you are talking about. Software is incredibly fluid, sometimes an idea or concept can become out-dated in a matter of months or a matter of hours. The biggest ones are the most difficult to challenge, because they often have the most dependent code around them, but this makes them the most important to constantly consider.

Keeping your conceptual design, implementation, and solution in lock step with a problem domain often involves some amount of change, that change can be hard to come by though. By challenging our standing beliefs we can find new avenues for exploration or strengthen the existing techniques and gain a better understanding of them.

toolmaker

[caption id="attachment_535" align="alignright" width="262" caption="Pictured Above: Programming"][/caption]

For about a year I had the unique experience of working at Abercrombie & Fitch's world headquarters in New Albany, Ohio as a Java Developer. It was a really interesting place to work, very stylish, loud music all the time, complex problems of incredible scale, and some of the best developers I've ever worked with. I was lucky enough to get placed into one of the most successful development teams, Planning. This team had developed a tool that was hugely popular and well regarded, and I was coming on board to maintain and improve this tool. The beautiful people and board-short dress code was just icing on the cake.

As a software developer this is a fairly basic scenario, there is some task that is complicated, business users want it to be less complicated, they give me some money, I make them some program, and everyone is happy. The odd thing though is that while programmers are completely used to making tools for others, they often forget that they can leverage this amazing skill for their own benefit. I distinctly remember a conversation I had with one of my colleagues there, Steve. It centered around the idea that most developers are happy to just get their assigned tasks done, but there are some that are toolmakers, and being a toolmaker is more valuable to a company than being a developer.

Fast forward to after my stint at A&F, I was on a new project and given the task of translating 80,000 lines of PL/SQL into T-SQL. The idea was to use...um...find and replace...or....something...and...um...get this to work. With such a fine technical specification and a mountain of PL/SQL staring at me, Steve's words bubbled up to my brain, "Make a tool." I had a time window of a some number of weeks to do the translation, it was a bit of a gamble, but I decided to spend a week writing a tool.

After remembering how C# worked after a year in the trenches with Java I had a program that took in the PL/SQL parsed each line and translated it to T-SQL. The task was made easier by the fact that I didn't have to write a generalized parser for PL/SQL as the 80,000 lines were written in a fairly consistent style, which greatly simpified the translation tool. After running the tool a few times, fixing some bugs, I had finished the task in a fraction of the projected time. Triumphantly, I went to my PM to show how I had just saved weeks of manual laborious work. "Great, now here is your next task...."

[caption id="attachment_538" align="alignnone" width="150" caption="Expected Reaction"][/caption]

[caption id="attachment_539" align="alignnone" width="150" caption="Actual Reaction"][/caption]

So I didn't get the praise I was expecting, that's fine, I'm not 4, I don't need a cookie. I continued on in my tasks quietly pleased with saving myself weeks of tedious work. Then the day came when the database requirements changed, the hundred or so CREATE TABLE statements would need to be altered. The task wheel was spun, and since only my name was on it at the time (thus defeating the need for the wheel) I was the lucky person to go through and change various aspects of each table.

I thought to myself, "Great, there goes a day or two down the grunt-work hole." Then the part of my brain that I put in charge of remembering important things woke up and slapped me. "You haven't altered the SQL yet, just alter your tool to do this too and re-translate, dumbass." After pondering the suggestion, and possibly more importantly the insult, leveled at me by my brain, I opened up my translation tool. A few keystrokes later the change to the translation mappings was made and I fired off the program. Bits flew, gears cranked, and out came the exact result I was looking for, the tool had just saved me another day or two.

Ultimately the project was a big hit, we delivered on-time and on-budget for what was a somewhat aggressive schedule. The tool saved us a good amount of time, let us get the application bootstrapped and running, which allowed us to start iterating and get things working. Over the course of that project I created tools here or there that ended up saving me, and some thankful members of my team, time and effort.

The lesson here is that you should look around you and see if you can make some tools. Tools can be scary though, they are a bit of an upfront cost, and if it doesn't work out then you've wasted even more time. With time and experience it becomes easier to recognize when it's a good time to make a tool and when it's a waste. Here are some good guidelines for tool making success.

• The best candidates are tasks with high amounts of repetition and little logic.


• Check to see if a similar tool already exists, don't reinvent the wheel.


• YAGNI (You Ain't Gonna Need It). You should be intimately familiar with the problem you are going to solve, solve exactly that problem, you can scale it out later. I didn't bother writing a general purpose PL/SQL to T-SQL translator, I only had one PL/SQL file to translate, if it worked on that then it worked on 100% of its input.


• If it would take less time to do the boring thing manually than to build a tool, just suck it up and do the boring manual task


• Share your tools, even if you don't think they will be useful to anyone else on your team, sometimes a tool can provide a surprising benefit to others


• KISS (Keep It Simple, Stupid). This is no time to become a highfalutin architect. Tools are normally quick and dirty one-offs, normal concerns like efficiency, user-friendliness, etc. can be brushed aside. If the tool is getting used often and by many people you can refactor it to make it more efficient or user-friendly, but don't pay that cost upfront

Developing software often requires a high degree of focus on the problem at hand, I wrote the other day about one approach to problem solving, simplifying the problem. This strategy can be coupled with toolmaking to dramatic effect. Once you realize the simpler actions you can automate those and write a tool to perform the task for you. The next time you are facing some mountainous problem, consider the toolmaking route.

division of labor

[caption id="attachment_280" align="alignright" width="216" caption="cartoon labor"][/caption]

When building or analyzing any system of significant complexity the most important thing to determine is a proper division of labor. The entire idea of n-tier architecture is built on dividing labor up into the appropriate tiers and keeping them cleanly separated. For anyone that's ever worked on a real world system half of the difficulty is caused by one layer leaking into another and the other half is getting around the separation caused by separation.

Despite my last sentence, division of labor is a necessary and good part of design. When done right it allows you to know where functions, variables, and constants should live, how things should interact, and who is the system of record. This is why it is imperative to decide beforehand or shortly into a project what the lines of demarcation are and who is responsible for what.

Prosper is one of a group of libraries, with well defined divisions of labor. This allows the libraries to be decoupled, and also has the added benefit of letting me know when I'm done. When I found myself adding functionality to Prosper the other day that I knew belongs in the next library, I realized that it was time to officially start the next library. So a new skunkworks project is moving ahead full steam, it contains 2 parts that work together, and so again requires a well defined division of labor.

Just because work has begun on a new library doesn't mean that Prosper is left out in the cold, yesterday I added support for the deprecated mysql library, standardized construction, and added better support for native functionality. If you are checking out Prosper, you can follow the latest development by pulling from the svn repo.

Back on topic though, figuring out the division of labor is the most important part of any analysis or design work. We recently at work had a 2 hour long discussion about order numbers. It went something along the lines of this:

BA: This order number comes from the Mainframe
Expert: No the Application creates that number
Dev: So, what does the SOA layer need to do?
BA: Generate the number
Expert: No, do nothing, the Application will automatically do it
Dev: Wait so who is responsible for creating the order number, and if its not the Mainframe why is it making one?
Expert: The Mainframe always made that number and the Application has always ignored it.
BA: So we will make it in the SOA Layer?
Dev: Who is in charge of making the unique identifier order number?!
Expert: [shrug]?
BA: [shrug]?
Dev: Let's decide that first

Explicit division of labor requires you to pull the unspoken assumptions everyone has about the system out of the silence of our own minds and puts it into the blinding daylight. Suddenly communication is smoother, arguments come to an end, because there is some record to consult to help guide decisions.

Draw out your explicit divisions of labor, don't carve it in stone though, if your original design ends up being too inflexible, allow it to change, but again, bring all the unspoken assumptions back out into the open. This allows you to cleanly define what piece is in charge of what function, keep layers from leaking into each other, and make for smoother development and analysis.

clean slate

[caption id="attachment_270" align="alignleft" width="300" caption="new desktop"][/caption]

Today is my first day at my new assignment. My company gave me a new(er) corporate laptop with a plain vanilla install of Windows XP Pro to use at my client site. Last night I had the joy of looking upon that fresh computer and thinking of the wonders that could be. I got ready to install the base programs that take a windows machine from factory to awesome. Here is my essential install list.

• Firefox - The first and only time I interact with Internet Explorer is to go download Firefox




• Heaven.cube theme


• Delicious Bookmarks extension


• Echofon extension


• Firebug extension


• FireGestures extension


• Flash plugin




• Programming Languages




• Haskell Platform 2009.2.0.2


• Java EE 5 SDK Update 8 (JDK 6 U17)


• CLISP LispBox


• Python 3.1.1


• Ruby 1.8.6 (I would have rather gone with 1.9.1, but the one-click windows installer is lagging behind and I one-click was all I wanted to do)




• Development Tools




• Dia - a drawing program


• ClipX - best clipboard manager


• Eclipse IDE for Java EE Developers


• PSPad - awesome free editor




• Tools Windows should come with




• 7-zip - small, fast, free


• Process Explorer - The task manager Microsoft should have built, but didn't, so they just bought it.


• Foxit Reader - small, fast, free (again)


• Pidgin - Take that MSN Messenger


• Paint.NET


• Royale Noir XP Theme

Then I spent plenty of time monkeying with the filesystem, the wallpaper, things that don't particularly matter, but make the laptop feel like home to me. I made my icons and text smaller, since I'm used to a giant monitor and the 1280x800 resolution felt a little cluttered to me. Now though I feel right at home with this machine, and feel like this is the best possible time of laptop ownership, right after you've got it set up just the way you like it, right before you add stupid things that pollute the registry and mess up your beautiful filesystem.

The other clean slate, the less fun one, is my brain. I have all kinds of knowledge in my brain about my last projects, but today I find myself swimming in a see of paths and acronyms and weird buildings, and a chair that isn't quite high enough to be comfortable. I'm starting to wrap my mind around the acronyms and getting a grip on what exactly we are doing here, but I fear it will take more time to feel as comfortable with this new position as I do with my new laptop.

Suffice it to say it could be much worse, I can clearly still push to my blog, so that's sweet. The people I am working with seem cool and fun, the project seems like a lot of work but nothing that we can't handle. I will probably have a whole different viewpoint in a week when I truly realize the scope of the project and settle into a nice little routine.

So far, so good, I'm off to read another IBM pdf that is half marketing / half documentation. Wish me luck.