usable in any place a human can be used

20100225

road trip

[caption id="attachment_772" align="alignright" width="300" caption="sitting... check. staring... check. boredom... check. Looks like we\'ve got ourselves a good old fashioned road trip!"]road trip sign[/caption]

I have a road trip coming up, one of those magical adventures where you sit quietly staring at the vast cornfields of the rust belt trying to keep from falling asleep and crashing into a cow or something. I've been preparing for this road trip and decided that instead of trying to find 5 hours worth of music to listen to that I could multitask and read while driving!


No I'm not some sort of superhero that can both read a book and drive, I am of course talking about audiobooks. Being a nerd I already have some audiobooks, I have Jon Stewart's America and Steven Colbert's I Am America (And So Can You!) but I thought that I could get some sweet programming knowledge in my brain while on the road. And so I have set out to find the best programming audiobooks / podcasts around, and I need your help!


So here is the challenge, if you so choose to accept it. Help me assemble a sweet list of programming audiobooks / podcasts, then I will subject them to my brain for 10 hours and come back and write a review of the good, bad, and ugly. Together we can produce the go to list for programmer road trip aural stimulation. Leave a comment with any audiobook / podcast related to programming, software development, or any other sufficiently nerdy subject. Let the miracle of crowdsourcing begin!!!

20100224

blackbox revisited

[caption id="attachment_767" align="alignright" width="275" caption="Oh shit they\'ve miniaturized HAL "]blackbox[/caption]

The other day I wrote about the concept of the blackbox. I focused specifically on the idea of a blackbox as us programmers think about it. I've been thinking about it more and more though and realized that there are blackboxes all around us. There are processes that our often complicated, confusing, manual messes that our non-programming friends and family have to suffer through. If they knew a little bit of ruby or maybe some batch programming they would be able to save mountains of time and effort, but they don't, so that's where programmers can shine. The problem is that to help our friends and family we have to shine some light into their blackbox tasks and figure out what they do, that's not the easiest task for a programmer to take on.


Recently at work they announced a change to the timecard system that we use, all timecards MUST be submitted by 10:00am Monday morning. The old system was, they should be in sometime on Monday and if not then our very nice receptionist will send out a sternly worded email shaming you into submitting you timecard. Being curious (and being that the explanation was just a few more lines down in the email) I wondered what change had occurred to the system to require this 10:00am deadline. Ends up that our receptionist used to enter this data by hand from the system we enter it in into the accounting software. This was a tedious and error-prone task, once someone bothered to look at it they realized that with a little bit of code they could automate this all and get these programs chatting back and forth like old bridge partners. A couple hours of programmer effort later and now our receptionist is able to answer calls and do all the other receptionisty things that she is great at instead of spending time copying numbers from one program to another.


The blackbox, timecards turning into accounting information, was illuminated and the process was horrible, a little bit of code applied and BAM new productivity abounds. It gets me to wondering, how many other blackboxes are all around me, waiting for someone to shine some light in, see the inefficiencies, and offer up some soothing code. I've given myself a new little game to play when talking to my friends and loved ones, when they start complaining about something that they have to do in their day to day, instead of just offering a sympathetic, "Sucks to be you loser" I start digging deeper. "These reticulated splines are giving you a lot of trouble, how exactly do you make them?" Normally they will reply with a bit of hesitation, maybe even a polite, "Oh, its boring, you don't need to know about that." But if you keep on gently nudging, you can find out enough of the process to help.


It doesn't always have to be by providing code, sometimes just the general nerdy computer knowledge that we have can be incredibly beneficial. It's easy to forget that not everyone knows the little tricks to make the computer easier and more fun to use. I was able to save my mother hours a day by explaining to her that despite what her boss told her, she does not have to shut down Photoshop between opening every file. Until then she had been instructed and believed that she had to wait a good minute for Photoshop to boot up in between every file, opening hundreds of files a day, you do the math. By asking my girlfriend some questions about something she does at work I was able to throw together an Excel spreadsheet to save her a ton of time. The magic of VLOOKUP and some well placed formulas was all that was needed to take a boring repetitive task and make it much simpler and less error-prone.


For those in the computer-know it's easy to look at the computer as a simple tool, something to be mastered and leveraged to make our lives easier. For many people though, computers are blackboxes, confusing little machines that spit out weird error messages and are constantly fighting them to get things done. And there is the other edge of this sword, by shining light into a process blackbox and making life easier you do the double duty of helping illuminate the blackbox for a computer novice. You help show by example that the computer is not mean or frustrating (well sometimes they are frustrating) but just a tool that with careful skill, some essential knowledge, and most importantly a lack of fear, you can make bend to your will.


Go find the blackboxes around you, help someone save 15-20 minutes a day, show someone that computers are not just useful in general but can be made personally useful. You will be a technology hero to them, you will find an interesting puzzle to play with for yourself, and at the end of the day everyone will be better off. Find the blackboxes and start pouring in light, you will be pleasantly surprised at what you find.

20100223

to tweet or not to tweet

[caption id="attachment_758" align="alignright" width="300" caption="web 2.0 kids these days with their tweeters and facespaces, I used to have to text on a 12-button phone and I liked it!"]twitter bird[/caption]

Ever since I started writing ihumanable.com so many many years ago (actually it was the beginning of October) I have tweeted the birth of every new blog post. For those who follow me (see the button at the top of the page to join the elite group of @ihumanable followers) I expect that you spend most of your day with bated breath waiting for the singular moment of glory that a new ihumanable blog post is ready for your consumption. I was pointed to an article today by Shawn Blanc about how to handle the tweeting of blog posts. The logic boils down to the following




  1. Some folks don’t care a dime about my nerdy posts, but have great concern about what I eat for lunch.

  2. Some folks are already subscribed to my RSS feed and would prefer to keep it there and nowhere else.



The solution Shawn Blanc comes up with is to have two separate twitter accounts @shawnblanc for personal "what I ate for lunch" tweets and @shawnblancnet for stuff about his blog. So the question that leaps to your mind is, should you immediately start following @ihumanablecom for all the updates about the great free content / ranting with oddly captioned pictures that I produce? No, no you should not, and I'm about to tell you why.


I have an RSS feed and twitter, some people would argue that I shouldn't tweet about blog posts because what if someone is both subscribing to my feed and following me (thanks to anyone who is so devoted). This poor unlucky bastard will get the grand news of a new post in gasp 2 different places.


This argument doesn't make much sense to me. Twitter is passive, it is the un-email. You follow people you like, you see their tweets, there is no "unread count" or really anything expected at all by the tweeter from the tweetee. This is why people love to tweet, its the best part of any conversation, the part where you are talking. Look at some of the recent important tweets.


Faught an old man for a parking spot at ihopp - AmandaSollenne

A man is a man when he can offer his hand. The Who - wealthmoneynow

Straight up doing nothing. Have a dentist appointment after school. then have to go to court for 5:30. Then have a bunch of homework. Great. - JamieBaskett

Now I'm not picking on these people (I don't even know them) I just went to the public timeline to see what was currently running through the tweet stream. The point is that these are low-value easily ignored communications. If something shows up in your tweet stream that you don't care about, at most its going to waste 140 characters of mental processing power.


The second argument is that somehow people could care more about what I had for lunch than my blog. What I have for lunch is some meaningless data point about my day it means nothing to me (although today's Grinders Chicken Parmesan Stromboli was amazing). This blog which I spend all kinds of free time and energy on actually means a great deal to me. I want to be out there promoting it and if you are following me on twitter I would imagine you would want to see the things that are important to me. If not then why are you following me.


I have people following this site on RSS and people following me on Twitter and I would imagine its not a perfect overlap. When I first started this blog I had no RSS subscribers because it was fresh and new, so I promoted it with a simple (usually less than 140 characters) tweet, one per day. Now I have people following me on twitter solely because of this website, and it would be a disservice to them to stop tweeting about the blog posts now, changing the rules all up midstream.


This blog is important to me, me @ihumanable. The things I write here are an expression of the frustrations, lessons, and victories that make up my life. I could easily start an @ihumanablecom (if its not taken) twitter account and tweet new posts out through that. But that doesn't make sense to me, @ihumanable is where I tweet things about me, ihumanable.com is about me, and so tweets about ihumanable.com will continue to be broadcast through @ihumanable.


The argument against blog post tweets fails to understand the very nature of twitter, it is a passive, non-blocking, stream of information. If someone is spamming hundreds of tweets a day about pointless blather (well then they are probably using twitter) then stop following them. If someone is trying to share something that they have worked hard on and care about once a day, then I would hardly think we need to erect walls of netiquette around it.

20100222

blackbox

[caption id="attachment_751" align="alignright" width="241" caption="Something happens in there... with gnomes I think."]blackbox diagram[/caption]

The Blackbox, cornerstone of computer science, thing of mystery and beauty. It is the fundamental concept that allows us to build bigger and better abstractions, making life easier for everyone. A blackbox is something that has well defined inputs and outputs but the manner in which it functions in unknown. The classic example of this is a library function, sqrt(9) takes the number 9 and returns the square root 3. There is a well defined input (in this case 9) and a well defined output (the square root, 3). How does the function produce this result, who cares?


As programmers we can focus on what we are good at, if someone loves programming math algorithms they can make a kickass math library, if they love web programming, here comes Rails. Then when I'm interested in writing a web app that needs to do some math, I don't have to worry about how to find the square root of some number, there is a handy blackbox all ready to go. This is a huge boon to productivity, I'm happy, math programmer man is happy, and the user is happy because they don't have to use my buggy homegrown math libs that say they owe a $47 tip on a $9 check. (And my math degree weeps at me)


This is all well and good, the problem comes when you need to know what's going on in a blackbox. There are any number of scenarios in which you may need to shine light into the depths of a blackbox. You have some strict performance or memory requirements, the blackbox's "well-defined" outputs are failing (what do you do when you call out to sqrt(9) and get 7.2 back) or the use of a blackbox is having some negative side-effect. In these situations the whole damn thing comes crumbling back down to earth, you now need to roll up your sleeves and get it done. Here is how you should approach the problem.



  • RTFM (Read the Fucking Manual) - This is the first step, if a blackbox is eating up too much memory see if there is a way to force a limit, if it is throwing an exception, check why that could be the case. Basically before you do anything else you should try to make sure that the blackbox is actually broken, because chances are that any sufficiently popular blackbox is going to have much more real-world use and rigorous testing than your new client code. This boils down to, don't blame the compiler for your shitty code.

  • Ask someone familiar with the blackbox - After you've actually RTFM you can move on to this stage, seek help from someone more familiar. If there are colleagues that have used this particular blackbox ask them first, they may have already gone through this entire list and can shortcut you to the answer. If you have no colleagues to turn to, try turning to the community or the creator, if that fails try throwing it up on a more generic community like, stack overflow.

  • Go to the source - If you are working with an open source blackbox then it isn't really a blackbox, just go read the code and see what it's doing. This is much easier said than done, so make sure you have exhausted the first two steps before diving into this one. If you are using a closed source blackbox you can either attempt to get the source through some sort of development channel (although this is unlikely) or you can decompile and look at that. Tools like RedGate's Reflector or the Java Decompiler Project can help to get at that closed source.

  • Workaround - Sometimes no matter how dogged your determination, you can't seem to figure out why the blackbox isn't working right, at this point you have exhausted the detective route, time to pull a MacGyver. If you can figure out someway to use the blackbox and workaround the deficiency then go ahead and do that. The important thing is to take this workaround and record it properly. Create an adapter, thin wrapper, or function. Make your own lightweight blackbox that is 1 part the old blackbox and 1 part workaround aggregated together.

  • Roll your own - Sometimes you get seriously desperate, you have researched your fingers to the bone and tried to work around the problem. You are at the end of your rope, the deadline is looming and the only thing standing between you and your project compiling is some pos third party closed source blackbox. Sure it would be nice to have a sqrt function nicely wrapped up for you, but hell you can dust off that part of your brain that remembers what a square root is and write your own function to find it. The important thing is that this should be the very last thing you do, after all other options are exhausted. Many a programmer will do this first or second, and that is Seriously Fucking Wrong©.


The power of a working blackbox is only matched by the frustration of a wonky blackbox. The good news is that a lot of blackboxes work really well sitting there quietly trimming our strings and rooting our squares. There is that every once and a while though when despite our best efforts we have to dive headlong into a blackbox and shine some light around.

20100216

telecommuting

[caption id="attachment_739" align="alignright" width="300" caption="Just let it warm up for a minute, you will be fine."]car filled with snow[/caption]

I am not in Indiana today, the trip has been postponed until the Snowpocolypse is over. I awoke this morning to find another 6-9 inches of snow had fallen on top of the unmelted foot and a half of snow already on the ground. I groggily turned on the TV, Columbus is under a Level 2 Snow Emergency and just about everything is closed. I check with the Franklin County Sheriff Office and see that for a Level 2 Snow Emergency the situation is as follows: "Roadways are hazardous with blowing and drifting snow. Only those who feel it is necessary to drive should be out on the roadways. Contact your employer to see if you should report to work."


With the threat of hazardous roadways and the fact that my job in no way, shape, or form requires my physical presence at any location, I shoot off an email, following the sheriff's advice to "Contact your employer to see if you should report to work." The sad news, of course, is to put on your business casual, bundle up, clear off the car, fight throw the blowing snow, and carry your laptop with you so you can plug in and get to work. The alternative of course would be to stay nice and warm, not endanger my life and property, and plug in my laptop at my desk at home. This alternative is clearly unacceptable, against the advice of the Sheriff's office and against all common sense I physical transported myself and my laptop several miles down the snow covered roads to plug in here at work today.


Here I am doing the same work I could be doing from home, as far as I know, the SOA book that I'm studying and the Message Broker Toolkit examples I'm working through are unaware of my physical location. I can easily access my corporate email, sametime account, and all relevant files from home, I can even participate in the conference call meeting that we have scheduled for 1 o'clock. The question is why, in spite of the rest of the city shutting down, was I required to come into work today? The rest of this post will present the Case For Telecommuting, the Case Against Telecommuting, and an Analysis. All of these sections are written with programming or some other suitable telecommuting profession in mind.


The Case For Telecommuting


Technology has long been on the march for decentralized development. Being able to reach resources from any location is more and more possible with web based solutions and DVCS. In addition to having resources available from any physical location our ability to communicate and collaborate despite physical distance has been steadily improving. Tools like Google Wave, Instant Messaging, and Collaborative editing have matured and become rich and expressive mediums. Phone conferences are now common place when working with outsourced resources and with physically distant resources.


For the particular case when travel is dangerous telecommuting has a distinct advantage over commuting. Commuting is another part of life that is ever growing.


In fact, so many people now commute more than an hour-and-a-half to work, the Census Bureau has given them a new name: Extreme commuters. ... It's growing five times as fast as the general growth in commuting — about three million, 3½ million [commute] more than 90 minutes a day, about ten million more than 60 minutes.

Commuting also has a negative effect on your physical, mental, and emotional health. The impact of Voluntary Solitary Confinement (VSC) are not pretty.



Long hours of commuting, especially if you’re driving, is associated with high blood pressure, musculoskeletal disorders, increased anger and resentment at work... Long commutes can also increase the risk of heart attacks, flu, depression etc.

- Effects of Long Commutes to Work

The Washington Post has an article from 2007 discussing the problems with long commutes.



As a consequence, more drivers will probably suffer the health effects of a commuter lifestyle, researchers and doctors said. "You tell someone they need to exercise or go to physical therapy, but how can they? They leave at 5 a.m. and get home at 7 or 8 p.m. at night," said Robert G. Squillante, an orthopedic surgeon in Fredericksburg who has treated patients for back pain and other commuting-related issues.
...
"It's just tiring," Hutson said of her daily drill. Someone who was never much for caffeine, she now bolsters herself with coffee in the morning and soda for the evening rush. But by midweek, "I'm running on fumes. That's the biggest toll. It's not enough sleep."

No one pays for the commute except the worker, and as we saw during the gas price spike up to $4/gal (even now its not much better, only because we have that high water mark to compare against does ~$2.50/gal seem reasonable) the cost of commuting can become a rather high "work tax." The Washington Post article hits on it briefly, but if you have to get up early to commute and get home late because of a commute you are also paying the price in terms of your time. You may only be getting paid from 9 to 5 but you are effectively giving your job all the time from the moment your commute begins in the morning to the moment it ends and you are home. If you have to adjust your sleep schedule to support this then any time you could be awake, doing something you enjoy, actually living your life, is forfeit to the needs of your employer and not you.


The Case Against Telecommuting


If telecommuting gives the working man more free time, more money in his pocket, and better health, why not let everyone telecommute? The arguments against telecommuting are many and we will examine a few that I have encountered.


The problem of split attention. When you are face to face with someone you know that they are giving you their full attention. With an IM or phone call the employee could easily be working on something else at the same time. They could be doing something work related, reading documentation, writing some code, discussing a problem with another colleague, but whatever it is they are splitting their attention and multi-tasking, and sometimes that is unacceptable.


The problem of accountability. This is a corollary to the problem of split attention. Split attention isn't so bad when the attention is split between listening to a meeting and writing code, it is pretty awful though if the attention is split between listening to a meeting and playing World of Warcraft. Keeping people accountable for the time they claim to be working is more difficult if they are unsupervised and physically distant.


The need for humanity. Often times phone conferences and IMs are more than enough to meet our communication needs, but this is not always the case. Sometimes there is no substitute for "pressing the flesh." Important meetings demand a level of attention that necessitates being physically present.


The problem of ineffective communication. As good as the tools have become there is still a gap between talking with someone, being able to read their facial expressions and body language, to fully understand what they are communicating. Telepresence and other techniques are getting us closer, but a face-to-face conversation conveys a wealth of unspoken communication.

The problem of bureaucracy. I am no fan of bureaucracy and there is a certain liberating power to having a team physically assembled. Difficult technical problems can be jotted out, ideas bounced around more easily. Communication tools are great, but still don't beat a whiteboard that everyone can see or sitting down with another programmer and sketching something out on a piece of paper. These impromptu mini-meetings are often incredibly productive, and they can be difficult to have when physically separated.


The problem of facility. Some of us have great work space set up for working from home. Fast internet connections, powerful machines, quiet space. But this isn't universal, some people can't work from home for one reason or another. A lack of connectivity or acceptable development machines, an environment that is not conducive to productivity, or some other issue. Some people require a more structured working environment in which to be productive. The lure of facebook and Judge Judy prove too much and workers that could be productive in an office environment fail when working from home.


Analysis


Telecommuting is increasing and is getting better. For now it is an option but societal and economic pressures and an increasingly agile work environment will make it more and more necessary to effectively optimize one's workforce. Many of the concerns and arguments against telecommuting boil down to a deficit in trust. The alternative though, longer and longer commutes coupled with increasing transportation costs is untenable.


As with an paradigm shift, new etiquette will have to be formed, the way we think about work and home will have to change, and tools will have to be built and improved to rise to the challenge. Although we shifted to the information age and a knowledge based economy in many ways, we continued to hold over traditions from a bygone era.


Telecommuting will take some time to get right, for anyone interested in adopting it I think a blended approach is the way to go. Start off small by offering anyone who would like to try it to telecommute one day a week or month. The important part will be follow up, getting feedback on what worked and what didn't. You can use this feedback to fine tune the experience and come up with a manageable system. Physical presence will always have a place in the workplace but as we move into a future with increasingly global concerns and clientele, increasing transportation costs, and a better understanding of the physical and emotional ramifications of chronic voluntary solitary confinement, telecommuting will become less science fiction and more day to day fact.

20100215

leaving on a jetplane

[caption id="attachment_735" align="alignright" width="300" caption="Oh captain poopy-pants is cranky, he didn\'t get his FAA regulated maditory nap-time."]baby riding plane[/caption]

Well as Central Ohio prepares for another 4 to 8 inches of snow today I just printed my boarding pass for tomorrow. I'm off to the most exciting place on earth, Indiana. I will be meeting up with the newly assembled SOA team, hopefully hitting the ground running making all kinds of fun message flows and translation logic. I'm excited because when I wrote climbing a mountain on November 6th, 2009 this was the mountain I was talking about. The road to get started has been mired in difficulties so far, we've accomplished a lot just none of the programming yet. This week will hopefully be when the tools get handed over and I get the keys to dad's car, so to speak.


What this means for me is the joys of post-9/11 nudity body scanners to take a 1 hour flight to Chicago to drive 45 minutes to a small town in Indiana. What this means for you is that between the jet-lag, long hours, corporate face time, synergy backflow, and spotty hotel internet there may not be any blog posts for a while. I know that this is a sad announcement, whatever will you do without me ranting everyday about technology?! Well this is my 91st post, and if you haven't been a reader from the beginning I suggest you dive into the archives, there are a ton of fun and interesting posts back there. If you don't want to dig around in the guts of this site like a blind man searching for a nickel, then look at the top of the right-hand sidebar, those are the most popular posts by hits. Other people, sentient human beings read those posts a lot, had nice conversations about them in the comments, and in general found them to be entertaining or informative.


Unlike other blogs comments never close at ihumanable.com and I haven't become jaded enough to stop reading the comments. Every time someone has a witty thing to add to the conversation I get an email about it, I read that email, sometimes I reply in the comment thread, if you are extra special lucky you may even get an email reply.


I hope to be able to post at least once in my absence, maybe while waiting for the snow to get cleared so that we can leave, but it may not happen. Fret not though, there is a wealth to read and enjoy, most of the stuff I write isn't really time sensitive (although some of the bad jokes might be). Thank you for reading, once I take care of this pesky job that allows me to pay for the hosting that provides you with free entertainment, I will be back to write more.

20100212

arbitrary

[caption id="attachment_730" align="alignright" width="199" caption="Fucking arbitrary decision, but a good one!"]Lion riding a horse[/caption]

Rant time, oh it's Friday rant time! You see we are making progress on our project at work up to the point that I will be traveling to beautiful Indiana next week to get some specialized training and toolkits. It's a big deal for our project and we are all very happy, I'm glad to get to be an important part of all this and actually look forward to it. I will be heading out there with two other guys from the company and they will be leaving on Wednesday while I stay until Friday. Here comes the fun part, how to drive around after Wednesday?


Rent a car! I hear you shouting at your computer (don't worry I can't really hear you, you are just very predictable). Well it may shock you to find out that I'm only 24 years old and because of the actuary tables I am not allowed to rent a car. You see you have to be 25 years old to do certain things, like rent a car or hotel room or get a nice break on your car insurance. Less than 25 and you are a hellbent young rogue ready to destroy any car you get your hands on. It doesn't matter that I've spent the last 6 years working hard studying at University to get 2 degrees or designing software for a Fortune 1000 company, no the table says 25 years old and by the hammer of Thor we are going by the table.


Now there is a monkey wrench, one I'm sure we'll get settled and figured, but it's not the first time arbitrary decisions have been made that have negatively impacted me, and it probably won't be the last. On this current project we deal with messages and messages have many pieces parts, in the old system you could have thousands and thousands of different pieces parts. In this new system you are limited to about 1,000 unique parts, it has been a challenge trying to figure out how to get everything to work. Is there some reason for this, NO! its completely arbitrary, they just chose a large number, and in the next iteration of the product that limitation will be lifted.


I remember hearing a story from a fellow developer about how new software was installed that all webforms had to send their data to so that it could be sanitized (despite the fact that it was already being sanitized by our program). This new cog in the machine had a limit of 100 fields per form, now only in the dreams of a fevered madman would you need more than 100 fields. Well it did, it was a complicated form dealing with complicated government requirements and to make it easy for the user it was dynamic and beautiful and everyone loved it. Then they had to completely break it, make it work in chunks and find some clunky workaround so that it would never submit more than 100 fields, the users hated it, the developers hated it, everyone hated it. Some digging went forth and the 100 field limit was completely arbitrary and you know in the code somewhere was something like this.


[java]
FormField fields[100]; //No one will ever need more than 100 fields
[/java]

This arbitrary decision causes a world of hurt. Now we don't want to go down the super configurable, just change this XML file which will regenerate this XML file which will be used to partially populate this .properties file which will be dynamically.... fuck you. Just that if you have limits there better be good fucking reasons, or else remove them.


If you are going to limit me to 140 characters there better be a good reason, oh because of the inherent limits imposed by SMS, fine Twitter I understand perfectly. You can't open two spreadsheet with the same name even if they are different files, what are you doing Excel?!


The next time you are about to code something that will result in a clearly arbitrary error message down the road, you better have a good reason to. There better be some sort of machine limitation or design limitation or something, because its not the limit that pisses people off, its their arbitrary nature.

20100211

password usability

A friend pointed me to an interesting article on A List Apart. If you don't want to read through the whole thing it basically says that a lot of password resets are caused by people remembering their passwords correctly but mistyping them. The concerns that once made masking passwords with asterisks are now being eclipsed by the usability problems this design has introduced. The post goes on to describe two potential alternatives, a toggle to show the password in plaintext (similar to the WiFi configuration screen in Windows) or to show the last character typed while masking the rest (similar to the iPhone or Android password inputs).


Both of these options are interesting and I personally would like to see either one gain greater acceptance, although with the rise of password managers built into Operating Systems and Web Browsers it seems less and less necessary. The problem with both of these techniques is discussed in the article, that changing the functionality of the password input undermines the user's confidence in your site's security. This is why I think that changing the nature of password inputs is dubious at best until it gains widespread adoption, maybe if Google were to implement them or some other web giant. Until that day I think a fine alternative would be Mattt Thompson's Chroma-Hash.


Chroma-Hash augments a password input with extra information. Something that is easy to remember and easy to notice when its wrong, a color swatch called the Chroma-Hash. Let's take a look at how it works.


[caption id="attachment_722" align="aligncenter" width="461" caption="The password (conveniently enough 'password') generates the colorful hash to the right."]chroma hash example[/caption]

The passwords match because the colors match, when entering your password you are informed of mistypings immediately by the hash being incorrect. Let's take a look at what happens if we carelessly fat-finger the confirmation typing "passworf" instead of "password" like it should be.


[caption id="attachment_723" align="aligncenter" width="465" caption="One little letter completely changes the Chroma-Hash, immediate feedback"]chroma hash with mismatch[/caption]

Small changes in the password generate big changes in the Chroma-Hash. The human brain is one of the best pattern matching engines in the world, Chroma-Hash leverages this fact. Very small changes in a sites design or color scheme are detectable, that's why people make a big deal when a site they commonly visit changes things, even slightly. This makes Chroma-Hash ideal for serving as a "password proxy." Others can see the Chroma-Hash and gain no information about your password and yet it instantly gives you a wealth of feedback about whether or not you have entered the correct password.


Take a look at Chroma-Hash, fork it on GitHub, implement it on your website. You get the advantage of recognizable feedback without needing to change the fundamental way in which the password input works.

20100210

baby steps

[caption id="attachment_714" align="alignright" width="200" caption="Today I conquer steps, tomorrow France!"]baby going up steps[/caption]

Last night I had the unadulterated fun of updating a client's blog. After handing off the reigns everything was going smoothly until some unknown combination of plug-ins was causing the mobile version of the site to be served to actual browsers. I was called in to triage the situation, nothing too dire, just two plug-ins with some configuration problems, once I told them to play nice, all was fine. It was then that I noticed there were 15 plug-in updates waiting and the whole WordPress install needed to be updated. So I took the time to take a backup and update everything, nothing too serious (for anyone that doesn't use WordPress, updating is a one-click affair now) just the day to day stuff that needs to be done to keep the lights on.


This got me thinking about upgrading and Google's decision to discontinue support for IE6. Some people are crying foul, saying that IE6 should still be supported, I'm decidedly of the I-hope-IE6-dies-in-a-fire camp. Lest this become a let's bash on IE6 rant, which it so easily could, fucking broken box-model CSS1 piece of shit, let's get back on track. The people that are upset about this are not technical luddites clinging to the familiar. Digg performed a user survey to figure out what they should do about IE6 and part of that was asking why people used IE6. Of IE6 users 70% responded that they either lacked administrative rights needed to upgrade IE6 or were told by someone at work that they could not upgrade. For anyone that has ever worked in Corporate America this should be no surprise to you. The nagging question though is why?


Why would the IT department, home of nerds always hell bent on trying out the latest and greatest, demand that you use a 9 year old piece of software? There are a number of reasons people point to.



  1. Mission Critical application is only compatible with IE6

  2. IE6 comes installed on all the machines

  3. IE6 is maintained by Microsoft, less of a headache for us

  4. Licensing / Partnership / Legal-mumbo-jumbo


If you fall into #4, I pity you. #3 is laughable because although the updates might be bundled with Windows Updates you aren't saving yourself any administrative costs by putting the most targeted web browser for malware on all your machines. #2 is true, but if you are big enough for an IT department you probably have some way to push software to machines (or you could, and this is a shocking idea, trust your employees to choose a browser that fits their needs). I really, really, really want to talk about #1 though, because that is the point of this whole post.


I've worked at a number of places that used some sort of Web Application that was deemed critical that only worked right in IE6. This was then used as a justification for never moving past IE6, the timecard application won't work or the spline flanger won't have those cool IE6 exclusive filters. I understand that no one wants to expend resources on a "working" application pulling it into the future. But what is the longterm idea here, are you just going to use IE6 forever? This prevailing consensus seems to be ultimately self-defeating. Google recognizes this and as they try to push what can be done with the web they find themselves expending far too much time and energy on a nearly decade old piece of technology.


The nice part about fixing an application on IE6 is that you avoid some cost in updating it to make it work with other browsers like IE7. You could have spent a week or two adding in some CSS fixes and some markup changes and bingo bango IE7 now works like a charm. Instead you decide to use Corporate Fiat to force the continued use of IE6. Well now IE9 is looming around the corner and instead of paying a week here or a week there you now have a mountain of technical debt to pay down to get it to work.


Although IE6 is a convenient example, we can see this happen in all parts of the software development world. This program compiles when linked to an older version of the library but not this new one, I don't have the time to figure it out, well just link in the old one. This is all well and fine to meet the deadline but you have just accrued some technical debt. Later when that library gets some great new speed-up or killer new feature, there you will be plodding along with version 0.91 because you can't get it to work with the new hawtness. The problem is that these debts are not additive, they compound on one another and make what would be a simple conversion to 0.92 a nearly impossible conversion to 3.76.


Keep your tools up to date, keep an eye on what's up and coming, you don't have to be the first to jump on the latest and greatest but when a new version has proven itself and become the new standard move to it. Technology doesn't stand still, if you don't keep up with the latest stable releases you will find yourself in a tight spot trying to do an overnight re-engineer of your product. Take the easy baby steps from version to version so you don't find yourself having to take a technological leap down the road.

20100209

time

[caption id="attachment_708" align="alignright" width="300" caption="Finally was able to get the clock to show me its good side, dirty girl"]clock[/caption]

Don't worry this won't be a rant about mortality or getting things done or any of the philosophy that has been dominating this blog as of late. This is back to basics, a discussion about software and a particularly tricky aspect of it, time. Not time as in, scheduling and managing time, but something far more fundamental representing time. It is an insidiously tricky problem and one that can be quite difficult to wrap your head around.


The problem comes from how we think about time as people living normal lives. "I'll meet you at 3 o'clock" is a rather dull and completely normal type of phrase to say to someone. As two normal people living normal lives, this simple phrase "3 o'clock" is plenty to convey when they should meet. This is because there exists a great deal of unspoken context between the two parties, if we are meeting for a business meeting I clearly mean 3:00pm not 3:00am. If we both work in the same building I probably mean relative to the our shared timezone, 3:00pm EST not 3:00pm GMT. There is a world of shared unspoken context that makes human-human time discussions easy and natural.


Computers are really stupid though, they need everything spelled out. If you were trying to store time you might take a naive approach at first and just store the string "3:00" maybe if you are really thinking it out you would store "3:00pm EST." This method soon starts showing its weaknesses as its hard to compare times, or perform operations on them. How many hours are between 2:00am EST and 5:30pm CST? There is a nasty problem to try to solve unless you have some sort of way to represent times in the abstract.


In steps a number of formats to represent time. There is the venerable Unix Timestamp which is the number of seconds from Jan. 1, 1970 as of my current writing it stands at 1265738039, but feel free to check for yourself. Then there are numerous proprietary formats like Microsoft's, Oracle's, etc. These all allow you to represent an exact moment of time in a portable abstract way with no dependence on the cavalcade of context us fleshy humans share.


Well problem solved, just bust out your favorite abstract representation and you are done. Not so fast, there are many other considerations to take into account when dealing with time. There are of course the tricky problems of Daylight's Saving Time, leap years, and the like. Imagine you are trying to add an event to a calendar system everyday at 5:00pm EST, you think you could just add it to today and then just add 24 hours and create a new event. DST hits your algorithm over the head at some point and everything is off an hour, oh no! Also now you have a ton of data to represent one basic fact, something happens everyday at 5:00pm EST. Its only one fact, you should need one record, not an infinite number of 5:00pm EST records. This hints at the next difficulty of time.


Humans think about repeatable events (sometimes with complex and insane rules) as commonplace and easy. This thing happens on the third Thursday of every month, unless of course Monday was a holiday and then it gets shifted to Friday. The problem with time and dates and repeating events is that human beings erected a ton of complex processing rules before they realized we were going to try and digitize them. These are difficult to represent and difficult to get right.


At first the task of representing arbitrary points and spans of time seems fairly straightforward, but it is a complex and nuanced task, like most things the devil is in the details. Before you go off half-cocked building up your own representation, take a look at some established formats, like Unix Timestamps, RFC5545, and ISO 8601.

20100208

experience as a force multiplier

[caption id="attachment_701" align="alignleft" width="300" caption="Oh DS cat you lovable scamp, don\'t use up all my healing potions"]cat saying "I weveled up yur guys"[/caption]

Over the weekend I experienced two things, one rather normal for a "computer guy" the other less in my wheelhouse. The first thing was that my girlfriend wanted me to help her register a web domain for her dad for his birthday. I have done this twice before, once for the website you are currently viewing and one other time for http://prosper-lib.com (which just redirects back here for the time being). I remember the first time I registered a domain and carefully read all the forms, making measured decisions, and hesitantly moved from step to step unsure of myself. It took an hour or two and then a few more to set up the hosting and get ftp access, set up wordpress and get everything up and running. In total it took a few hours. I didn't have such a luxury of time, she needed me to get the site up and running, preferably with a custom "Coming Soon" page before lunch, it was 11:00 so I only had around an hour. I clicked over to the site, click click click type type type, and it was up and running. I looked over at the clock to see how badly I had overshot the envelope, 11:15... wait what?


The second experience I had this weekend was tearing up some carpet at my new place. I have undertaken tearing up 850 sq ft of carpet to install new wood flooring in my house. The first night I was able to remove carpet from two rooms in just under 4 hours. It was my first time, and I went slowly following the instructions and making sure that I did everything just right. The second night I worked with an experienced floor guy, we finished taking out the mats, staples, tack strips and carpets from 5 rooms. Over the weekend there was one room left to do, I was able to knock it out, carpet, mats, tack strips, the whole nine yards in about an hour.


After these two events it solidified a point in my head, experience is the best force multiplier. In the first case it was my own hard won experience (not that there is really anything that difficult about registering a website). After performing the task a few times I gained an experience that multiplied my force by an order of magnitude, what once took a few hours now only took a few minutes. In the second case I was able to leverage someone else's experience to augment my own. I learned countless tricks to getting the job done quickly and efficiently, but more importantly I gained a huge amount of confidence. Having someone that already had mastered the task work with me and confirm that I knew what I was doing gave me all the confidence in the world to go in on Saturday and tear up the last room lickety split.


There were two take away points from these experiences. When doing something new seek out someone more experienced than you to pair with, even if its just temporary. Fully participate with them, ask questions, and do not let them do everything for you. This is the best time to fail, you will get immediate feedback from the more experienced party about how you are failing, why you are failing, and how to avoid failure next time. This will give you a foothold to quickly learn by doing. The second point to take away is to experience lots of things, even if you suspect you will fail. Life is full of failures, but those failures impart experience which is the best force multiplier to get to success.


I would be unable to count the number of times I've ruled out some great idea because it took me down some unknown avenue, and I was afraid of failing. I have no experience charging people money, I don't know how to start up a business, I don't know how to make a website. These are all things I've said to myself at one point or another, the thing I realize now is that no one is born knowing how to do these things, there is only one way to get experience, try new things. Don't be afraid to fail, work hard to succeed, and know that no matter what happens the experience is valuable and worth it.

20100205

persist

[caption id="attachment_696" align="alignright" width="300" caption="measure twice, turn once"]semi truck[/caption]

I just witnessed an interesting even unfold in front of my current client's building. Today in beautiful Columbus, Ohio we are experiencing one hell of a winter storm, lots of fun. A semi truck decided that our parking lot was a great place to turn around in, he just apparently forgot to measure. CRASH BOOM and the lamppost has changed from its normal 90° to a more precarious 78ish°. After striking the lamppost and ramming a concrete barrier the semi was fairly well wedged, but the driver continued revving and the engine struggled and moaned. This persistence paid off and after about 10 minutes of maneuvering he was free. Now to call the insurance companies.


This got me thinking about the nature of difficult situations (self-imposed or otherwise). I also had a breakthrough largely due to persistence today, so happy coincidence. After months of working on documentation, being told that I would never be allowed to program the sacred system, the tides have turned (thanks to an excessively high quote) and the coach has called me up, and I'm ready to play under the bright lights.


I've just completed printing out 400 pages of documentation (my weekend is going to be so much fun!) and I will prepare myself for the difficult task ahead. My persistence has paid off, but the bigger realization is that persistence in general pays off.


To borrow one of my favorite quotes from Scrubs, "People are bastard-coated bastards with bastard filling." People don't ever want to think that you can succeed, and they are going to tell you no over and over again. There are two reasons for people to shut down your dreams:



  1. They have tried and failed, the task is impossible, your success would undermine them. If you succeed where they have failed then it isn't an impossible task, they just weren't good enough to do it.

  2. The have achieved it, but only by their virtue of their amazing talents and pure force of will. You don't have what it takes!


This is why it's vitally important to ask people for advice, sage wisdom, realistic assessments, but always keep in mind that very few people actually want you to succeed, especially if its in something they can't or already have. Don't worry though, these aren't bad people, it's human nature, and when you succeed you will act the same way. Success through hardship and against the odds plays a fun trick on the human brain, it makes you believe you pulled off the impossible. This feels amazing and so you will do anything in to hold on to it, and you don't want some dirty unwashed masses pulling off the same feat and diminishing it.


The big lesson here though is persistence, the most successful people in the world are those that were willing to have the door slammed in their faces 1000 times before making that sale. This is what you need to do, steel yourself for the struggle, prepare to be emotionally drained, and realize that this is what living is. Life is a stream of people telling you no, with the occasional yes. What separates the successful from the mediocre is that the successful are willing to wade through all the noes to get to the sweet succulent yes.


I've been paying my dues on this project for 3 months, and finally I get to go back to what I love the most, coding. There are a hundred stories of people trying and failing and trying and failing and trying and failing to finally succeed. It is definitely the path less taken, but it will make all the difference.

20100204

finite

[caption id="attachment_689" align="alignright" width="300" caption="all that eggnog caught up to him... the cocaine addiction didn\'t help either."]santa claus' tombstone with child crying[/caption]

Here's something unpleasant to think about, you are finite. Think about it deeply, drink it in, sit with it, feel like you've come to understand it, then wake up in a cold sweat thinking about it some more. There is only one certainty in this life, it will end. It gets worse, your time, your effort, your experiences, they are all finite. It's one of those big huge things in life that is too inconvenient to think about, so we don't. This is the core reason why most religions exist, coming to terms with one's own limited nature is terrifying. I've always been an Atheist, intimately connected to this notion that one day the unique thing that is me will be no more, just as for billions of years before I popped into existence I wasn't yet. It's something I tell myself I have come to terms with, but it is a daunting challenge and one that must be revisited from time to time.


As an aside: although I'm more than happy to discuss my personal spiritual beliefs I attempt to keep such charged matters off of this blog, if you would like to chat about it just post a short comment indicating so and I will contact you. I am an Atheist but I respect others' belief systems and don't intend to offend anyone.

I was driving home the other day and an interesting story came up on the radio. The main point being that as we age we sense that time is moving faster and faster. This is the spark that has led me to revisit one of the least comfortable aspects of our shared humanity. I've been pondering it for the last few days, thinking about my very real finiteness and trying to evaluate if I'm acting like a man with an expiration date. It was a mixed evaluation, in some ways I am, in others I am not, not a terrible fate by any stretch.


In my personal life I feel like I'm running full steam ahead. It might be the last week or so of 14 hour days working on getting my new house ready to move into. (After years of snacking and typing my body isn't as suited to the tasks of tearing up carpets and fixing walls as it once was). I've set real goals in terms of building skills, building relationships, and creating code I can be proud of, and I'm exceeding my expectations. I finally feel like my wheels have caught solid ground and aren't just spinning anymore.


There are still some areas I'm not happy with, I'm taking stock and taking action to address these. On the whole though, I feel like I'm spending my limited time here the right way for now. There are a thousand and one clichés about this subject: live each day like it could be your last, what would you do if you knew you would die tomorrow, etc. The problem is that they all take an outlook that is far too limited. No one could live each day like it could be your last, that's fucking stupid. If I knew I was going to die I'd spend all my money and live with complete abandon, you can't do that everyday of your life. It is the routine of life that makes this so difficult.


Life is finite but there is also a whole big pile of it. We like to think that we can wait and do something next week, month, year, etc. This is where all that "I always thought we'd have more time together" thing comes from. Want to open a bakery, learn to play the drums, fight a bear, there's no time like the present. If you are unhappy, you have to fix that. Other people that aren't you are probably going to tell you that you are foolish for trying to fight a bear, well those people can waste their lives while you're out punching Yogi in the mouth.


Don't let anything stand in your way, take it personally because nothing could be more personal. This is your precious finite life and its ticking away. If you are unhappy or feel like it's being wasted, do something about it. Nothing could be more important. It's not comfortable, and it can be frightening, but take stock of your life today and the whole time keep in mind, I only have so many days left, is this how I want to spend them.

20100203

bureaucracy

[caption id="attachment_677" align="alignright" width="300" caption="Wait until he meets the Undersecretary for Reduction Planning and Appropriation\'s new Assistant"]pentagon bureaucracy cartoon[/caption]

Bureaucracy, long the scourge of people who want to git-r-dun. There are some people out there that hate bureaucracy with a passion I can only really muster up for the pending zombie apocalypse. Make no mistake about it, I dislike bureaucracy, I think it is a wasteful but sometimes necessary evil. I've found myself ensnared in a bureaucratic nightmare for the last few months so I thought I would jot down some observations on bureaucracy.


Zombie Bureaucracy


(Wow lots of zombies in this post so far) A Zombie Bureaucracy is a bureaucracy that had some reason (however flimsy) for existing in the past but no longer needs to exist. Not unlike a zombie this bureaucracy manages to live long past its usefulness shambling into the future pointless and frustrating. This is caused by the ease that more bureaucracy can be created, at the stroke of an email someone can dream up a committee or process, but the difficulty in disbanding bureaucracy. It is disproportionately difficult to reduce bureaucracy because it looks like work, and people don't like having their work taken away, it reduces their sense of job security.



Dev: Why do I need 3 developers to sign off on every commit?
Mgr: Because 4 years ago two of the lead developers got into a pissing war over code formatting
Dev: Why didn't they just work it out?
Mgr: Because they had huge egos and management was too scared to fire either one
Dev: Is this still going on?
Mgr: No, Frank left 3.5 years ago because he didn't want to put up with Mark anymore
Dev: So why do I still need 3 developers to sign off on every commit?
Mgr: Because 4 years ago.... I guess it doesn't make much sense anymore.... That's the way it is!

Just like misery, bureaucracy love company


Bureaucracy is an attempt to control something, to take organic chaotic processes and make them orderly. The problem that often arises is that bureaucracy is its own organic chaotic system, that then requires more bureaucracy ad infinitum. If you've ever been in a meeting where all you decided was the schedule of meetings, congratulations, you are in the Matryoshka doll hell of bureaucracy.


Meetings


I loathe meetings, a bunch of people talking about the work they could be doing if they weren't in meetings all the damn time. Meetings are seductive, they sound and feel and look like work, but they are occupational masturbation. No one has ever brought a product to market because they were able to make it to 10 meetings a day. Meetings have almost no value, sometimes they are necessary, but not nearly as common as corporate culture would have you believe.


And the rest...


[caption id="attachment_683" align="alignleft" width="289" caption="Forgot to file the A34C-Bh amendment releasing liability for tongue trauma"]hot tamales candy box[/caption]

I could go on and on, but I will cut the rant short and get to the point. Bureaucracy at its core is about trust, or more importantly the lack thereof. I don't have my girlfriend fill out the A34C-B form (Confirmation of Confection Purchase Agreement) before running to the store to ensure that she understands that I would like her to pick me up some Hot Tamales. I trust her at her word, there is no need for such ridiculous formalities.


When an organization has enshrined itself in a monument of bureaucracy what it is really saying is, "We've been burned before and now we don't trust you." We don't trust you to deploy your code correctly, we don't trust you to design things properly, we don't trust you to do X, so let's get a bunch of people together to review it. This is fine in small doses, frankly I want people to review my code and my designs, I make mistakes like everyone else. But when taken to the extreme it takes a toll on your motivation, on your creativity, and on any preconceptions that you knew what you were doing. Bureaucracy is the surest way to crush your workforce into a homogeneous mix, you will catch the awful at the expense of the great.

20100202

stupid ideas

[caption id="attachment_671" align="alignleft" width="263" caption="No caption needed"]stupid burn[/caption]

I want to be honest with you for a minute, I have lots of stupid ideas. I have them all day, in the shower, while eating a sandwich, while writing my blog posts, stupid ideas come streaming into my head. I used to just ignore them, "No one needs a peanut powered lawnmower" I would tell myself. The interesting turn was when I started writing these stupid ideas down.


Some stupid ideas will always remain stupid ideas, some though will change over time. That dumb idea might actually just be so brilliant it looks stupid. It might be so big and scary that you've convinced yourself it's stupid. It might not be a bad idea at all, after some careful reflection. It might even be a wonderful idea.


Why do we throw away our stupid ideas? Is it because we don't have time for them, bah, that can't be it, you are currently wasting precious time reading the stupid things I have to say. Is it because we are so swamped with brilliant ideas that we have to prioritize, probably not, or else you'd be sipping margaritas next to your perpetual motion machine. No, the answer is much simpler, its because we are afraid or failing. Stupid ideas are failure babies, a dumb idea grows up to be a hairy ugly failure, and we have learned in the school of life that failures are bad, so we avoid them.


Have you ever talked to a 4 year old? I have on various occasions, interesting little buggers those 4 year olds. They will tell you every little thought that bounces into their overly energetic little brains, no matter how wrong or foolish. They don't mind failing over and over again, they will insist the moon is made of mashed potatoes and not feel bad at all when you tell them the truth. And most importantly they learn a ton of information.


Now don't take this the wrong way, I don't think you should just go around like some half-cocked lunatic spouting off every little thing that pops into your head. There was a time for that and if you go into the board room now insisting the moon is made of mashed potatoes you will be roundly mocked and rightfully so. What I am advocating is the preservation and analysis of your stupid ideas. Let's take a look at some stupid ideas quickly to see why this is useful.



  • Idea #1: A blanket that has arm holes and you wear it backwards or something... yea like a backwards robe! Pretty fucking stupid, but better known as the Snuggie, which has become a huge success and sold tons.

  • Idea #2: Instead of executing the code on this machine, let's build another machine that doesn't actually exist and have that fake machine live on the real machine and have code execute on the fake machine.... What? What are you rambling about Jenkins, get out of my office! Jenkins just invented the Virtual Machine architecture that power things like Java's JVM and .Net's CLR.

  • Idea #3: Wouldn't it be awesome if you could tape together 4 iPhones and have like a really big iPhone. I'm sorry, did you just smoke a bunch of peyote?! That's dumb.... Or its the new iPad.


You get the picture I'm trying to paint here. A lot of the amazing things we take for granted and depend upon everyday, things that seem like great ideas now, all had a moment in time when they were some far-fetched pie-in-the-sky dream. Most ideas start off as stupid ideas. Stupid ideas are important and valuable, not all of them, but there will be some diamonds in those dirt clods. You just have to be smart enough to sift through them.


My advice to you is to jot down those stupid ideas you normally throw by the wayside. Revisit them every once and a while, re-evaluate their "stupidness." Sometimes you will find a diamond in the rough, sometimes a stupid idea can be the spark needed to have a brilliant idea, and sometimes you will just get a good laugh out of the stupid things you think up.

20100201

open format

[caption id="attachment_663" align="alignright" width="300" caption="We\'ll just lock your data up in here, don\'t worry we\'ll open it up later if you need it. This is what a closed format sounds like"]bank safe[/caption]

Back in the days when a computer had 64k of RAM and high-capacity meant you had a double-sided floppy disc, there were these funny little things called binary file formats. We still have them today, they are the "pattern" that a program will write and read to and from disc to save and load a file into a program. Some are open, like image file formats, and some are closed, like Microsoft's binary blob formats for Office. As the world has advanced and storage has become dirt cheap people started looking for an easier way, and whenever there is a problem that we don't have a tool for yet, we reached for XML.


XML is actually not a bad fit for this problem domain, its a little on the ugly side, but that's ok, very few people crack open a raw file to read it. The people that are cracking open files in a text editor to peer inside are probably tech-savvy enough to read a little XML. The big bonus of switching to XML, like ODF or DOCX, is that there are very mature XML parsers and writers for almost every programming language. That means that a determined programmer can grab a copy of your format spec, her favorite XML parser, and write a program that operates on your file format. This is the essence of that oh-so-marketing-speak word synergy.


Now I would love to take credit for this awesome idea, but if you've read The Art of Unix Programming you will recognize this as nothing more than an extension of the Rule of Composition. Now that you've read the Rule of Composition (and you should make the time to read the rest of the Art of Unix Programming, even if you never plan on programming anything for Unix, the lessons within are just in general great to know), you will recognize the inherent advantage of having parsable file formats. Now that I have cunningly converted you over to my way of thinking, what format should you use?


XML


XML (eXtensible Markup Language) is the old standby, it is reliable, standardized by the W3C, well-supported, and there are tons of tools around for playing with XML. XML is "human-readable" for those of you unfamiliar with it here is an example.


[xml]
<book>
<title>Example Book</title>
<pages>
<page>
<![CDATA[
...page stuff here...
]]>
</page>
<page>
...you get the idea...
</page>
</pages>
</book>
[/xml]

Its a bit tag-heavy and that means a slightly large file size, this isn't a huge concern since storage is so cheap, but you should be aware of it. XML has a neat feature called DTD (Document Type Definition), armed with this most parsers will be able to tell right away if a document is well formed. XML is big and clunky but a fine choice for many applications, especially if you need typing information.


YAML


YAML (YAML Ain't Markup Language) is the format of choice for the ruby community. YAML is well supported by most mainstream programming languages, it is a more lightweight choice than XML. Here is the same thing as above in YAML.


[ruby]
book: Example Book
pages:
- page: >
...here goes my page data...
- page: >
...you get the idea....
[/ruby]

YAML uses the structure of the text to indicate the structure of the data. Ending tags are dropped and indentation becomes more important. YAML looks simplistic at first but has a wide-array of functionality hiding below the simple hello world examples. References, hashes, arrays, and much more are possible with YAML. The specification allows you to make concise documents that contain an astounding amount of information.


JSON


JSON (JavaScript Object Notation) is a lightweight way to represent data structures. JSON excels by being incredibly simple to learn and use. There is native support for it in JavaScript which makes it ideal for use in AJAX (which would then technically by called AJAJ), and there are JSON parser available in most mainstream languages. Here is the example from above in JSON.


[javascript]
{title: "Example Book", pages: [ page: "...page stuff goes here...", page: "...you get the idea..." ] };
[/javascript]

Just like in JavaScript everything in JSON is a Hash or Array. JSON is a simple typeless data recording system, perfect for dynamic languages. JSON is a proper subset of YAML 1.2 so most YAML parsers can also parse JSON. JSON's incredibly lightweight nature lends itself for being used when sending data over a wire or when space is at a premium.


BSON


BSON (Binary JavaScript Object Notation) is a binary form of JSON. It is meant to be more space efficient than JSON but maintain the ease of parsing. It is described as "A General-Purpose Data Format" and was devised as the serialization medium for the MongoDB project. It is an interesting format to look at if space is at a premium and there are already parsers for C, C++, PHP, Python, and Ruby.




File formats no longer need to be gnarled, tangled messes. We have new standards that allow us to freely share data created in one program to be analyzed, manipulated, and turned into something new in another program. Being able to freely interoperate is the future of computing, it is the driving force behind web standardizations, micro formats, and open file formats. The next time you need to persist data to the file system, resist the urge to roll your own serialization scheme, and check out one of the technologies presented above. The world will thank you.