One Year Later

[caption id="attachment_930" align="alignright" width="300" caption="Where I've hung my hat this last year"][/caption]

For the last year I've worked as the Director of New Media and Web Development for 614 Media Group. It's a lofty title, meant to inspire all who read my email signature that, "Hey this guy is a bigshot." In reality what I did was much more mundane, I wrote LAMP applications. For the last year I've sat down everyday at this Mac, popped into TextMate, and wrote PHP. I was king of the castle, and being king is good, I got to choose my tools: Mac + TextMate + Git + MAMP + MacGDBp + GitHub + GitHub Issues + Flourish. I got to work on interesting products, if you had told me a year ago that in a year's time I would have written both a Groupon Competitor and a full CMS system, I would have told you that was crazy talk.

Over the last year I have grown as a developer and as a system administrator and as someone trying to get traction for a new product and as a million other hats I got to wear. For the first time in my career I was able to go from concept to "functioning-putting-money-in-the-bank" web application. This is huge. Having been a consultant for years before this job, I spent most of my time maintaining other peoples systems, throwing in a feature here or there, coding up a new report, stuff like that. At 614 I was able to start from nothing, build up a database schema, create the framework and the code, paint it all pretty, launch, iterate, relaunch, iterate, tear my hair out, wake up to fix problems at 3am, curse the server for being slightly different than development, drink more caffeine than any human should drink, but most importantly learn and grow.

With learning and growing in mind, I have an announcement to make, at the end of this month I will be leaving my job at 614 Media Group. I leave on a good note though, the software is stable and quietly humming along bringing in revenue. I am leaving behind a framework with a proven track record, a stable foundation that can be built upon. But most importantly I'm leaving as a friend and supporter of 614, wishing them nothing but the best for the future. 614's curse is that they put a lot of responsibility into your hands and if you are good at it, other people will take notice, and those other people may come in and make you an offer you can't refuse.

[caption id="attachment_935" align="alignleft" width="245" caption="They made me an offer I couldn't refuse"][/caption]

I'm not sure how much I'm allowed to say about the new position, but I will be moving out to the West Coast this month to start a new chapter in my career. For the PHP folks out there, do not worry, I will still be rocking PHP (but also probably rocking some ruby and java as well and anything else they want me to learn). This is a big, stressful, somewhat scary decision to make, but I believe in the company that I will be joining, and after meeting with their people am nothing but excited to work with and learn from them.

Growth and change are good things, they can be painful and stressful and sad sometimes, but the minute you stop growing as a developer you've given up, and I love programming too much to give up. This next month is going to be a blur of packing and moving and unpacking and learning a new area and learning a new codebase and new people and new places. Parts of it will be uncomfortable, parts will be stressful, but overall it will be a learning experience I couldn't get anywhere else, and I wouldn't have it any other way.

Excel and PHP

[caption id="attachment_920" align="alignright" width="380" caption="You got Spreadsheet in my Web App, you got Web App in my Spreadsheet... and it's delicious!"][/caption]

Note: I may seem a bit harsh on Excel, but I actually think it's an amazing piece of technology, I just don't like how people abuse it.

Web development for business people follows a fairly predictable path. Here is a generalized form of it.

1. Business person encounters problem


2. Grabs Excel, Excel being the magic fix for everything


3. Creates abomination of a solution in Excel, it grows too large, concurrency and consistency issues creep in


4. At this point normally a web developer will be brought in to "Make this Excel sheet work on the internets"


5. Work work work work work...


6. Everyone is delighted


7. Hey is there anyway we can dump these value back into Excel?!


8. At some point they will use the dump to create some new spreadsheet, go back to step 3.

Now of course it's not always this bad and normally there are really good reasons to dump things back into Excel, instead of bothering the web developer with a bunch of business analysis, they can dump the data and perform the Excel wizardry they love so much. I actually think this is a good idea, instead of me baking in calculated values or reinventing Averages and Means and all the other cool stuff Excel can already do, I just drop them off some nice raw data and let them have their business fun. They get exactly what they need, I get exactly what I want (less fragile code with less crazy business rules) and everyone is happy.

Now in PHP there are various solutions to dumping out Excel. The first would be the venerable Spreadsheet_Excel_Writer. I've used it before and it works, it's a port of a Perl library called Spreadsheet::WriteExcel, but there are a few drawbacks. The first is that it's a PEAR module, you use PEAR to install it and it brings along all kind of PEAR stuff like PEAR_Exception and the like. The second is that on the homepage for Spreadsheet_Excel_Writer is this very happy message, "Spreadsheet_Excel_Writer is outdated and needs a complete rewrite. If you want to help with this task please get in touch with us. Otherwise we don't recommend this package for new development." So in short, we should avoid Spreadsheet_Excel_Writer.

The next thing you might find in your googling is PHPExcel which writes out OpenXML 2007 files. This is another large and much more well maintained library for working with Excel files. If you need any advanced features I would highly suggest looking into this one.

But I want something simpler than this, I just want to dump some data out. I don't need to have all these bells and whistles. Also I hate the fact that to write out a few cells in most of these Excel Libraries requires the following.

[php]
$objPHPExcel = new PHPExcel();
// Add some data
$objPHPExcel->setActiveSheetIndex(0);
$objPHPExcel->getActiveSheet()->SetCellValue('A1', 'Hello');
$objPHPExcel->getActiveSheet()->SetCellValue('B2', 'world!');
$objPHPExcel->getActiveSheet()->SetCellValue('C1', 'Hello');
$objPHPExcel->getActiveSheet()->SetCellValue('D2', 'world!');
[/php]

The way these libraries work are just, ugh. They are very heavy and when you are just trying to plop some values into a spreadsheet so someone can have at them, they are overkill. I found myself rewriting the same solution again and again, it's a fancy little solution using some pack() magic that I found over here. Here's what a simple little spreadsheet looks like.

[php]
require_once('Excel.php');
$sheet = new Excel('My Awesome Spreadsheet');
$sheet->label('Hello'); //Write out 'Hello'
$sheet->left(); //Move the cursor left
$sheet->label('World'); //Write out 'World'
$sheet->down(); //Move the cursor down
$sheet->home(); //Head as far left as possible
$sheet->number(1337); //We are so 1337
$sheet->send(); //Melt the user's face off
[/php]

This has met my needs and you can find it as a Gist right over here. I wanted an object oriented way to use those functions and I wanted some help in moving the cursor around and some simple sanity checks on things so that I didn't ruin anything. It's got plenty of comments in code and should be pretty simple to pick up. You can write out labels or numbers and you can move the cursor all around, left, right, up, down, top, and home (top being the first row, home being the first column).

For right now this is meeting all my needs and more, I've recently been playing around with the fact that Excel will happily turn HTML Tables into Spreadsheets, I've got a library cooking for this but have been playing around with some interesting ideas for styling and other advanced features. Until then hopefully this will help any PHP programmer who just wants to dump out some data without going into dependency hell.

Happy Coding

Get my handy dandy Excel Writer here

Fuck You Internet Explorer

[caption id="attachment_907" align="alignright" width="360" caption="This is for you IE"][/caption]

Be Warned: This post has very little technical value, more just a rant against my most hated enemy

I hate Internet Explorer, with a fiery passion. Normally I just code along in beautiful browsers and use cross browser CSS and JavaScript frameworks. It's gotten to the point that I can code something up in Firefox and make it pretty and go over to old IE-saurus Rex and load it up and it looks 90% of the way there and a few hacks later it's fine. You see I still hate IE, but I've ingrained in my brain somewhere what things are "safe" and will probably work in that POS browser and which will require hours of tweaking, reading, googling, hacking, gnashing of teeth, and general misery. I try to avoid the latter.

So If I've developed this system to insulate myself from the misery of IE, why the hate today? IE9, that's why. I've read several reviews both positive and negative, but at the end of the day I'm not switching browsers, so I really care about one thing and one thing only, does IE9 make my job as a web developer better or worse. I've come to the conclusion that it's worse, and for that I have a lovely pile of "Fuck You" cake sitting right here with IE's name on it.

Not Available for Windows XP

First of all, IE9 is not available for Windows XP. Great, game over right there you fucking idiots. I don't think people should use 10 year old operating systems, but people are stupid, see: the continued popularity of American Idol for proof. Firefox is taking the responsible route and helping these poor souls bring their browser into the 21st century even if they are fine with an ancient OS. As Firefox Engineering Director Johnathan Nightingale says about XP, "...the best metrics that we’ve got say 40 to 50 percent of the web is still on XP. That’s too big for us to just leave them behind"

Missing a metric ton of shit

IE9 is just missing a ton of stuff. Stuff that web developers care about, stuff that would make our lives easier and happier, stuff that IE9 rapes right before sleeping soundly on a pile of skulls. Paul Rouget of Mozilla makes this case, let me quote some of the juicier parts.

No CSS3 for you

Here's a list of CSS3 stuff IE9 won't support: CSS3 Transitions (for animations), CSS3 Text Shadow, CSS3 Gradients, CSS3 Border Image, CSS3 Flex box model. Great news, don't put photoshop away just yet, you are going to need 1px wide gradients and text with shadows baked in, and fuck it.

Suck it UX

Hoping you might get HTML5 History API, Drag'n Drop from Desktop, or Web Workers to speed up your applications. Not with IE9 you won't.

Not the point though

Let's not get caught up in a feature list shoot out though, because that's not the point. Let's move onto the next subject.

Fractured even further

Does IE9 make my job easier, nope not at all. Microsoft's all consuming obsession with backwards compatibility means that they won't force anyone to upgrade. While other major browsers are moving to a model of strongly-recommended / automatic background updates to keep everyone on the latest and greatest, Microsoft is still playing the game by the old rules. Now the browser landscaped is fractured even further, I have to make sure my code works in IE6, IE7, IE8, and IE9, and if I want to be really sure I should also check in IE7-compatibility, IE8-compatibility, and Quirks Mode. Microsoft has never made it easy to run different IE versions side by side, I've used several products to do this and they all work to varying degrees. Now I will have to run another different virtual machine with either Vista or Win7 so I can test in IE9. And with no / terrible developer tools for all of these browsers, well its a fucking nightmare.

Conclusion

I wish I had one. At this point the only thing I've concluded is Fuck You Very Much Internet Explorer. Here's an idea, stop already Microsoft. I don't care how good you make IE10, the fact of the matter is that since you won't force upgrades, it isn't fucking helping. The only thing you are doing is making more work for the people that actually make the web beautiful. I fucking hate you. Fuck you very much.

Lipsum Lives

[caption id="attachment_903" align="alignright" width="300" caption="This is how I feel right now"][/caption]

I've been hard at work for the last 7 weeks, sprinting from start to finish on a custom CMS system that will manage our company's various online publications. It has been a grueling experience. When I started my friend, Jeremiah, told me, you are crazy, don't do that, I'm going to hit you in the face. And yet, undeterred, I soldiered on.

I was not so foolish or prideful to think that other CMS systems were inferior or stupid or ugly. I use WordPress to power this blog, and let me tell you, WordPress is friggin sweet. Drupal 7 just came out, it looks to be an amazing piece of software as well. I looked at various PHP CMS systems, but at the end of the day I decided to build my own for the following reasons.

• I have no experience with developing for WordPress / Drupal / Insert favorite CMS here


• Time was at a premium


• Flexibility is a must

Now I'm sure there are Drupal Ninjas and WordPress Wizards out there, thinking, if you need to get a flexible website up fast, just use [the CMS system I know and love] and it will be easy peasy. I'm not against learning, and I try hard to fight NIH syndrome. The long story short version of this was I was told, "5 weeks we need A, B, C, D, E, F, G and it needs to work in X, Y, Z, (several other alphabets worth of letters) ways." I knew that I didn't have the time to climb up the learning curve of a CMS and become a WordPress Guru or Drupal High Priest. I did have time to write software, and since I know PHP and Flourish like a fat kid knows cake (aka, like I know cake), I decided that the route with the highest possible success was to write something.

The next most important point is that I work for an amazing company. We are growing fast and the future is anything but certain. Ideas for the publication sites come fast and furious, sometimes radical, sometimes insane, sometimes brilliant, having total control of the software is a must. Getting something up was important, but building a solid foundation that would allow us to build new engaging features in the future was even more important. With our custom framework we will be able to pivot and experiment in a way that we simply couldn't with off-the-shelf CMS software given the engineering skills at hand.

Here's what I learned building a "successful so far" CMS system. CMS systems are hard. They are really really really friggin hard. They are hard in stupid ways and hard in smart ways. Getting it just right is impossible, getting it half-way good is a tremendous challenge. Every feature is 10x more complicated than it looks on the surface and you won't know how until you talk to the person that cares about that feature. Every feature, no matter how much you talk to people while building it and no matter how much you model and poke and prod, is going to be wrong in some way. You have to simultaneously be so cocksure that you look at this mountain of a task and say, I can take this on, and humble enough to listen to bug reports, user frustrations, and endless visual tweaks so that it's actually worth using.

I have a new found respect for the tremendous task software like WordPress and Drupal and Joomla and so many others try to tackle. I want to go and learn one, Drupal 7 looks nifty, just to see how the pros do it. I have also been reminded that even when you are Inventing it Here, you don't need to go NIH crazy. Building this software was an exercise in standing on the shoulders of giants. Our avatars are powered by Gravatar which easily has the most straightforward and easy API ever. Our comments are protected by ReCaptcha cutting edge captcha technology that also helps read unOCRable books. Our email campaigns are being handled by the amazing MailChimp. And of course the whole project is built on my favorite unframework, Flourish.

So now it's live, you can go view the fruits of my labor at DIG Baton Rouge. It's the first of four sites that will be moved over to the new Lipsum CMS engine. I will spend the following weeks, tweaking, adding new features, fine tuning and bug smashing. I'm incredibly proud of what our small team was able to accomplish in such a short time. Lipsum represents a solid technical foundation on which we will build the future of our publication sites. It was not easy, but it was necessary, and I for one am glad we took the road less traveled. As Joel Spolsky famously wrote, "If it's a core business function -- do it yourself, no matter what."

Form Generator for Flourish

[caption id="attachment_892" align="alignright" width="300" caption="Machines are complicated, so is software"][/caption]

A while back I saw a post in the Flourish discussion boards requesting a Form Generation system, see here. The short discussion that follows basically argues that form generation is too subjective and although it may at some point be added to Flourish it won't be anytime soon. I work in Flourish all day, everyday, and I love every minute of it. I'm probably beating a dead horse at this point, but Flourish really is just a tremendously solid core to build off of. Working with Flourish so much and finding myself performing the same form mark-up again and again, I've given some serious thought to building a Form Generation system and have been using a prototype system for a week or two. Having built a working Form Generation system and used it for a while I want to talk about the good, the bad, and the ugly.

The Good

As a programmer there is nothing more frustrating than finding a library or an API you want to use and having it just be an absolute nightmare to integrate with. I spend a lot of time, ever since Prosper, thinking very hard about how to provide great APIs that make people want to use my software. After a lot of thought about how to go about building a Form Generator library, I've come to what I think is the right balance. The ideal API should fit well with the Flourish way of doing things, be flexible, powerful, expressive, and extensible. So without any further bloviating, here is my proposed syntax, in this example I will use a simple sign-in form.

[php]
echo fForm::post()
->add(fForm::text('Username'))
->add(fForm::password('Password'))
->add(fForm::submit('Sign In'));
[/php]

This would produce the following output

[html]
<form action="http://the/current/url/" method="post">
<label for="username">Username</label>
<input type="text" id="username" name="username" />
<br>
<label for="password">Password</label>
<input type="password" id="password" name="password" />
<br>
<input type="submit" id="sign_in" name="sign_in" value="Sign In" />
</form>
[/html]

The API is fluid, simple, and concise. It follows the Flourish style and uses fGrammar::underscorize() to turn labels into ids. It produces clean simple markup and doesn't mangle any ids, allowing for JavaScript to be easily attached. As far as the API goes, I'm really quite happy. The other powerful thing this allows you to do is tightly integrate your Form Generation with value repopulation, for automatically refilling in values when an error occurs. With this tight integration I've been able to create a prototype system that automatically repopulates across posts when an error occurs, can use an existing fActiveRecord instance as a seed to make editing amazingly easy, and display beautiful inline errors on the form.

The Bad

Having used the prototype for a while I can say it's not all great. Simple CRUD style forms are incredibly easy to mock up, having error values persist automatically is really nice, and having inline error message (especially those from fActiveRecord::validate()) work with little fuss has been a huge speed boost. I can knock out simple pages in a fraction of the time. The problem comes when I want to do some edge case, PHP's greatest strength is easily outputting whatever markup you want, by introducing this layer of indirection you give up this inherent strength for whiz-bang features. It's still incredibly easy to output whatever markup you like, but the disparity between the automatic features the fForm library gives you and your own simple markup is stark and will bewilder a normal user. The problem here isn't that fForm lacks power, it's that it imbues certain elements with too many features and makes creating a one off element prohibitively expensive.

The Ugly

The architecture that worked best for implementing the prototype was a tangled mess of inheritance, and no matter how much I tried to refactor I still haven't found an inheritance graph that I find acceptable. Using my prototype has considerably cut down on development time, but the amount of memory used holding a object graph in memory to represent complex forms is a trade-off. The API has several inconsistencies that bother me, but don't really impact the performance or usability of the library.

The Future

So where to go from here? That's the question I keep coming back to. I want to introduce a layer between object graph and view, this way a plugin system can be used to change the rendering of elements and the form itself in a fundamental way. As soon as possible I plan on releasing a clean room open source version of the fForm library for use with Flourish on GitHub. I believe I've come up with a way to make an easily extensible widget system, now I just need to get something out the door. I will release it with my alpha framework, Rigby, in the coming weeks. Comments, and of course forks, are welcome. I will release some screencasts about how to use Rigby once I get it into a stable state. Stay Tuned!

Second System

[caption id="attachment_883" align="alignright" width="300" caption="Just a little more to the left...."][/caption]

I've been working, both on the job and as a freelancer, right now my 9-5 is starting to enter that ever so exciting phase where one project enters maintenance mode and another project starts to spin up. The new project starting to spin up is to rewrite the CMS system that powers our various publication websites. Now for those of you not in the know, a CMS system is crazy complicated. I'm writing this current blogpost in a very specialized CMS system called WordPress, just for blogging. This software probably has had somewhere upwards of eleventy-billion man hours go into it (once you factor in themes and plugins and all that jazz).

Of course whenever you are faced with a complex problem that's been solved to death your best bet is to look around and see if anyone has done the heavy lifting for you already. So I started looking around, WordPress, Joomla, Drupal, all mature PHP CMS systems, none that I'm familiar with even a little. I started reading, out of the three I ended up liking Drupal the best, it seems well engineered, flexible, and very stable. I read and read and read about nodes and the way Drupal thinks about content and I started to think, ok maybe we can use this. The problem is that management is used to a level a flexibility that I won't have with Drupal for months, maybe even years. They want this flexibility from the beginning and that was making using Drupal seem less and less likely. Then some requirements filtered in and out and it looks like a custom solution is going to be best... c'est la vie.

Now this is in no way a slight against Drupal or Joomla or any other CMS system out there, I'm sure in the hands of Drupal Ninja or Joomla Gurus this would be easy as pie. For me though, building a solid system on my time tested and well worn toolbox of PHP, SQL, and Flourish will probably be the best way to deal with the constantly changing features and requirements that we need to support. Now I'm going to start building, start sketching out how things will work, how data will be thought of in the system, user models, documents, revisions, all kinds of fun stuff. It's a bit intimidating, but as a Software Developer this is really the stuff I live for, big gnarly problem, blank TextMate buffer, well known tools, get cracking.

The big problem I'm fighting right now is the dreaded Second-system effect. Watch out for this my dear friends, it's not a nice trap to get caught in. It works a little something like this.

1. Build a system, this is the First-system, it works, celebrate and have a shot of Jack.


2. That stupid "real world" comes in and messes up your data model, you add a little hack to compensate.


3. Oh, it has to be able to do what? No, you said it didn't have to do that!! Fine, hack hack hack.


4. Maybe some cleanup, probably some more hacking and bending.


5. You've tried your best to keep First-system shiny and new, but the shine's off the apple.


6. Crazy requirement, requires rewriting huge parts of the system, oh noes!


7. Alright, now that we know so much more let's build Second-system.

Herein lies the danger. You get the go ahead to build Second-system, and you have all this experience from First-system, you really think you understand the problem domain a lot better, so far so good. Here's where things get hairy though, you start thinking about how you would build the system so that you don't run into the same problem First-system had, you start abstracting and abstracting, it gets more and more complicated, but you are now a master in this domain, you can handle it. Users who have been frustrated for months or years using First-system start throwing in their advice, things that have always bugged them, their nice-to-haves, and hell, you haven't built anything yet, chuck that automated-frog-boiler in your system plan.

Before you know it you've got a beautiful masterpiece, it handles everything, all the problems encountered in the First-system, all the nice-to-haves, everything, it is a theoretical thing of beauty. Now all you need is 6 maybe 7 years to build it.... oh shit. But there's something even more wrong about the Second-system, it has buried in it a dirty lie, that you understand the problem domain. You probably have a pretty good understanding of the problem domain, as it stands right now. Having already tackled this problem once does not make you clairvoyant, there will be changes to the system and your system will have to adapt to those new realities. This compounds the Second-system effect, because your Second-system is almost inevitably more complicated than the First-system and therefore harder to change and bend and make work in that pesky "real-world."

Right now I have to fight the urge to over-engineer, to outwit this problem, because I'm just not smart enough to solve this problem forever and ever into the future. So I'm going to focus on building a solid platform that let's me build out modules and snap them off when they aren't needed anymore. Something robust but simpler than the current system. Doing this will mean trading things off, it will mean making hard decisions upfront about architecture to meet our current needs but also look to the future, and this is what a good Software Developer is supposed to do. And once my current project is safely resting in maintenance mode, quietly humming along making my company money, I will begin building a Second-system and hopefully avoid Second-system effect.

Auto-timestamps in Flourish

This will be a short post, I just wanted to share a protip for anyone using Flourish and in particular using the fActiveRecord ORM. fActiveRecord is pretty awesome, with it and fRecordSet I find myself very rarely having to drop down to raw SQL, although for some reports it can't be avoided and then fORMDatabase::retrieve()->query() is your friend. Anyways one of the really cool features about fActiveRecord is that it has a plug in module based off of hooks and I wanted to pass along something that I've started doing as a convention that's worked out really well for me.

I like all of my model classes to have a created_at and an updated_at timestamp that automatically do the correct thing to provide a very basic audit trail. This does not let you off the hook for doing actual auditing, but it is a nice way to do sanity checks and get some instant information about records. fActiveRecord makes this pretty easy to do, let's dive into some code.

[php]
class someModel extends fActiveRecord {
/**
* The fActiveRecord class provides the configure function to let you set-up Active Record hooks. Let's do that now
*/
protected function configure() {
fORMDate::configureDateCreatedColumn($this, 'created_at');
fORMDate::configureDateUpdatedColumn($this, 'updated_at');
}
}
[/php]

Easy as that you now have your created_ats and updated_ats working great for the someModel class. Here's the cool part though, because boilerplate code is stupid ugly code, if you want to use this across your project simply do the following.
[php]
class ActiveRecord extends fActiveRecord {
/**
* For this project every model should record created_at and updated_at
*/
protected function configure() {
fORMDate::configureDateCreatedColumn($this, 'created_at');
fORMDate::configureDateUpdatedColumn($this, 'updated_at');
}
}
[/php]

Now when you need to create a new model simply extend ActiveRecord instead of fActiveRecord and auto timestamps are done.