open format

[caption id="attachment_663" align="alignright" width="300" caption="We\'ll just lock your data up in here, don\'t worry we\'ll open it up later if you need it. This is what a closed format sounds like"][/caption]

Back in the days when a computer had 64k of RAM and high-capacity meant you had a double-sided floppy disc, there were these funny little things called binary file formats. We still have them today, they are the "pattern" that a program will write and read to and from disc to save and load a file into a program. Some are open, like image file formats, and some are closed, like Microsoft's binary blob formats for Office. As the world has advanced and storage has become dirt cheap people started looking for an easier way, and whenever there is a problem that we don't have a tool for yet, we reached for XML.

XML is actually not a bad fit for this problem domain, its a little on the ugly side, but that's ok, very few people crack open a raw file to read it. The people that are cracking open files in a text editor to peer inside are probably tech-savvy enough to read a little XML. The big bonus of switching to XML, like ODF or DOCX, is that there are very mature XML parsers and writers for almost every programming language. That means that a determined programmer can grab a copy of your format spec, her favorite XML parser, and write a program that operates on your file format. This is the essence of that oh-so-marketing-speak word synergy.

Now I would love to take credit for this awesome idea, but if you've read The Art of Unix Programming you will recognize this as nothing more than an extension of the Rule of Composition. Now that you've read the Rule of Composition (and you should make the time to read the rest of the Art of Unix Programming, even if you never plan on programming anything for Unix, the lessons within are just in general great to know), you will recognize the inherent advantage of having parsable file formats. Now that I have cunningly converted you over to my way of thinking, what format should you use?

XML

XML (eXtensible Markup Language) is the old standby, it is reliable, standardized by the W3C, well-supported, and there are tons of tools around for playing with XML. XML is "human-readable" for those of you unfamiliar with it here is an example.

[xml]
<book>
<title>Example Book</title>
<pages>
<page>
<![CDATA[
...page stuff here...
]]>
</page>
<page>
...you get the idea...
</page>
</pages>
</book>
[/xml]

Its a bit tag-heavy and that means a slightly large file size, this isn't a huge concern since storage is so cheap, but you should be aware of it. XML has a neat feature called DTD (Document Type Definition), armed with this most parsers will be able to tell right away if a document is well formed. XML is big and clunky but a fine choice for many applications, especially if you need typing information.

YAML

YAML (YAML Ain't Markup Language) is the format of choice for the ruby community. YAML is well supported by most mainstream programming languages, it is a more lightweight choice than XML. Here is the same thing as above in YAML.

[ruby]
book: Example Book
pages:
- page: >
...here goes my page data...
- page: >
...you get the idea....
[/ruby]

YAML uses the structure of the text to indicate the structure of the data. Ending tags are dropped and indentation becomes more important. YAML looks simplistic at first but has a wide-array of functionality hiding below the simple hello world examples. References, hashes, arrays, and much more are possible with YAML. The specification allows you to make concise documents that contain an astounding amount of information.

JSON

JSON (JavaScript Object Notation) is a lightweight way to represent data structures. JSON excels by being incredibly simple to learn and use. There is native support for it in JavaScript which makes it ideal for use in AJAX (which would then technically by called AJAJ), and there are JSON parser available in most mainstream languages. Here is the example from above in JSON.

[javascript]
{title: "Example Book", pages: [ page: "...page stuff goes here...", page: "...you get the idea..." ] };
[/javascript]

Just like in JavaScript everything in JSON is a Hash or Array. JSON is a simple typeless data recording system, perfect for dynamic languages. JSON is a proper subset of YAML 1.2 so most YAML parsers can also parse JSON. JSON's incredibly lightweight nature lends itself for being used when sending data over a wire or when space is at a premium.

BSON

BSON (Binary JavaScript Object Notation) is a binary form of JSON. It is meant to be more space efficient than JSON but maintain the ease of parsing. It is described as "A General-Purpose Data Format" and was devised as the serialization medium for the MongoDB project. It is an interesting format to look at if space is at a premium and there are already parsers for C, C++, PHP, Python, and Ruby.

---

File formats no longer need to be gnarled, tangled messes. We have new standards that allow us to freely share data created in one program to be analyzed, manipulated, and turned into something new in another program. Being able to freely interoperate is the future of computing, it is the driving force behind web standardizations, micro formats, and open file formats. The next time you need to persist data to the file system, resist the urge to roll your own serialization scheme, and check out one of the technologies presented above. The world will thank you.

unlicense'd

[caption id="attachment_606" align="alignright" width="300" caption="Freedom has never kicked so much ass"][/caption]

I wrote yesterday about the importance of open source software. I got one comment saying that prosper would be awesome if it were in the public domain. Here is the comment from Arto Bendiken

Prosper looks *sweet*. It’d be awesome if Prosper indeed were in the public domain. I think such a move would make for a decisive selling point over the numerous competing PHP libraries, setting Prosper clearly apart from the herd and allowing it to be easily embedded into any new emerging PHP web frameworks.

It’s worth mentioning that a key reason that SQLite, which is in the public domain, has prospered (no pun intended) has been the fact that it can be so easily (both technically and legally speaking) embedded into any other software. The end results of technical excellence combined with singularly liberal licensing speak for themselves: http://www.sqlite.org/famous.html

I think a lot of these same benefits could come into play also in such a potentially key higher-level web app infrastructure piece as Prosper. It certainly seems that you have the technical side of things well in hand, already, so here’s hoping for a bright and prosperous future.

Well I completely agree with Arto and want to clear up any confusion about prosper and its license. Prosper was original hosted on Google Code, which requires you to choose a license, at the time I selected the MIT/X11 license based off of my belief that it was the most free. A little further down the road I saw the WTFPL - Do What The Fuck You Want To Public License and thought about switching to that. There is a bluntness to the ideal of that license that appealed to me.

The problem with the WTFPL is of course its name, I want prosper to power lots of things and that means taking into account a wider audience. While hackers and rock star programmers can easily adopt something with WTFPL, I didn't want to be responsible for some corporation turning their back on an otherwise useful library because they were squeamish about a word in the license.

[caption id="attachment_607" align="alignright" width="220" caption="I always thought people just hated the letter C... then I found out this means no copyright"][/caption]

Then I found this post on Hacker News (ironically by the same Arto Bendiken) and it lead me to the Unlicense. The Unlicense is exactly what I was looking for and simple to adopt. If you go to the GitHub repo you will see that the project is now officially Unlicense'd. The 0.8 release will be the first official Unlicense'd release, and so will every release going forward. I believe in free and open source software and so I'm publicly declaring now that prosper will always be free, open source, and unencumbered. The mechanism that I will use to ensure this going forward will be Unlicense.

What's the big deal, plenty of things are open source, why is this any different? The difference is that I'm giving this away fully, without any restrictions. Want to build a cool new library that uses prosper as the low-level database abstraction layer, go for it. Want to build a proprietary ORM that utilizes prosper, go for it. Want to repackage prosper and try to sell it (as underhanded as that is), go for it. Want to smother prosper with jam and eat it, if you can find a way to, go for it. Want to.... enough already, you can do whatever you like.

It is easy to talk about things like freedom and support them in the abstract, it becomes more difficult when you have something concrete. People could very well take prosper and take it in directions I don't care to see it go, but that is life, that is creation, that is the essence of open source. It is my goal to see prosper provide a solid foundation for building new and better tools and frameworks in php. I want as many people as possible to use it, and so this was a logical choice.

The next time you have some useful chunk of code lying around and you want to give it to the public domain for the good of the public, think about using Unlicense. I can't say it any better than they can so I will just quote their section about why you should use Unlicense. "Because you have more important things to do than enriching lawyers or imposing petty restrictions on users of your code."

open source

[caption id="attachment_602" align="alignleft" width="300" caption="insert stupid star wars pun here"][/caption]

I've been getting into the guts of the SimpleTest trying to make it bend to my will. So far I've been pleased with the framework, I've been able to set up a testing structure that works for me. To understand the testing structure better you need to know a little about prosper's structure.

Prosper uses a classic adapter pattern to handle the different backends it supports, so it has a directory structure similar to the following

• Query.php


• Adapters/




• BaseAdapter.php


• PreparedAdapter.php


• MySqlAdapter.php


• etc...

Since I wanted the unit tests structure to make some sense I decided that there should be one point of entry that I could run to test every adapter and then entry points for each adapter individually. I ended up with a test structure like so.

• All Tests (Suite)




• MySQL Tests (Suite)




• Select Statements


• Insert Statements


• etc...




• Oracle Tests (Suite)




• Select Statements


• Insert Statements


• etc...




• etc...

[caption id="attachment_597" align="alignright" width="300" caption="All my tests pass but its just so boring"][/caption]

This structure is working nicely and now I get to have the fun of writing hundreds and hundreds of unit tests, it really puts into perspective the shear size and scope that prosper has grown into. The next thing though was that I wanted beautiful results. If I was going to have to look at these things all day, I wanted them to caress my eyeballs and make them feel all warm and fuzzy.

SimpleTest (like all PHP code) is open source and even has a way to extend the reporters, although its easier said than done. I fired up my favorite php editor and got down to work rewriting the HtmlReporter class so that the results were more to my liking. It was not the easiest task, but there is some ok documentation about it and the names were all fairly sensible. I hacked away at it, changing a little bit here or there and then finally decided to just scrap what they had wrote and redo most of it. It took some time to get everything squared away but with some inspiration ...cough...blatent theft...cough... from QUnit, I was able to make the following.

[caption id="attachment_599" align="aligncenter" width="300" caption="such hotness! unit tests have never looked so good"][/caption]

This is why I love open source so much, I wasn't stuck wishing that this thing looked better, I didn't have to resort to some sort of greasemonkey magic, I was able to fundamentally alter simpletest by changing the code. This is also why prosper is released under the MIT License, although recently I've been thinking about moving it to the Unlicense. I want people to be able to fix bugs, play around and change things, and ultimately make prosper a better tool.

Right now there are some Prosper specific hacks going on in the new reporter I wrote and its only been tested in Firefox, since its what I develop in when I'm on windows. I plan on cleaning up the code a bit and then submitting a patch to SimpleTest containing my new QHtmlReporter. This is part of being a good open source citizen, you can reap the benefits but then you should also try to pay it forward or back.

Open source software is empowering, not only in what it can do, but how it empowers the users to make them better. I'm going to continue writing up unit tests and with this awesome new display, my eyes will be happy to watch the results scroll by.

php pain

[caption id="attachment_353" align="alignright" width="210" caption="he never forgets, even the stupid decisions"][/caption]

I am a fan of php, ever since my first software development job where I was thrown in the deep end and had to sink or swim. I swam in the sea of php and actually think it is a very nice language (begin receiving hate mail.... now!). You see, I never knew the non-OOP php, I was brought into a well managed project with Classes and Interfaces, it was basically Java with dollar-sigils thrown in. I never suffered through the spaghetti nightmare that so much php is, so I'm admittedly wearing some rose-colored glasses.

There is one feature that I love about php, it is the height of elegance, the associative array. The associative array can do anything, need a stack just use array_push and array_pop, need to do some functional programming array_map, array_filter, and array_walk are your friends. The beauty of the associative array in php is that it is ubiquitous, its a hash map, array, stack, object, all rolled into one.

The problem with the associative array though is that its create syntax is ugly. Here is how you create an array in php

[php]
$example = array(1, 2, 3);
[/php]

That doesn't seem so bad, what am I complaining about, well the problem comes when you have nested arrays (which is somewhat common).

[php]
$configuration =
array(
'name' => 'My Configuration',
'server' => array (
'name' => 'localhost',
'port' => '8080',
'users' => array (
array (
'username' => 'example',
'password' => 'example'
),
array (
'username' => 'user01',
'password' => 'pass01'
)
),
'overrides' => array (
'https' => false )
)
);
[/php]

The biggest problem is not that I have to keep writing the word array its that arrays don't look like arrays. They look like function calls, with other nested function calls, of course the newlines and tabs help out, but they don't solve the problem. Let's look at how this would appear in JavaScript using super-awesome JSON.

[javascript]
var configuration = {
name: 'My Configuration',
server: {
name: 'localhost',
port: '8080',
users: [ {username: 'example', password: 'example'},
{username: 'user01', password: 'pass01'} ],
overrides: {https: false}
}
};
[/javascript]

That's much nicer and the more important part is that hashes look like hashes and arrays look like arrays. There was a proposed php syntax change that was discussed to death and an analysis here which would have made the following legal php

[php]
$configuration = [
'name' => 'My Configuration',
'server' => [
'name' => 'localhost',
'port' => '8080',
'users' => [ ['username' => 'example', 'password' => 'example'],
['username' => 'user01', 'password' => 'pass01'] ],
'overrides' => ['https' => false]
]
];
[/php]

But this is not the case, and more importantly, may never be the case. The issue has been brought up and rejected several times, with the following reasoning

I see no advantages here, only another way to do already possible thing and yet another way to confuse people.

- Anthony Dovgal

For the record: I'm -1. array() is enough. Ridiculous idea to begin with.

- Jani Taskinen

There is no reason to add another syntax for *exactly* the same thing, especially because it's ungoogable

- Derick Rethans

Then there is this

I'm ok with it as well. Like I said over a year ago (*), it is a syntax very familiar to web developers and it feels natural to most people.

(*) http://marc.info/?l=php-internals&m=117060700805108&w=2
- Rasmus Lerdorf (the guy who created PHP)

The problem I see here is that the vote finally breaks down to 1 / 3 contributors in favor and 17 / 20 users in favor, it was then rejected. There is no recourse and as long as the people maintaining php see this as a useless feature, as a "backward incompatible syntax that duplicates already existing one, but 5 characters shorter" change request, it will never move forward.

This is a shame and it is myopic and dishonest. To trivialize this change supported by the vast majority of the php users voting as some sort of luxury people want so that they can avoid typing 5 characters is intellectually dishonest. The major point is that square brackets mean arrays in php, unless you are creating one, then it looks like a function.

[php]
$array = array(0, 1, 2, 3, 4, 5, 6); //Create the array
$element = $array[5]; //Get the element at index 5
$array[3] = "hello"; //Set the element at index 3
$array[] = "example"; //Push an element onto the array
[/php]

The dishonest part about the argument against the change is that it would be some sort of crazy zany backwards incompatible change and we would never break backwards compatibility. First off, if you are going to classify any addition to the language as backwards incompatible, then stop having a core development team, you are done, close up shop you can't add anything to the language anymore. Secondly, php has a history of removing and adding things that break older code, anyone remember the first stab at classes, or ever written $stringvar{4} to access a character, that is going away and you should write $stringvar[4] from now on. It's deprecated as of php 5.3.0 and will most likely be removed as of php 6.

But we are left as a community without recourse, the array function to create an array exists and by that fact alone it will remain forever. The most disheartening thing about the whole misadventure is that because this has been brought up and rejected so many times any time it is brought up now it is automatically rejected as being rejected before. Discussion has attached to this idea a Scarlett Letter, most undeservedly.

At the end of the day I won't jump ship, but it's one of those things that makes you stop and scratch your head. Why be so against it? I don't buy the arguments against it, I don't understand the fervor that the devs fight against it with, and I don't see the harm.

As people push the boundaries of what the language can do and the concept of the DSL (pushed by those ruby people) becomes more and more prominent, language cruft will have to fall by the wayside. Not improving array literal syntax is not the end of the world, but the attitudes and egos on display fighting a harmless change that the user base wants could be.

---

Edit: My arrays were using non-literal keys, which is incorrect as pointed out by Mewp in the comments, they have been corrected. Sorry about that, thanks for the correction Mewp.

discomfort

[caption id="attachment_284" align="alignright" width="300" caption="was the collander really that hard?"][/caption]

I am an American, which means that my life is constant bombardment of people offering to make it simpler, faster, easier, and more comfortable. Watch an hour of tv and count the number of times people are trying to make your life easier and more comfortable, its quite a few. The problems they solve aren't even real problems, they are imagined problems that we've been convinced are real. I used to think these problems were real too, I didn't have enough time, I didn't have enough skill, I needed the pre-packaged solution that was quick and ready in 5 minutes.

Then I woke up one day and realized that I had been swindled by a bunch of bullshit.

Who has 15 minutes to make pasta for dinner?! [Show woman dumping pot of scolding hot water and noodles all over herself]
There's just not enough hours in the day!! Buy our microwave-licious pasta bowls!!!

You know who has 15 minutes to make pasta, everybody does. Want to write a comment saying there is no way you could free up such time, well fuck you Mr. I-have-time-to-read-a-blog-and-post-a-snarky-comment-but-somehow-can't-find-15-minutes-with-both-hands-and-a-flashlight. At this point though, we are all thinking, am I trying to make some sort of point about pasta or is this going to get around to programming at some point?

Well thanks for going on that side-rant with me, let's look at what in the world I'm trying to get to. Being a programmer I like to be comfortable, I like to work with the languages I like, and the tools I know, and the frameworks I understand. It makes sense, people are more productive when they are working with the tools we know, that's why we have holy wars about what language is better and which framework to use, we invest some time getting our nice comfy language set-up just the way we like it and we look at people sitting on uncomfortable languages and wonder what the hell is wrong with them.

The problem with being comfortable is that its hard to learn and grow when you are comfy. You know the language, you might learn some edge cases here or there or uncover a new feature or bug every once in a while (and this is important too, its important to be a master of something), but new learning is rarely done this way. Its very hard to learn new things by doing the thing you already know how to do day in and day out. Yesterday I decided to do something new, and it was uncomfortable, but I've started learning and hope to learn a bunch more.

What was this horrible discomfort that I suffered, moving my code for prosper from Google Code to GitHub. You see, I've been using Subversion since college, when one of the smartest software developers I know, Ian Potter, introduced me to it. I had heard some stuff about svn before and compared to the source control system we were using (a folder called test and a folder called production) had determined svn would be a step up. He set up svn and patiently answered my questions of which code was on my machine and which code was on the server over and over again until my brain finally wired it up.

Moving to svn was much more complicated than the tried and true 2 folder methods, but the safety, history, and concurrency that svn offered was more than worth the week or two of learning the new system. In my career so far I've gotten to use cvs, svn, and vss, they are all similar concepts, there is a central repo and you can commit, update, branch, merge, blame, etc. This is how source control works in my brain, and I am completely comfortable with how this all works.

Well svn is no longer the new hotness, git is in big time, and a good portion of its popularity can be attributed to GitHub. GitHub is the facebook of source hosting, its a really great site to host your project on, most interesting new projects are hosted there, and the social aspect allows people to easily interact with you and your project. I've had a GitHub account for a while, and I tried it out before, only to go back to the comfortable embrace of Google Code and svn.

[caption id="attachment_285" align="alignleft" width="300" caption="who could resist octocat?"][/caption]

So why move to git, just because its shiny? Well, GitHub is almost enough of a reason, but the distributed nature of git, the increased popularity of git, and the ability to quickly make new branches and merge them back into trunk, make it worth learning. If I want to continue being active in a community that more and more relies on git, its probably best to learn it.

I've taken the first step, I've moved my stuff over to GitHub and I've got git up and running on my laptop, I've been able to push some changes upstream and over the next few days I will be reading the excellent help that GitHub provides you. It will not be comfortable, I will probably be scratching my head for a while, wishing to go back to the centralized repo that I know and love. I will more than likely treat git like svn for a while until I "get it." But I've decided to stick it out, to live with and own my discomfort and to learn this new thing.

Here is my challenge to you: maybe you've read a few things about a new technology and its interested you, maybe you've even taken a few trembling steps into the new technology, and you've walked away or put it aside. Go back to it, jump in head first, don't look back and don't give up. The great thing about technology is there is usually a manual to read, a community to turn to, and our old friend Google. Make yourself uncomfortable, you might just learn something.

dogfood, terror, and tweets

[caption id="attachment_251" align="alignright" width="300" caption="too literal"][/caption]

Yesterday I released my new library Prosper (in case you hadn't noticed the new menu entry at the top of this blog). It was a ton of work, way more work than I ever expected, but it was worth it. Prosper is now out the door and ready for human consumption, I've been using it all day on a fun little project, mostly playing around with the twitter API (which I've done before but never from php).

I was talking the other day with the esteemed Rick Kierner about the upcoming CodeMash convention. Since I work for an amazing company that is sponsoring the event, I get to go for just about free. I plan on going to the precompiler sessions and that day comes out of my pocket, but its a small price to pay. Rick is taking care of the nitty gritty details but in the end I will have to give him some cash money to pay for the precompiler and the first night's hotel room, he decided to use BillMonk to record the debt and keep everything straight.

After perusing BillMonk for a while I decided that I really liked the site, its a cool idea, and I thought to myself, "hey self, now that tipjoy is defunct could there be a space for me to make my mark." (Side note, I often think hyperlinks at myself). Rick and I bounced some ideas off each other and at the end of the day decided that there really wasn't a good business in what we wanted to do, so we abandoned the thought and moved on, c'est la vie.

Today though the idea kept bouncing around in my head, really its been bouncing since we first started talking. Today I finished my proof of concept for my new minimalist bill tracking system, currently dubbed cha-tweet (a portmanteau of cha-ching and tweet). I'll outline quickly the concept behind it, a rough approximation of its implementation, its status, and what the title has to do with anything so far in the post.

The idea is basically this, you can tweet a specially formed tweet and cha-tweet will aggregate those into a nice little summary of who owes you cash and who you owe cash. The tweets look like this

The debt cha-tweet

@I_owe_this_guy_money > $5 lunch: delicious sandwich!

The give cha-tweet

@I_gave_this_guy_money < $5 lunch: mcdelicious

The basic idea is that money comes out of or goes into the username to or from you, respectively. It's a simple syntax that lets you sort by debtor/debtee and by type of debt. Since it starts off with the @username syntax it only gets displayed (by default) to people that know both of you and would maybe care, and its easy to find by the service because it is treated by twitter as a reply. The system works fairly well, and I'm happy enough with the concept.

The problem was that to create an accurate summary it had to go get and process the entire tweet stream, which can take a long while, and assuming that most of the tweets in a users tweet stream will not be cha-tweets, there is no need to do all this processing everytime (and in steps sexy miss rdbms). So, throw a database at the problem, store the seen cha-tweets and operate on them, this has now been implemented, it took one hour, and I used prosper to do it.

So how was it? Really easy, I stole the configuration from the todo list project, switched around the credentials and the schema name, and was off and running. Everything was working well until I tried to update something and Prosper spit up a bunch of non-sense at me, and my heart sank. Did I just release some library full of horrible bugs and will be mocked endlessly, the cries of n00b and pwned reverberating so loudly as to drive me mad?! No, I just don't know how to type so instead of writing this
[php]
Prosper\Query::update('user')
->set(array('since' => $since))
->where('id = :id', $_POST)
->execute();
[/php]
I wrote
[php]
Prosper\Query::update('user')
->set(array('since' , $since)) //WHOA!
->where('id = :id', $_POST)
->execute();
[/php]
Changing that double arrow to a comma makes a big difference, and causes prosper to correctly try to set `0` to 'since' and `1` to 'some_huge_twitter_id_number' which is what I told it to do.

Crisis averted, my library still functions properly, hip hooray!

• Is it easy to use? Yes, I found that it cut down on my development time considerably.


• Is that because I wrote it and have no learning curve? Quite possibly, I'm not going to rule that out.


• Is the code easy to read? Yes, its fairly straightforward crud stuff, made much simpler by prosper.


• Did I learn anything eating my own dogfood? The verbose function is awesome! I wrote the verbose function in the beginning as a simple debug / sanity check of sorts to make sure that things were being translated and serialized correctly (pro tip: the magic __toString function will print the sql statement that will be executed). During development though it was incredibly useful to be able to switch the call to execute for a call to verbose and immediately see at a fairly granular level what was going on. This got me to thinking how much prosper needs some official documentation, and that will be my project for the next week or two.

In the end I have a nifty little twitter gadget, I will probably be hosting it somewhere soon so keep an eye out for that, but more over I have some more experience as an end-user with prosper and a renewed energy to write the official documentation.

---

I'm also adding some stuff to the blog, there are now social media icons, the rss feed link got a new friend mr. twitter link, and there are some backend changes for speed and stability. This is another round of blog tweaks and that is the best time to have your opinion incorporated into the blog's overall look and feel, so comment away!

announcing prosper

[caption id="attachment_248" align="alignright" width="128" caption="prosper"][/caption]

I've been talking and teasing and hinting about my side project. Well today is the day I can finally announce my project, prosper.

Prosper has been in the works for a few months in my spare time, it is a database abstraction layer written in php. For anyone who has ever written a web application in php you know that they are written for one database and one database only. It is a MySQL application or an Oracle application or an X application. If the application gets popular enough, the developer might get around to adding some support for other backends, and when they do it is always some homegrown solution. With prosper you can write your database code once and have it work across 18 different backend databases.

Prosper came about when I was looking for a quality ORM layer, the one thing that struck me as odd is that everyone was rolling their own database abstraction layer. Doctrine has DQL, Flourish has F-SQL, php.activerecord has an adapter architecture, and so each library has its own quirks and compatibility.

Following the Unix philosophy, prosper does one thing and tries to do it well. Here are some things that prosper is not. Prosper is not an ORM. Prosper is not an Active Record implementation. Prosper is not an MVC framework. Prosper will not pluralize names, will not handle sequences for Oracle, will not slice and dice your onions. Prosper is one thing and one thing only, a library that turns function calls into sql.

Official documentation is coming soon, but for now you can view the case study, learn why you should use proper, and get the code

I read the documentation, I wrote the code, now its your turn. Give it a try, submit issues to the Google Code Page, and let me know what a great / terrible job I did.

Note that prosper is not ready for primetime yet, this is version 0.5, lots of the backends should work but only the MySQL one has gotten any kind of real testing. The code is open source currently under an MIT License, although the license may change for the 1.0 release, it will always be free to use and open source though. Here is the list of things that still need to happen.

• Thoroughly test the backends


• Add any backends that I missed


• Write up official documentation


• Move code to github to facilitate collaboration


• Forge partnerships with other open source projects


• Shamelessly promote

Go get the code and let me know what else I need to get to work on.

i can't stop you from being stupid

I've been working on a skunkworks side project that is nearing release, I'm down to the last 5%, which is the most difficult part. Suffice it to say that this project is a library meant to be used by other developers (and myself) to develop neat and nifty things. One of the most complicated things is trying to define an API that makes doing the right thing easy and doing the wrong thing really hard or impossible. The problem is that the as I pressed on I realized that almost half my code was checking for some stupid thing and protecting against it, attempting to build up state machines and syntax trees in the hope of reporting to you that you had done something stupid, and then it dawned on me, I can't stop you from being stupid.

The other thing that dawned on me is that I shouldn't try to, there are already built in errors that will point out what you did wrong, I don't need to add another layer on top. So I came up with some simple guidelines, applied it to my project, and watched unnecessary code melt away.

• Pick sane defaults


• Assume the developer knows what they are doing


• Make it easy to do the right things


• Make it hard to do the wrong things

The beautiful part of all of it is that my code became easier to read and understand, which means when there is a problem and you need to drop into my code (since it is open source), you will be able to understand what's going on. Let's take a look at what I mean with a contrived example, an HTML building library with 4 functions for clarity.

[php]
class HTML {
static function open_html() {
echo "<html>";
}

static function open_body() {
echo "<body>";
}

static function close_body() {
echo "</body>";
}

static function close_html() {
echo "</html>";
}
}
[/php]

The way I was programming did a bunch of handholding and sanity checks, which were nice but unnecessary. The class became bloated with so much state and sanity checking that it was becoming unwieldy, let's take a look at an overly protective incarnation of the above code.

[php]
class HTML {
private static $html_open = false;
private static $body_open = false;

static function open_html() {
if(self::$html_open) {
self::handle_error("html is already open");
}
echo "<html>";
self::$html_open = true;
}

static function open_body() {
if(self::$body_open) {
self::handle_error("body is already open");
} else if (!self::$html_open) {
self::handle_error("body must be contained in html");
}
echo "<body>";
self::$body_open = true;
}

static function close_body() {
if(!self::$body_open) {
self::handle_error("cannot close unopened body");
} else if(!self::$html_open) {
self::handle_error("body must be contained in html");
}
echo "</body>";
self::$body_open = false;
}

static function close_html() {
if(self::$body_open) {
self::handle_error("cannot close html, unclosed body exists");
} else if(!self::$html_open) {
self::handle_error("cannot close unopened html");
}
echo "</html>";
self::$html_open = false;
}

static function handle_error($message) {
//What is the correct behavior, should we attempt to fix the error, report it, who knows.
//We'll just stop execution
die($message);
}
}
[/php]

Those sure are some nice error messages and it keeps you from creating malformed html, but what's the point of it all. The browser is more than happy to tell you that your markup is invalid, or run the output through a lint checker or W3C validator, why should this class have some buggy half-implemented validator inside of it. The answer is, it shouldn't.

If you write stupid code you should get stupid results, Garbage In - Garbage Out. Library code shouldn't hold your hand making absolutely sure that you never make a mistake, what's the point of it. If you write the following code

[php]
HTML::open_body();
HTML::close_html();
HTML::open_html();
[/php]

You are clearly missing something about how markup works, it's not the library's job to hold your hand and guide you through this crisis, the browser will slap you in the face and you will have to learn something. Now there is a caveat, here we are generating HTML, there are great utilities for finding errors in HTML, so we don't need to reinvent the wheel. If you are writing a library or application code where if something goes wrong the error displayed is fine, don't reinvent the wheel. If the error is unacceptable you may need to do some error reporting.

At the end of the day you can't keep the user of your creation from being stupid, they are going to do stupid things and stub their toes and curse your name and you need to make sure that they had to go far off the beaten trail to do so, overlooking obvious better ways to do it, so that it's their fault and not yours. Make it easy to do the right thing and hard to do the wrong thing, but if the user wants to open the body tag before the html tag, let them, they have to learn sooner or later.

just because its better doesn't mean its good

I recently wrote a blog post called why no love for scripting languages lamenting the lack of open source scripting environments in Windows 7. I got some interesting feedback, most of which went along the lines of "POWERSHELL!!!1!!one!"

Being an interested fellow I took it upon myself to look up PowerShell, and it looks like a nice language, really good for administrative tasks. So I read some of the manual and it looked ok, and I started looking for some scripts that would compare old school .bat to the new PowerShell. I found what I was looking for here PowerShell Examples. We will be looking at an example that displays the current date.

@ECHO OFF
IF NOT "%1"=="" GOTO Syntax

:: Use BATCHMAN to retrieve day
BATCHMAN DAY
:: Errorlevel 0 means BATCHMAN was not found
IF NOT ERRORLEVEL 1 GOTO NotFound
FOR %%A IN   (1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL  %%A SET DD=0%%A
FOR %%A IN (0 1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL 1%%A SET DD=1%%A
FOR %%A IN (0 1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL 2%%A SET DD=2%%A
FOR %%A IN (0 1)                 DO IF ERRORLEVEL 3%%A SET DD=3%%A

:: Use BATCHMAN to retrieve month
BATCHMAN MONTH
FOR %%A IN (1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL  %%A SET MM=0%%A
FOR %%A IN (0 1 2)             DO IF ERRORLEVEL 1%%A SET MM=1%%A

:: Use BATCHMAN to retrieve year
BATCHMAN YEAR
FOR %%A IN (0 1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL  %%A SET YYYY=198%%A
FOR %%A IN (0 1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL 1%%A SET YYYY=199%%A
FOR %%A IN (0 1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL 2%%A SET YYYY=200%%A
FOR %%A IN (0 1 2 3 4 5 6 7 8 9) DO IF ERRORLEVEL 3%%A SET YYYY=201%%A

:: Store in variable and clean up temporary variables
SET SortDate=%YYYY%%MM%%DD%
SET YYYY=
SET MM=
SET DD=

:: Display the result
ECHO.
ECHO SortDate = %SortDate%
GOTO End

:Syntax
ECHO.
ECHO SortDate.bat,  Version 1.00 for MS-DOS
ECHO Display the current date in YYYYMMDD format
ECHO.
ECHO Usage:  SORTDATE
ECHO.
ECHO This batch file uses BATCHMAN, a utility by Michael Mefford
ECHO.
ECHO Written by Rob van der Woude
ECHO http://www.robvanderwoude.com

:End

Rough, if you've ever had to write anything non-trivial in batch you will start to feel that pain at the back of your eyes right now, this is your brain trying to eat your memories. Batch is hideous, and difficult to write, and gross, and everyone hates it. Don't believe me, well you don't have to, Microsoft agreed with me and began development on Monad in 2003, this project would become PowerShell. Now let's look at the PowerShell equivalent of this code

""
"Date / Format   YYYYMMDD        DD-MM-YYYY        MM/DD/YYYY"
"============================================================"
"Yesterday       " + (get-date (get-date).AddDays(-1) -uformat %Y%m%d) + "        " + (get-date (get-date).AddDays(-1) -uformat %d-%m-%Y) + "        " + (get-date (get-date).AddDays(-1) -uformat %m/%d/%Y)
"Today           " + (get-date -uformat %Y%m%d)                        + "        " + (get-date (get-date)             -uformat %d-%m-%Y) + "        " + (get-date (get-date)             -uformat %m/%d/%Y)
"Tomorrow        " + (get-date (get-date).AddDays(1)  -uformat %Y%m%d) + "        " + (get-date (get-date).AddDays(1)  -uformat %d-%m-%Y) + "        " + (get-date (get-date).AddDays(1)  -uformat %m/%d/%Y)

That is much better. I see some stuff that looks like objects in there (I'm definitely seeing the dot operator). I'm a n00b to PowerShell and yet this code is easy to grok and all and all very pretty. Nicely done Microsoft, you get a cookie.

Well this would be a pretty boring post if I just went around patting Microsoft on the head for doing a good job. The interesting thing about PowerShell Examples is that they go on to provide the same example in other languages as well. Here is the same thing in perl.

#! perl

# SortDate.pl,  Version 1.00
# Display "sorted" date (YYYYMMDD)
# Written by Rob van der Woude
# http://www.robvanderwoude.com

# Parse time string
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

# Add "base year"
$year = $year + 1900;

# Add 1, since moth seems to be zero based
$mon  = $mon  + 1;

# Add leading zeroes if necessary
if ($mon < 10) {
 $mon = "0".$mon
}
if ($mday < 10) {
 $mday = "0".$mday
}

# Concatenate substrings
$sortdate = $year.$mon.$mday;

# Display result
print "\nSortDate = $sortdate\n";

Certainly could use a text formatter of some kind to get rid of the icky manual padding, but for someone who has never written a line of perl, this looks quite nice and readable (although I am quite aware that that is not always the case for perl). So what's got a bee in my bonnet? Well I almost didn't write this until my friend Jeremiah tweeted the following

Writing powershell. This is happily like writing perl. 7:31am Oct 6th

And a little bit later

ZOMG This is seriously just like writing perl... I <3 you #PowerShell 10:41am Oct 6th

Side note: Jeremiah is about the smartest man in the world when it comes to SQL follow him here and view his blog facility9.

Why go about reinventing the wheel, if you are going to make a language similar to perl, just host perl. What's the harm? Its hard to call it harm, but its inefficient and tastes a little bit too much like embrace, extend, extinguish to me. PowerShell becomes the defacto scripting language in Windows enticing open source programmers because it is similar to their language of choice. So why fret, PowerShell, perl, batch, bashscript, who cares? Well there are legion reasons why software developers should care.

1. CPAN, PEAR, GEMS, etc. - These are huge repositories of tested code that can be leveraged quickly and easily by open source developers. CPAN (Comprehensive Perl Archive Network) contains 16,600 modules. PEAR (PHP Extension and Application Repository) contains 536 different packages with 1,255,213 lines of code. RubyForge hosts 8406 different projects. These established languages have a gigantic ecosystem of usable code.
2. Learning curve - Do you know the intricacies of your chosen language? Did you spend two hours debugging a wily error and because of it have forever learned some dark corner of your language? Can you tell me in your sleep the difference between $foo and $$foo in php? Well none of that will help you in the new PowerShell.
3. Brain drain - Come up with a really clever way to solve a problem in PowerShell, great, keep it to yourself. Where is the community? There is powershellcommunity.org. But there is yet to be an established authority for the community

Now these problems I outline are true of any emerging language. They are normally offset by some inherent positives in the language. I haven't examined PowerShell in-depth enough to find out its intrinsic value. On first blush and with my limited exposure, it seems to be a competent enough pseudo object oriented scripting language well suited for administrative tasks. Nothing ground breaking, nothing that knocked my socks off, in the words of Homer Simpson:

I saw you desperately trying to cram one more salty treat into America's already bloated snack hole. So I did what I could. I did what any loving husband would do! I reached out to some violent mobsters.

PowerShell is just one more salty treat that Microsoft is cramming into America's already bloated snack hole. It would be fine if they had a level playing field and allowed other scripting languages to be first class citizens, but they don't. After the mind numbing pain of batch scripting, PowerShell seems great. It really starts to lose its shine when you view it against the cornucopia of free mature open source scripting languages.

I want to end with a real world example (anonymized). There was a system that managed blerns. Blerns could be collected in collections called Bars, Bars could be collected in collections called Foos. These things were numbered and a system was built to manage them in COBOL. Because COBOL likes to make pictures, the original architects thought to themselves, we will pack the information into an 8 digit number like so FFBBbbbb, so that 01010001 would be the first blern, in the first bar, in the first foo, 01010002 would be the next and so on.

This worked great, the company was selling blerns left and right, and pretty soon foo:01 had 98 bars. Then things started to break down, 3 more bars, and the whole house of cards would come crashing down. What to do, what to do?! The engineers were assembled and solutions were offered.

We only have 8 foos, just assign the overflow into foo:10.

Seemed reasonable enough, but that would have resulted in weird code springing up all over the place like this

if foo == 1 or foo == 10
  return "foo the first"
else
  ...
end

Surely there was a better solution, what could it be? Well, suffice it to say I wasn't present for these meeting and I only saw the terror of the last solution being halfway implemented and this new solution coming in. The solution was a technical sounding concept called field widening. It amounted to storing the identifiers like so FFFBBBbbbb, 10 digits now capable of holding up to 1000 Foos, 1000 Bars per Foo, and 10000 Blerns per Bar. Surely this was better, but was it any good. I guess it might be, as long as we don't live in an exponential world.

Oh...

why no love for scripting languages

Windows 7 is about to drop 20091022. And the internet is buzzing with comments like this:

Definitely the best OS of all time. - WindowsFTW

Snow Leopard for me please! - Sam

It's sad that people are excited by this level of mediocrity only because XP is archaic and Vista is a disaster. Talk about lowered expectations.

Get your anti-virus, and registry cleaners, and disk defragmenters, and other assorted utilities ready. You'll need them. Modern OS? Not hardly. - Terry

Well putting religion aside, Windows 7 looks to me what Vista should have been. Microsoft is doing a great job with the ad campaign, they have awesome buzz going, and the computing world will probably be better off thanks to Windows 7.

To be fair I use Vista Business at work and Vista Home Premium at home with an Ubuntu partition that I dual boot into. I don't hate Windows or Microsoft, but I do feel that both deliver underpowered products at overblown prices, their greatest strength is that with such a high market share other manufacturers make their products to easily inter-operate. That is really the thing that keeps me from dropping Vista at home, after 4 hours of trying I still couldn't get my PS3 to play nice with my Ubuntu box for media streaming, it worked in 2 minutes with Vista.

So here is my question for Windows 7, why no love for scripting languages?

Apple's new Mac OS X Snow Leopard has built in support for Ruby, PHP, Perl, and Python.

Ubuntu can install almost any hosting environment with the click of a button or by issuing a simple sudo apt-get ruby

Windows is supposed to be the OS for everyone, easy to use, so where as the Ubuntu nerds are more than happy with firing up a terminal and apt-getting some fun, why doesn't Windows have these scripting environments ready to rock out of the box?

Could it be that Windows is philosophically opposed to hosting scripting environments? I don't think so as they build in a J-Script interpreter, try double clicking a .js file.

Could it be that Windows foot print would be gigantic if they included these languages? Ruby has a 23.7MB installer, Python 2.6.3 is 14.2MB, Perl 6's Rakudo source is 314KB, and PHP 5.3.0 has a 23MB installer. So in total we are talking about 60MB of disk space.

So why no love? I'm guessing that it is three fold.

1. Not invented here syndrome. These aren't Microsoft technologies, so why bundle them in with a Microsoft product. This is somewhat valid, if these environments expose a security hole or need lots of updates, this is overhead for Microsoft that doesn't add any money to their bottom line but costs them in maintenance.
2. Conflicts with the .Net platform. If these environments are installed by default, then people could make HotCocoa like bindings and create Windows programs outside of the .Net platform. Microsoft wants people to use IronRuby, IronPython, CLR based scripting languages. Really they want you to use C# and Visual Basic.Net so that you will buy the Visual Studio licenses, they are after all a software business.
3. Unsuitable hosting environment. This one is more conjecture as I'm not an OS Engineer, but it is informed conjecture. Windows lacks the fundamental security model that is at the heart of Unix systems. Windows 7 will more than likely continue with the bolted on solution of User Account Controls, although hopefully less annoying in this latest release. This makes the idea of sandboxing a scripting environment much more difficult in the Microsoft world than in a Unix based OS.

But Windows 7 definitely should include these scripting languages, and others (lua is the first to come to mind), because it will ultimately benefit them. Steve Ballmer once famously shouted on stage "Developers, developers, developers, developers!"

There is a world of Ruby, Python, PHP, Perl, Lua, Ocaml, Erlang, Haskell, etc. developers that have a huge hurdle to jump in getting their programs onto the Windows platform. They can either bundle the interpreter into their program a-la py2exe, they can try to get their program to run on a CLR based implementation of their language (if one exists), or they can try to get the user to install the necessary platform.

If Microsoft made it easier to release to these platforms, a world of new and innovative software could flourish on the PC platform. Rails development wouldn't have to suggest that if you have the misfortune of owning a windows-box to do all your development inside of a virtual linux-box.

If you’re working on Windows, you may find it easier to install Instant Rails. Be aware, though, that Instant Rails releases tend to lag seriously behind the actual Rails version. Also, you will find that Rails development on Windows is overall less pleasant than on other operating systems. If at all possible, we suggest that you install a Linux virtual machine and use that for Rails development, instead of using Windows.

From what I understand it is not currently baked in to Windows 7, but maybe we could throw developers a bone in SP1. If they could standardize the convention of scripting languages installation on the Windows platform, they could become a friendly release environment to a large segment of the open source movement. If Microsoft wants to engage the Open Source community in a meaningful way, this would be a great first step. Anything is better than the sham CodePlex Foundation Microsoft recently tried to pass off as an Open Source haven, but that really warrants its own post.

why no love for scripting languages

Windows 7 is about to drop 20091022. And the internet is buzzing with comments like this:

Definitely the best OS of all time. - WindowsFTW

Snow Leopard for me please! - Sam

It's sad that people are excited by this level of mediocrity only because XP is archaic and Vista is a disaster. Talk about lowered expectations.

Get your anti-virus, and registry cleaners, and disk defragmenters, and other assorted utilities ready. You'll need them. Modern OS? Not hardly. - Terry

Well putting religion aside, Windows 7 looks to me what Vista should have been. Microsoft is doing a great job with the ad campaign, they have awesome buzz going, and the computing world will probably be better off thanks to Windows 7.

To be fair I use Vista Business at work and Vista Home Premium at home with an Ubuntu partition that I dual boot into. I don't hate Windows or Microsoft, but I do feel that both deliver underpowered products at overblown prices, their greatest strength is that with such a high market share other manufacturers make their products to easily inter-operate. That is really the thing that keeps me from dropping Vista at home, after 4 hours of trying I still couldn't get my PS3 to play nice with my Ubuntu box for media streaming, it worked in 2 minutes with Vista.

So here is my question for Windows 7, why no love for scripting languages?

Apple's new Mac OS X Snow Leopard has built in support for Ruby, PHP, Perl, and Python.

Ubuntu can install almost any hosting environment with the click of a button or by issuing a simple sudo apt-get ruby

Windows is supposed to be the OS for everyone, easy to use, so where as the Ubuntu nerds are more than happy with firing up a terminal and apt-getting some fun, why doesn't Windows have these scripting environments ready to rock out of the box?

Could it be that Windows is philosophically opposed to hosting scripting environments? I don't think so as they build in a J-Script interpreter, try double clicking a .js file.

Could it be that Windows foot print would be gigantic if they included these languages? Ruby has a 23.7MB installer, Python 2.6.3 is 14.2MB, Perl 6's Rakudo source is 314KB, and PHP 5.3.0 has a 23MB installer. So in total we are talking about 60MB of disk space.

So why no love? I'm guessing that it is three fold.

1. Not invented here syndrome. These aren't Microsoft technologies, so why bundle them in with a Microsoft product. This is somewhat valid, if these environments expose a security hole or need lots of updates, this is overhead for Microsoft that doesn't add any money to their bottom line but costs them in maintenance.


2. Conflicts with the .Net platform. If these environments are installed by default, then people could make HotCocoa like bindings and create Windows programs outside of the .Net platform. Microsoft wants people to use IronRuby, IronPython, CLR based scripting languages. Really they want you to use C# and Visual Basic.Net so that you will buy the Visual Studio licenses, they are after all a software business.


3. Unsuitable hosting environment. This one is more conjecture as I'm not an OS Engineer, but it is informed conjecture. Windows lacks the fundamental security model that is at the heart of Unix systems. Windows 7 will more than likely continue with the bolted on solution of User Account Controls, although hopefully less annoying in this latest release. This makes the idea of sandboxing a scripting environment much more difficult in the Microsoft world than in a Unix based OS.

But Windows 7 definitely should include these scripting languages, and others (lua is the first to come to mind), because it will ultimately benefit them. Steve Ballmer once famously shouted on stage "Developers, developers, developers, developers!"

There is a world of Ruby, Python, PHP, Perl, Lua, Ocaml, Erlang, Haskell, etc. developers that have a huge hurdle to jump in getting their programs onto the Windows platform. They can either bundle the interpreter into their program a-la py2exe, they can try to get their program to run on a CLR based implementation of their language (if one exists), or they can try to get the user to install the necessary platform.

If Microsoft made it easier to release to these platforms, a world of new and innovative software could flourish on the PC platform. Rails development wouldn't have to suggest that if you have the misfortune of owning a windows-box to do all your development inside of a virtual linux-box.

If you’re working on Windows, you may find it easier to install Instant Rails. Be aware, though, that Instant Rails releases tend to lag seriously behind the actual Rails version. Also, you will find that Rails development on Windows is overall less pleasant than on other operating systems. If at all possible, we suggest that you install a Linux virtual machine and use that for Rails development, instead of using Windows.

From what I understand it is not currently baked in to Windows 7, but maybe we could throw developers a bone in SP1. If they could standardize the convention of scripting languages installation on the Windows platform, they could become a friendly release environment to a large segment of the open source movement. If Microsoft wants to engage the Open Source community in a meaningful way, this would be a great first step. Anything is better than the sham CodePlex Foundation Microsoft recently tried to pass off as an Open Source haven, but that really warrants its own post.