Data Driven Apps

Web applications are typically driven by data and interactions between it. A typical situation involves: Querying data from the database, formatting as HTML, outputting. A few years ago it was just that simple. Today you have JSON, RSS, and other various formats.

Going by the standard model, you have to create each page separately for all of the various formats, duplicating code. I can’t say I solved this problem entirely, but I’m getting close.

Grab the Steaks, It’s Time for a bbQ

The goal is to be functionally similar to SQL but include a formatting layer that intelligently determines the way to display the data. The following snippet show how the old model can be transitioned to bbQ.

$DB('Children')
   ->select('children_id', 'dob', 'fitness', 'status')
   ->format(function ($row) {
      echo "${row['dob']} - ${row['fitness']} - ${row['status']}";
   });

A simple example using the gFonted database. The first line selects the table from the database. On the second we select the columns we want in the query. The query results are then returned in to a callback from the format function. Very quickly you could create predefined formatting functions.

$DB('Children')
   ->select('*')
   ->left_join('Template', 'children_id')
   ->format(bbQuery::format_table); // predefined format functions

But thats far from automatic, intelligent maybe. Its impossible to guess what format anyone will need. There are a few generic formats (JSON and XML) that could be dealt with automatically.

$DB('Children')
   ->select('*')
   ->left_join('Template', 'children_id')
   ->limit(5)
   ->order_by('fitness', bbQuery::asc) // bbQuery::asc == "asc"
   ->set_type("application/json")
   ->format();
 
$DB('Children')
   ->select('*')
   ->left_join('Template', 'children_id')
   ->limit(5) // is a variable that gets overwritten
   ->order_by('fitness', bbQuery::desc) // order by's are string
   ->order_by('dob', bbQuery::asc) // and gets appended to
   ->set_type("application/xml")
   ->format();

Going the Other Direction

Sometimes its useful to put data into a database. I guess thats an understatement. Its also an understatement to say that doing so is a simple task. An example of standard data insertion:

$DB('Correct')
   ->insert_into('children_id', 'correct')  // returns an insert object
   ->values(12, false);

However this could be greatly simplified. MySQL already knows the datatype for the column, so validation could be automated. The data could even be automatically inserted from a form.

<form>
   <input name="children_id" type="text" value="12" />
   <input name="correct" type="checkbox" />
</form>
 
<?
$DB['Children']
   ->insert_into('children_id',  'correct')  // returns an insert object
   ->from_form();
?>

And finally some other useful snippets:

$DB->query("select * from Children")
    ->format(bbQuery::format_table);
 
$DB('test')
   ->create_table() // returns a create_table object
   ->column('test_id', bbQuery::id_column()) // id is special if set as a primary key, unsigned integer autoincrement
   ->column('test_foreign_id', bbQuery::id_column()) // just a unsigned integer
   ->column('test_char', bbQuery::char_column(10)) // field length as parameter
   ->column('test_int', bbQuery::integer_column(bbQuery::big), 0) // default as third parameter to column
   ->column('test_enum', bbQuery::enum_column('val', 'val3', 'val2')) // enum is varidaic
   ->primary_key('test_id')
   ->foreign_key('test_foreign_id', 'Correct');
 
$DB('test')
   ->alter_table() // returns a alter_table object
   ->add('test_timestamp', bbQuery::timestamp_column, bbQuery::now)
   ->drop('test_int');
 
$DB->query("truncate table test"); // format is not appropriate here

If you are an artist, astronimer, or a space enthusiast you can help contribute.  A list of desired space objects are at Google Docs.  If your interested leave a comment, or just fork the get repository and send me a Pull Request.

Hybrid Means Better Gas Mileage

Unlike Billy’s technique I chose to use PHP instead of Perl.  Many of the language constructs are identical between PHP and JavaScript.  However there are few snags.

• PHP scripts have to start and end with ‘<?’ and ‘?>’ respectively.
• PHP variables must start with a ‘$’.
• Most of JavaScript’s standard functions are wrapped in objects.  Math, String, etc.
• PHP uses the ‘.’ for concatenation, while JavaScript uses ‘+’.
• PHP uses C++ style definition for objects, while JavaScript uses a prototype definition.
• JavaScript uses the C++ style const keyword to define constants, while PHP uses the define() function.

The Lucky Ones

Not all of these problems can be worked around, but enough of them can be worked around that functioning code can be written.

PHP scripts have to start and end with ‘<?’ and ‘?>’ respectively.

The workaround for this is to run the script using the eval() function.  The PHP’s eval() function doesn’t require the opening and closing tags as feature.

PHP variables must start with a ‘$’.

The dollar sign is a valid character in JavaScript variables, so starting them with the dollar sign poses no problem.

Most of JavaScript’s standard functions are wrapped in objects.  Math, String, etc.

PHP.js can be used to wrap the standard JavaScript functions.  Acting as a compatibility layer.

JavaScript uses the C++ style const keyword to define constants, while PHP uses the define() function.

PHP version 5.3.0 allows for the const keyword.

Taking the Plunge

As a proof of concept I wrote a small function to calculate  the great circle distance.  It computes the distance in both PHP and JavaScript.  You can find it here and hybrid PHP / JavaScript code here.

The Revelation

The name, a single dollar sign, is what struck me.  I never knew that such a character could be used in a variable name.  As I, and many others, were taught the only character allowed were:

1. letters
2. underscores
3. and numbers so long they are not the first character

I was interested and decided to embark on quest to figure out what characters were allowed and which weren’t.  This took me to the ECMA-262 specification, the standard for JavaScript, or ECMA Script as its formally called. The standard states:

Identifiers are interpreted according to the grammar given in Section 5.16 of the upcoming version 3.0 of the Unicode standard, with some small modifications.

The Unicode Standard allows for nearly 14,000 characters to be used in variables, as opposed to the measly 63 that are commonly used.

The Disappointment

I instantly tried to stuff Sigmas (Σ) and Double Struck N’s (ℕ) in to my code, but to my dismay, they didn’t work.  Disappointing? Yes.  Surprising? No.  It seems to be this way with all web standard’s, either implemented wrongly or incompletely (or both, heres looking at you Microsoft).  So, to test compliance I wrote a small JavaScript that creates a variable, using one of the Unicode characters, and assigns a value to it.  If throws an exception, then that means the browser doesn’t support that character.  Simple enough, but who wins this browser compliance battle.  So far Opera, it supports 99.5% of all characters.  Heres the data I’ve collected so far:

Yeah, I know thats a pathetically small dataset, but you can help run the test script and then post the last three lines in the comments along with the browser, version, and OS.  WARNING: This script takes a very long time to execute, it may appear to lock your browser up, but be patient.

On an unrelated note, Chrome’s Javascript engine was suprisingly fast.

Get it at: http://code.sachimp.com/ding/

• code.sachimp.com There’s some cool stuff including Inkscape extensions, Buildings API and, and some cool form widgets.
• hire.sachimp.com My portfolio and resume, In case you need a really good developer / designer.

Don’t forget to check out getcorkd.com if you need a model or photographer.