Executive Summary: Who knew serving up JSON for a public API was so complicatedly anti-hillbilly?

Our next major phase at BookedIN’s plan for world-except-for-Edmonton-and-certain-parts-of-Winnipeg-domination is underway. That’s the public-facing site which will provide a way for YOU THE PUBLIC to book appointments online at your favorite…ahem…”service providers”. Let me explain our marketing strategy in detail.

Ha ha, I jest of course. That last sentence is the sum-total of what I know about our marketing strategy. I have a hard enough time trying to keep myself entertained through code. (As a general rule, I start with the code reviews.)

To populate the public site, we’re building an API around our appointment manager. And we’ve opted for JSON as the default format mostly because I love the way people pronounce it, accenting both syllables like badly-translated anime.

In this app, we are making two API calls from two different places. When we first load the page for a particular vendor, the server makes a call to retrieve the company details and a list of services. After the page is loaded, jQuery comes along and populates the list of available appointment times. In this way, we get the benefit of SEO being able to see the vendor details and services and a snappy user interface when the user navigates to different dates within a particular vendor page, as outlined on the Google Webmaster Blog (though I didn’t actually discover that link until after we decided on the structure).

In the appointment manager, serving up JSON is pretty simple. Configure the servlet, get the data, convert to JSON (we’re using Jackson), and write it to the response. This is working just fine with the server-side company details call.

For the client-side call, it’s not. Depending on how you configure the AJAX call in jQuery, we get one of the following:

  • The API call is never made
  • The call is made but cancelled
  • The call is made and returns but has no data

All are symptoms of the same issue: cross-domain client calls, which aren’t allowed. I read that it’s for security reasons which, due to my loathing of all things security-related, was enough for me not to read further.

From what I can tell, you can’t make a call to another domain in jQuery (or likely any JavaScript library) and expect to get JSON back.

Here’s an example. Follow this link in your browser: https://github.com/api/v2/json/repos/search/zenboard

You’ll probably get this back:

   1: {"repositories":[{"type":"repo","username":"amirci",

   2: "url":"https://github.com/amirci/zenboard","watchers":5,"owner":"amirci",

   3: "has_wiki":true,"open_issues":0,"score":6.994658,"followers":5,"forks":3,

   4: "has_issues":true,"language":"Ruby",

   5: "description":"Companion to agile zen to provide extra calculations and functionality",

   6: "pushed":"2011/05/18 15:25:54 -0700","fork":false,"size":1348,

   7: "created_at":"2010/11/24 17:28:33 -0800","name":"zenboard","has_downloads":true,

   8: "private":false,"pushed_at":"2011/05/18 15:25:54 -0700",

   9: "created":"2010/11/24 17:28:33 -0800","homepage":""}]}

(Side note: If you’re using AgileZen, the project above, ZenBoard, is an awesome companion for it.)

Now let’s try this in jQuery:

   1: function loadProjects( ) {

   2:     var url= "https://github.com/api/v2/json/repos/search/zenboard";

   3:   $.ajax({

   4:       url: url,

   5:       type: "GET",

   6:       dataType: "json",

   7:       success: function(data) {

   8:           alert( 'moo' );

   9:       }

  10:   });

  11: }

Throw this into a $( document ).ready call, load it up, and you get nothing. The browser developer tools give you a vague hint of what’s going on:

CanceledRequest

 

The request to the GitHub API was canceled. Let’s make one small change to the JavaScript:

   1: function loadAppointmentTimes( ) {

   2:     var url= "https://github.com/api/v2/json/repos/search/zenboard";

   3:   $.ajax({

   4:       url: url,

   5:       type: "GET",

   6:       dataType: "jsonp",

   7:       success: function(data) {

   8:           alert( 'moo' );

   9:       }

  10:   });

  11: }

The only difference here is the datatype is now jsonp instead of json. Load this page up and we get a hearty “moo” alert.

But take a look at the headers for the request we’ve made:

RequestWithURL

There’s an extra parameter: callback. jQuery added this. Furthermore, here’s the response:

RequestWithResponse

This ain’t quite JSON. It’s a JavaScript function call wrapped around JSON.

The upshot of all this: When you make a request without the callback parameter, GitHub will give you JSON. Unless it’s called from JavaScript from the browser in which case, I believe it’s the browser itself that says “Papa don’t preach that way” and cancels the request completely because it won’t allow JSON to come back.

But when you tell jQuery to make the call as jsonp, it adds an auto-generated callback parameter (I believe you can specify the name of the callback if you so desire). GitHub is nice enough to adjust the response accordingly, wrapping the JSON in the callback function. Further, jQuery is nice enough to strip it off again and give you back the intended JSON.

When I first started this post, it was going to lead up to this point where I ask: When creating an API, is it normal to support both JSON and JSONP requests? In fact, I ask the question in the title already.

But it appears the answer is yes based on other services that offer APIs. Obviously, GitHub supports it. So does BitBucket, Twitter, AgileZen, and Flickr (though Flickr uses a different callback parameter name). So…thanks for listening, I guess…

Final Note

JSONP (and any cross-domain request, I believe) is read-only. I.e. it supports GET requests only, not POSTs (or PUT or DELETE, I suppose). The odd thing is: I only discovered this today while researching for this post. This baffles me because I’ve been doing AJAX apps since 2000 and don’t remember ever having to deal with this. I suppose they were all same-domain, or if they were cross-domain, it was read-only. Anyway, score one for blogging because within the next two days, we would have run into this very issue at BookedIN and wasted a bunch of time tracking down the cause.

To get around this limitation, there are two options (probably more depending on how academic you want to make your research):

Use a proxy

That is, you POST the request to the same domain and in the server code, forward the request on to the other domain. Remember, cross-domain security is only on the browser.

Cross-Origin Resource Sharing

This is new to me as well as of this morning. It’s a draft specification to get around the same origin policy I’ve spent these many minutes describing. I know nothing about except what I read on AgileZen’s documentation. The salient points are:

  • Works on newer browsers only. This essentially translates into “Works on IE8 or higher and all versions of Firefox and Chrome that you’re likely to see in the wild.”
  • May have issues with some proxy servers and firewalls

This was enough for us to postpone this route to a future release when our client isn’t the only one for the API.

Kyle the Applicable