Introduction and Disclaimer

This post is going to describe how I've ending up designing, what I consider to be a fairly RESTful web API. I'm far from being an expert, and this is definitely the closest thing to a RESTful API that I've ever created, so I'm not even experienced with REST APIs. So if I were you, I wouldn't take anything here as REST gospel. If you've got some criticisms, let me know, I'm all about the learning. There's also so much more to a RESTful architecture than what I describe in this blog post, this is simply what it took for me to find my way on to the path.

Until about 6 months ago, I'd always been sceptical of creating RESTful APIs, but I think I've had a few pennies drop since then that have made me fairly confident that I grasp the basics pretty well.

One of the biggest problems I've had was finding really good concreate examples, but recently the huddle-apis documentation popped up on my twitter timeline (via @bwaine, @dzuelke and @darrel_miller,) and I found it really useful.

Authentication

Something I wanted to clear up straight away was authentication. One of my biggests misconceptions about REST was the 'Stateless' constraint. I took that way too literally (or maybe I didn't and I now I'm just ignoring it). I'm now of the opinion that a session cookie is a valid way to authenticate a consumer for a REST API. That said, I wanted to go better than that, so I went and implemented WSSE authentication as described by the ATOM API. Now my application supports both, so the API can be consumed via XHR from the websites pages, and also by the more 'stateless' WSSE authenitcation headers. I've actually implemented this using Symfony's security component, which could be an entire blog post unto itself. In retrospect, I think I probably should have considered OAuth, but I always thought that was more about authorisation than authentication.

Richardson Maturity Model

Richardson Maturity Model

The Richardson Maturity model is where I started with regards to setting goals for designing the actual web service. My basic knowledge of REST web services would have me meeting level 2 fairly easily, so I needed to brush up on the two things that would take me fully to level 3.

HTTP Verbs - PUT vs POST

Admittedly I had a rough idea of correct use of the HTTP verbs, but the penny didn't fully drop on the POST vs. PUT until I really started digging in to articles. My take on it is now, if you're use PUT, you'll always be sending a full representation of the resource you expect to find at the URI for that resource. A POST would generally be to create something where you wouldn't necessarily know the URI of the newly created resource or where you aren't sending a full representation of the resource.

  • POST /messages - Add this message resource to the /messages collection (create)
  • PUT /messages - Store this collection of messages at /messages
  • PUT /messages/123 - Store this message resource at /messages/123 (create or update)
  • POST /messages/123 - Store these attributes in the message resource at /messages/123

I don't think my interpretation would ring completely true with the purists, but it makes sense in my mind and it gave me the chance to move on to more important things...

HATEOAS and Defining Media Types

I'm aiming to hit level 3 of the model, which means conforming with the Hypermedia as the engine of application state (HATEOAS) guiding principle. I'm not overly convinced by the whole principle within the context of my requirements, but I thought I'd give it a shot.

The first step for is to document the media types we're going to accept and serve through the API.

I decided to solely support JSON as the data format I'll be providing to consumers. Every time I provide or consume XML, I hope it will be the last, so I won't be bothering with that. I've no real interest in HTML either, although it is nice to be able to browse APIs in a browser. With that in mind, I did a bit of reading about HAL in JSON, and liked most of what I found, so decided to take a similar approach.

I attended the PHPNW conference last year, and picked up some good points about media types from Ben Longden's talk, mainly:

  • Don't version the API, version the media types
  • Leave room for expansion and you might not have to version the media types

You see a lot of APIs with URIs like /1/messages, where 1 would be the API version. This goes against the REST grain, affectively describing a messages resource in version 1 of the API, whereby /2/messages would actually be the same resource, but in version 2 of the API. I'm hoping to never have to take that route, if my API changes that drastically, I'll introduce a new version of the media types that are affected. Thereby I'll be able to serve either the version 1 representation of the /messages resource, or the version 2 representation.

Leaving room for expansion is an easy one. Rather than:

    {
        "to": ["dave@atstsolutions.co.uk"]
    }

We'd do:

    {
        "to": [{"address": "dave@atstsolutions.co.uk"}]
    }

If we wanted to add more information to each recipient at some point, we could do so without affecting existing consumers of the API.

Example - API Resource (our API's entry point)

Now, one of the HATEOAS principles is that the consumer should be able to navigate to the resource they require from an original endpoint. I'm not really too sure how narrow or wide that can be, but for argument's sake, I'm going to have an API resource at the root URI of my API, with a media type of application/vnd.acme.api+json. This way consumers should be able to hit the root uri to start with and navigate to the resource they need. Personally, I find it very hard to imagine myself doing it, I'd be looking up the resource uri once and then hard-coding it, but maybe that's just me.

I'm currently documenting the media types on a single in my technical documentation, so as an example, the API resource would look something like this:

application/vnd.acme.api+json media type:

    {
        "_links": {
            "self": {"href": "/", "type": "application/vnd.acme.api+json"},
            "messages": {"href": "/", "type": "application/vnd.acme.messages+json"}
        },
        "version": "1.0",
        "author": "Dave Marshall"
    }

I got introduced to Behat when I worked at Sky Betting and Gaming, and it's been an invaluable tool for me in the new job, particularly as it's enabled me to quickly create automated tests for rickety old legacy php scripts, where unit testing is out of the question. It's further worked out as a nice tool to both test and, coupled with the media types documentation, document the API's usage as well as behaviour.

Cuking it Wrong: I've been using behat to write integration tests for my API, rather than acceptance tests. They're written by a developer, for developers and speak REST over HTTP rather than the language of the domain. They're also fairly declaritive, I find it easier to write specific tests and cover edge cases. I think it goes against the grain of most people doing BDD with cucumber/behat, but it works for me.

Given our new endpoint, I can write a couple of simple scenarios covering the end point and authentication:

Feature: Acme example API
    In order to make things happen
    As an API consumer
    I need to make API calls

    Scenario: Authentic consumer checks API details
        Given the following API users exist:
            | Username    | Password    |
            | username123 | password123 |
        And I am API user "username123" with password "password123"
        And I send "wsse" authentication
        And I accept "application/vnd.acme.api+json"
        When I send a GET request to "/"
        Then the api response status code should be "200"

    Scenario: Authentic consumer checks API details with incorrect password
        Given the following API users exist:
            | Username    | Password    |
            | username123 | password    |
        And I am API user "username123" with password "incorrectpassword"
        And I send "wsse" authentication
        And I accept "application/vnd.acme.api+json"
        When I send a GET request to "/"
        Then the api response status code should be "401"

    Scenario: Non-Authentic consumer checks API details
        Given the following API users exist:
            | Username    | Password    |
            | username123 | password    |
        And I am API user "incorrectusername" with password "incorrectpassword"
        And I send "wsse" authentication
        And I accept "application/vnd.acme.api+json"
        When I send a GET request to "/"
        Then the api response status code should be "401"

Example - Messages Resource

As I'm sure you noticed, the API resource representation example above includes a link to a messages resource. Given that link, we can right some gherkin to test the messages resource API.

Feature: Messages Resource API
    In order to manage messages
    As an API consumer 
    I need to make create, read, update and delete messages resources

    Background:
        Given the following API users exist:
            | Username    | Password    |
            | username123 | password123 |
            | username456 | password456 |
        And I am API user "username123" with password "password123"
        And I send "wsse" authentication
        And I accept "application/vnd.acme.api+json"
        When I send a GET request to "/"
        Then the api response status code should be "200"
        And I remember the JSON response value at "_links/messages/href" as "messages_uri"

    Scenario: Authentic user retrieves a collection of messages
        Given the following messages exist:
            | From        | To          | Subject        | Body           |
            | username456 | username123 | Test Message 1 | Blah Blah Blah |
            | username456 | username123 | Test Message 2 | Blah Blah Blah |
        And I accept "application/vnd.messages.api+json"
        When I send a GET request to "<messages_uri>"
        Then the api response status code should be "200"
        And the JSON response should have 2 "messages"
        And the JSON response value at "messages/1/_embedded/message/subject" should be "Test Message 2"

This scenario above uses a background to 'navigate' to the messages resource uri via the API resource described earlier, and then tests the messages resource API. Let's take a look at the application/vnd.acme.messages+json media type documentation.

{
    "_links": {
        "self": { 
            "href": "/messages", 
            "type": "application/vnd.acme.messages+json"
        },
        "send-message": { 
            "href": "/messages/outbox", 
            "type": "application/vnd.acme.message+json",
            "method": "POST"
        }
    },
    "messages": [
        {
            "_embedded": {
                "_links": {
                    "self": {
                        "href": "/messages/1",
                        "type": "application/vnd.acme.message+json"
                    }
                },
                "from": {
                    "name": "username456"
                },
                "to": {
                    "name": "username123"
                },
                "subject": "Test Message 1",
                "body": "Blah Blah Blah"
            }
        },
        {
            "_embedded": {
                "_links": {
                    "self": {
                        "href": "/messages/2",
                        "type": "application/vnd.acme.message+json"
                    }
                },
                "from": {
                    "name": "username456"
                },
                "to": {
                    "name": "username123"
                },
                "subject": "Test Message 2",
                "body": "Blah Blah Blah"
            }
        }
    ]
}

Whoa, a little verbose! This is definitely the downside to trying to stick with the HATEOAS principles, but you kind of get used to it. A few things to note in the example:

  • Links - we've got a send-message link, pretty self explanatory. I've added the type and the method as attributes, as it's one thing I actually found to be missing in HAL+JSON. Most of the time the HTTP method to use can be assumed, but everything else is so explicit I thought why not. I'd do the same in a HTML form, so why not in my JSON representation.

  • Embedded Message - Rather than just a list of URIs to the individual messages, I've embedded some of the message contents

  • The from and to attributes of the messages are both JSON objects. Right now they only have a name attribute, but somewhere down the line we might introduce a user resource, at which point we could add links at the very least, if not embedded information such as full name and other attributes a user may have.

Now we know where to find the hyperlink to send a message, we can write an integration test that will discover the URI and create a message


    Scenario: Authentic user sends a message 
        Given the following messages exist:
            | From        | To          | Subject        | Body           |
            | username456 | username123 | Test Message 1 | Blah Blah Blah |
            | username456 | username123 | Test Message 2 | Blah Blah Blah |
        When I send a GET request to "<messages_uri>"
        Then the api response status code should be "200"
        And I remember the JSON response value at "_links/send-message/href" as "send-message-uri"
        Given I accept "application/vnd.message.api+json"
        And I send "application/vnd.message.api+json"
        When I send a POST request to "<send-message-uri>" with the following:
            """
            {
                "from": {
                    "name": "username123"
                },
                "to": {
                    "name": "username456"
                },
                "subject": "Hello",
                "body": "Blah Blah Blah"
            }
            """
        Then the api response status code should be "201"
        # and check the response attributes etc.

And that's about it. I'm really happy with the progress I've made, but please feel free to tear in to my practices in the comments below. I'll be back with some more REST related posts, probably on my implementation with Silex and probably some more details on the custom Behat contexts I'm using.