Epiphany: Replace HATEOAS With "Hypermedia Describes Protocols"

As a few of my friends know, I live for epiphanies. I love to connect concepts. So I'm really happy to be having one now (it's been a while as regular readers of my blog -- if any remain -- can tell).

For a LONG time, I've been talking about how all interfaces can be defined in terms of IFaPs (Identifiers, Formats, and Protocols). My canonical example of an interface composed of IFaPs is of course the Web: URL (I), HTML (F), and HTTP (P). All three intersect in a particular instance of HTML, say my blog's home page. The HTML for my blog's home page is filled with URLs, HTML tags, and even HTTP "verbs" (though these are quite rare, mostly in an HTML form or embedded JavaScript).

Then along came REST and with it the concept of HATEOAS: Hypermedia As The Engine of Application State. And everyone, myself included, spent a lot of time trying to grok it and explain it to others. We're still trying. One way I try to explain it is by highlighting that HATEOAS requires that each server response must contain not only the requested data -- but also control information (in the forms of specially tagged URLs) describing the next set of permitted interactions with the server. It is this additional control information (at a bare minimum just some links to more data) that turns mere media into hypermedia.

Now along comes Jim Webber with a much better (dare I say brilliant) way of explaining HATEOAS and hypermedia: "Hypermedia Describes Protocols!" (See slide 26.) At first this might seem counterintuitive, since I said that HTTP is the Protocol and HTML is the Format in the WWW. But URLs, HTML, and HTTP are just generic description languages for describing domain-specific identifiers, formats, and protocols. Thus, think of a web of specific HTML pages as a domain-specific protocol. Jim Webber uses the example of ordering a Starbuck's coffee. (What's important is that each hypermedia DSL is composed using the generic languages of URL, HTML, and HTTP.)

This notion of bringing together identifiers, formats and verbs to describe a protocol is not new. One of the best descriptions of this was in the WS-BPEL 1.1 spec:

In thinking about the data handling aspects of business protocols it is instructive to consider the analogy with network communication protocols. Network protocols define the shape and content of the protocol envelopes that flow on the wire, and the protocol behavior they describe is driven solely by the data in these envelopes. In other words, there is a clear physical separation between protocol-relevant data and "payload" data. The separation is far less clear cut in business protocols because the protocol-relevant data tends to be embedded in other application data.

So if WS-BPEL was already thinking about mixing protocol data with "payload" data, what's so new about HATEOAS? The fundamental difference is that WS-BPEL is based on the concept of providing an entire static protocol description up front once and for all -- and providing it out of band. But HATEOAS is based on the notion of progressive description (don't bother Googling the term, I coined it; and not to be confused with progressive disclosure). More and more of the description of the protocol is provided to the client (in band in the protocol itself) as the client executes its part of the protocol. I guess another good term might be JIT Protocol Description (Just In Time). Another good term might be "self-describing protocol". So now when explaining HATEOAS, instead of saying "each server response must contain control information" (huh?), I can say "each server response progressively self-describes the current protocol."

Now there are pros and cons to static/complete vs dynamic/progressive protocol descriptions. How can I program a client to execute its part of a protocol if I don't have a full description of it up front? But if I encode the complete static description of the protocol into my client up front, how can I change the protocol dynamically?

Love to hear others' thoughts. I'm going to think about this some more. That's why I love epiphanies -- they make you think about things in new ways.