Wednesday, January 28, 2009

Rest Exploration: Part 1. PUT versus POST

I have been working on a new set of Rest web services for and I keep coming across interesting challenges in defining a meaningful, REST-compliant API. I'll post more details as I work through the whole API.

Now, for the first challenge. I've discovered that PUT and POST are pretty complicated. Well, not complicated, but their proper use is not always obvious. I've read several articles that discuss PUT versus POST. Here's a brief list:
After all that reading, I think that I have a pretty decent understanding of the basics. Specifically, PUT is an idempotent method to create/set the full state of the resource specified at a particular URI. POST is not idempotent, nor is its operation stated so simply. Paul suggests that we think of POST as "create new URI as child of the current URI." In other words, POST is meant to add a new resource to a collection. In fact, the HTTP 1.1 spec nicely describes the distinction:
The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource.

As you can see, the true meaning of POST is pretty unclear. It can do, more or less, anything that the service wants. I think that Paul suggests we use POST sparingly because it can be made to do pretty much anything we want. And, like Spiderman's learns,
"with great power comes great responsibility".

This part makes total sense to me. But, here's where I get confused. What if I want to modify only part of the state of a resource?

As I understand the "official" thinking, PUT can really only be used if I send the entire state of the resource. If I want to update just part of the state, then PUT doesn't seem like the right verb. Since POST is the more flexible verb, perhaps I could use POST to modify just the part of my state that I care about. But somehow, this seems exceedingly unsatisfying. First, it's definitely violoating Paul's suggestion of using POST for adding child resources. Second, it feels like I'm shoe-horning a portion of the PUT function into a POST.

So, let's make it a bit more concrete with the real example I'm working on. In AboutMyBaby, you create a scrapbook. A scrapbook has a bunch of state or metadata - e.g., title, owner, url, and "template" or "skin". The "template" or "skin" defines the look and feel of that particular scrapbook. There are hundreds of templates, each of which is its own resource in REST-speak. So, each scrapbook resource "has a" template resource. The API needs a way to change the template of a given scrapbook. It's really that simple.

So, what are my options?

Option 1: Use PUT in the traditional way.
Meaning: In my client, I must retrieve the entire current state of the scrapbook resource, modify the one item I care about (namely the template URI to use), and PUT it to the URI of the scrapbook.
PUT /scrapbook/1234/metadata HTTP/1.1

<amb_template>URI TO new template</amb_template>

Pros: Well, it's definitely REST and it definitely follows all the precepts people talk about.
Cons: It seems totally bizarre and overly complicated to force the client to retrieve the entire state of the scrapbook just to modify 1 part of it. This means that there must be additional work on both the client (to get the state prior to updating) and the server (to service the "get state" request).

Option 2: Use POST
Meaning: In my client, I just create a POST message that encapsulates the change I want to make.
POST /scrapbook/1234/metadata HTTP/1.1

<amb_template>URI to new template</amb_template>

Pros: The client need not create the entire state of the scrapbook metadata to alter just the template. The resulting XML message (or other format) is relatively compact - at least as compared to the message representing the full state of the resource.
Cons: It just doesn't seem RESTy. I know the precise thing I want to have happen. The operation should be idempotent. Moreover, this feels like the easy way out, and frequently, the easy way is not the architecturally correct way.

Option 3: Make a new URI and use PUT
Meaning: Instead of having the URI represent the "metadata" resource associated with a scrapbook, make a new URI that is just the template associated with the metadata associated with the resource.
PUT /scrapbook/1234/metadata/template HTTP/1.1

<amb_template>URI to new template</amb_template>

Pros: This feels more RESTy. The action is very clear-cut. The operation is definitely idempotent.
Cons: Although it is RESTy, it also seems like a slippery slope. That is, what if I want to change the title of the scrapbook? Should the title be it's own resource with its own URI? This seems like it would cause an explosion of URIs associated with each little characteristic of a scrapbook.

Option 4: Filter the URI and use PUT
Meaning: Use a URI that represents the metadata of the scrapbook, but "filter" it by using a query string. Then, the client can use PUT but only update one part of the metadata.
PUT /scrapbook/1234/metadata?template HTTP/1.1

<amb_template>URI to new template</amb_template>

Pros: This feels RESTy. The action is very clear-cut. The operation is definitely idempotent. This option prevents the runaway resource creation of option 3.
Cons: Although it seems RESTy, I'm not sure that query strings were meant for PUT operations. I think this may lead to strange results down the road because it almost feels like I'm tweaking the true meaning of PUT. Also, I defined PUT at the outset as a method which requires that you send the entire state of the resource. Clearly, in option 4, we are modifying that dictum.

As you can see, a seemingly simple operation ("update the template associated with a scrapbook") has turned into 4 potential service definitions. I am still thinking through this problem, but I would love to hear the advice, thoughts, and comments on the structure of this API.

Tuesday, January 20, 2009

Blind repetition and learning from the past

I think that this post - "The 37Signals Effect" by Louis Brandy - is a great article and really points out a useful lesson. In short, Louis argues that by simply following the precise advice of a successful company or blogger, you will achieve your own success.

I firmly believe that far too often, people blindly follow the exact words of successful predecessors. Just think about it. Someone else, who was solving some other problem, took a series of steps that helped to solve that problem. Now you, facing a different set of problems, think that you can solve your problem by doing the exact same thing? That just doesn't make any sense. Blindly repeating action items is a sure recipe for failure. Or at least, doing so doesn't really get you any closer to success. You might still become successful - just not because of these repeated steps.

On the other hand, those who don't know history are doomed to repeat it. So, to assume that there are no lessons to be learned from past experiences of others is equally foolhardy. Surely, successful people were successful for some reason. Maybe they can't articulate that reason very clearly or maybe the reason was pure chance (as Malcolm Gladwell argues in Outliers), but either way, they were successful, you want to be successful, and so it's worth considering why they succeeded. Perhaps, buried somewhere in there is a nugget that will help you succeed in your business.

With this in mind, let's revisit Louis' primary example of blind repetition.

First, he quotes Getting Real:
Do less than your competitors to beat them. Solve the simple problems and leave the hairy, difficult, nasty problems to everyone else.

Then, Louis comments that "
doing 20% of the work for an 80% product, and solving only the easy problems" leaves the consumer wanting more and lets someone else duplicate your product or service quickly and easily.

There are actually 2 issues intertwined here - first, the "do less" strategy, second, the "easy solutions are vulnerable to duplication" challenge.

I think that the "do less" strategy misses the real point. Consumer-facing businesses succeed when they solve a problem or pain that consumers experience, and solve that pain "better" than the current solutions available to the consumer. Some consumer pains are solved with simple solutions and some require more complex solutions. But the complexity of the solution doesn't matter. What matters is how well it solves the problem people experience.

However, I also think that the "do less" strategy can be useful under certain circumstances. For example, new technology can fundamentally change the possibilities for solving a certain problem. Frequently, products that have traditionally solved a problem are eclipsed by newer technologies and businesses are slow to react. Or, if existing products are not eclipsed, then the newer technologies enable new solutions that solve the problem more successfully. When this happens, the "do less" strategy can be extremely successful. New, clean, simple products can quickly replace large, unwieldy, complex products. (
Usually, newer products are simpler, but only because they are newer and don't have days/weeks/months/years of cruft built up around them.) But, the critical issue isn't the product itself or its complexity - it's how the product addresses the pain felt by users. If your product addresses the user pain better than the competitor's, then you will likely succeed. Doing less or more work than your competitor per se doesn't make a difference.

Now, the "it's easy to duplicate easy solutions" challenge is different story. I think that there is an underlying flaw in the logic of this line of thinking. It suggests that your product is somehow novel or not duplicative or won't be copied if it's complex. Well, guess what. Pretty much every idea is a copy of some other idea. Jason Nazar points this out quite dramatically in his presentation "Mistakes People Make Before Starting a Business".

Mistake 3: Thinking no one else has ever done it before
... just go to page 3 of the Google search result... It doesn't make a difference if someone else is doing it, just do it better.
I think that Jason capture the point exactly. Don't worry if someone else is going to copy you or if you are copying someone else! Being successful is only partially about the idea. And the chances are, someone else either has done it before or is doing it now. So, who really cares if your idea is easy to duplicate? You are more likely to be successful if you focus on the solution to the problem faced by users. If you execute on your product better than anyone else, then your business is likely to succeed. (Or, at least, it won't fail just because someone copies you.) Eventually someone else will copy you, so you must do it better, cheaper, faster, etc. And if Google copies you, well, take it as a complement and move on to the next idea. Or walk into the Googleplex and pitch your business to them as a better solution that their simple demo app.

So, in conclusion, don't blindly repeat someone else's steps. But don't blindly ignore the learnings buried in others' experiences. Read the blog posts and articles from successful and unsuccessful people. Think about how they apply to you and your business. Shape those lessons and mold them into something that's useful for you. If you can't figure out how to do that, then file them away as interesting, but not clearly useful. Revisit that list periodically to see if you can make sense of those ideas you can't apply immediately. Of course that's not simple. But no one said that being successful in business is easy. If it was that easy, then everybody would be successful.

Friday, January 16, 2009

What web video could be like....

My day job is the Director of Technology for 60Frames Entertainment, a digital studio making, distributing, and monetizing original video content for the web. A lot of people have told me that video online will never be more than low quality, "watch me as I set myself on fire and ride my bike around" junk. Some people have duly noted that there is that kind of low-grade fair, and then there is porn, and that's it.

I think that it's possible to do more. I took this job because I think that there are people who are really good at making original video content and I firmly believe that the future of video is online. It's not that every video will be great. I mean, look at feature films - is every film good? No way. But, there are good films. In the same way, not every web video will be good. But, I haven't really seen a lot of great web videos.

If you don't believe that web video can be high quality, watch this video. It's a pilot that 60Frames produced with John August. If you're not in the entertainment biz, you might not know who John August is. But, if you've seen Charlie and the Chocolate Factory, Charlie's Angels, Go, or Big Fish, then you've seen John's work. You can read more of his bio on IMDB or read his blog about screenwriting.

I think you'll find this video is engaging, funny, fast-paced, and actually tells a story. Hopefully, it even leaves you wanting more..

So, spend a few minutes and check out this pilot. I think you'll agree that it's really pretty damned good.

The Remnants from John August on Vimeo.

Hi world. Nice to see you again!

After much delay, I've decided it's time to blog again. I have about a zillion mini-posts in my head that I just never bothered to write down. I know they won't all be fascinating, but what blog is 100% interesting all the time?

My posts will be a mix of technology, business, politics, sociology, and god knows what else comes into my mind.

So, enough of the introductory crap... let's get on with the getting on!

By the way, if you want to see my old blog posts, you can read them at