Comparing data with RESTful API?

The whole concept behind REST is that you leverage the power of the underlying HTTP protocol.

The whole concept behind REST is that you leverage the power of the underlying HTTP protocol. In this case there are two HTTP headers that can help you find out if the list on your mobile device is stale. An added benefit is that the client on your mobile device probably supports these headers natively, which means you won't have to add any client side code to implement them!

If-Modified-Since: check to see if the server's copy has been updated since your client first retrieved it Etag: check to see if a unique identifier for your client's local copy matches that which is on the server. An easy way to generate the unique string required for ETags on your server is to just hash the service's text output using MD5. You might try reading Mark Nottingham's excellent HTTP caching tutorial for information on how these headers work.

If you are using Rails 2.2 or greater, there is built in support for these headers. Django 1.1 supports conditional view processing. And this MIX video shows how to implement with ASP.Net MVC.

Good idea! But you still need to fetch the entire list if it was modified. Or can you get around that?

– Rasmus Kaj Nov 9 '09 at 7:26 The caching tutorial was very informative. However I believe at the end it will be my code that must handle all the fields and so I basically would implement the fourth option I outlined in my PS. – Steve Nov 9 '09 at 16:03.

There is nothing in HTTP that says that POST must update the server. People seem to forget the following line in RFC2616 regarding one use of POST: Providing a block of data, such as the result of submitting a form, to a data-handling process; There is nothing wrong with taking your client side wishlist and POSTing to a resource whose sole purpose is to return a set of differences. POST /Bookstore/WishlistComparisonEngine.

I think the key problems here are the definitions of Book and Wishlist, and where the authoritative copies of Wishlists are kept. I'd attack the problem this way. First you have Books, which are keyed by ISBN number and have all the metadata describing the book (title, authors, description, publication date, pages, etc.) Then you have Wishlists, which are merely lists of ISBN numbers.

You'll also have Customer and other resources. You could name Book resources something like: /book/{isbn} and Wishlist resources: /customer/{customer}/wishlist assuming you have one wishlist per customer. The authoritative Wishlists are on the server, and the client has a local cached copy.

Likewise the authoritative Books are on the server, and the client has cached copies. The Book representation could be, say, an XML document with the metadata. The Wishlist representation would be a list of Book resource names (and perhaps snippets of metadata).

The Atom and RSS formats seem good fits for Wishlist representations. So your client-server synchronization would go like this: GET /customer/{customer}/wishlist for ( each Book resource name /book/{isbn} in the wishlist ) GET /book/{isbn} This is fully RESTful, and lets the client later on do PUT (to update a Wishlist) and DELETE (to delete it). This synchronization would be pretty efficient on a wired connection, but since you're on a mobile you need to be more careful.

As @marshally points out, HTTP 1.1 has a lot of optimization features. Do read that HTTP caching tutorial, and be sure to have your web server properly set Expires headers, ETags, etc.Then make sure the client has an HTTP cache. If your app were browser-based, you could leverage the browser cache.

If you're rolling your own app, and can't find a caching library to use, you can write a really basic HTTP 1.1 cache that stores the returned representations in a database or in the file system. The cache entries would be indexed by resource names, and hold the expiration dates, entity tag numbers, etc. This cache might take a couple days or a week or two to write, but it is a general solution to your synchronization problems. You can also consider using GZIP compression on the responses, as this cuts down the sizes by maybe 60%.

All major browsers and servers support it, and there are client libraries you can use if your programming language doesn't already (Java has GzipInputStream, for instance).

If I strip out the domain-specific details from your question, here's what I get: In your RESTful client-server application, the client stores a local copy of a large resource. Periodically, the client needs to check with the server to determine whether its copy of the resource is up-to-date. Marshally's suggestion is to use HTTP caching, which IMO is a good approach provided it can be done within your app's constraints (e.g. , authentication system).

The downside is that if the resource is stale in any way, you'll be downloading the entire list, which sounds like it's not feasible in your situation. Instead, how about re-evaluating the need to keep a local copy of the Wishlist in the first place: How is your client currently using the local Wishlist? If you had to, how would you replace the local copy with data fetched from the server?

What have you done to minimize your client's data requirements when building its Wishlist view(s) and executing business logic?

Good questions, but I am afraid they have already been checked. The client does need the full wishlist and needs to have it updated regularly (it's just an example application). – Steve Nov 9 '09 at 7:07.

Your third alternative sounds nice, but I agree that it doesn't feel to RESTfull ... Here's another suggestion that may or may not work: If you keep a version history of of your list, you could ask for updates since a specific version. This feels more like something that can be a GET operation. The version identifiers could either be simple version numbers (like in e.g. Svn), or if you want to support branching or other non-linear history they could be some kind of checksums (like in e.g. Monotone).

Disclaimer: I'm not an expert on REST philosophy or implementation by any means. Edit: Did you ad that PS after I loaded the question? Or did I simply not read your question all the way through before writing an answer?

Sorry. I still think the versioning might be a good idea, though.

I'm afraid I had the PS in the original text, though admittedly it's not featured very prominently :) – Steve Nov 9 '09 at 7:08.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.