This post is also available in: Russian

In this modern era of growth and development of mobile applications, enabling of efficient client-server interaction is gaining more importance. In this context, the following factors are critical: flexibility, scalability, documenting, support for older API versions, and caching. In this post, we are going to discuss the most important aspects of service development and design that will help you attain better results.

Flexibility and Extensibility

To provide for flexible and scalable client-server interaction, we use our RESTful API. A good RESTful API is that which can be modified quickly and easily. We can call a server RESTful, if it adheres to the REST concepts. When you are developing an API to be mainly used by mobile devices, the following key principles should be kept in mind. This applies not just to the API development, but to its support and enhancement.

State Agnostic

A RESTful server should in no way track, store, and use the current client context. The client should take this task on itself. In other words, do not make the server remember the state of the mobile device that is using the API.

Let’s imagine that we are developing a social application. In this context, a developer’s error might be providing of an API call allowing a mobile device to identify the last read item in the feed. The API call normally returning the feed (let’s call it /feed), will return items that are newer than the last read one. Sounds sophisticated, is not it? But have you "optimized" data exchange between the client and the server? Well, it looks like you have not.

What might go wrong in this particular case, is that if your user is accessing the service from two or three devices, then when one of the devices is setting its last read item, the others are not able to load their own last read items.

State agnosticism means that the data returned by a given API call should be agnostic of earlier calls.

A proper way to optimize the call is to use an HTTP header called If-Modified-Since. But let’s discuss it a bit later.

On its part, the client should store parameters generated on the server on accessing the server, using them for subsequent API calls, if needed.

Cached and Multi-Layered Architecture

The second principle is to alert customers that the server response may be cached for a certain period of time and re-used without any new requests to the server. As a client, you can use either the mobile device or a proxy server. More details on caching are given below.

Client-server separation and a single interface

The RESTful server should conceal its implementation from the customer as much as possible. The client does not need to know about which database is used on the server or how many servers currently process requests etc. Enabling of proper separation of functions is critical in the context of scalability, if your project starts to quickly gain popularity.

So those are, perhaps, the core three principles of RESTful server development.

Documenting

The worst documentation that might be written by a server developer is a long, monotonous list of API calls with parameters and return data. The main problem with this approach is that the changes to the server and the returned data format becomes a real nightmare as the project grows.

First and foremost, we recommend you to consider the basic, high-level data structures (models) used by your application. Then, try to conceive of the actions that can be applied to these components. The Foursquare API documentation is a nice case to be studied before you start writing your own documentation. This API consists of a set of high-level objects, such as locations, users, etc. Also, they define a set of actions that can be applied to these objects. As soon as you know your high-level objects and their operations applicable to them, creating of API call structure becomes easier and more comprehensible. For example, to add a new venue, it would be logical to invoke a method like /venues/add

Make sure to document all the high-level objects. Then, document your requests and responses to them based on high-level objects instead of plain data types. Instead of writing "This call returns three string fields, the first fields contains the id, the second contains the name, and the third contains description", write "This call returns a structure (or model) describing location."

Older API Support

Before the advent of mobile applications, in the era of Web 2.0, creating of different API versions was in no way a problem. Both the client (JavaScript /AJAX front-end) and server were deployed simultaneously. Your users (customers) always used the latest client version to access the system. As you were the only company developing both the client and server parts, you had a full control of how your API was used, and all API changes were immediately applied to the client. However, this is not applicable to multi-platform client applications. You may deploy ver. 2 of your API expecting that everything would be OK, but this would entail failure of iOS applications based on the old version. Users of older apps may still exist in spite your updated version upload to the App Store. You have to be always ready to maintain your API versions and discontinue support of some of them as soon as needed. However, your should support each of your API versions for at least three months.

URL based versioning paradigm

The first solution, is URL-based versioning. It means that, for example, api.example.com/v1/feeds will be used by version 1 of iOS application, while api.example.com/v2/feeds will be used by version 2. However good this all sounds, you can not continue to create copies of your server-side code for each change in the returned data format. We recommend this approach only for global API changes.

Model based versioning paradigm

Above we have described how you can document your data structures (i.e., models). Think of this documentation as an agreement signed between the developers of server and client parts. So, you should not make changes to the models without changing the version. This means that in the previous case, two models should be used: #1 and #2. Behavior of model #1 is the same as in the documentation.

Caching

Let’s now discuss caching. Caching is considered by many as mainly a client task (or a task run by the intermediate proxy.) However, while developing the server side at little effort you can make your API fully compliant with the intermediate caching proxy. It means that you get a free-of-charge load balancing from them. All you may need is described in Chapter 13 of the HTTP specification.

The two main principles we recommend, are:

Do not try to implement custom caching models in the client application.
Understand the basic principles of caching defined in the HTTP 1.1 RFC. It specifies two models of caching. They are: expiration model and validation model.

In any client-server application, the server is a trusted source of information. When you request a resource (i.e., a page or response) via the server API, the server sends to the client, among other things, some additional advice on how the client can cache the resulting resource. The server indicates to the client when the cached information would expire. Such advice can be sent to the client either programmatically or using the server configuration. The expiration model is usually implemented through the server configuration, while the validation model requires coding on the server side. It is for the developer to decide when to use either the expiration or validity models, based on the type of the resource returned. The expiration model is commonly used when the server can uniquely determine how long a particular resource will be valid. The validity model is used in all the other cases.

The expiration model

Let’s take a common caching configuration. If you’re using nginx, you probably have something similar in your config file:

location ~ \. (jpg | gif | png | ico | jpeg | css | swf) $ {expires 7d;}

nginx converts these settings in the appropriate HTTP header. In this case, the server sends either the "Expires" field or "Cache-Control: max-age=n" in the header for all images, assuming that the client will cache them for 7 days. This means that you would not have to request that data for 7-days. Each of the common browsers (and intermediate proxies) processes this header and works as expected. However, most of the Open Source frameworks enabling image caching for iOS, including the popular SDWebImage, use built-in caching, simply removing the images after n days. The problem is, that such frameworks are in conflict with the validity model.

Validity model

Both Facebook and Twitter solve the issue of replacing outdated profile images after a new image has been uploaded, using the validity model. In the validity model, the server sends to the client a unique resource identifier and the client caches both the identifier and the answer. In terms of HTTP, such a unique identifier is called ETag. When you make a second request to the same resource, you should send its ETag. The server uses this ID to validate whether the requested resource was changed since the last access (remember, the server is the only trusted source of information.) If the resource has actually changed, the server delivers its final copy. Otherwise, it sends 304 Not Modified. Implementation of the cache validity model requires additional development efforts, both on the client and server sides. Let’s now describe both of these models in detail.

Support at the Client Side

In fact, under iOS, if you use MKNetworkKit, it does the entire job automatically.

[[MKNetworkEngine sharedEngine] useCache];

But for the Android and Windows Phone developers, we will dwell on the details of how this should be implemented. The cache validity model uses HTTP headers called ETag and Last-Modified. Implementation of the client part is easier than of the server part. If you received ETag with a resource, when making a second request for it, send ETag to "IF-NONE-MATCH" header field. Similarly, if you have got "Last-Modified" with a resource, include it in the "IF-MODIFIED-SINCE" header of your subsequent requests. On its part, the server will decide when to use "ETag" and when to use "Last-Modified".

Implementation of the expiration model is quite simple. Simply calculate the expiration date based on header fields, such as "Expires" or "Cache-Control: max-age-n", and clear the cache on that date.

Server-side implementation

Use of ETag

Generally, ETag is calculated on the server using a hashing algorithm. (Most high-level server languages, such as Java/C#/Scala have object hashing tools). Prior to generating a response, the server should calculate the object hash and add it to the ETag header field. Now, if the client has really sent IF-NONE-MATCH in the request and this ETag is equal to what you have calculated, send 304 Not Modified. Otherwise, create a reply and send it with a new ETag.

Using Last-Modified

Implementation of Last-Modified is not particularly easy. Let’s suppose that our API has a call to return a list of friends.

http://api.socialnetwork.com/friends/

When you use ETag, you should calculate a hash of the array of friends. When using Last-Modified, you should send the last modified date of a given resource. Since this resource is a list, such date should reflect the last time a friend was added. For this purpose, the developer should enable saving of the last modified date for each user in the database. A little more complicated than ETag, this method gives a great advantage in terms of performance. When a client requests a resource the first time you deliver the full list of friends. Afterwards, requests from the client will have "IF-MODIFIED-SINCE" field in the title. Your server-side code should only deliver the list of friends added after that date. Prior to database access, the code of accessing the database was like:

SELECT * FROM Friends;

After the modification it has become:

SELECT * FROM Friends WHERE friendedDate> IF-MODIFIED-SINCE;

If the query does not return any records, send 304 Not Modified. Thus, if the user has 300 friends and only two of them have been added recently, the response will contain only two entries. The query processing time on the server and relevant resources involved, are hence reduced significantly. Of course, this is a highly simplified code. The developers will have even more headache when they decide to support removal or blocking of friends. The server should be able to advice the client on which friends have been added or removed. This technique requires additional server-side development efforts.

Caching model selection

Well, this was not an easy topic. Now, let’s try to summarize and formulate the basic rules for the use of a particular caching model.

All static images should be served based on the expiration model.
All the data generated dynamically should be cached using the validity model.
If your dynamically generated resource is a list, you should use the validity model based on Last-Modified. (For example, /friends). In the other cases, use the validity model based on ETag. (For example, /friends/firstname.lastname).
Images or any other resources that can be changed by the user (e.g., avatars), should also be cached by the validity model based on ETag. However these are images, they are not as permanent as, say, a corporate logo. In addition, you may just not be able to accurately estimate the expiry term of such resources.

Another way (which although it is simpler to implement, but it is somewhat hackerish), is the use of "URL error". When the response contains avatar’s URL, part of it should be made dynamic. So, your should replace URL

http://images.socialnetwork.com/person/firstname.lastname/avatar

with

http://images.socialnetwork.com/person/firstname.lastname/avatar/ <hash>

If the user changes an avatar, the hash should change. The call delivering your friend list, will now send modified URLs for users that have changed their avatars. This way, changes in the profile images would proliferate almost instantly! If your server and client applications comply with the generally adopted caching standards, your iOS app and your product would run smoothly.

In this post, we have provided a simple explanation of the standards to which the vast majority of developers fail to adhere.

DENIVIP Media

Video apps and services, technologies and business

Best Practices for Server and Application Interaction