Before you read on this isn’t a post devoted to image caching. This is a post about data caching in general with image caching being an extreme form of data caching. It comes from a bit of work I did recently caching data from a tracking feed. It’s based around why you want to cache, what data you might need to cache and how you might cache (I used .NET but you can do it in all major web development languages). Caching has often been the premise of web sites that want to be, and I’m using a technical terms here, ‘screamingly fast’ and not ‘snail slow’.
Caching before caching was famous.
For many people developing with ArcGIS server caching means one thing, map caching. This has become the panacea for many scalability issues with applications. As if the maps are going to be the only slow part of your application. Whilst removing the bottlenecks of pretty maps away from your site is very important it often can form one part of your data caching strategy. There are often other areas that you might look at when deciding what to cache and when.
In web applications caching can occur in at a number of levels, specifically at the data level (caching data to avoid making costly requests) or the page output level (caching the output of web pages so they don’t have to be built in real time every time). Combining these areas can help an application scale but there are a number of considerations you should take into consideration before implementing any caching into any system.
Often with spatial systems, especially in the enterprise (within the firewall) the need for data to be current often outweighs the extra benefits that might be gained through getting a few more users onto the system. The difficulty is understanding with any application is how much it will scale and whether building this stuff into the project is worth it, the problem is that usually when you realise that it might be useful it’s too late to refactor your application and you’re stuck in a world of application fail. Now it’s easy to crow at the big internet applications when you see they are struggling with the amount of users on the system, most of us though will never have the same architectural issues that plague Twitter, Flickr or Facebook, if you do I suggest having a read of the Art of Capacity Planning to understand how to monitor the load you’re going to be under and to mitigate appropriately!
For the rest of us it shouldn’t mean that the use of caching won’t help our applications, even under modest scale and if you’re a .NET developer there are easy ways to implement it within the system, either at the code level (for data caching) or the server level (for output caching).
So what does to cache mean?
The process of caching is basically the storage of the output of a request or set of processing; so that once it has been done it may be reused for a number of people. The basic premise being that the output of the work is valid for a number of users and any changes to the underlying data will not impact the usage of the information within the time period that the cache is valid. This means that you need to understand the nature of the data and how it is used within any workflow. Failure to do so can mean your users potentially seeing data that is temporally incorrect, something that wouldn’t possibly be out of place in a episode of Doctor Who. So the two main questions with caching are the concepts of what to cache and how long the data can be validly stored within the cache.
What to cache?
The question of what to cache is always going to be a tricky one that will be down to the application that is being developed, down to the need for currency of the data being used. If you think that data can be delayed 15, 30 or 60 minutes, or even longer, without affecting the users operation of the site then it’s probably ready for caching. This could save endless requests for the same data that in turn reduces the processing cycles on your servers, reducing the number of machines you need to use thereby reducing the cost of the implementation. Even in this world of unlimited cloud machines, there is still a cost associated with every processing cycle and every saved resource is saved money, again good in these times of thrift.
In terms of exactly what to cache these three areas given by Microsoft in their enterprise caching block information are a good start:
You must repeatedly access static data or data that rarely changes.
Data access is expensive in terms of creation, access, or transportation.
Data must always be available, even when the source, such as a server, is not available.
In order to make your site as robust as possible you need to take a pessimistic view of software and servers (no comments please) and admit that at some time they will fail. If you want to protect yourself from such failure using a cache to store data and to make sure it never expires if you don’t get new information will allow your site to give the impression that it’s bullet-proof even though things are blowing up at the back end!
How to cache?
Microsoft make it amazingly straight forward to implement data caching, although as I said deciding what to cache and for
how long is another matter. On the web there are ways of adding to the data cache of an application using the HttpRuntime.Cache property that can be used to get access to the applications cache. The report manager example given on this MSDN page gives a starting point for implementing a class that can control access to a piece of cached information.
There are a number of choices that need to be made whilst using the cache object. Especially around the timing of objects, an absolute or sliding scale can be used to control how long an item stays in the cache. Long running static objects that don’t change can be kept alive using the sliding scale, as long as they are accessed within a time period they stay alive. Objects that need to be updated periodically can be made to expire at a certain time so as to make sure they are current to users but also to not require the server to process to much or request data from remote services too often. Further information about caching and ASP.NET can be found here.
Timing the Updates
Have a look at this picture. You can see the red line, which is the route the person is riding. You can see the person, WTF? I hear you say… well maybe not. But it shows a good point about getting the timing right with any bit of cached data. Here we have two cached pieces of information. The position and the route. We get both in WGS 84 and want to project them into Web Mercator to lie over our map tiles.
The point request can be performed very quickly but might not get updated very often; the route get’s updated even less but can take longer to project. We can see that if we update the position every 15 minutes, which is quiet timely enough when tracking someone on a bike, and the route every hour say, which again should be fine, then the application should be able to scale quiet well as the amount of requests we are actually making is quiet low. Why the problem?
Well obviously there may be a point whereby the position is not updated in the cache between the time that the the route is updated and the position is updated. This might be because the feed you are using (if it’s not your own) in turn is updated at different intervals. In the picture (at the start) we have the rider’s position slightly to the left of the end of the route as the caches are slightly out of sync. Five minutes later and the rider is back where they should be (and the corresponding rift in the space time continuum is closed, phew!), as shown in the picture to the right.
This raises an interesting point about how to time your cache. Do you have one cache for everything; do you cache various pieces of data at different rates depending upon how often the information get’s updated?
Do I need to care for my application?
As with any architectural decision within your application the need to use caching or not should be down to the actual amount of users you are going to get and the complexity of service calls you’re going to make. Often though it’s the unknown unknowns that with affect the application over its lifecycle that will make the difference, will someone decide to roll your nice heavy weight intranet application out over the internet (it’s happened I’ve seen it, it wasn’t pretty)? Will your obscure application suddenly be tweeted about by Stephen Fry (he loves to watch websites die!)?
There are many issues whereby having an understanding of the use of caching might be important, and in fact site saving! With the ease of implementation with today’s tools then there is little reason, apart from ignorance (usually mine…), whilst a certain level of caching can’t be built into even the most innocuous application.
Cycling to the Ashes
For me getting back into caching was all about developing a simple website for a charity bike ride for the ashes. Anyway whilst it might seem a different world from what we usually do at ESRI(UK) the actual scalability challenges that might be found in a consumer application that might be mentioned on a national radio or television show (or even by Mr Fry!) are the same in a large enterprise application and if designed correctly should be able to scale appropriately.
Anyway if you have read this far and are still awake (once again this post wasn’t meant to be so long!), please hop over to Oli’s site and go sponsor him, it’s a worthy cause.