Background
Once a page has been cached in the browser (or proxy), the server cannot make the cached copy be utilized in place of the updated version. Pages that are likely to change cannot, therefore, be cached at the client for very long. There is nothing we can do to change this.
We should have greater control at the server, but sadly, the built-in response caching prevents cache invalidation. Additionally, it employs the same cache duration for asking the browser to cache the page and for server-side caching. Some apps cannot use the built-in response caching due to these limitations.
We are going to investigate developing a unique response caching system. We’ll presum the following conditions:
- Allow server and client cache durations to be different
- Allow an arbitrary page to be removed from the cache at will
- Allow multiple cached pages to be removed based on a common criterion (tags)
The fundamental prerequisite is the capacity for endless caching at the server and the ability for application code to invalidate current cache entries when content changes.
Additionally, we’ll prioritize efficiency and quickness over adaptability. IMemoryCache will be used to implement the cache for a single web server. It is simple to go from using IMemoryCache to IDistributedCache in web farm scenarios, however because of this interface’s less features, the third criteria (tagged) cannot be met without a redesign.
Implementation
We will implement our caching as middleware that runs before MVC and can short-circuit the entire request pipeline if a cached page is found in order to enable it to be as quick as feasible.
We have the following flow for a page that is not in the cache:
- Fail to retrieve page from the cache
- Execute inner middleware (the MVC pipeline) redirecting the response to a buffer
- Cache the page (if conditions are met)
- Render page
The sequence is substantially shorter for a page that has already been cached:
- Retrieve page from cache
- Render page
The quickest way to return cached material is using middleware, but it is more challenging to allow specific page cache setup. The middleware might be utilized independently if all pages were cached using the same criteria, but for the majority of real-world sites, we require the ability to control which pages are cached and for how long. Although middleware configuration might be used, using attributes on controller actions is considerably simpler.
Controlling server caching
We’ll use a very basic action filter attribute to let us manage how specific pages are cached:
public class CacheAttribute : ActionFilterAttribute
{
public int? ClientDuration { get; set; }
public int? ServerDuration { get; set; }
public string Tags { get; set; }
public override void OnActionExecuting(ActionExecutingContext context)
{
// validation omitted
if (ClientDuration.HasValue)
{
context.HttpContext.Items[Constants.ClientDuration] = ClientDuration.Value;
}
if (ServerDuration.HasValue)
{
context.HttpContext.Items[Constants.ServerDuration] = ServerDuration.Value;
}
if (!string.IsNullOrWhiteSpace(Tags))
{
context.HttpContext.Items[Constants.Tags] = Tags;
}
base.OnActionExecuting(context);
}
}The filter’s simple implementation only adds the attribute values to the collection of HttpContext Items. To communicate between the operation and the middleware for caching, we are using the collection.
The middleware
The primary Invoke method of the middleware is quite readable:
public async Task Invoke(HttpContext context)
{
var key = BuildCacheKey(context);
if (_cache.TryGet(key, out CachedPage page))
{
await WriteResponse(context, page);
return;
}
ApplyClientHeaders(context);
if (IsNotServerCachable(context))
{
await _next.Invoke(context);
return;
}
page = await CaptureResponse(context);
if (page != null)
{
var serverCacheDuration = GetCacheDuration(context, Constants.ServerDuration);
if (serverCacheDuration.HasValue)
{
var tags = GetCacheTags(context, Constants.Tags);
_cache.Set(key, page, serverCacheDuration.Value, tags);
}
}
}Your requirements will determine how you construct the cache key, however it may only require the use of context.Request.Path.
The request is finished if the page can be retrieved from the cache and written to the response.
private async Task WriteResponse(HttpContext context, CachedPage page)
{
foreach (var header in page.Headers)
{
context.Response.Headers.Add(header);
}
await context.Response.Body.WriteAsync(page.Content, 0, page.Content.Length);
}If a cached page cannot be located, additional effort must be done. In order to determine if we may cache the request, we first set any necessary client caching headers. If another method is used, we call the following middleware component and stop processing the request because we only wish to cache GET methods.
Capturing the request output from internal middleware components is the next step. In the part after this, we go over how this is accomplished. If we’ve set up server side caching (using the action filter mentioned above), the last step is to save the page to the cache.
Capturing the response
You must replace the response body stream’s default with a MemoryStream in order to capture the page response:
private async Task<CachedPage> CaptureResponse(HttpContext context)
{
var responseStream = context.Response.Body;
using (var buffer = new MemoryStream())
{
try
{
context.Response.Body = buffer;
await _next.Invoke(context);
}
finally
{
context.Response.Body = responseStream;
}
if (buffer.Length == 0) return null;
var bytes = buffer.ToArray(); // you could gzip here
responseStream.Write(bytes, 0, bytes.Length);
if (context.Response.StatusCode != 200) return null;
return BuildCachedPage(context, bytes);
}
}We return null and make no attempt to cache the response if nothing has been written to it or if the status code is not 200. If not, a CachedPage instance is returned:
internal class CachedPage
{
public byte[] Content { get; private set; }
public List<KeyValuePair<string, StringValues>> Headers { get; private set; }
public CachedPage(byte[] content)
{
Content = content;
Headers = new List<KeyValuePair<string, StringValues>>();
}
}The content of the cached page is combined with a subset of the response headers, some of which should be removed (for example, the date header).
Controlling client caching
We choose the straightforward method when it comes to providing caching headers to the browser:
public void ApplyClientHeaders(HttpContext context)
{
context.Response.OnStarting(() =>
{
var clientCacheDuration = GetCacheDuration(context, Constants.ClientDuration);
if (clientCacheDuration.HasValue && context.Response.StatusCode == 200)
{
if (clientCacheDuration == TimeSpan.Zero)
{
context.Response.GetTypedHeaders().CacheControl = new CacheControlHeaderValue
{
NoCache = true,
NoStore = true,
MustRevalidate = true
};
context.Response.Headers["Expires"] = "0";
}
else
{
context.Response.GetTypedHeaders().CacheControl = new CacheControlHeaderValue
{
Public = true,
MaxAge = clientCacheDuration
};
}
}
return Task.CompletedTask;
});
}Keep in mind that our action filter determines the ClientDuration value. There are three possibilities:
- Unset – do not send headers
- Zero – set various headers instructing downstream clients not to cache the response
- Other – set the cache headers to the provided value
The cache
We haven’t yet talked about the actual cache, which is one thing. The _cache references in the code above really correspond to a wrapper class that contains the IMemoryCache implementation that is built-in.
public class CacheClient : ICacheClient
{
private readonly IMemoryCache _cache;
public CacheClient(IMemoryCache cache)
{
_cache = cache ?? throw new ArgumentNullException(nameof(cache));
}
internal bool TryGet<T>(string key, out T entry)
{
return _cache.TryGetValue(Constants.CacheKeyPrefix + key, out entry);
}
internal void Set(string key, object entry, TimeSpan expiry, params string[] tags)
{
var options = new MemoryCacheEntryOptions
{
AbsoluteExpirationRelativeToNow = expiry
};
var allTokenSource = _cache.GetOrCreate(Constants.CacheTagPrefix + Constants.AllTag,
allTagEntry => new CancellationTokenSource());
options.AddExpirationToken(new CancellationChangeToken(allTokenSource.Token));
foreach (var tag in tags)
{
var tokenSource = _cache.GetOrCreate(Constants.CacheTagPrefix + tag, tagEntry =>
{
tagEntry.AddExpirationToken(new CancellationChangeToken(allTokenSource.Token));
return new CancellationTokenSource();
});
options.AddExpirationToken(new CancellationChangeToken(tokenSource.Token));
}
_cache.Set(Constants.CacheKeyPrefix + key, entry, options);
}Expiration tokens are used by the Set function to enable bulk removal of cache entries.
The concept is that we store a CancellationTokenSource for every entry in the cache and additional CancellationTokenSources for each tag we define. If you have never used CancellationTokenSource before, this may be a little difficult to understand. When setting the cache entry, we produce a token by using these CancellationTokenSources. In the following part, we’ll look at using the CancellationTokenSources to invalidate cache items in bulk.
Cache invalidation AKA cache busting
This unique kind of response caching was implemented primarily to meet the need for the ability to delete cache items before they naturally expire. The following methods are exposed by our CacheClient class for this:
public void Remove(string key)
{
_cache.Remove(Constants.CacheKeyPrefix + key);
}
public void RemoveByTag(string tag)
{
if (_cache.TryGetValue(Constants.CacheTagPrefix + tag, out CancellationTokenSource tokenSource))
{
tokenSource.Cancel();
_cache.Remove(Constants.CacheTagPrefix + tag);
}
}
public void RemoveAll()
{
RemoveByTag(Constants.AllTag);
}As you can see, deleting a single cache entry just requires that you know the key. You could want to allow action, controller, and route values to be given instead to make it more user-friendly.
The RemoveAll and RemoveByTag methods invoke the Cancel() function, which expires all tokens issued by the source, after retrieving the CancellationTokenSource from the cache. These cache entries are eliminated as a result.
Limitations
This is a very simple illustration of response caching and is devoid of many features found in native response caching middleware. The lack of the option to alter caching depending on headers, cookies, etc. is perhaps the most noticeable missing. It would not be extremely challenging to incorporate VaryBy. In reality, all we’re altering is how the cache key is generated and adding support for CacheAttribute setting.
Additionally, we are using IMemoryCache rather than IDistributedCache, which is more scalable. As was already said, modifying the usage is simple but will reduce functionality. Since IDistributedCache does not allow expiration tokens, the method described here cannot be used to remove pages in bulk. Naturally, nothing prevents you from coming up with a different approach, but unsophisticated implementations will almost surely have a negative impact on performance.
Summary
The creation of simple response caching middleware that enables manual invalidation of items both individually and in bulk using tags was covered in this post. To store pages (or other action results like JSON) in an in-memory cache, we combined an action filter with a middleware component. Then, we made a number of methods available on our CacheClient class to enable the deletion of cache entries.
The example code should not be used in place of the built-in response caching, which offers far more capability, although it might be helpful in some circumstances.
