Finally! A Shared-Cache Solution Based on SQL Server Cache!

source code:

https://github.com/amiriltd/hybrid-output-cache

Its no secret that Output caching can significantly improve the performance and scalability of an app by reducing the work required to generate responses from the server. Theoretically, Microsoft’s implementation of output caching provides flexibilities for the developer including:

1. The caching storage mechanism

2. The caching behavior

3. Rules around caching validation and invalidation

4. Configuration with minimal code needed

Realistically however, if you wanted to scale from a one-server setup to a multi-distributed or serverless architecture, the options were scarce: You either paid for some Redis cache cloud SAAS solution, or spend time creating your own custom implementation. Whichever option you chose, their common denominators lied in the fact the both approaches did not support the IDistrubutedCache interface mainly because of its the lack of atomic features which are pre-requisites for tagging. For those developers invested in the idea of having the freedom to choose a caching storage mechanism like SQL Server cache, this was a tough pill to swallow until now.

Introducing HybridCache

HybridCache is Microsoft’s new caching solution that provides the performance, scalability, and flexibility developers were looking for when they invested in the IDistributedCache interface for their application. For many developers, the added support for different caching storage mechanisms such as SQL Server cache is the game-changer because it saves time by not having to implement too much code and money since you no longer have to invest in Redis cache to achieve a distributed caching solution.

Using HybridCache with Output Caching

So we are eager to use HybridCache with our existing Output Caching implementation right? Just add the package from nuget package manager and register the services in your container right?

dotnet add package Microsoft.Extensions.Caching.Hybrid --prerelease

dotnet add package Microsoft.Extensions.Caching.SqlServer --prerelease

Not so fast! There are a few things Microsoft left out as I’m writing this article. Currently when you enable Output Caching with “AddOutputCache()” extension method Microsoft will, by default, creates an MemoryOutputCacheStore as the caching storage mechanism.

So where is the HybridOutputCacheStore?? As of now, there is no concept of a HybridOutputCacheStore in the aspnetcore repository so we will have to create our own custom HybridOutputCacheStore based on the IOutputCacheStore interface. Also since we are not using the default Output Caching implementation “AddOutputCache()” in our application pipeline, we will need to create our own “AddHybridOutputCache()” extension method in order to register the services into our application pipeline.

Creating your HybridOutputCacheStore with HybridCache and IDistributedCache

Creating your own IOutputCacheStore is not as complicated since HybridCache is available. The interface is made of of three methods:

1. EvictByTagAsync

2. GetAsync

3. SetAsync

My implementation makes use of HybridCache and IDistrubtedCache:

  public class HybridOutputCacheStore : IOutputCacheStore
 {
     private readonly HybridCache _cache;
     private readonly IDistributedCache _distributedCache;
     private readonly object _tagsLock = new();
     private readonly ILogger _logger;

     public HybridOutputCacheStore(HybridCache cache, 
                                    IDistributedCache distributedCache, 
                                    ILoggerFactory loggerFactory)
     {
         ArgumentNullException.ThrowIfNull(cache);
         _cache = cache;
         _distributedCache = distributedCache;
         _logger = loggerFactory.CreateLogger<HybridOutputCacheStore>();
     }

     public ValueTask EvictByTagAsync(string tag, CancellationToken cancellationToken)
     {
         ArgumentNullException.ThrowIfNull(tag);
         lock (_tagsLock)
         {
             _logger.LogInformation("Evicting By Tag");
             _cache.RemoveByTagAsync(tag, cancellationToken);
         }
         return ValueTask.CompletedTask;
     }

     public async ValueTask<byte[]?> GetAsync(string key, CancellationToken cancellationToken)
     {
         ArgumentNullException.ThrowIfNull(key);
         var entry = await _distributedCache.GetAsync(key, cancellationToken);
         _logger.LogInformation("Getting value by key: {key}", key);
         return entry;
     }

     public ValueTask SetAsync(string key, byte[] value, string[]? tags, 
        TimeSpan validFor, CancellationToken cancellationToken)
     {
         ArgumentNullException.ThrowIfNull(key);
         ArgumentNullException.ThrowIfNull(value);
         if (tags != null)
         {
             lock (_tagsLock)
             {
                 _logger.LogInformation("Setting value with key: {key} and tags: {tags}", key, tags);
                 SetEntry(key, value, tags, validFor);
             }
         }
         else
         {
             _logger.LogInformation("Setting value with key: {key} No tags", key);
             SetEntry(key, value, tags, validFor);
         }
         return ValueTask.CompletedTask;
     }

     void SetEntry(string key, byte[] value, string[]? tags, TimeSpan validFor)
     {
         Debug.Assert(key != null);
         var hoptions = new HybridCacheEntryOptions() { Expiration = validFor };
         var options = new MemoryCacheEntryOptions
         {
             AbsoluteExpirationRelativeToNow = validFor,
             Size = value.Length
         };
         _cache.SetAsync(key, value, hoptions, tags, default);
     }
 }

Creating the AddHybridOutputCache() extension method

In order to add our implementation of Hybrid output caching to our application, we need to create two additional files:

1. HybridOutputCacheOptionsSetup.cs

2. OutputHybridCacheServiceExtensions.cs

Note: HybridOutputCacheOptionsSetup.cs is needed since OutputCacheOptionsSetup is inaccessible by Microsoft.

 internal sealed class HybridOutputCacheOptionsSetup : IConfigureOptions<OutputCacheOptions>
 {
     private readonly IServiceProvider _services;

     public HybridOutputCacheOptionsSetup(IServiceProvider services)
     {
         _services = services;
     }

     public void Configure(OutputCacheOptions options)
     {
     }
 }

And finally OutputHybridCacheServiceExtensions.cs to add Hybrid output caching to the application with Dependency Injection (DI):

  public static class OutputHybridCacheServiceExtensions
  {
      public static IServiceCollection AddHybridOutputCache(this IServiceCollection services)
      {
          ArgumentNullException.ThrowIfNull(services);
          services.AddTransient<IConfigureOptions<OutputCacheOptions>, HybridOutputCacheOptionsSetup>();
          services.TryAddSingleton<ObjectPoolProvider, DefaultObjectPoolProvider>();
          services.TryAddSingleton<IOutputCacheStore, HybridOutputCacheStore>();
          return services;
      }

      public static IServiceCollection AddHybridOutputCache(this IServiceCollection services, Action<OutputCacheOptions> configureOptions)
      {
          ArgumentNullException.ThrowIfNull(services);
          ArgumentNullException.ThrowIfNull(configureOptions);

          services.Configure(configureOptions);
          services.AddHybridOutputCache();

          return services;
      }
  }

Putting it all together

Since HybridCache is still in preview mode, I will not create a package in nuget for this. With the Microsoft team adding to the package on a weekly basis, I am certain this article will be obsolete in a year. While we wait for Microsoft, simply add all three files into your project under a new folder ex: “OutputHybridCacheServiceExtensions” and register your services using dependency injection as you normally would.

Next, register the services in your application and use them. In this example, I’m using DistributedSqlServerCache as my primary cache store and automatically caching the HTTP responses from my catalog service. Notice that my AddHybridOutputCache() method take in all the options available with AddOutputCache() and the UseOutputCache() method works the same.

builder
      .Services.AddDistributedSqlServerCache(options =>
      {
          options.ConnectionString = builder.Configuration.GetValue<string>(CacheSettings.ConnectionString);
          options.SchemaName = CacheSettings.SchemaName;
          options.TableName = CacheSettings.TableName;
      })
;

#pragma warning disable EXTEXP0018 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
builder
    .Services.AddHybridCache(options =>
{
    options.DefaultEntryOptions = new Microsoft.Extensions.Caching.Hybrid.HybridCacheEntryOptions()
        { Flags = Microsoft.Extensions.Caching.Hybrid.HybridCacheEntryFlags.DisableLocalCache };
});
#pragma warning restore EXTEXP0018 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
builder.Services.AddHybridOutputCache(options =>
{
    options.AddBasePolicy(builder =>
        builder.Expire(TimeSpan.FromMinutes(10)));
});

var app = builder.Build();
app.UseOutputCache();

app.MapRemoteBffApiEndpoint("/catalog", builder.Configuration.GetValue<string>(HttpClientBaseAddresses.CatalogApi) + "/api/catalog")
    .CacheOutput();

If I call my endpoint now, the first call is cached and subsequent calls read from the SQL server cache which in this case is much faster than making an HTTP API request to a cold endpoint. Notice the first call took over 18 seconds from my cold starting HTTP trigger endpoint. The next calls ran in under 59 milli seconds (ms). Not bad!

Conclusion

HybridCache is the answer for developers looking for a flexible caching solutions that will scale with your application without having to invest additional time and money on services like Redis Cache. By the time .NET 10 rolls around, I am counting on a vast majority of the concepts we talked about in this article like the HybridOutputCacheStore to be baked into to the nuget package in some way. Until that time comes around, feel free to use our implementation of Hybrid Output Caching with HybridCache and IDistributedCache.

source code:

https://github.com/amiriltd/hybrid-output-cache

👏 Give this article a clap or a few if you like it

✅ Don’t forget to follow to receive more insight

😉 Feedback is always appreciated!