Ph.D. Alumnus: Kia Shakiba

Reference:

Kia Shakiba
Efficient In-Memory Cache Fleet Orchestration
Ph.D. Thesis, University of Toronto, 2026.

Supervisor(s):

Michael Stumm

Download Thesis:

Abstract:

In-memory caches, such as Redis and Memcached, play an important role in reducing data access latencies and the load on backend data stores by serving frequently accessed data from main memory. The effectiveness of such a cache is largely dependent on its configured parameters, such as its size and eviction policy. Unfortunately, selecting these parameters is an arduous task and therefore atypical in-memory cache often simply uses a default, non-optimal configuration which remains static for the lifetime of the cache.

The thesis of this dissertation is that significant performance and resource usage benefits can be realized through periodic cache reconfigurations and that the existing cache orchestration techniques are limited in their abilities to configure caches under modern workloads. This is primarily evident when managing large fleets of hosting servers, each running multiple caches. Previous cache orchestration techniques are limited to managing a small number of caches running on a single host.

In this dissertation, we present a comprehensive analysis of several real-world publicly-available in-memory cache workloads from which we obtain several insights. First, we outline that an effectivecache orchestrator requires the accurate and efficient modeling of a cache's performance as a function of its configuration parameters. We show that a cache's eviction policy can have significant effectson its performance, though existing modeling techniques are limited in their abilities to efficiently demonstrate these effects. We propose a novel modeling technique, Kosmo, which accurately andefficiently models a cache's performance under various eviction policies, online. Second, we show that existing in-memory caches are unable to exploit the findings of our modeling technique as they arelimited in their abilities to switch eviction policies at runtime. To address this, we propose a novel in-memory cache, PaperCache, which is capable of efficiently switching between any eviction policyat runtime. Finally, we demonstrate the effects of periodically modifying a cache's configuration parameters on its performance and show that significant benefits can be achieved through globaloptimizations across a fleet of hosting servers. We propose Flux, a novel online in-memory cache orchestrator which uses our proposed modeling technique, Kosmo, and in-memory cache, PaperCache, to significantly improve the resource usage and performance of the caches it manages.

Keywords:

In-memory caching, cloud computing, resource management, eviction policies, optimization, memory management, cache orchestration

BibTeX:

@phdthesis(Shakiba-PhD26,
    author = {Kia Shakiba},
    title = {Efficient In-Memory Cache Fleet Orchestration},
    school = {University of Toronto},
    supervisors = {Michael Stumm},
    month = {May},
    year = {2026},
    keywords = {In-memory caching, cloud computing, resource management, eviction policies, optimization, memory management, cache orchestration}
)