How Drupal caches work under the hood

The journey of the http request in Drupal:

STEP 1. Page Cache module.

HTTP request will hit the Page Cache (page_cache) module to check if the path is in cache. 

How this is computed is via Drupal cache tags only, ignoring cache contexts and cache max-age.  For more information about cache tags, contexts, max-age (https://www.drupal.org/docs/8/api/cache-api/cache-api).

X-Drupal-Cache: HIT (This means that this page was served by page_cache module)

X-Drupal-Cache-Tags: block_content:1 block_content:2 ....

The X-Drupal-Cache-Tags is the secret sauce on how Drupal knows when to serve the cached page or when to invalidate cache.  Drupal cache headers can be enabled using this guide (https://www.drupal.org/docs/8/api/responses/cacheableresponseinterface#debugging)

Drupal computes this via the DatabaseBackend class:

Essentially, it compares cachetags table's number of invalidations (summing all the relevant invalidations for this page) vs checksum count in cache_page table.   If the totals match, it will retrieve from cache.  

Cachetags table:

Cache_page table: 

Take note that whenever you save a block content or node it will increase the number of invalidations in cachetags table.  On the next request, the totals would be different, making the cache invalid.

Also take note that step 1 are only applicable for anonymous users.

If the page is cached, it will return the response.  Otherwise, it will proceed to the next step.

STEP 2. Dynamic page cache.

This cache layer is for all user types (anonymous and logged in users).  This layer also honors all 3 cache metadata: cache tags, cache context, and cache max age.

From the photo below, we can see in the header:

X-Drupal-Dynamic-Cache: HIT (This means that this page was served by dynamic_page_cache module)

Unlike page_cache module, it does not return a truly full page, it returns a "full" page with the more dyamic parts replaced with place holders.  The placed holder content will still have to go through specific code to generate, before the response is sent back to the browser.  

The place holder content is what makes it possible to cache the page in spite of the variation.  

The invalidation computation is similar to how to page_cache module computation is done.  The only difference is that it uses the cache_dynamic_page_cache table vs cachetags table.  Take note as well that the expire value would no longer be always -1 and will be used in the computation (cache max-age).  Moreover, the cid would no longer be just the url, but other things like language, theme, timezone, etc (cache contexts).

If there is no dynamic_page_cache found, it will now go through the next step.

STEP 3. Render Cache and other cache implementations.

All blocks of the page, and other parts of the page are cached individually as shown.  Render will consult the 3 cache metadata: Cache tags, cache contexts, cache max-age before deciding to re-render or use a cached version.  

Please take note that a cached content can contain one or more smaller cached content inside it.

Summary:

Caveats and notes:

1. Cache tags bubble up. 

This means that smallest piece of cached content will pass it up to its parent piece of cached content until it reaches the page (top most). To ensure cache bubles up, take a look at this blog https://www.previousnext.com.au/blog/ensuring-drupal-8-block-cache-tags-bubble-up-page . 

2. Why is X-Drupal-Dyanamic-Cache set as Uncacheable?

This because of one of cache contexts in your content is either user or session.  Dynamic page cache will not cache user cache context because it is too expensive, there are too many variations.  To ensure that this is cached, use placeholdering https://bryanmanalo.com/auto-placeholdering-drupal .  

3. For more information about cache tags, contexts, max-age (https://www.drupal.org/docs/8/api/cache-api/cache-api).