Cloudflare stores the file I marked no-store, and my own rule is why

A cache rule is a blanket instruction. It matches a shape of request — a method, a path prefix, a header — and applies one decision to everything that fits. The Cache-Control header is the opposite: it is per response, written by whatever generated that response, and it can be different on every URL. Both of those are reasonable systems. The trouble starts the moment they disagree, because the disagreement is silent, and you discover it one file at a time.

The nine files, and what each one is for

None of these is a page a person visits. They exist so that a crawler, an agent, or a language model can find out what this site offers without reading the HTML. Three of them are conventions old enough to be boring; the rest arrived in the last two years and most of them are still drafts.

The measurement is two requests per file: one to fill the edge, one twenty seconds later to see what it kept. Both are real GETs with the IP pinned, because a HEAD is not answered from the same path and reports a different cache status entirely.

IP=$(dig +short @1.1.1.1 A visibleorigin.com | head -1)
curl -s -o /dev/null -D- --resolve visibleorigin.com:443:$IP https://visibleorigin.com/sitemap_index.xml \
  | grep -iE '^(cf-cache-status|age|cache-control|last-modified|etag):'

The file I marked no-store is stored right now

The sitemap is emphatic. Its response carries no-cache, no-store, must-revalidate, max-age=0 and an Expires: 0 for good measure. The specification leaves no room to interpret the second of those. RFC 9111 section 5.2.2.5 says it in one sentence:

The no-store response directive indicates that a cache MUST NOT store any part of either the immediate request or the response and MUST NOT use the response to satisfy any other request.

The edge stores it, and I watched one stored copy of it age without a break for as long as I cared to watch:

20:25:31Z  sitemap(no-store): age 53   HIT
20:26:31Z  sitemap(no-store): age 113  HIT
20:27:31Z  sitemap(no-store): age 174  HIT
20:28:31Z  sitemap(no-store): age 234  HIT
20:29:31Z  sitemap(no-store): age 294  HIT
20:30:32Z  sitemap(no-store): age 354  HIT
   ...
21:04:36Z  sitemap(no-store): age 1202 HIT

There is one detail in that response worth stopping on: the edge passes the no-store along to the client unchanged. So a browser receiving that sitemap is told not to store it, and obeys. The instruction is honored by everyone downstream and ignored by the one cache in the middle that is actually holding a copy.

RFC 9111 anticipates a cache that does not obey, and the wording is worth reading against this case, because it does not fit:

This directive is not a reliable or sufficient mechanism for ensuring privacy. In particular, malicious or compromised caches might not recognize or obey this directive, and communications networks might be vulnerable to eavesdropping.

Malicious or compromised. Mine is neither. It is a correctly functioning cache doing precisely what its operator configured, and its operator is me.

The rule that overrides it is mine, and Cloudflare documents that it can

Two things about Cloudflare’s defaults make it clear this is configuration rather than product behavior. First, by default the CDN would not store most of these files at all: “Cloudflare only caches based on file extension and not by MIME type. The Cloudflare CDN does not cache HTML or JSON by default. Additionally, by default Cloudflare caches a website’s robots.txt.” The published extension list runs to about seventy entries — CSS, JPG, PDF, WOFF2 and so on. XML is not in it. TXT is not in it. MD is not in it. By default my sitemap and my llms.txt would go straight through uncached, and robots.txt would be cached by an explicit special case rather than by any rule of mine.

Second, respecting the origin’s headers is not an upgrade I failed to buy. Cloudflare’s Origin Cache Control page says the feature “indicates that Cloudflare should strictly respect Cache-Control directives received from the origin server,” and then, plainly: “Free, Pro and Business customers have this feature enabled by default.” This zone is on the Free plan. The default behavior is to obey me.

The same page names the thing that changes it, and it is the thing I built: “Cloudflare’s Cache Rules allows users to either augment or override an origin server’s Cache-Control headers or default policies set by Cloudflare.” I wrote a cache rule so that this site’s HTML would be cached at the edge, because without it every page view reached the origin. That rule marks matching requests eligible for cache and sets an edge lifetime. It matches on method and path shape. It has no idea which of the things it matches were carrying instructions of their own.

The proof that the rule beats the header, in one file

There is a cleaner demonstration than the sitemap, because it does not involve no-store at all. One of the manifests sets a deliberately short lifetime:

/.well-known/agent-skills/index.json    cache-control: public, max-age=300

Five minutes, and the specification is unambiguous about what that means: RFC 9111 section 5.2.2.1 defines it as “the response is to be considered stale after its age is greater than the specified number of seconds.” The number is a choice: the skills index changes when I add a skill, and I did not want an agent holding a stale one for long. If the edge were honoring the origin, the stored copy would be replaced somewhere around age 300. Here it is at twenty past eight in the evening:

20:44:34Z   MISS                  <- I purged the zone here
20:45:34Z   age 60     HIT
20:49:35Z   age 300    HIT        <- the lifetime the origin asked for ends about here
20:54:35Z   age 601    HIT
21:00:36Z   age 962    HIT
21:04:36Z   age 1202   HIT        <- same stored copy, still climbing when I stopped

Twenty minutes on a five-minute lifetime, four times over, and still climbing when I stopped watching. By the arithmetic in the sentence above that response was stale at 301 seconds, and the edge kept handing out the same copy for another fifteen minutes. The MISS at the top deserves an honest label: that is me purging the zone, not a lifetime expiring. Nothing in this run expired on its own.

The edge lifetime in my rule replaced the one in the header, and the header is the only place the intent was written down. That is not a bug in anything. It is the documented precedence, applied to a file I was not thinking about when I wrote the rule.

And there is something else in that transcript that I did not go looking for, which settles the question of cause better than anything else here. I was watching two files at once: the sitemap, whose origin says no-store, and this manifest, whose origin says max-age=300. Two files, two different instructions, nothing in common but the domain.

20:45:34Z   agent-skills (max-age=300): age 60      sitemap (no-store): age 60
20:49:35Z   agent-skills (max-age=300): age 300     sitemap (no-store): age 300
20:54:35Z   agent-skills (max-age=300): age 601     sitemap (no-store): age 601
21:00:36Z   agent-skills (max-age=300): age 962     sitemap (no-store): age 962
21:04:36Z   agent-skills (max-age=300): age 1202    sitemap (no-store): age 1202

Identical, second for second, for forty minutes. A file told to live five minutes and a file told never to be stored at all are being kept for exactly the same length of time, because neither instruction is what decides. One lifetime, written once, applied to both. That is what a blanket looks like from underneath, and it is the clearest evidence in this note that the disagreements above have a single cause rather than four.

The two files that are never stored, and five requests that locate why

/auth.md and /index.md both send public, max-age=3600 and both come back DYNAMIC, which is the edge saying it did not store the response at all. These are the two files an automated client is most likely to fetch twice, so they are the worst two to leave uncacheable. Something is bypassing them, and I can narrow down what without opening the dashboard. Five requests:

A  /auth.md   with Accept: text/html       -> text/markdown   DYNAMIC
B  /about/    with Accept: text/markdown   -> text/markdown   DYNAMIC
C  /about/    with no special header       -> text/html       MISS
D  /index.md  with Accept: text/html       -> text/markdown   DYNAMIC
E  /llms.txt  with Accept: text/markdown   -> text/plain      DYNAMIC

E is the informative one. A plain text file that is normally a HIT becomes DYNAMIC the moment I send an Accept header asking for Markdown — even though the response is still text/plain and nothing about it changed. That is a bypass keyed on the request header, firing on a URL that has nothing to do with Markdown. It is supposed to exist: it is what stops a cache from serving a browser the Markdown an agent asked for, which is a failure I have written about before and nearly shipped.

But A and D show that the .md routes are bypassed with the header explicitly set to HTML, so the header rule is not what is catching them. Either the bypass also matches on the path, or my caching rule simply never matched .md in the first place — which would be enough on its own, because MD is not one of the extensions Cloudflare caches by default. From outside I cannot tell those two apart, and I am not going to guess between them.

The validator the edge made up

This one is small and I like it the most. At the origin, /robots.txt is generated by the application and carries no Cache-Control and no validator at all — no ETag, no Last-Modified, three requests in a row. Through the edge it arrives like this:

age: 127
last-modified: Sun, 06 Sep 2026 19:37:21 GMT
cf-cache-status: HIT

That date is not when the file changed. My own transcript has the answer: I purged the zone at 19:36 and made the first request to that URL at 19:37:20. The Last-Modified being served is the second the edge happened to fetch the file, one second later.

Which is not what the field means. RFC 9110 section 8.8.2 defines the value as “the date and time at which the origin server believes the selected representation was last modified.” The origin believes nothing of the sort; it declined to have an opinion, and something downstream supplied one. A client doing a conditional request against that date is asking “has this changed since a CDN node fetched it?” while believing it asked “has this changed since it changed?” Those give the same answer often enough that the difference never announces itself.

The file nothing can revalidate

The last one is entirely mine and there is no configuration involved. /index.md is generated by a plugin I wrote, and at the origin it carries no ETag and no Last-Modified. Nothing downstream can ask whether it changed, because there is no value to ask with. Every read is a full transfer of the whole document, forever, and since the edge never stores it, every one of those transfers reaches my server.

The neighbouring file gets it right — /auth.md ships an ETag — which makes this a plain omission rather than a design decision. It is on the list to fix, and I am publishing it before the fix rather than after, because a note that only contains solved problems is a brochure.

What this costs, and why I do not feel it

The consequence of the sitemap case is a stale sitemap served to crawlers after I publish something. The consequence of the manifest case is an agent holding a skills index for longer than the five minutes I asked for. Neither has bitten me, and the reason is uncomfortable: I purge the entire zone every time I publish.

That habit papers over all of it. It also happens to be the only purge available here, for reasons I measured separately: the plugin that does the purging can only ever tell Cloudflare to clear everything, and the command that names a single URL never reaches the edge at all while still reporting success. So the mechanism that hides today’s problem is the expensive one, and the cheap one does not work. That is not a happy equilibrium; it is two faults that happen to cancel.

It also means the failure is waiting for anyone whose deployment does not end in a full purge, which is most people, and it will present as “Google is showing an old page” rather than as anything resembling a cache header problem.

The sharpest case, found while writing this

Everything above is about files. This one is about a feature, and I found it two hours before publishing by re-running a command I had already run.

This site serves Markdown to clients that ask for it with an Accept header. That is content negotiation, it is the mechanism I wrote a whole note about, and on the day I measured this it was switched on. Here it is working, immediately after a purge, through Cloudflare:

$ curl ... -H 'Accept: text/markdown' https://visibleorigin.com/about/
content-type: text/markdown; charset=utf-8      cf-cache-status: DYNAMIC

Now two ordinary browser requests for the same page, which is all it takes to put an HTML copy in the origin cache. Then the same Markdown request again:

$ curl ... https://visibleorigin.com/about/
content-type: text/html    x-litespeed-cache: miss     cf-cache-status: MISS
$ curl ... https://visibleorigin.com/about/
content-type: text/html    x-litespeed-cache: hit      cf-cache-status: HIT

$ curl ... -H 'Accept: text/markdown' https://visibleorigin.com/about/
content-type: text/html    x-litespeed-cache: hit      cf-cache-status: DYNAMIC
$ curl ... -H 'Accept: text/markdown' https://visibleorigin.com/about/
content-type: text/html    x-litespeed-cache: hit      cf-cache-status: DYNAMIC

The negotiation did not fail. It never happened. The origin cache had a stored HTML copy and answered from it before the application ran, so the code that inspects the Accept header was never reached. The edge is not involved — it says DYNAMIC throughout, correctly refusing to store anything. This is the same class of problem as the sitemap, one layer lower: a cache in the middle deciding what the origin would have said.

The uncomfortable part is what a third-party checker sees. The agent-readiness scanner that grades this site tests exactly this, and while my own requests to the same URL were coming back as HTML, it reported:

markdownNegotiation: pass — "Site supports Markdown for Agents"
details: {"contentType": "text/markdown; charset=utf-8"}

Both of those are true at once, and I do not yet know why. Something about how that scanner asks puts it in a different cache bucket from the one my request lands in, and I tried and failed to reproduce the difference with the header shapes I could think of — the cache-busting Cache-Control it sends, compression, the URL. So I am publishing an unfinished result: the check that certifies this feature passed at a moment when a client asking the way I ask would have received HTML.

What I can say without qualification is the operational part. Whether an agent gets Markdown from this site depends on whether that page is currently in the origin cache, and that is not a property anyone should have to reason about.

The part that is about me and not about caches

I wrote the paragraphs above believing I had found something. Then I searched properly, and this is what came back: the same failure was diagnosed and written down on 26 August, three weeks ago, with the mechanism described correctly and a working fix beside it. The comment sitting above that fix describes what I spent this evening measuring, in fewer words than I have used, and ends with a warning about the specific way the fix silently does nothing if you write one directive wrong.

The workaround below is still the one that works, for a reason that survives the correction: the edge is a second cache, and Cloudflare does not honour Vary either. What you can do is rewrite the request URI so that the cache key misses on purpose, which forces the application to run, while leaving the path the framework sees unchanged so it still resolves the right page. It is a workaround rather than a correction, and it is the one that works.

And there is a second half to the lesson that only appeared when I went to apply the fix. Applying it here would not have worked either. The fix assumes the document root’s .htaccess is where rewriting is decided. On this site it is not: WordPress’s rewrite rules exist there, but a second copy of them also sits inside the virtual host configuration, and the last of that copy is a catch-all matching every path which is not a real file, ending in [L]. That [L] terminates rewrite processing before the document root’s .htaccess is ever consulted, so a rewrite added there would do nothing at all, silently. The rules are in both files; only one of the two files is being used.

The document root’s .htaccess already contains two rules that deny access outright, so I did not have to change anything to look:

/litespeed/debug/x.log      404      the .htaccess rule for this path says 403
/.litespeed_conf.dat        404      so does this one
/zzz-nada/                  404      control: no rule anywhere

Neither denial happens. That is consistent with the catch-all sending the request to the application before the file is ever consulted — but I want to be honest about how much this proves, because it is less than it looks. Both of those paths are paths that do not exist, which is exactly the condition the catch-all matches. The measurement cannot separate “the .htaccess was never reached” from any other reason those particular rules did not fire.

The clean version of the test needs a rule that matches a file which does exist, so that the catch-all’s condition fails and the request has to continue. I do not have one: every deny rule in that file points at a path with nothing behind it, and I am not going to create a file in a public document root to satisfy a test. So the config is unambiguous and the measurement is only consistent with it. I am flagging that rather than presenting the two as if they were the same strength of evidence.

So the fix has to go somewhere else on this machine than it does on the others, and finding that out was a second measurement after the first one had already told me the answer. A fix that worked somewhere else is a hypothesis about your system, not a conclusion.

This site never got that rewrite, and the reason is not that the lesson was missing. It is that nothing carried it here. It sat in notes I did not search, and I ran the measurements, drew the diagram and wrote the section before typing the search that would have saved all of it.

So the honest summary of this section is not “look what I found.” It is: a known failure, with a known fix, went unapplied on one site for three weeks because knowing something and finding it again are different problems. The measurement discipline this whole site is about does nothing for you if the answer was already written down and you did not look. I am publishing before the fix because the fix needs a change I will not make at midnight with two posts in the queue, and because dated evidence of my own gap is worth more than a clean story.

How it ended, added 8 September 2026

The fix did not work, and neither did the second one. Both were applied to the live server and both were measured inert:

RewriteCond %{HTTP:Accept} text/markdown  +  [E=LSCACHE_VARY_VALUE:md]   → no effect on the cache key
reqHeaderVary Accept   (in the vhost's own cache module)                 → no effect on the cache key

The reason is in the server binary, and it is the kind of thing worth knowing before you spend an evening on it: on this build the cache key cannot include a request header at all. The only key-modifying facility operates on the query string — its own debug line reads [CACHE] modified QS in cache key is [%s] — and the query string is confirmed to be part of the key. So the only remaining route was to rewrite the URI on every matching request, immediately before the catch-all that routes the entire site. That is the rule shape which, when I got it wrong once before, returned 404 for every URL on the site and left nothing in any log.

So I switched the feature off instead. Asking this site for Markdown with an Accept header now returns HTML, consistently, rather than returning Markdown only when the page happens to be uncached. The parallel routes at /slug.md are unaffected and always were — they do not depend on the cache, and they are the mechanism the HTTP specification prefers when shared caches sit in front of you.

What that costs is one checkbox on an agent-readiness scanner. What it buys is that the site no longer advertises a behaviour it only sometimes had. I would rather fail a check honestly than pass it on a cold cache.

What I could not establish

I did not read the current cache rules. I have the expression and placement I documented when the markdown rule was created, but that was a week ago, the rule order has changed at least once since, and the number of rules recorded in my own notes disagrees with itself. Publishing the rule text from memory is exactly the mistake that cost me a note I had to rewrite once already.

So: the behavior above is measured, the precedence is documented by the vendor, and the identification of which specific rule produces each row is an inference from five requests. If the rules turn out to say something else, the measurements do not change and the explanation might.

How to check yours

Two requests per file, and a comparison you have to make by hand because nothing does it for you:

IP=$(dig +short @1.1.1.1 A example.com | head -1)
for u in /robots.txt /sitemap.xml /llms.txt /.well-known/api-catalog; do
  echo "== $u"
  # what the origin asked for
  curl -s -o /dev/null -D- https://ORIGIN-IP-OR-HOST$u | grep -iE '^(cache-control|etag|last-modified):'
  # what the edge did, second request
  curl -s -o /dev/null --resolve example.com:443:$IP https://example.com$u
  curl -s -o /dev/null -D- --resolve example.com:443:$IP https://example.com$u \
    | grep -iE '^(cf-cache-status|age|cache-control|last-modified):'
done

Three things to look for. A HIT on anything whose Cache-Control says no-store or private. An age that climbs past the max-age the origin set. And a Last-Modified at the edge that the origin never sent. Any of the three means a rule is overriding a header, which is allowed, and worth knowing you did.

Cache headers behind a CDN: frequently asked questions

Does Cloudflare respect Cache-Control from my origin?

By default, yes: Origin Cache Control is documented as enabled by default for Free, Pro and Business plans, and it means origin directives are followed as sent. What changes that is a Cache Rule, which Cloudflare documents as able to augment or override both the origin’s headers and Cloudflare’s own defaults. If your headers are being ignored, a rule of yours almost certainly says so.

Why is a no-store response being cached at the edge?

Because a cache rule marked it eligible for cache and gave it an edge lifetime, and that takes precedence over the response header. The tell is that the no-store is still present in what the client receives: the header travelled, it just did not govern the hop where the copy is being kept.

What does cf-cache-status DYNAMIC mean?

That the response was not stored, and it is not the same as MISS. MISS means the resource was cacheable and simply was not in cache at that moment, so the next request may well be a HIT. DYNAMIC means nothing was stored at all. Watching one URL go MISS and then HIT is how you tell them apart in ten seconds.

Does Cloudflare cache .txt, .xml, .json or .md by default?

No. The published default list is by file extension and contains none of those, and the docs state that HTML and JSON are not cached by default. There is one explicit special case: robots.txt is cached by default. Everything else on that list of nine files here is cached because a rule of mine says so.

Should my sitemap be cacheable?

Briefly, yes, and that is a defensible thing to configure. What is not defensible is having the response say no-store while the edge keeps a copy, because then the file is telling every downstream reader one thing and the network another. Pick one and make them agree: either relax the header or exclude the path from the rule.

How do I find which rule is responsible?

From outside, by elimination: change one thing per request and see what moves the cache status. A header, a path, a method. From inside, by reading the rules in order and remembering that the last matching rule wins, which is the opposite of the order most people assume.

A later note takes the other half of this: why a cache-buster does not skip the cache but hands you a private entry inside it, measured on this domain with the commands printed.

The general shape of this

Everything in this note is a system working. The origin sets headers correctly. The CDN applies rules in the documented order. The rule does what it says. The specification is clear, the vendor documentation is accurate, and the outcome is still that three files behave the opposite of how they were labelled.

What produced it is a category difference nobody flags: a rule is written about a shape of request, and a header is written about one response. One is authored once and applies to things that do not exist yet; the other is authored per file by whoever generated it, often a plugin, often not you. They are both correct and they answer to nobody in common, so when they disagree the only place the disagreement is visible is in a response you have to go and look at.

Keeping those files right is one half of the problem; keeping a single fact consistent across all of them is the other, which is its own note on adding one service and watching it land on seven surfaces.

Every command here ran against this server on the day this was published, and every quotation is from a specification or a vendor page you can open. If one of them returns something other than what I said it does, that is worth an email. And if you would rather find out from somebody else whether your own edge is doing what you asked, that is what the free check is for.