My cache rule asks a different question from the one it looks like it asks

By John Patzi · Published · 11 min read

Diagram of a cache rule that misses capitalisations of the media type it looks for and fires on strings that merely contain its characters, including a different media type entirely

The short version. A Cloudflare cache rule on this site decides whether a request is exempt from the cache by looking at the Accept header. It looks like it is asking “does this client want Markdown?” It is not. It is asking “does this header contain these thirteen characters, in this exact case?” — and those are different questions in two directions at once.

It misses TEXT/MARKDOWN, which the HTTP standard says is the same media type as the one it is looking for. And it keeps application/vnd.foo+text/markdown out of the cache, which is not Markdown at all and merely contains the letters.

I wrote that rule. I also started this note with the wrong explanation of it, and found out by measuring rather than by thinking harder.

What the rule is for

This site can serve two representations of the same page: HTML for people, Markdown for agents that ask for it. That arrangement only survives if a cache in front of it keeps the two apart, because a cache that stores one copy per URL will hand the second visitor whatever the first one got.

The guard is a rule at the edge. I do not have to guess what it says, because I wrote its expression down in the plugin it protects, in a comment that has been sitting there since August:

output
Name:       Bypass cache on Accept: text/markdown
Expression: any(http.request.headers["accept"][*] contains "text/markdown")
Then:       Cache eligibility -> Bypass cache
Order:      last

contains. Not equals, not matches a media type: a substring test over the raw header value. Everything in this note follows from that one operator, and I could have known it without a single request.

The intent was: if the request looks like it wants Markdown, do not serve or store a cached copy for it. The negotiation was switched off on this site when these requests were made, for reasons I wrote up separately, which is precisely what made the rule safe to poke at: every request below got HTML regardless, so nothing could be poisoned by measuring. It has since been switched back on, which is why the table is dated.

Eight requests to the same URL

Same page, same minute, nothing changing but one header. Here is the command, so the table below is repeatable rather than reported:

shell
IP=$(dig +short @1.1.1.1 A visibleorigin.com | head -1)
curl -s -o /dev/null -D - --resolve visibleorigin.com:443:$IP \
     -H 'Accept: text/markdown' https://visibleorigin.com/services/ \
  | grep -iE '^(cf-cache-status|age):'
Accept sentcf-cache-statusageStored at the edge?
text/markdownDYNAMIC—no
text/markdownzzDYNAMIC—no
zztext/markdownDYNAMIC—no
application/vnd.foo+text/markdownDYNAMIC—no
xxtext/markdownxxDYNAMIC—no
text/html, text/markdown;q=0.9DYNAMIC—no
text/markdowHIT1392yes
TEXT/MARKDOWNHIT1392yes

The two HIT rows share an age, which is the part that matters: they are not merely both cacheable, they are reading the same stored object. DYNAMIC is Cloudflare’s word for “not eligible for cache, no lookup happened”. The last column reports what was stored, not which rule decided it — a distinction that turns out to matter, and that has a section of its own below.

Diagram of a Cloudflare cache rule with two failure edges: it misses TEXT/MARKDOWN and other capitalisations of the same media type, and it keeps out strings that merely contain the characters, including a different media type entirely
Both columns are wrong, and both follow from the same substitution.

The two errors are the same error

Read the table as two halves and the shape appears.

It refuses a media type the standard calls identical

RFC 9110 defines a media type as type "/" subtype where both are tokens, and then says plainly: “The type and subtype tokens are case-insensitive.”

So TEXT/MARKDOWN and text/markdown are not two things. They are one thing spelled two ways, and a client that sends the first is asking for exactly what a client sending the second asks for. My rule treats them as unrelated.

It accepts a media type that is not Markdown at all

The other half is worse and I did not expect it. application/vnd.foo+text/markdown is a made-up vendor type; it is not text/markdown and no negotiation logic should treat it as such. The edge keeps it out of the cache all the same, and does the same for zztext/markdown and xxtext/markdownxx, which are not media types at all.

And the cut-off is exactly where a substring test would put it: text/markdow, one character short, is stored normally. Whatever is making the decision is not evaluating a media type: it is matching a literal run of thirteen characters anywhere in the header value — which is exactly what the expression in my own comment says it does.

Under-matching and over-matching look like two bugs and are one: a case-sensitive substring search standing in for content negotiation. Everything the edge gets wrong here follows from that single substitution.

And a third artefact that disagrees with both

There is a loose end here that I cannot tie, and leaving it out would be the dishonest option.

My own note says the rule’s action is Bypass cache. Cloudflare has a status for exactly that outcome, and this zone emits it — just not here:

output
/services/?customize_changeset_uuid=x   -> cf-cache-status: BYPASS
Cookie: wordpress_logged_in_x=1         -> cf-cache-status: DYNAMIC
Accept: text/markdown                   -> cf-cache-status: DYNAMIC

The first line proves the zone reports BYPASS when a bypass action matches. The second is a session cookie, which is the ordinary reason a response is simply not eligible for caching — no rule involved. And the header I am investigating behaves like the second, not like the first.

So I have three artefacts and they do not agree. My notes say a bypass rule exists with that expression. The measurements say requests carrying that substring are not stored. And the status code says the outcome looks like ineligibility rather than like a bypass.

At least three things could explain it: the rule may not exist in the form I recorded, its action may have been changed since, or Cloudflare may report DYNAMIC in cases I do not understand. I cannot tell you which, because reading the zone’s rules requires dashboard access I do not have. What I can tell you is that the behaviour in the table is real, that it discriminates by case-sensitive substring, and that attributing it to a specific rule would be an inference dressed as a measurement.

That last part is not a new lesson here. A note of mine from last week ends with the admission that I had not read the current cache rules and that identifying which one produced each row was inference. It still is. The difference is that this time I checked whether the status code supported the story before telling it, and it did not.

Why the case sensitivity is there

It is documented, and it is not a Cloudflare defect. Their rules language documentation states it directly: “A common use case for transformation functions is the conversion of a string of characters to uppercase or lowercase, since by default, string evaluation is case-sensitive.” There is a lower() function for exactly this, described as converting “a string field to lowercase”.

So the platform behaved as documented and the rule was written without reading that sentence. That is the ordinary version of this failure and it is the one worth writing down: the defect is not in a system behaving unexpectedly, it is in a default I did not look up.

What I got wrong while writing this

The first version of this note said that only the exact lowercase string fires the rule. That reading fitted five measurements and was wrong, and I only found out because the draft was put through a round of deliberate attack before publishing.

Three requests killed it: text/markdownzz, zztext/markdown and application/vnd.foo+text/markdown all behave like text/markdown. Had I published the exact-match version, the first reader with a terminal would have refuted it in thirty seconds, on a site whose entire argument is that its claims can be checked.

This is the fourth note in a row where that pass changed something substantial before publication, and in this case it changed the finding into a better one. An exact-match rule that is merely too strict is a curiosity. A substring rule that is both too strict and too loose is a defect with two edges, and I would have published the curiosity.

Testing what should fail

The boundary was found by three requests that were designed to be rejected. That is the whole method and it is worth separating from the finding, because it transfers to any rule in any dashboard.

My first five inputs were all plausible spellings of the thing the rule looks for. Every one of them behaved as I expected, which felt like confirmation and was not: a test that only sends valid inputs measures the middle of a rule and never its edges. Rules fail at edges. That is where they are defined.

So the useful inputs are the deliberately wrong ones, and they cost nothing:

Kind of probeWhat it answersHere
The thing itselfDoes the rule work at all?text/markdown — yes
The same thing, spelled differentlyIs the comparison normalising anything?TEXT/MARKDOWN — no, it is not
The thing with noise around itIs it a substring test?xxtext/markdownxx — yes, it is
The thing one character shortWhere exactly is the boundary?text/markdow — at thirteen characters
A different thing that contains itDoes it over-match on real input?application/vnd.foo+text/markdown — yes

Five probes, four of which are supposed to be rejected. Two of the four were not, and those two are the finding.

The last row is the one that turns a curiosity into a defect, because it is not a synthetic string: structured suffixes like +json and +xml are a normal part of media type names, and text/markdown is a registered type in its own right — RFC 7763 registers it with IANA. A vendor type that ends in something containing those characters is not an absurd thing to invent.

The fix, and why it is not just lower()

Wrapping the field in lower() closes the half that loses requests, and does nothing about the half that catches things it should not. Both halves come from matching a substring, so a rule that keeps matching substrings keeps the second problem no matter what case it uses.

There is a deeper limit here worth stating rather than hiding. Accept is not a string, it is a list with preferences attached. RFC 9110 describes the q parameter as assigning “a relative ‘weight’ to the preference for that associated kind of content”, so text/html, text/markdown;q=0.9 is a request that prefers HTML and will tolerate Markdown — and a substring test cannot tell that apart from a request that only wants Markdown. In my table that row comes back DYNAMIC, meaning this rule would route a browser-style request around the cache on the strength of a fallback it listed at lower priority. Matching substrings against Accept approximates content negotiation; it does not perform it. An edge rule can be made much better and cannot be made correct.

Which is a reasonable argument for not putting the decision there at all. On this site the negotiation now lives on separate URLs — /services/ and /services.md — where no cache has to be persuaded of anything, because the two documents have two names. That is less elegant than negotiation and it has the advantage of not depending on a string comparison in a dashboard.

Accept, Cloudflare cache rules and case sensitivity: frequently asked questions

Are HTTP media types case-sensitive?

No. RFC 9110 states that the type and subtype tokens are case-insensitive, so text/markdown, TEXT/MARKDOWN and Text/Markdown are the same media type. Parameter names are case-insensitive too; parameter values may not be, depending on the parameter.

Are Cloudflare rule comparisons case-sensitive?

Yes, by default, and their documentation says so in the section on transformation functions. Wrap the field in lower() and compare against a lowercase literal if you want case-insensitive matching.

Does any real client send Accept in uppercase?

Browsers do not, and the common crawlers do not. That is not the same as it never happening: anything that constructs the header itself can send any spelling, and a rule that decides caching behaviour on a string comparison is a rule whose behaviour can be selected by whoever sends the request. Whether that matters depends on what the rule protects.

How do I tell whether an edge rule fired?

Read cf-cache-status on a GET, not a HEAD, and read age alongside it. DYNAMIC means nothing was stored or looked up; HIT with a matching age across several requests means they are all reading the same stored object. The response will not tell you which rule produced the outcome, so attribute carefully.

Is substring matching on Accept ever correct?

It is an approximation that is often good enough and never exact, because Accept is a list with preferences rather than a single value. If the consequence of getting it wrong is that someone is served the wrong representation of a page, prefer separate URLs, where correctness does not depend on parsing anything.

Why is this measurable safely on a live site?

Because the negotiation it guards was switched off when these requests were made, so every request in the table received HTML whichever way the rule went. On a site where negotiation is live, the same experiment can write the wrong representation into a shared cache. Check what the rule guards before you test the rule, and check it again on the day you run the test rather than trusting what you remember.

The general shape of this

A rule in a dashboard reads like a sentence in English — if the client accepts Markdown, skip the cache — and executes as a string comparison. The gap between those two readings is where this lives, and it is invisible in the interface, because the interface shows you the sentence.

The habit that would have caught it earlier is not reading documentation more thoroughly, which nobody does. It is testing the rule with inputs it should reject as well as inputs it should accept. Five requests that all matched taught me nothing about the boundary. The three that found it were deliberately wrong ones, and they took under a minute.

Every request above was made against my own domain on the day this was published, and the command that made them is printed so you can repeat it against your own. If you want the same reading taken on a rule of yours — including the ones that came switched on and that nobody chose by hand — that is part of the work I do, and it starts with the free check.