CloudFlare + nginx, or save with the help of a “coffee maker” (upd2: super-short dynamics cache!)

  • Tutorial

Good Friday, dear% username%, avid reader and fighter for justice on the Internet!

We all remember (Google remembers exactly!) That there was such an article CloudFlare + nginx = cache everything on a free plan . In which the basic principles of saving on tariffs and servers were considered, through omnivorous caching on the CloudFlare side of files up to 512MB.

In this article, we will play with the response codes of our server to save even to build a zigguratand don’t switch to an “enterprise plan” which offers us a similar result in their “offers”.

At a free rate, TTL is 2 hours. This is what they have written in the free tariff plan. If we dig deeper , we get:

  • 200 301 120m;
  • 302 303 20m;
  • 403 1m;
  • 404 5m;
  • any 0s;

More detailed: redirects are cached for 20 minutes, authorization errors for a minute and classic 404 for 5 minutes. Other codes are not cached.

This means that if you have a cool dynamic site that, for example, provides an API for some operations, etc. etc. and this is wrapped up through CF, since it generates both files and pictures and a lot of traffic, and the dynamics hurt ... then if someone understands how your answers are generated (bare count or in base64 or your base or hashes, but it’s more difficult to pick up the latter ), then he (a person from the dark side of the room without lighting) can deceive other users with a false answer, namely - 404.

For example:

  • A simple service that pulls EXIF ​​from photos that were flooded. An example of a link what it means to give EXIF ​​as TXT from a photo with id = 1.

Suppose you have 1336 pictures there. This means that the 1st, 2..1336th will give out the 200th answer, which will be cached on the cloudflare side.

Solution # 2, without a crutch with 418 code (a consequence of this comment).

Solution # 1, crutch through the 418 "i'm teapot" code.

Here is our emulation (nginx):

location ~* "^/api/image/exif/(.*?)\.txt$" {
	default_type text/plain;
	set $testimageid $1;
	if ($testimageid ~ 1337){
		return 404 "Image $testimageid not found";
	return 200 "EXIF for $testimageid image";

And so on ...

Until we get to the photo with ID 1337, which is not yet on the server. What will our dynamics do? It is logical that she will give out some text about the error and code 404.

Which will be next cached . And all subsequent requests to this URL will return a response from the CF cache for the next 5 minutes .

Is this critical? Yes and no. It depends on how many users you make miserable and how many millions you save on CDN (thanks to me).

What to do? We look at the official material and understand that we need to change the code ... the server response code in the case of 404.

But not everything is so simple. You cannot_just_take_take_and_change_response_code.jpg, since each code carries some kind of semantic load, which will be interpreted or they simply beat it!

So for this, a code was invented that does not carry any semantic load, this is code 418, which is described in rfc2324 .

Change the code of our emulation (and clear the cache for this link in the CF panel):

location ~* "^/api/image/exif/(.*?)\.txt$" {
	default_type text/plain;
	set $testimageid $1;
	if ($testimageid ~ 1337){
		return 418 "Image $testimageid not found";
	return 200 "EXIF for $testimageid image";

And we get the following picture:

This signifies the fact that until there is no picture, CF does not cache the response from our server, passing to the clients each time a fresh response with an error 418 [from our server] . Next, as the picture appears on the server, and our dynamics gives an answer of 200, CF caches this answer for 2 hours.

Code 418 belongs to the 4xx group of codes. And even if the end user does not have the definition of this code in the browser (or in the self-written software), then there is at least (should be!) A check for 4xx as in the screenshots of this article (firefox, red square).

This is how you can simply report an error (4xx) and ask cloudflare not to cache this response when it’s policy, when we all cache. This protects against false 404 responses from the server when someone requested future material before publication.

upd2: Solution # 2, without a crutch with 418 code. Corollary of this comment.

First we set the condition for Browser Cache Expiration:

Then we create this Page Rule:

The main point: the absence of Browser Cache TTL and the presence of Cache Level: Cache Everything !

Then, if your TTL for valid answers is 315360000, then you need to add
add_header Cache-Control "public, max-age=315360000";

in location'y, without adding it to the server!
location / {
	add_header Cache-Control "public, max-age=315360000";

Location with the dynamics is:
location ~* "^/api/image/jexif/(.*?)\.txt$" {
	proxy_pass http://localhost:8080/headerstest/headerstest.jsp?imgid=$1;
	proxy_set_header Host $host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Following makes our dynamics give errors with the code 404 with such a header (using the JSP as an example)
response.setHeader("Cache-Control", "private, max-age=0");

As a result, Cloudflare does not cache errors on the above settings, but that's not all!

If you need frequent cache updates for 200 responses of your dynamics, then you need to specify (again on the example of JSP)
response.setHeader("Cache-Control", "public, max-age=20");

And thus you get a 20 second cache of the answers of your dynamics! What is less than the enterprise rate!

In fact, while this works, you can completely remove the site on cloudflare and hide the IP server (provided that entries other than A / AAAA / CNAME / TXT are located on different ip).

May the force come with you!

ZY, it is set $testimageid $1;used because nginx cannot compare $ 1 in an if expression (but can operate on $ 1 in a config or in content), for this you need to enter a variable, but YourChief showed a more beautiful solution .. For the second solution, you need to remove
add_headerfrom server and leave it in location so that the first does not “stick” nginx to the second (this is such a miracle in the answer “public, max-age = 20, public, max-age = 315360000”).
Chippedboilersthe pictures are aligned and compressed in png- 8 , perfectionists on dialup can smile!

Also popular now: