CDN

CDNs (e.g. Cloudflare, Cloudfront...etc) caching your site content can cause hard to identify issues. We've seen cases when the customer had a CDN hosted in front of their website, and it intercepted the request before it could even make its way to the Prerender middleware.

Here is what happened in details:

  1. The customer had a well working website with CDN added.
  2. Customer installed Prerender on their apache/nginx server, expecting it to prerender pages for bots.
  3. Random bot (e.g. Google bot) visited the page
  4. The page that was requested existed in the CDN cache, so the CDN responded not even invoking the backend servers (including the Prerender middleware)
  5. The bot received an unrendered response.

What could be a solution?

The CDN must be HTTP header aware and not responding for those user agents that you want your site to be prerendered for.

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.