How does Prerender.io work?

How Prerender.io works:

When a crawler visits, Prerender needs to identify it. This is done by checking a special code crawlers carry (user agent string).
Prerender also checks if the crawler is looking for your main content (excluding things like images or code).
If it's a crawler after your content, Prerender fetches a special pre-rendered version of your page, ready for the crawler to quickly understand and index.

How Prerender Works with Your Tech

Prerender helps search engines see your content instantly, boosting your SEO. Integrating it involves letting Prerender know when a search engine crawler visits your site.

Two Integration Options:

The Prerender integration can be adapted to how you are delivering your pages' content.

The best practices for integrating Prerender are:

CDN Integration : This is the best option if you already use a Content Delivery Network (CDN) in front of your server. It's fast to set up but might have less flexibility depending on your CDN provider.
Backend Integration : This gives you more control over which requests get sent to Prerender. However, it involves some coding on your server.

Consider how much control you need and your comfort level with coding. Backend integration offers more control, while CDN is quicker to set up.

You can find more about integrations in our Available integrations page.

Direct User Access

This is how your system operates by default.
Requests are coming from your clients and you serve them. Prerender doesn't take part in the flow.

First-Time Crawler Request

If a crawler requests a page, your backend identifies it as a crawler and sends a request to Prerender. The first time this happens, the page won't be in the cache, called a cache miss. In this case Prerender.io needs to fetch the page from you, render it, store it in the cache, and return it to you. This process may take several seconds.

Ultimately, the Prerender integration issues the following curl command, if the request is coming from a crawler:

curl -H "X-Prerender-Token: <YOUR\_PRERENDER\_TOKEN>" https://service.prerender.io/https://www.example.com/

Repeated Crawler Request

From the next occasion on, the page will be in our cache and we will serve it from there.

Content Update

Your content changes. To follow the changes, Prerender.io needs to periodically refresh it's cache. This is called recaching and you can set the maximum age of your pages in the cache (called cache freshness). In this case, the trigger of the render is a timer without external event.

Video Overview

Here's a quick video explainer of how the process works.

Prerender.io improves SEO by intercepting crawler requests and delivering clean, pre-rendered HTML versions of your pages. This helps ensure that search engines like Google and Bing can index your single-page applications (SPAs), JavaScript-heavy content, and dynamic routes accurately and efficiently.

By caching and serving pre-rendered pages only to bots, Prerender boosts crawlability and performance while minimizing overhead for human visitors.

Detailed Explanation / How It Works

When Crawlers Visit

Prerender checks the user agent string in each request to identify if it's a bot.
If it’s a search engine crawler and the content is intended for indexing (not images or JS files), the request is rerouted to Prerender.io.
Prerender then returns a pre-rendered HTML snapshot of the page.

How Prerender Works With Your Stack

Prerender must be integrated with your web server, CDN, or middleware so that it knows when to intercept crawler traffic. Once in place, Prerender ensures that bots are served fully rendered pages, enhancing their ability to read metadata, rich content, and structured data that would otherwise be missed.

Integration Options

There are two common ways to integrate Prerender:

CDN Integration: Ideal for users already behind a CDN like Cloudflare or Akamai. Quick to set up, minimal coding, but slightly less flexible.
Backend Integration: Offers more control over logic and request conditions. Ideal if you need fine-tuned control over how/when Prerender is triggered.

For setup details, refer to our Available Integrations page.