Overview of Prerender.io Crawlers
  • 11 Mar 2024
  • 1 Minute to read
  • Contributors
  • Dark
    Light
  • PDF

Overview of Prerender.io Crawlers

  • Dark
    Light
  • PDF

Article Summary

User Agent


Prerender.io crawler User-Agent will always contain:

Prerender (+https://github.com/prerender/prerender)
CrawlerFull user agent string
Prerender.io desktopMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/W.X.Y.Z Safari/537.36 Prerender (+https://github.com/prerender/prerender)
Prerender.io mobileMozilla/5.0 (Linux; Android 11; Pixel 5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 Prerender (+https://github.com/prerender/prerender)
Prerender.io sitemap workerPrerender (+https://github.com/prerender/prerender)

Wherever you see the string Chrome/W.X.Y.Z in the user agent strings in the table, W.X.Y.Z is a placeholder that represents the version of the Chrome browser used by that user agent: for example, 117.0.5938.62-1.

These values can be spoofed. If you need to verify that the visitor is Prerender, you should use reverse DNS lookup.

Prerender.io crawler IP address ranges


See: IP Addresses

To verify that Prerender.io is the crawler:

Run a reverse DNS lookup on the accessing IP address from your logs, using the host command. Verify that the domain name is prerender.io. Run a forward DNS lookup on the domain name retrieved in step 1 using the host command on the retrieved domain name. Verify that it's the same as the original accessing IP address from your logs.

Example:

> host 142.54.167.130
130.167.54.142.in-addr.arpa domain name pointer us-mci1-38.renders.prerender.io.

> host us-mci1-38.renders.prerender.io
us-mci1-38.renders.prerender.io has address 142.54.167.130

Was this article helpful?

What's Next