Prerender.io detects requests from search engine crawlers and serves them a cached, fully rendered version of your page. This process involves:
- Crawler Detection: Prerender identify requests from bots like Googlebot based on their user-agent strings.
- Page Rendering: Fetching your page, rendering it in a headless browser, and caching the result.
- Serving Cached Content: Delivering the cached HTML to the crawler, ensuring quick and accurate indexing.
This setup allows human visitors to experience your site normally, while bots receive optimized content for indexing.
Here are the four main steps:
1. Complete Signup
- Visit Prerender.io and click the Start For Free or Create an Account button.
- Fill in the required information to create your account.
2. Integrate Prerender.io with Your Website
- Check our full list of Available Integrations to identify which integration guide to use. Prerender must be integrated as close to the visitor as possible via CDN or reverse proxy.
- Test the integration by changing your browser's user agent to a crawler (e.g., Googlebot) and verifying that the prerendered page is served. Check this page for more detailed instructions.
Note: Prerender only serves pages to bots—human visitors should not be affected.
3. Adjust Query Parameters
It's important to review your query parameters so that you don't waste your renders on low-value pages.
How this works:
Prerender caches each unique URL individually. If two URLs share the same path but differ in their query parameters, Prerender treats them as separate URLs. This works well for parameters that influence the page's content, but it can lead to inefficiencies when parameters don't affect the page's content.
To address this, Prerender offers an option to ignore specific query parameters. When a request includes an ignored parameter, Prerender will serve the base URL without that parameter. For example, if a crawler requests https://example.com/category?param=1 and "param" is set to be ignored, Prerender will respond with the cached version of https://example.com/category.
Step-by-step guide:
- Access your Prerender.io dashboard and navigate to Cache Manager > URL Parameters.
- Configure which query parameters should be ignored to prevent unnecessary caching of duplicate pages:
- Ignore specific query parameters: Exclude parameters that don't affect page content.
- Ignore all query parameters: Exclude all parameters unless specified otherwise.
- Only cache specific query parameters: Include only parameters that change page content.
Note: Changes take effect within an hour, and affected URLs are removed from the cache two hours after the update.
4. Review and Submit Sitemaps
By submitting your sitemaps to Prerender, your pages are precached before bots access them. This approach also adds new pages to the cache automatically as they are created. This proactive caching approach enables you to leverage the benefits of Prerender more quickly.
When a bot requests a page, Prerender serves it directly from the cache for faster response times. If the requested page isn’t cached, Prerender fetches it from your server, renders it, and then delivers it to the bot. This process, known as a cache miss render, results in slower response times compared to serving a cached page.
Adding a sitemap helps alleviate cache misses, ensuring your pages are ready to be served when bots request them.
Step-by-step guide:
- In your Prerender.io dashboard, go to the Sitemaps section.
- Add your sitemap URL to allow Prerender to pre-cache your pages.
- Use the Crawl Now feature to verify that your sitemap is accessible and that pages are being cached. If the crawl is unsuccessful, you will see "Failed" in the Status column, and with the button next to it, you should see the reason that might stand behind the failed crawl.
- Regularly updating your sitemap ensures that new pages are promptly cached and served to crawlers.
Note: Once your sitemap is added, Prerender will crawl the Sitemap by the interval you set up and add any new, uncached URLs to the cache. Pages already cached will not be recached when the sitemap is recrawled.
You can find more details and information about how Prerender Sitemap works on this page.
Common Pitfalls /Tips
- Bot Blocking: Ensure that your server or firewall settings do not block search engine bots, as this can prevent Prerender from crawling and serving cached pages.
- Integration: Integrate Prerender into the part of your system that handles incoming requests. Check this page for a detailed guide on that.
- Geo/IP Restrictions: Avoid geo-based or IP-based restrictions that might prevent Prerender.io servers from accessing your site.