Akamai

Akamai

Akamai works easily with a prerendering service like prerender.io. Since search engines can only see Javascript tags when they crawl sites, a prerender service can improve your SEO by rendering your content in Javascript.

This article covers the basics of a prerender service and explains how to get started.

You can easily add the Prerender.io service to your delivery:

1. Detect crawlers
2. Send crawler requests to prerender service
3. Modify request path
   
   

Detect crawlers

Detecting the key crawlers can be done based on the user-agent request header. (If you use Bot Manager, you’ll have more flexibility and better results).

You can find a list of these key bots below. 

 'Google-InspectionTool',
  'googlebot',
  'Yahoo! Slurp',
  'bingbot',
  'yandex',
  'baiduspider',
  'facebookexternalhit',
  'twitterbot',
  'rogerbot',
  'linkedinbot',
  'embedly',
  'quora link preview',
  'showyoubot',
  'outbrain',
  'pinterest/0.',
  'developers.google.com/+/web/snippet',
  'slackbot',
  'vkShare',
  'W3C_Validator',
  'redditbot',
  'Applebot',
  'WhatsApp',
  'flipboard',
  'tumblr',
  'bitlybot',
  'SkypeUriPreview',
  'nuzzel',
  'Discordbot',
  'Google Page Speed',
  'Qwantify',
  'pinterestbot',
  'Bitrix link preview',
  'XING-contenttabreceiver',
  'Chrome-Lighthouse',
  'TelegramBot',
  'SeznamBot',

Send crawler requests to prerender service

Whenever the user agents are detected, we forward them to `service.prerender.io`. Make sure to set these two key settings to origin hostname:

Cache key hostname: Origin Hostname

Forward host header: Origin Hostname

Add your Prerender token using the Modify Outgoing Request header behavior.

In practice, this means that you need to set the x-prerender-token header in the forwarded request to the token. You can find it at the top left corner of your Prerender dashboard.

Modify request path

We still need to change the outgoing request path to the full URL. You can do this by using property variables and modifying the outgoing request path behavior.

Do not cache

Crawlers typically generate a lot of traffic. Prerender visits each page only once per crawl; by the time our service crawls the same page for the second time, the cached version is either evicted or no longer up to date.

So, checking the edge cache, the parent cache, and going forward to the origin is slower than going straight to the origin. Going to the origin gives you the benefits of SureRoute and persistent connections.