This Confluence instance is now read-only, please head over to the Algolia Confluence instance for the same more up-to-date information

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Current »

Does the crawler automatically crawl my website content?

Answer

Yes, the Sajari crawler will visit your website periodically to update or remove existing content in your collection. However, if you're have the ping-back code installed on your website, any changes will be applied to your collection instantly when that page is visited for the first time and the ping-back code is triggered.

If a page is in a collection, and then its HTTP status code changes to 404, Sajari will immediately remove this page from your collection as soon as the page is visited and the ping-back code is triggered.

Similarly, when a new page is published and viewed for the first time, this will be added to your collection instantly.

  • No labels