...
Locate the
<header>tag of the page you want to prevent from being crawled.Add the following code within the header:
<meta name="robots" content="noindex" data-sj-noindex />Save the changes. The crawler will ignore this page next time it comes across it.
Additionally you can use crawling rules to programmatically exclude sections or certain pages of your web site. You can also set individual pages to not be indexed from the data sources tab of the admin Console.
Related articles
https://www.sajari.com/docs/user-guide/indexing-data/advanced-crawler
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| Page Properties | ||
|---|---|---|
| ||
|