...
I want to prevent certain pages from being indexed and appear appearing in search results.
Solution
You can add can add an HTML element of data-sj-noindex anywhere in a page and it will not be indexed. Most commonly this will be defined in the <head> of an HTML page as follows:
...
Additionally you can use crawling rules to programmatically exclude sections or certain pages of your web siteyour website. You can also set individual pages to not be indexed from the data sources tab of the admin Console.
...
| Filter by label (Content by label) | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Documentation
Advanced crawler documentation
...