Spaces
Apps
Templates
Create
Search.io Knowledge Base
All content
Space settings
Shortcuts
Troubleshooting articles
Troubleshooting articles
This trigger is hidden
How-to articles
How-to articles
This trigger is hidden
Content
Results will update as you type.
Fundamentals
Account Management
Indexing Data
Indexing Data - Crawler
•
How can I fix PDFs and DOCs that fail to index or have the wrong title or description?
•
Why are 404/403 webpages in my collections index and returned with search results?
•
Can a collection or site be re-indexed?
•
How long does it take for boost and exclude rules to process?
•
Can I index Doc/Docx/PDFs?
•
How to instruct the crawler to visit all subdomains or prevent subdomains being crawled?
•
Can I index dates using the crawler?
•
Does instant indexing remove pages that return a 404/403 response?
•
How does the crawler handle 301 or 302 redirect?
•
How does crawling and indexing work?
•
What customizations can be applied to the crawler?
•
How often does the crawler crawl my content?
•
How do canonicals impact indexing?
•
Does the crawler automatically crawl my website content?
•
How to whitelist the crawler if its being blocked
•
How can I test search in a password-protected staging environment?
•
How to prevent and remove duplicate webpages from search results
•
How to add custom fields based on meta tags
•
How to remove unwanted pages from search results
•
How do I prevent pages from being crawled?
•
How to index a sitemap?
•
How to setup instant indexing <test>
•
Can I remove a page from a Collection?
•
Why are there no records or pages missing from my website collections index? <test>
•
Can I index and crawl password protected sites?
•
How do I whitelist or blacklist so the crawler only adds certain webpages to my index?
•
How do I obtain a list of all the records or field values that are currently indexed?
•
How do I see when a webpage was last crawled and its crawl status?
•
The Domains menu: How to use the Search From Domain feature to have search functionality on a different domain to your content
•
What HTML elements does Search.io crawl?
•
Custom meta tag not splitting array when indexed
Synonyms, Spelling and Autocomplete
Search - Querying
Search - Filtering and Exclusions
Search - Relevance and Ranking
Display Settings
Analytics
Search.io Knowledge Base
/
Indexing Data - Crawler
Summarize
This Confluence instance is now read-only, please head over to the Algolia Confluence instance for the same more up-to-date information
Indexing Data - Crawler
Richard Davidson
Owned by
Richard Davidson
Last updated:
Jul 21, 2022
Loading data...
{"serverDuration": 26, "requestCorrelationId": "706950d5ddd2495eb01a4bc9c7b784b0"}