What is a Spider?
A spider, also known as a bot, robot, or crawler are software programs that scan the web. They vary in purpose from indexing web pages for search engines to harvesting e-mail addresses for spammers. (TIP: Get your emails encrypted on your webpages to avoid getting emails collected as spam).
What is a Spider Trap?
A spider trap refers to either a continuous loop where spiders are requesting pages and the server is requesting data to render the page or an intentional scheme designed to identify (and “ban”) spiders that do not respect robots.txt
For an SEO consultation, please contact us