2, direct the output
1, reduce the spider crawling efficiency love Shanghai, resulting in Shanghai in the process of love spider crawling and some do not exist 404 page
analysis found that the website links and no tool points out of the 404 page links, and then through further investigation, found that the output of RSS pages. In the background, I set the output RSS displays only the abstract, rather than text, so when the program will automatically cut. Cut into incomplete part. Love Shanghai is to identify the text link, hence the 404 page.
1, modify the program to remove all links after truncation output
how to solve this problem:
reduced the love Shanghai spider web opportunity, when the love of spiders in Shanghai found 404 pages and will not be directly discarded, but not regularly repeated crawling to confirm whether the page does not exist, so to Shanghai spider crawling opportunity wasted in the 404 page
404 is the page itself does not affect the weights of the website, but the 404 error occurs because the system is harmful:
reduced the website trust in the search engine, if a website is often large, often the 404 error occurs, will undoubtedly reduce the degree of trust in the search engine that can affect the weight of
today through love Shanghai webmaster tools found love Shanghai spiders in the grab website has 404 pages:
this problem many sites will appear, hope this can arouse the attention of webmaster! This paper starts from the chapter light blog: 贵族宝贝zhangliangblog贵族宝贝/ Shanghai dragon /alert_output_desc.html, please indicate the copyright.