There is error in crawling in the SharePoint environment and the error message is “The SharePoint item being crawled returned an error when attempting to download the item for example .aspx files”

Crawling Error:

SharePoint Crawl Log Error: The SharePoint item being crawled returned an error when attempting to download the item for example .aspx files

Solution:

1. Open Regedit on your search server.
2. Navigate to this registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Global\Gathering Manager
3. Change Value “UserAgent” from “MSIE 4.01” to “MSIE 8.0”.
4. Restart the SharePoint Search Service.
5. Open a SharePoint PowerShell.
6. Get-SPSessionStateService.
7. If this returns false then we need to deploy one
   Enable-SPSessionStateService -DatabaseName “NameOfDatabase”