Crawl error Processing this item failed because of an unknown error when trying to parse its contents sharepoint

During various search troubleshooting i came across the following crawling error in the Crawl log of a SharePoint 2013 environment.

Processing this item failed because of an unknown error when trying to parse its contents. (Error parsing document ‘http://********.*****.com/Project/abcd/Q_M/ABX/SitePages/Homepage.aspx’. Sandbox worker pool is
closed.; ; SearchID = *******************)

In order to fix this you can try to perform the following action plan:
Open “Local Policies
Click on “User rights assignment

user-rights-assignment

Make sure that the search service account has the following rights:
Replace a process level token

adjust-memory-quotas-for-process

Adjust memory quotas for a process

adjust-memory-quotas-for-process-properties

Impersonate a client after authentication

impersonate

Please make sure that the policies don’t get changed afterwards.

After implementing the above changes please run a clear configuration cache
After clearing the cache, start a full crawl and the errors should be gone.

SharePoint item crawled returned error when attempting to download the item example aspx file

Error:

SharePoint Crawl Log Error: The SharePoint item being crawled returned an error when attempting to download the item for example .aspx files

Solution:

1.Open Regedit on your search server/s
2.Navigate to this registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Global\Gathering Manager
3.Change Value “UserAgent” from “MSIE 4.01” to “MSIE 8.0
4.Restart the SharePoint Search Service.
5.Open a SharePoint PowerShell
6.Get-SPSessionStateService
7.If this returns false then we need to deploy one
Enable-SPSessionStateService -DatabaseName “NameOfDatabase”

search diagnostics and reports sharepoint

We can access and analyze several query and crawl health reports, logs and usage reports from the Search service application in the SharePoint Central Administration to monitor the health of the search system.

The health reports and logs only contain information after a full crawl has completed. To run a full crawl, we have to set up a Search service application, add at least one content source, and then start a full crawl.

To view the health reports and the crawl log, one have to be an administrator of the Search service application. Alternatively, an administrator who is a member of the Farm Administrators group can grant user accounts Read permissions on the Search service application. A user account that has Read permissions can only view the Search service application status page, the health reports and the crawl log.

Query health reports:

  1. Trend
  2. Overall
  3. Main Flow
  4. Federation
  5. SharePoint Search Provider
  6. People Search Provider
  7. Index Engine

To view query health reports:

  1. Verify that the user account that is performing this procedure is an administrator of or has Read permissions to the Search service application.
  2. In Central Administration, under Application Management, click Manage service applications.
  3. On the Service Applications page, click the Search service application.
  4. On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Query Health Reports.
  5. On the Search Service Application: Query Latency Trend page, click the query report that you want to view.

The following table shows which reports are available.

query-health-report

Crawl health reports:

SharePoint 2013 provides the following reports about crawl health:

  1. Crawl Rate
  2. Crawl Latency
  3. Crawl Queue
  4. Crawl Freshness
  5. Content Processing Activity
  6. CPU and Memory Load
  7. Continuous Crawl

To view crawl health reports

  1. Verify that the user account that is performing this procedure is an administrator of or has Read permissions to the Search service application.
  2. In Central Administration, under Application Management, click Manage service applications.
  3. On the Service Applications page, click the Search service application.
  4. On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Crawl Health Reports.
  5. On the Search Service Application: Crawl Reports page, click the crawl health report that you want to view.

The following table shows which reports are available.

crawl-health-report

Crawl log:

The crawl log tracks information about the status of crawled content. This log lets you determine whether crawled content was successfully added to the index, whether it was excluded because of a crawl rule, or whether indexing failed because of an error. The crawl log also contains information such as the time of the last successful crawl and whether any crawl rules were applied. You can use the crawl log to diagnose problems with the search experience.

To view the crawl log

  1. Verify that the user account that is performing this procedure is an administrator of the Search service application, or has Read permissions to it.
  2. In Central Administration, under Application Management, click Manage service applications.
  3. On the Service Applications page, click the Search service application.
  4. On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Crawl Log.
  5. On the Crawl Log – Content Source page, click the view that you want.

crawl-log-views

Additional columns in the Content Source, Host Name and Crawl History views:

content-source-host-name-crawl-history-view

Usage reports (search report):

To view usage reports

  1. Verify that the user account that is performing this procedure is an administrator of or has Read permissions to the Search service application.
  2. In Central Administration, under Application Management, click Manage service applications.
  3. On the Service Applications page, click the Search service application.
  4. On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Usage Reports.
  5. On the View Usage Reports page, click the usage or search reports view that you want view.

usage-report-search-report