'DisableLoopbackCheck' to fix - Error: Access is denied. Verify that either the Default Content Access Account has access to this repository .........

Error:

Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.

Root Cause :

This issue occurs when the Web site uses Integrated Authentication and has a name that is mapped to the local loopback address.

This is because your operating system(Windows 2008 in my case) include a loopback check security feature that is designed to help prevent reflection attacks on your computer. Therefore, authentication fails if the FQDN or the custom host header that you use does not match the local computer name.

Resolution:

Following instructions would disable the LoopBackRequest in registry. 

1. Run regedit.exe from the command prompt

2. In the registry editor navigate to "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa"

3. Right click on "Lsa" and create a new 32bit DWORD value called DisableLoopbackCheck

4. Right click  and Modify the value of "DisableLoopbackCheck" as "1"

6. Close the registry editor.

You may now restart the crawl and hopefully it will work.