Tag Info

New answers tagged


The GitHub robots.txt does explicitly disallow crawling of the wiki pages, for example in the Googlebot section: User-agent: Googlebot Allow: /*/*/tree/master Allow: /*/*/blob/master ... Disallow: /*/*/wiki/*/* As this is the site-wide robots file, there isn't any getting around it. It is an interesting choice, since GitHub describes wikis as a place to ...

Top 50 recent answers are included