.NET程式遮蔽一些無效蜘蛛的訪問配置

uxinxin發表於2024-07-17
1  <rule name="Block spider">
2           <match url="(^robots.txt$)" ignoreCase="false" negate="true" />
3           <conditions>
4             <add input="{HTTP_USER_AGENT}" pattern="Apache-HttpClient|SemrushBot|Webdup|AcoonBot|AhrefsBot|Ezooms| EdisterBot|EC2LinkFinder|jikespider|Purebot|MJ12bot|WangIDSpider|WBSearchBot|Wotbox|xbfMozilla|Yottaa|YandexBot| Jorgee|SWEBot|spbot|TurnitinBot-Agent|curl|perl|Python|Wget|Xenu|ZmEu|facebookexternalhit"   ignoreCase="true" />
5           </conditions>
6           <action type="AbortRequest" />
7       </rule>

記錄一下，簡單測試可用

相關文章