1
0
Fork 0

New blog post: Guide - How to Block User Agents With Nginx

Also updated style.css for this blog post
main
will 1 year ago
parent 8178b57a16
commit 7660d36f87

@ -8,6 +8,14 @@
<name>Will</name>
<uri>https://isopod.cool/</uri>
</author>
<entry>
<title>Guide - How to Block User Agents With Nginx</title>
<id>guide_blocking_user_agents_nginx</id>
<link rel="alternate" href="https://isopod.cool/blog/posts/guide_blocking_user_agents_nginx/" type="html" title="Guide - How to Block User Agents With Nginx"></link>
<published>2023-02-23T00:00:00+00:00</published>
<updated>2023-02-23T00:00:00+00:00</updated>
<summary>How to configure Nginx to tell Cloudflare and their shitty centralized Fediverse server to go fuck themselves. And other unwelcome stuff too, I guess.</summary>
</entry>
<entry>
<title>Guide - Things I Keep Having To Google</title>
<id>guide_things_i_keep_having_to_google</id>

@ -0,0 +1,75 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Guide - How to Block User Agents With Nginx</title>
<link href="../../../style.css" rel="stylesheet" type="text/css" media="all">
<style type="text/css">
h1 {
background-image: url('nginx.svg');
}
/*summary > * {
margin-bottom: 0;
display: inline-block;
}
details[open] > summary h6 {
display: none;
}*/
</style>
</head>
<body>
<h1>guide:</h1>
<h2 id="caption">How to Block User Agents With Nginx</h2>
<nav>
<a href="../../../">home</a>
<a href="../../">blog</a>
</nav>
<p>Recently, Cloudflare has launched a clumsy, poorly-executed attempt to centralize the fediverse on their platform known as Wildebeest. There are <a href="https://blog.cloudflare.com/welcome-to-wildebeest-the-fediverse-on-cloudflare/" title="This one's kind of a joke, but it explains in detail just how much Wildebeest is fundamentally dependent on Cloudflare's platform, and if you agree with me about anything you should already know why that's bad.">a</a> <a href="https://stop.voring.me/notes/9bka8dyjjo" title="Gleason, of getting-kicked-off-the-dev-team-for-the-fedi-server-he-created-for-sucking-so-much fame, is now contributing code to Wildebeest.">few</a> <a href="https://glitterkitten.co.uk/@doot/109910496299181873" title="This one's the most important. Wildebeest literally just publically displays messages marked as direct. Think admins being able to read your &quot;direct messages&quot; was bad? Try everyone.">reasons</a> not to want to use it, and you probably don't want to be federating with it either. However, blocking every instance running it on sight would be both tedious and ineffective.</p>
<p>The way I've chosen to deal with this is to just configure my reverse proxy, Nginx, to deny connections from anything with "wildebeest" in the user agent string. There are several other good reasons to do this, such as blocking bots that ignore robots.txt, or adapting this approach to serve specialized pages to old browsers, or just denying access to anything that isn't Chrome if you want to earn your place in the 9th circle of Hell.</p>
<p>I'm assuming here that you already know the basics of configuring Nginx, otherwise this article won't really be of much use to you.</p>
<h2>How to do it</h2>
<p>Paste this into the main <code>server</code> block of your Nginx config to instantly drop the connection to any client with a "Wildebeest" (case insensitive) user agent:</p>
<code>
<h4>/etc/nginx/sites-available/yoursite</h4>
if ($http_user_agent ~* (wildebeest)) {<br>
&nbsp;&nbsp;&nbsp;&nbsp;return 444;<br>
}
</code>
<h2>More configuration</h2>
<p>If all you want is to block Wildebeest and forget about it, you can leave now. If you want to customize this behavior further, read on.</p>
<h3>Different HTTP responses</h3>
<p>444 is a custom response code in Nginx that just drops the connection immediately. You can use any other HTTP error code you like, for example to return a <code>403 Forbidden</code> error:</p>
<code>
return 403;
</code>
<p>To issue a permanent redirect to some other URL:</p>
<code>
return 301 https://example.org/;
</code>
<p>Or just to be silly :3</p>
<code>
return 418;
</code>
<h3>Block multiple user agents</h3>
<p>You can also block multiple user agents in one statement like so:</p>
<code>
<h4>/etc/nginx/sites-available/yoursite</h4>
if ($http_user_agent ~* (wildebeest|googlebot)) {<br>
&nbsp;&nbsp;&nbsp;&nbsp;return 444;<br>
}
</code>
<h3>Includes</h3>
<p>One thing I'd recommend doing is moving all your user agent blocks to a separate file that you can include in all your Nginx sites. I place this file at <code>/etc/nginx/includes/bans</code>:</p>
<code>
<h4>/etc/nginx/includes/bans</h4>
if ($http_user_agent ~* (wildebeest|googlebot)) {<br>
&nbsp;&nbsp;&nbsp;&nbsp;return 444;<br>
}
</code>
<code>
<h4>/etc/nginx/sites-available/yoursite</h4>
include /etc/nginx/includes/bans;
</code>
<p>The <code>include</code> statement goes in your <code>server</code> block as before.</p>
</body>
</html>

@ -0,0 +1,2 @@
<?xml version="1.0" encoding="utf-8"?><!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg width="800px" height="800px" viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg"><title>file_type_nginx</title><path d="M15.948,2h.065a10.418,10.418,0,0,1,.972.528Q22.414,5.65,27.843,8.774a.792.792,0,0,1,.414.788c-.008,4.389,0,8.777-.005,13.164a.813.813,0,0,1-.356.507q-5.773,3.324-11.547,6.644a.587.587,0,0,1-.657.037Q9.912,26.6,4.143,23.274a.7.7,0,0,1-.4-.666q0-6.582,0-13.163a.693.693,0,0,1,.387-.67Q9.552,5.657,14.974,2.535c.322-.184.638-.379.974-.535" style="fill:#019639"/><path d="M8.767,10.538q0,5.429,0,10.859a1.509,1.509,0,0,0,.427,1.087,1.647,1.647,0,0,0,2.06.206,1.564,1.564,0,0,0,.685-1.293c0-2.62-.005-5.24,0-7.86q3.583,4.29,7.181,8.568a2.833,2.833,0,0,0,2.6.782,1.561,1.561,0,0,0,1.251-1.371q.008-5.541,0-11.081a1.582,1.582,0,0,0-3.152,0c0,2.662-.016,5.321,0,7.982-2.346-2.766-4.663-5.556-7-8.332A2.817,2.817,0,0,0,10.17,9.033,1.579,1.579,0,0,0,8.767,10.538Z" style="fill:#fff"/></svg>

After

Width:  |  Height:  |  Size: 1.0 KiB

@ -139,13 +139,26 @@ code {
background-color: black;
padding: 0.15rem;
}
main > code, body > code {
:is(main,body) > code {
display: block;
width: 32rem;
margin: auto;
margin-bottom: 1rem;
padding: 0.7rem;
border: 1px solid #00ffd5;
}
:is(main,body) > code > h4 {
margin: -0.7rem;
margin-top: -0.4rem;
margin-bottom: 0.7rem;
padding-bottom: 0.2rem;
font-weight: normal;
top: 0;
left: 0;
right: 0;
text-align: center;
border-bottom: 1px solid #00ffd5;
}
tr:hover {
background-color: #2222227c;
}

Loading…
Cancel
Save