The Anubis surge
Long time readers on this BBS would know I don't really do javascript. Dillo has served me well for most of my document reading needs on the web. However! In this past month I've been hit by a surge of bot detection pages on previously accessible websites! stackoverflow, the arch wiki just to name a few. So my fellow javascript haters: What is your experience in recent month?
2025-04-25 · 1 year ago · 👍 norayr
2 Comments ↓
🦂 zzo38 · 2025-04-25 at 04:55:
I have disabled my HTTP server for now; other services remain (please do not proxy them using a public service unless it requires some kind of user authentication). However, I had not had problem with Anubis because by default it is bypassed if you do not claim to be Mozilla-based. Since the badly behaved scrapers all seem to claim to be Mozilla-based, this works.
(I had briefly added port knocking to the HTTP server, but it caused a kernel panic so now the HTTP server is not available at all.)
🚀 RubyMaelstrom · 2025-04-25 at 20:17:
One site that's really disappointing that doesn't seem to work on non-Javascript browsers anymore is archive.org. When you connect without Javascript it *claims* to be forwarding you to a "lite'" version of the site, but the forward never happens. I've even emailed them about it but haven't received a reply.