Comment by 🚀 MikeK
Alvus was easily 1/4 of the traffic on my site before I firewalled it. There was an 'observable gopher project' that had a similar level of requests on gopher before I blocked them too. Worst two by a long way.
Mar 09 · 8 weeks ago
Original Post
Known crawler IPs? — I've noticed shortly after posting about my video game music stream, there's been a connection streaming for nearly 20 hours straight. I can disconnect it, and figure it it's an actual person they'll reconnect after some time. But I suspect it may be a crawler that doesn't check robots.txt, doesn't have logic to abort if a request takes too long or has too many bytes, etc. I know one crawler (Kennedy) publishes their IP addresses. Totally Legit Gemini Search doesn't but...