Comment by 🛸 bluesman
Ha! Nice. I'm just glad I caught it. It might still be going otherwise.
Jan 26 · 3 months ago
2 Later Comments ↓
I have seen the alvus.nl crawler on my gemini server. I tried the search engine, it was not working 2 days ago.
Here a some crawler ips seen on my gemini server:
193.70.85.11 lupa
23.88.52.182 tlgs
116.202.128.144 freeshell.de
64.149.155.184 135.148.41.168 kennedy
77.161.107.142 alvus.nl
There are more crawlers but I haven't identified them.
Alvus was easily 1/4 of the traffic on my site before I firewalled it. There was an 'observable gopher project' that had a similar level of requests on gopher before I blocked them too. Worst two by a long way.
Original Post
Known crawler IPs? — I've noticed shortly after posting about my video game music stream, there's been a connection streaming for nearly 20 hours straight. I can disconnect it, and figure it it's an actual person they'll reconnect after some time. But I suspect it may be a crawler that doesn't check robots.txt, doesn't have logic to abort if a request takes too long or has too many bytes, etc. I know one crawler (Kennedy) publishes their IP addresses. Totally Legit Gemini Search doesn't but...