Alvus Search

Anyone know who runs the Alvus search engine? I'd like a word.

gemini://alvus.nl/

🛸 bluesman

Jan 10 · 4 months ago

2 Comments ↓

🍀 gritty · Jan 10 at 15:42:

TIL what bash.org is.

🛸 bluesman [OP] · Jan 11 at 14:15:

On the off chance the admin sees this, it's good that you're checking robots.txt but it needs to be queried on each crawler pass. I recently added robots.txt to exclude a new gemini streaming resource. The crawler did check once for robots.txt weeks ago but it hasn't looked for it since. Without the exclude directive, the search engine tries to download an "infinite" audio stream (it shouldn't do that anyway, IMO). I had to kick the connections after many hours on several occasions. I blocked the crawler IP for the moment.