Comment by 🐙 norayr

yes, anubis is used by many projects, that don't have very strong infrastructure.

irony is, ai crawlers are created by corporate world.

to scan and “know” “everything”.

but normal non-corporate servers can't stand the load. so they need to protect themselves via tools like anubis.

thus ai crawlers will get mainstream information from strong servers. they won't get marginal data about marginal projects.

ai won't know anything: as a product of corporate world it will be best used with corporate tech by corporate workers.

for example: maemo-leste (linux for phones), freepascal compiler pages use anubis.

both infras can't stand ai bot load. so ai won't know much about them.

Apr 10 · 4 weeks ago

SourceHut has been under DDoS attack. SourceHut Pages is also affected by this.