Fresh search results on gemini ?
Do you think your gemini capsule would support a higher crawl rate from search engines like Kennedy or AuraSearch?
I know my capsule can handle a more agresive crawl rate and , of course, more often crawls.
I see it took Kennedy more than 1 month to revisit my capsule and almost 2 for AuraSearch...
or maybe ... it's hard on the servers hosting these search engines
Apr 03 · 5 weeks ago · 🤔 1
3 Comments ↓
🐦 JustASillyBird · Apr 03 at 16:09:
This is Gemini, so most crawls are going to be pure text. Pretty light load. It's just an annoyance that the protocol's deliberate limitations include no IMS mechanism, so there's no way to check if a page has changed other than to re-download it all.
🚀 lars_the_bear · Apr 03 at 18:04:
My "new" capsule has been around for about two months, and does not appear in Kennedy searches at all. My old one did, and there are links between the two. So, yes, I don't think it wouldn't hurt if the search was a bit more vigorous.
As @JustASillyBird says, though, the lack of any way for a crawler to know whether content has changed since the last download probably makes it difficult to implement an efficient crawler. So I'm not surprised that crawls are infrequent.
💀 StrokedOut · Apr 05 at 17:27:
Kennedy crawls are triggered manually. If triggered, the crawler crawls everything it knows and shuts down afterwards.