Comment by πŸš€ clseibold

Re: "Is there a smolnet search engine/search aggregator for…"

In: s/SmallWeb

AuraGem Search supports gemini, nex, scroll, and spartan. It does not yet support gopher, but I intend to add this at some point in the future.

However, please note that today my search engine is being moved over from Firebird to Postgresql, so the searching will not work for today. I'll have it fixed/finished by tomorrow.

β€” AuraGem Search

πŸš€ clseibold [πŸ›‚ Code of Conduct rule 1 violations]

2025-06-20 Β· 11 months ago

4 Later Comments ↓

πŸš€ Copenhagen_Bram [OP] Β· 2025-06-21 at 02:15:

Awesome! Hope you get it back up soon. Looking forward to seeing it search gopher too.

πŸš€ clseibold [πŸ›‚] Β· 2025-06-21 at 04:05:

@Copenhagen_Bram The search engine is back up now, but the crawler is going atm to rebuild the database back up. Also, the search querying has changed so that search terms are ANDed together rather than ORed, because this is the default in postgres, but I intend to fix this soon.

The stats page provides info on the crawler and feed aggregator crawler anytime they are running:

β€” AuraGem Search Stats

It also shows the current capsule being crawled by the capsule on-demand crawler. It goes one capsule at a time. You can submit your capsule to be crawled on-demand by submitting it to this link:

β€” Crawl Capsule On-Demand

☯️ dragfyre · 2025-06-22 at 11:45:

@clseibold are there significant changes that need to be done to get it searching on gopher, or is it more about flipping the right switches and running the crawler?

πŸš€ clseibold [πŸ›‚] Β· 2025-06-22 at 22:50:

@dragfyre Aside from writing a gopher client and gopehrmap parser in Golang, the only other thing is to figure out how I'm going to store the info in the database.

Some gopher search engines will store all of the itemtypes of a gophermap in the db, but that's a different model from how my db works for gemini, nex, scroll, and spartan.

Currently, I save every *url* and details of its content, one row for each url. Whereas a gopher search engine might save multiple rows of gophermap items per path. So it's definitely a little different, particularly because a gophermap item doesn't necessarily correlate to a URL, let alone a gopher resource (there are other itemtypes like text and WAIS, etc.).

But I think I can adapt the gopher model into the current model that I use. Idk, we'll see.

Original Post

πŸŒ’ s/SmallWeb

πŸš€ Copenhagen_Bram:

Is there a smolnet search engine/search aggregator for multiple protocols at once? (gemini, gopher, nex, etc) β€” I was wondering if there were a multi smolnet protocol search engine I could set as default for Lagrange.

πŸ’¬ 5 comments Β· 2025-06-20 Β· 11 months ago