[ANN] New Gemini Server: Space Age

Howdy Geminauts,

  Some of you may know me as lambdatronic at floss.social on Mastodon. I've
been having a lot of fun playing around in Geminispace for the past
several weeks, and I decided it was about time that I wrote my own
Gemini server to add to the software list on the Project Gemini capsule.

So without further ado, here it is:

  https://gitlab.com/lambdatronic/space-age

Space Age is a Gemini server written in Clojure (https://clojure.org/).

It implements Gemini protocol specification v0.14.2 (July 2nd 2020), all
except for client certificates (which I'm still working on).

The entire code base (including build files) clocks in at a cool 200
lines with absolutely no dependency libraries.

At present, Space Age simply works as a read-only file server over
Gemini. It will serve up any readable files under its document root
directory (specified at startup) or from the public_gemini directory in
any user's home directory.

If a requested URL maps to a directory, the server looks for an
index.gmi or index.gemini file in that directory. If neither of those
exist, it automatically generates a directory listing in text/gemini
format with links to each of the files and directories within it for
easy filesystem navigation.

Take a look, read the README, and give it a spin. I'd be happy to hear
any feedback you folks might have or ideas for additional features.

Happy hacking,
  Gary

-- 
GPG Key ID: 7BC158ED
Use `gpg --search-keys lambdatronic' to find me
Protect yourself from surveillance: https://emailselfdefense.fsf.org
=======================================================================
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

Please avoid sending me MS-Office attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

On Fri, Jul 10, 2020, 00:39 Gary Johnson <lambdatronic at disroot.org> wrote:

> Space Age is a Gemini server written in Clojure (https://clojure.org/).
>
> It implements Gemini protocol specification v0.14.2 (July 2nd 2020), all
> except for client certificates (which I'm still working on).
>
> The entire code base (including build files) clocks in at a cool 200
> lines with absolutely no dependency libraries.
>

Great work!

I write Clojure at $dayjob and consider it a great fit for full-stack web
(and Gemini by extension). Besides a static Gemini site, I've written one
CGI app in Rust (twinwiki) and one server/app in Python/Jetforce (
ansi.hrtk.in). Both could have been in Clojure if there was a nice library.

For my use RAM consumption is important, though, so I'd need to measure a
bit with GraalVM or possibly a tiny JVM instance. Practically Rust will
always be the better (best?) choice, but it's a bit heavy for prototyping.

Briefly reading the code I was left wondering if java.net.URI wouldn't
parse URIs out of the box. Probably it's just not ergonomic and/or has
weird corner cases...

Happy hacking!

-Hannu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200710/aa14
b6fa/attachment.htm>

Gary Johnson <lambdatronic at disroot.org> writes:

> Space Age is a Gemini server written in Clojure (https://clojure.org/).

Nice, now there are two parenthesis-oriented Gemini servers ;)

-- 
+-----------------------------------------------------------+
| Jason F. McBrayer                    jmcbray at carcosa.net  |
| A flower falls, even though we love it; and a weed grows, |
| even though we do not love it.            -- Dogen        |

On Fri, Jul 10, 2020 at 09:30:20AM -0400, Jason McBrayer wrote:

[...]

> > Space Age is a Gemini server written in Clojure (https://clojure.org/).
>
> Nice, now there are two parenthesis-oriented Gemini servers ;)

And a client too ;-)

Bye!
C.

Hi Hannu,

  Thanks for the feedback. I also write Clojure+Clojurescript for a
living doing full stack web development and scientific modelling in the
environmental arena.

To control RAM consumption by the JVM, you can set -D-Xmx in Space Age's
toplevel deps.edn file. Outside of embedded environments (e.g.,
Raspberry Pis), I suspect the default of a few hundred MBs of RAM
shouldn't be too much of a burden for most laptops, desktops, or VMs,
but with that flag, you can set the JVM's max memory to whatever you
want. GraalVM could be an interesting build target though, so let me
know if have any success compiling Space Age on it.

I'll take a look at java.net.URI to see if it provides any benefit over
my current approach. I suspect I'll still need to perform string
trimming, selective lower-casing for some URI components, and default
values for missing elements as I am doing currently, but if it shaves a
few lines off the code base, that's certainly nothing to complain about.
;)

In other news, I just added a primitive CLJ script running facility to
Space Age. Now all you have to do is drop an executable <whatever>.clj
file into your ~/public_gemini folder (or the document root directory),
and you're off to the races! Any query params from the request URL are
available to the script in a global variable called (creatively)
"request-params". Anything you print to stdout in your script will be
returned as the body of a text/gemini response to the client.

I'm still thinking through just how much additional info I want to pass
to these scripts and how/whether I should let them send non-success
responses, but that's half the fun of it after all.

(hack 'the :planet)
  Gary


Hannu Hartikainen <hannu.hartikainen+gemini at gmail.com> writes:

> On Fri, Jul 10, 2020, 00:39 Gary Johnson <lambdatronic at disroot.org> wrote:
>
>> Space Age is a Gemini server written in Clojure (https://clojure.org/).
>>
>> It implements Gemini protocol specification v0.14.2 (July 2nd 2020), all
>> except for client certificates (which I'm still working on).
>>
>> The entire code base (including build files) clocks in at a cool 200
>> lines with absolutely no dependency libraries.
>>
>
> Great work!
>
> I write Clojure at $dayjob and consider it a great fit for full-stack web
> (and Gemini by extension). Besides a static Gemini site, I've written one
> CGI app in Rust (twinwiki) and one server/app in Python/Jetforce (
> ansi.hrtk.in). Both could have been in Clojure if there was a nice library.
>
> For my use RAM consumption is important, though, so I'd need to measure a
> bit with GraalVM or possibly a tiny JVM instance. Practically Rust will
> always be the better (best?) choice, but it's a bit heavy for prototyping.
>
> Briefly reading the code I was left wondering if java.net.URI wouldn't
> parse URIs out of the box. Probably it's just not ergonomic and/or has
> weird corner cases...
>
> Happy hacking!
>
> -Hannu


-- 
GPG Key ID: 7BC158ED
Use `gpg --search-keys lambdatronic' to find me
Protect yourself from surveillance: https://emailselfdefense.fsf.org
=======================================================================
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

Please avoid sending me MS-Office attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

> On Jul 10, 2020, at 20:38, Gary Johnson <lambdatronic at disroot.org> wrote:
> 
> Any query params from the request URL are
> available to the script in a global variable called (creatively)
> "request-params". Anything you print to stdout in your script will be
> returned as the body of a text/gemini response to the client.
> 
> I'm still thinking through just how much additional info I want to pass
> to these scripts and how/whether I should let them send non-success
> responses, but that's half the fun of it after all.

Perhaps The Common Gateway Interface (CGI) could be of guidance?

Sounds similar in principle.

Yes, these are CGI scripts in practice. I'm just trying not to shell out
to subprocesses in the interest of server responsiveness. I'm currently
just thinking through how to provide as simple and intuitive an
interface as possible for the script writer. Probably it will amount to
some kind of handler function API that takes a request map and returns a
response map in the classic tradition of all ring-based Clojure web
apps.

Scripts are currently run within their own auto-generated temporary
namespaces, which prevents them from accidentally polluting the server's
namespaces with def* forms. Any I/O to stdout is captured in a string
and currently returned as the body of a "20 text/gemini; charset=utf-8"
response. This is fine as a starting point, but I think I want to create
something that feels more functional and less state-oriented pretty
soon.

I'll keep you posted as it progresses.

Happy hacking,
  Gary


Petite Abeille <petite.abeille at gmail.com> writes:

>> On Jul 10, 2020, at 20:38, Gary Johnson <lambdatronic at disroot.org> wrote:
>> 
>> Any query params from the request URL are
>> available to the script in a global variable called (creatively)
>> "request-params". Anything you print to stdout in your script will be
>> returned as the body of a text/gemini response to the client.
>> 
>> I'm still thinking through just how much additional info I want to pass
>> to these scripts and how/whether I should let them send non-success
>> responses, but that's half the fun of it after all.
>
> Perhaps The Common Gateway Interface (CGI) could be of guidance?
>
> Sounds similar in principle.


-- 
GPG Key ID: 7BC158ED
Use `gpg --search-keys lambdatronic' to find me
Protect yourself from surveillance: https://emailselfdefense.fsf.org
=======================================================================
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

Please avoid sending me MS-Office attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

It was thus said that the Great Gary Johnson once stated:
> Yes, these are CGI scripts in practice. I'm just trying not to shell out
> to subprocesses in the interest of server responsiveness. I'm currently
> just thinking through how to provide as simple and intuitive an
> interface as possible for the script writer. Probably it will amount to
> some kind of handler function API that takes a request map and returns a
> response map in the classic tradition of all ring-based Clojure web
> apps.
>
> Scripts are currently run within their own auto-generated temporary
> namespaces, which prevents them from accidentally polluting the server's
> namespaces with def* forms. Any I/O to stdout is captured in a string
> and currently returned as the body of a "20 text/gemini; charset=utf-8"
> response. This is fine as a starting point, but I think I want to create
> something that feels more functional and less state-oriented pretty
> soon.
> 
  GLV-1.12556 [1] uses handlers to generate content.  The builtin handlers
include ones to serve up a file (yup--just one file), a filesystem (handles
serving up data from a directory) and user directories [2].  The API I use
is simple.  The optional functions are:

	okay,err = init(conf)

  This function does any intialization for the handler.  It is given the
configuration parameters from the configuration file and returns 'true' if
success, or 'false' and an error message if it fails.  Typically, if I have
this function, it verifies the configuration passed in to make sure it will
handle requests.  This configuratio block can also be used for state
information for the handler if need be.

	okay,err = fini(conf)

  This function is called when the server is being shut down, and is
intended to do any cleanup (that isn't not already handled by the garbage
collector) or saving any persistent data [3].

  The mandatory function is:

	status,mime,data = handler(conf,auth,location,match)

  This handles a request.  It's given the configuration block, the client
certificate (if any [4]), the parsed URL and the 'match' array, which I
don't know how to explain other than an example, in this case, the
filesystem handler (from my development system configuration file):

	{
	  path      = "^(/cgi%-bin/)(.*)",
          module    = "GLV-1.handlers.filesystem",
          directory = "/home/spc/projects/gemini/non-checkin/cgi-bin",
          cgi       = true,
	}

  The path component is a Lua pattern (a type of regex) to determine which
handler to call for a given request, and it has two parts, the first (which
is captured in ()) determines the "base" path of the request, and the second
is the part the handler is reponsible for.  It's these two captures that are
passed in the 'match' section.

  This function returns a status code, meta information (named mime here)
and a body (which for errors, has to be "") as a string.  Handlers do not
have direct access to the socket.  And a given handler can be instantiated
multiple times, each with its own configuration block.

  As handlers in GLV-1.12556 are also Lua modules, that means they get their
own "global" namespace.  It works and I've had no issue with this structure. 
It also means it's easier for me to write a handler than it is a CGI (or
SCGI) script, although it does mean I have to add a configuration block and
restart the server when adding a new handler.

  -spc

[1]	https://github.com/spc476/GLV-1.12556

[2]	I run more custom handlers than I do builtin ones.

[3]	My "quote of the day" handler will save which quote was last served
	in order to pick up from where it left off.

[4]	Authorization has already been done by the time the request is
	handled so the handler doesn't need to do it, but some handlers
	(like the filesystem handler, which deals with CGI/SCGi) can do some
	additional processing.

On Fri, 10 Jul 2020 at 21:38, Gary Johnson <lambdatronic at disroot.org> wrote:
> To control RAM consumption by the JVM, you can set -D-Xmx in Space Age's
> toplevel deps.edn file.

True, but I've had clojure and java apps crash with OOM in production
by giving them too little. In my experience you need to measure the
RAM consumption with maximum expected load, then add a bit for the
unexpected. Of course for a typical Gemini server that *should* be
very little. Yet I'll need to measure to know. :)

> Outside of embedded environments (e.g.,
> Raspberry Pis), I suspect the default of a few hundred MBs of RAM
> shouldn't be too much of a burden for most laptops, desktops, or VMs,

True, for a single server. But my VPS is soon running four Gemini
servers, some web ones and I have plans for more. The way JVM works it
tends to stay at max allocation, so each JVM instance is a burden.
Thankfully VPSes are huge these days. I used to have one with 64MB
RAM...

But yeah, I'll keep your project in mind. Maybe I'll find the
inspiration to hack with it on GraalVM one of these days!

> (hack 'the :planet)

::awesome!