Script to test gemini servers

Hi all,

I have been working on a python script that can be pointed to a gemini server,
and will automatically send various types of requests to surface errors and
other strange server behavior. This was suggested in the mailing list a while
ago by solderpunk(?). I thought it was a neat idea so I took my own stab at it.

I am not attempting to cover 100% of the gemini specification. Some of the
checks aren't even related to the spec, and are strictly informational.
Take all results with a grain of salt.

For example:

- Does your server support IPv6?
- What version of TLS do you negotiate?
- Is your certificate's CA trusted?
- How do you respond to a http:// URL?
- How do you respond to a URL with an invalid UTF-8 byte sequence?
- How do you respond to a request with a newline but no carriage return?

The program can be found here:

https://github.com/michael-lazar/jetforce/blob/master/jetforce_diagnostics.py

I've included it in the jetforce repo out of convenience, but the script can
be downloaded and run independently of jetforce. The only hard dependency is
Python 3.7+. The python "pyca/cryptography" library is an optional requirement,
if you want to inspect server TLS certificates.

Usage:
  ./jetforce_diagnostics.py host [port]

This is very much still considered a work-in-progress. If you have any
suggestions or would like to contribute additional tests to run, please feel
encouraged to respond here or open an issue or PR in the github repo.

To save you all the trouble, I have already run this against all known gemini
servers and published the results on my server. This is not an attempt to
shame anybody, my own server exposed several failures that I now need to fix.
But I think there are some interesting insights to be gleaned about how servers
are implemented in practice, and how the gemini spec might be improved:

gemini://mozz.us/diagnostics/

Some of the results:

- All servers use "\n" line endings on their home page instead of "\r\n"
- No server will accept a URL without a scheme specified (per the spec, this
  should be implied as gemini://)
- Only half of servers currently support TLS version 1.3

- mozz

It was thus said that the Great Michael Lazar once stated:
> Hi all,

  Hello.

> I have been working on a python script that can be pointed to a gemini server,
> and will automatically send various types of requests to surface errors and
> other strange server behavior. This was suggested in the mailing list a while
> ago by solderpunk(?). I thought it was a neat idea so I took my own stab at it.

  Thanks for doing this.  I find the results interesting, and yes, you did
find three bugs in GLV-1.12556.

> For example:
> 
> - Does your server support IPv6?

  Mine (GLV-1.12556) does, but my hosting provider doesn't have IPv6 yet (or
maybe they do---it's been awhile since I last checked).

> - What version of TLS do you negotiate?

  1.2---it's a limitation of the library I'm using.

> - Is your certificate's CA trusted?

  No.  I'm the one using a custom CA.

> - How do you respond to a http:// URL?

  I reject them.

> - How do you respond to a URL with an invalid UTF-8 byte sequence?

  Here's one of the bugs, and I have no idea how I'll fix this as it's a
cross-library concern.  Sigh.

> - How do you respond to a request with a newline but no carriage return?

  I respond.  I made CRs optional everywhere internally---it makes it easier
for me to test.  Now, you did make one test labeled "RequestMissingCR" where
you did send a LF.  Per the spec, I would expect this to return a 59 Bad
Request, but since I treat CRs as optional, that's why I passed that one.

> Some of the results:
> 
> - All servers use "\n" line endings on their home page instead of "\r\n"
> - No server will accept a URL without a scheme specified (per the spec, this
>   should be implied as gemini://)

  This was another bug.  I've already fixed this, and the one other bug (I
allow URLs longer than 1024 characters, which has been fixed for now).

  Again, thanks for this.

  -spc

On Thu, Jan 09, 2020 at 02:11:32AM -0500, Sean Conner wrote:
> 
>   Again, thanks for this.
>

Yes, thanks a lot for this, because I'm continuing to do a bad job of
pushing this idea to completion!

If you're feeling motivated, some things which my rough stab at this
tested which you didn't mention are:

* What happens if the client sends a totally empty request?
* What happens if the client sends a non-empty request which is not a
  URL of any kind, like "Hello, Gemini!\r\n"?
* What happens if the client sends a non-empty request with a valid URL
  which is longer than 1024 characters?

All of these should result in a status code of 59.

Cheers,
Solderpunk

On Thu, Jan 9, 2020 at 2:11 AM Sean Conner <sean at conman.org> wrote:
>
> It was thus said that the Great Michael Lazar once stated:
> > - How do you respond to a request with a newline but no carriage return?
>
>   I respond.  I made CRs optional everywhere internally---it makes it easier
> for me to test.  Now, you did make one test labeled "RequestMissingCR" where
> you did send a LF.  Per the spec, I would expect this to return a 59 Bad
> Request, but since I treat CRs as optional, that's why I passed that one.

It's interesting to hear that this was intentional on your part. You're not
alone on this, 3/5 servers that I tested didn't care about the CR at all. My
own server does require the CR, but I didn't like writing it that way and felt
like I had to jump through unnecessary hoops to conform to the specification:

    data = await self.reader.readuntil(b"\r\n")

instead of the more straightforward:

    data = await self.reader.readline()

I would prefer it if CR was optional. Similar to how <whitespace> is already
defined as "any non-zero number of consecutive spaces or tabs". The same
argument applies here. Content authors can't visually distinguish newline
characters in text/gemini files, and I can't tell the difference in my terminal
when I'm inspecting a gemini request.

   <newline> = [<CR>]<LF>

- mozz