CGI

Hello all,

I was excited to learn about Gemini recently. I ported some web content of
mine over (gemini://pboyd.io), and I've been enjoying looking through
what's out there.

The Gemini sites (err.. capsules?) remind me of the web in another era, so
I thought it would be fun to re-create some "classic" CGI scripts. I
started with a Guestbook (gemini://pboyd.io/guestbook/, source: gemini://
pboyd.io/guestbook/guestbook.pl). I got it working, but it felt like I was
in the weeds from the very start.

The first problem I hit is that I'm limited to one input at a time. I
wanted a few inputs (name, location, comment). Of course, plenty of command
line programs work fine with a single argument list, so, in theory, that
should be enough. But flags or comma-separated input wasn't what I was
going for. Successive input prompts didn't seem quite right either.

I ended up with a "form" made out of links to input URLs. The data is
collected in a temp file named with a random session ID, and there's a
final "submit" link that updates the generated guestbook (in true retro CGI
fashion it probably fails horribly when two people submit at once).

This works, but like I said, kinda in the weeds. Is there a better way to
get multiple inputs that I missed?

The other problem I ran into is that dynamic responses were cached in my
client. In my case, the user would submit the form and be presented with an
error page. They'd fix the error and resubmit only to get the same error
again because of the client cache.

I worked around that by making sure each generated link is unique (by a
counter). Conditional GET behavior is probably too complicated, but it
would be nice if there were a "Pragma: no-cache" type flag on dynamic pages.

Anyway, am I making this harder than it needs to be? Am I just trying to do
something that shouldn't be done on Gemini?

Cheers,
Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200717/c6a6
09a0/attachment.htm>

One way to do it would be to have multiple inputs in a row,
and store the results in a temp file based on IP address. It
still fails if multiple people from the say IP address submit,
but that's much rarer.

makeworld

That's a good idea. It would probably work fine for this particular case.
Though I'd worry about it for anything more complicated. But I guess that
could be solved by client-side certs.

As an aside, I had a co-worker who made a web game once and added IP-based
rate limiting. The only trouble was that the only people who played his
game were his co-workers who mostly played from the office network, totally
triggering his rate limit.

On Fri, Jul 17, 2020 at 11:35 AM <colecmac at protonmail.com> wrote:

> One way to do it would be to have multiple inputs in a row,
> and store the results in a temp file based on IP address. It
> still fails if multiple people from the say IP address submit,
> but that's much rarer.
>
> makeworld
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200717/f6f3
5f97/attachment.htm>

I like the form idea and I wrote something similar for gopher a while back [0].

https://gopher.floodgap.com/gopher/gw?=mozz.us+7005+312f64656d6f2d666f726d

In my case, I used URL query parameters to keep track of previously submitted
form values. This worked because query params do not have any special meaning
in gopher URLs. For gemini the query component is reserved, but you could
probably accomplish the same thing using a special path component with the
encoded fields in it.

gemini://example.com/cgi-bin/form/name%3Dpaul%26location%3Dcanada/

- mozz

[0] https://github.com/michael-lazar/flask-gopher/blob/master/demo/run_server.py#L154

Hmm.. interesting. I'm already packing data into the URL path, so yeah, I
can just put the rest of the data in there. I'll have to try it out when I
get a minute.

The protocol spec has a max URL size of 1024 bytes, but as long as the
inputs are short I think it will work.

Paul

On Fri, Jul 17, 2020 at 1:37 PM Michael Lazar <lazar.michael22 at gmail.com>
wrote:

> I like the form idea and I wrote something similar for gopher a while back
> [0].
>
> https://gopher.floodgap.com/gopher/gw?=mozz.us+7005+312f64656d6f2d666f726d
>
> In my case, I used URL query parameters to keep track of previously
> submitted
> form values. This worked because query params do not have any special
> meaning
> in gopher URLs. For gemini the query component is reserved, but you could
> probably accomplish the same thing using a special path component with the
> encoded fields in it.
>
> gemini://example.com/cgi-bin/form/name%3Dpaul%26location%3Dcanada/
>
> - mozz
>
> [0]
> https://github.com/michael-lazar/flask-gopher/blob/master/demo/run_server.py#L154
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200717/513a
d82b/attachment.htm>

A word of warning - as you probably have surmised, query string or 
otherwise manipulated URLs are not good for all types of application. 
They work just like HTTP GET queries and as such should really be 
limited to idempotent interactions with the server (like search queries 
or other peristent URLs).

There is nothing yet equivalent to a non-cacheable data submission to 
the server (like HTTP POST) yet. I think it would be great if there 
could be one, but nothing on the horizon at the moment.

So the complex URLs you can assemble in interactions with Gemini are 
subject to being linked to and replayed.

For example if I have a page on my site with 100 parametrised links to a 
gemini based guest book, every time GUS comes to visit, there could end 
up being 100 new entries posted!

Best wishes

  - Luke

On 17-Jul-2020 19:26, Paul Boyd wrote:
> Hmm.. interesting. I'm already packing data into the URL path, so 
> yeah, I can just put the rest of the data in there. I'll have to try 
> it out when I get a minute.
>
> The protocol spec has a max URL size of 1024 bytes, but as long as the 
> inputs are short I think it will work.
>
> Paul
>
> On Fri, Jul 17, 2020 at 1:37 PM Michael Lazar 
> <lazar.michael22 at gmail.com <mailto:lazar.michael22 at gmail.com>> wrote:
>
>     I like the form idea and I wrote something similar for gopher a
>     while back [0].
>
>     https://gopher.floodgap.com/gopher/gw?=mozz.us+7005+312f64656d6f2d666f726d
>
>     In my case, I used URL query parameters to keep track of
>     previously submitted
>     form values. This worked because query params do not have any
>     special meaning
>     in gopher URLs. For gemini the query component is reserved, but
>     you could
>     probably accomplish the same thing using a special path component
>     with the
>     encoded fields in it.
>
>     gemini://example.com/cgi-bin/form/name%3Dpaul%26location%3Dcanada/
>     <http://example.com/cgi-bin/form/name%3Dpaul%26location%3Dcanada/>
>
>     - mozz
>
>     [0]
>     https://github.com/michael-lazar/flask-gopher/blob/master/demo/run_server.py#L154
>

I put all the collected data in the URL. It works pretty well and got rid
of the temp files. It event fixed the cache problem since each generated
URL is unique to the data submitted. But it generates some nasty URLs (see
the bottom).

Luke, good points about the ability to replay a post. Temporary session
files didn't have that problem at least. I suppose I could bring back a
session ID just to counter replays, but I'll wait until it becomes a
problem.

Anyway, I'm probably not going to write something like this again. The
guestbook is kind of fun (Hello K?vin and Timur!), so I'll leave it up. But
I'll stick to things that fit better in the future.

Case in point about replays, here's an ugly URL to a pre-filled form:

gemini://
pboyd.io/cgi-bin/guestbook.pl/link/S2luZyBBcnRodXI=;Q2FtZWxvdA==;VGhlIExhZH
kgb2YgdGhlIExha2UsIGhlciBhcm0gY2xhZCBpbiB0aGUgcHVyZXN0LCBzaGltbWVyaW5nIHNhb
Wl0ZSwgaGVsZCBhbG9mdCBFeGNhbGlidXIgZnJvbSB0aGUgYm9zb20gb2YgdGhlIHdhdGVyIHNp
Z25pZnlpbmcgYnkgRGl2aW5lIFByb3ZpZGVuY2UgdGhhdCBJLCBBcnRodXIsIHdhcyB0byBjYXJ
yeSBFeGNhbGlidXIuIFRoYXQgaXMgd2h5IEkgYW0geW91ciBraW5nIQ==;?none

Paul

On Fri, Jul 17, 2020 at 2:59 PM Luke Emmet <luke at marmaladefoo.com> wrote:

> A word of warning - as you probably have surmised, query string or
> otherwise manipulated URLs are not good for all types of application.
> They work just like HTTP GET queries and as such should really be
> limited to idempotent interactions with the server (like search queries
> or other peristent URLs).
>
> There is nothing yet equivalent to a non-cacheable data submission to
> the server (like HTTP POST) yet. I think it would be great if there
> could be one, but nothing on the horizon at the moment.
>
> So the complex URLs you can assemble in interactions with Gemini are
> subject to being linked to and replayed.
>
> For example if I have a page on my site with 100 parametrised links to a
> gemini based guest book, every time GUS comes to visit, there could end
> up being 100 new entries posted!
>
> Best wishes
>
>   - Luke
>
> On 17-Jul-2020 19:26, Paul Boyd wrote:
> > Hmm.. interesting. I'm already packing data into the URL path, so
> > yeah, I can just put the rest of the data in there. I'll have to try
> > it out when I get a minute.
> >
> > The protocol spec has a max URL size of 1024 bytes, but as long as the
> > inputs are short I think it will work.
> >
> > Paul
> >
> > On Fri, Jul 17, 2020 at 1:37 PM Michael Lazar
> > <lazar.michael22 at gmail.com <mailto:lazar.michael22 at gmail.com>> wrote:
> >
> >     I like the form idea and I wrote something similar for gopher a
> >     while back [0].
> >
> >
> https://gopher.floodgap.com/gopher/gw?=mozz.us+7005+312f64656d6f2d666f726d
> >
> >     In my case, I used URL query parameters to keep track of
> >     previously submitted
> >     form values. This worked because query params do not have any
> >     special meaning
> >     in gopher URLs. For gemini the query component is reserved, but
> >     you could
> >     probably accomplish the same thing using a special path component
> >     with the
> >     encoded fields in it.
> >
> >     gemini://example.com/cgi-bin/form/name%3Dpaul%26location%3Dcanada/
> >     <http://example.com/cgi-bin/form/name%3Dpaul%26location%3Dcanada/>
> >
> >     - mozz
> >
> >     [0]
> >
> https://github.com/michael-lazar/flask-gopher/blob/master/demo/run_server.py#L154
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200718/9399
e640/attachment.htm>

> On Jul 18, 2020, at 15:46, Paul Boyd <boyd.paul2 at gmail.com> wrote:
> 
> I put all the collected data in the URL. It works pretty well and got 
rid of the temp files. It event fixed the cache problem since each 
generated URL is unique to the data submitted. But it generates some nasty 
URLs (see the bottom).

Ugly indeed.

Also, keep in mind that URLs are limited to 1024 bytes in Gemini.

On Saturday, July 18, 2020 3:46 PM, Paul Boyd <boyd.paul2 at gmail.com> wrote:

> I put all the collected data in the URL. It works pretty well and got 
rid of the temp files. It event fixed the cache problem since each 
generated URL is unique to the data submitted. But it generates some nasty 
URLs (see the bottom).

Consequence is heavy url, possibly bigger than the resulting server response.
Session token and server side temporary file is nicer on network usage...

Maybe in the future, the spec could evolve to allow efficient form, 
managed by the gemini client and submitted to the server without the need 
of multiple connection.

freD.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200718/47bd
f7e3/attachment.htm>

Hi!

On Sat, 18 Jul 2020 at 16:47, Paul Boyd <boyd.paul2 at gmail.com> wrote:
> Anyway, I'm probably not going to write something like this again. The 
guestbook is kind of fun (Hello K?vin and Timur!), so I'll leave it up. 
But I'll stick to things that fit better in the future.

I think it's really valuable to *try* all kinds of things and
different approaches to see *what* fits well. IMHO we have only
scratched the surface of what is possible with the Gemini protocol
(and what makes for good UX).

To summarize what has been discussed here:
1. Filling in multiple values is not trivial with the input response type
2. One way is to generate a session id on the server. (This is
stateful, which has some downsides but avoids replay issues.)
3. Another way is to embed all the responses in the URL.

Some ideas that haven't been mentioned:

a. If you include all the fields at once, you could just ask for them
in one request. Eg. "10 name;location;link;comment" which the user is
smart enough to parse and respond with "Hannu;;;This is cool!". Or for
a better UX, make the fields newline-separated (ie. %0A in the URL)
and have multiline-capable input fields in browsers. Note: this is
against a strict interpretation of the spec but hasn't been discussed
AFAIK.

b. Instead of having different links for different fields, you could
ask for the fields in succession (but this only works well for the
session id based approach). Eg. /guestbook/sign responds with "30
/guestbook/sign/aF3d", then /guestbook/sign/aF3d responds with "10
Name", /guestbook/sign/aF3d?Hannu with "10 Location (optional)", then
"10 Link (optional)" and "10 Comment" in sequence, saving each query
string to the server-side session. After the final response either
save the comment or show the responses with a link to save.

I personally think the Gemini protocol (as it currently is) can be as
powerful as the UNIX command line (note: only CLI, not TUI). In fact,
you could serve a remote shell over Gemini. Of course we're all
spoiled by interactive software even in a terminal so Gemini feels
quite limited.

-Hannu

I did consider a succession of inputs. I think we can all agree that a
succession of modal inputs is irritating, and the UI would be exactly that
on most clients.

Asking the user to use a separator seemed error prone, so I didn't want to
do that either. I know it can work. makeworld's gemlikes has a prompt of:

> Enter your username, a space, and your comment

Usernames don't have spaces, and comments do, so this works great here. I'm
just not sure people are going to get "name;location;link;comment" right
consistently. I certainly wouldn't, I'd end up with "paul;;my comment"
instead of "paul;;;my comment" and get my comment in the link field.

Your UNIX command-line comment is interesting. One set of input args is
enough for UNIX, so maybe getopt style flags?

--name John --location "From Beyond" --comment "This is what I have to say"

A form still seems right for a guestbook (the real-life version is a form
after all). But there ought to be something interesting we can do with
flags on the input line. Maybe an ASCII art drawing program? With inputs
like "line --from 0,0 --to 10,10".

Paul

On Sun, Jul 19, 2020 at 5:28 AM Hannu Hartikainen <
hannu.hartikainen+gemini at gmail.com> wrote:

> Hi!
>
> On Sat, 18 Jul 2020 at 16:47, Paul Boyd <boyd.paul2 at gmail.com> wrote:
> > Anyway, I'm probably not going to write something like this again. The
> guestbook is kind of fun (Hello K?vin and Timur!), so I'll leave it up. But
> I'll stick to things that fit better in the future.
>
> I think it's really valuable to *try* all kinds of things and
> different approaches to see *what* fits well. IMHO we have only
> scratched the surface of what is possible with the Gemini protocol
> (and what makes for good UX).
>
> To summarize what has been discussed here:
> 1. Filling in multiple values is not trivial with the input response type
> 2. One way is to generate a session id on the server. (This is
> stateful, which has some downsides but avoids replay issues.)
> 3. Another way is to embed all the responses in the URL.
>
> Some ideas that haven't been mentioned:
>
> a. If you include all the fields at once, you could just ask for them
> in one request. Eg. "10 name;location;link;comment" which the user is
> smart enough to parse and respond with "Hannu;;;This is cool!". Or for
> a better UX, make the fields newline-separated (ie. %0A in the URL)
> and have multiline-capable input fields in browsers. Note: this is
> against a strict interpretation of the spec but hasn't been discussed
> AFAIK.
>
> b. Instead of having different links for different fields, you could
> ask for the fields in succession (but this only works well for the
> session id based approach). Eg. /guestbook/sign responds with "30
> /guestbook/sign/aF3d", then /guestbook/sign/aF3d responds with "10
> Name", /guestbook/sign/aF3d?Hannu with "10 Location (optional)", then
> "10 Link (optional)" and "10 Comment" in sequence, saving each query
> string to the server-side session. After the final response either
> save the comment or show the responses with a link to save.
>
> I personally think the Gemini protocol (as it currently is) can be as
> powerful as the UNIX command line (note: only CLI, not TUI). In fact,
> you could serve a remote shell over Gemini. Of course we're all
> spoiled by interactive software even in a terminal so Gemini feels
> quite limited.
>
> -Hannu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200719/840f
7d8a/attachment.htm>

On Sun Jul 19, 2020 at 2:59 PM CEST, Paul Boyd wrote:
> I did consider a succession of inputs. I think we can all agree that a
> succession of modal inputs is irritating, and the UI would be exactly
> that
> on most clients.

To my mind this is the canonical way of getting multi-part input: use
client certificates to establish a persistent session and then just have
the server send successive responses with status code 10.

I agree this would be irritating in a GUI client that pops up a box akin
to a HTTP basic auth login box every time, but that's more an argument
against that being a good UI design than against this being a good way
to handle this kind of input.  This approach is what the protocol makes
natural/obvious, and so it makes sense to design UIs to match it.

In an UI like AV-98, this approach works exceptionally well and provides
a look and feel that's almost indistinguishable from using a shell, or a
terminal application with a REPL interface.  That's a very powerful
thing which we still haven't even begun to experiment with the power of.

I realise something like AV-98 is not a comfortable "daily reading"
interface for most people.  Of course, there are probably ways to make
this approach pleasant to use in a GUI client - people are very welcome
to work on that.  I also still think that having separate clients for
reading static text and using remote applications makes a lot of sense.
It's not necessarily a good match to quick and easy, low-effort
interactivity - but then, that was never the goal.

Cheers,
Solderpunk

You could have a sort of landing page for the form, using client 
certificates, that looks something like

Foo: $foo
=> /form/foo Enter Foo

Bar: $bar
=> /form/bar Enter Bar

=> /form/submit Submit Form

where it displays the current value (if it exists), each associated link 
allows that value to be input/changed, and then the submit link marks the 
whole form as ready to be processed. You could even substitute "Please 
fill out all fields" or something for the last link until the required 
information is all available.

On July 19, 2020 10:01:51 AM EDT, Solderpunk <solderpunk at posteo.net> wrote:
>On Sun Jul 19, 2020 at 2:59 PM CEST, Paul Boyd wrote:
>> I did consider a succession of inputs. I think we can all agree that
>a
>> succession of modal inputs is irritating, and the UI would be exactly
>> that
>> on most clients.
>
>To my mind this is the canonical way of getting multi-part input: use
>client certificates to establish a persistent session and then just
>have
>the server send successive responses with status code 10.
>
>I agree this would be irritating in a GUI client that pops up a box
>akin
>to a HTTP basic auth login box every time, but that's more an argument
>against that being a good UI design than against this being a good way
>to handle this kind of input.  This approach is what the protocol makes
>natural/obvious, and so it makes sense to design UIs to match it.
>
>In an UI like AV-98, this approach works exceptionally well and
>provides
>a look and feel that's almost indistinguishable from using a shell, or
>a
>terminal application with a REPL interface.  That's a very powerful
>thing which we still haven't even begun to experiment with the power
>of.
>
>I realise something like AV-98 is not a comfortable "daily reading"
>interface for most people.  Of course, there are probably ways to make
>this approach pleasant to use in a GUI client - people are very welcome
>to work on that.  I also still think that having separate clients for
>reading static text and using remote applications makes a lot of sense.
>It's not necessarily a good match to quick and easy, low-effort
>interactivity - but then, that was never the goal.
>
>Cheers,
>Solderpunk

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200719/b5c5
4698/attachment.htm>

On Sun, 19 Jul 2020 at 17:08, Solderpunk <solderpunk at posteo.net> wrote:
> In an UI like AV-98, this approach works exceptionally well and provides
> a look and feel that's almost indistinguishable from using a shell, or a
> terminal application with a REPL interface.  That's a very powerful
> thing which we still haven't even begun to experiment with the power of.

I was going to post something like this. The paradigm chosen by Gemini
basically prevents GUI or TUI like applications, but it's great for
console use. The web is already a universal GUI application platform,
so Gemini can be something else.

An interaction method like the one in AV-98 would be impossible for
the web but it's amazing for Gemini. I believe this is a strength
we're only beginning to understand. For example, consider the huge
rise of podcasts and audiobooks: people love to listen to information.
With Gemini it would be trivial to browse with a headset while
commuting. The web, on the other hand, is horrible for screen reader
use (ask any blind person).

On Sun, 19 Jul 2020 at 15:59, Paul Boyd <boyd.paul2 at gmail.com> wrote:
> Your UNIX command-line comment is interesting. One set of input args is 
enough for UNIX, so maybe getopt style flags?
>
> --name John --location "From Beyond" --comment "This is what I have to say"

I like this idea! It also makes me wonder if single-line-capable data
formats would be useful. Eg. edn {:name "Hannu" :comment "Hello
world!"} and YAML {name: Hannu, comment: Hello world!}. (Both do
suffer from unfamiliarity, though.) There must be a huge array of
alternatives, most of them better than my CSV idea.

-Hannu

* Sunday, 2020-07-19 at 16:01 +0200 - Solderpunk <solderpunk at posteo.net>:

>To my mind this is the canonical way of getting multi-part input: use 
>client certificates to establish a persistent session and then just 
>have the server send successive responses with status code 10.
>
>In an UI like AV-98, this approach works exceptionally well and provides 
>a look and feel that's almost indistinguishable from using a shell

*Except* that you can only have one line of output between each line of 
input, because 10 doesn't allow a body. This is rather limiting.

Other than adding a new response code, the nicest way I see around this 
is a convention that clients should make it easy to rerequest the last 
URI with a new query parameter. This could be done for example with 
a user-enabled "REPL mode" in which the client repeatedly accepts a line 
of input from the user and uses it as a query parameter in a new request 
based on the current URI.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200719/3430
29d9/attachment.sig>

On Sun, 2020-07-19 at 17:50 +0200, mbays at sdf.org wrote:
> *Except* that you can only have one line of output between each line
> of 
> input, because 10 doesn't allow a body. This is rather limiting.
> 

I think this can work because you use redirects in between (which is
also what you need to do if you want to encode the answers in the URL).

/story/123/comment/add

20 text/gemini\r
Hello! We're going to ask you for a temporary certificate to establish
a session, your name, and the comment, OK? If you're ready, click the
link below!
=> ready

/story/123/comment/ready

60 Please tell your client to generate a temporary certificate\r

/story/123/comment/ready

20 text/gemini\r
Excellent! First your name, then the comment, OK? If you're ready,
follow the link below.
=> name

/story/123/comment/name

10 Your name? \r

30 gemini://example.org/story/123/comment/name-ok

/story/123/comment/name-ok

20 text/gemini\r
Excellent, Alex. Thanks! Now for the comment. Please remember that you
have about 900 characters. Follow the link below when you're ready.
=> comment

10 Let's have that comment!\r

30 gemini://example.org/story/123/comment/comment-ok

/story/123/comment/comment-ok

All-right! Got your comment. This is what we're going to save:

"I love your post. Viagra cheap! -- Alex"

Click the link below to save.
=> save

30 gemini://example.org/story/123

20 text/gemini\r
Bla bla bla...