Uploading content over gemini

Hey all —  
I’ve been thinking about how since I’ve followed Gemini, there have been a 
number of ideas for “writing” gemini — several different alternative 
protocols. I run flounder.online, and I’ve found these ideas compelling, 
but I haven’t really seen any of them take off. 

However, one idea I had was to use Gemini itself, without modification, to 
handle uploading content, ie a file. I have a couple of ideas:

1. sending several  <META> In 1024 bytes to write an entire file.
2. For a text file, sending a diff including a line number and a line of 
text up to 1024 bytes. Repeat this for each diff.

While both of these are technically feasible, they are a bit wonky, and 
for large files would involve many requests. What I’m wondering is whether 
these are ideas worth considering — ways to upload files without extending 
or replacing Gemini. Any thoughts?

Thanks

Alex
alex.flounder.online

On Fri, Nov 12, 2021 at 5:03 AM Alex <alex@alexwennerberg.com> wrote:
> However, one idea I had was to use Gemini itself, without modification, 
to handle uploading content, ie a file. I have a couple of ideas:
>
> 1. sending several  <META> In 1024 bytes to write an entire file.
> 2. For a text file, sending a diff including a line number and a line of 
text up to 1024 bytes. Repeat this for each diff.
>
> While both of these are technically feasible, they are a bit wonky, and 
for large files would involve many requests. What I’m wondering is whether 
these are ideas worth considering — ways to upload files without extending 
or replacing Gemini. Any thoughts?

Skyjake linked something Solderpunk wrote about 1,5 years ago
(gemini://gemi.dev/gemini-mailing-list/messages/001672.gmi) in
another thread:

> It should also be very easy to write an app targetted at "non-technical"
> authors which lets them submit chunks of writing up to 2 KiB or so, with
> an "append" link at the submission confirmation page to submit a follow
> up chunk.  It wouldn't necessarily be the smoothest experience in the
> world, but if most content could be written in a single request and 99%
> with one or two "append" requests, I think it would be usable enough.
> Heck, this is the "slow internet", right?  A little bit of inconvenience
> as part of a careful and deliberate process should not scare us away.
>
> In general, solving perceived problems with the limitations that Gemini
> imposes by combining the "primitives" which are already there in
> creative new ways, even if they are very slightly clunky, makes me far,
> far happier than adding in additional more advanced features to remove
> those limitations.  If we discover really useful and generally
> applicable constructions that can be built in this way, we can give them
> names, standardise them, and clients can choose to impelement them in
> ways which hide the clunkiness from the user.  It would be wonderful,
> though, if they were still usable in a clunky way by a knowledgable
> users in clients which didn't support them explicitly.

What his and your ideas have in common is that they push complexity to
the client. In my opinion, this complexity should be in the
server-side application, keeping both servers and clients simple.
Simplicity of client and server implementation is one of the core
ideas of Gemini. If there are multiple competing clients and servers,
each with at least some market share, this ensures that no client or
server can unilaterally extend the protocol or the gemtext format
(with e.g. tracking, making multiple requests behind the user's back
etc.). If a capsule - a wiki or a gemlog or whatever - decides to
expect chunked input or diffs that only certain clients can provide,
then some clients will be "more equal than others", to quote a
classic.

But if we want the complexity of uploading/writing/editing content to
reside in the application, and we also want to avoid the problem
detailed above, we have two options left:
* Use a "back channel" different from Gemini (e.g. an HTTP API, Titan,
FTP, etc.) to upload
* Introduce a new, POST-like way to upload via the Gemini protocol itself.

While ideas to do the second have been consistently rejected in the
past, let me try to argue for why we should reconsider it. The FAQ
says:

> Alternative, simple-by-design protocols like Gopher and Gemini create
> alternative, simple-by-design spaces with obvious boundaries and
> hard restrictions. You know for sure when you enter Geminispace,
> and you can know for sure and in advance when following a certain
> link will cause you leave it. While you're there, you know for sure and
> in advance that everybody else there is playing by the same rules. You
> can relax and get on with your browsing, and follow links to sites you've
> never heard of before, which just popped up yesterday, and be
> confident that they won't try to track you or serve you garbage because
> they *can't*. You can do all this with a client you wrote yourself, so you
> *know* you can trust it. It's a very different, much more liberating and
> much more empowering experience than trying to carve out a tiny,
> invisible sub-sub-sub-sub-space of the web.

I believe this is another core value of Gemini - that it should be a
self-contained space. But if the protocol itself doesn't allow a
simple way to author content, people will need to either use back
channels or purpose-built clients that implement chunking, chunked
diffs or whatever. If the protocol itself defined a (very simple but
convenient for both implementers and end-users) method to do so, it
would solve the self-containment problem. Making the protocol a tiny
bit more complex is well worth it in my opinion to avoid ad-hoc,
convention-based client implementations that would make simpler
clients incompatible with some server side apps (or at least extremely
inconvenient to use - you would need to manually chunk the content,
create a diff etc.).

I'm relatively new to Geminispace so maybe this discussion has already
been had. If so, please disregard what I wrote, I don't want to repeat
a debate that has already come to a conclusion.

What do you think?

I think that Titan exists and is on the same level of complexity as 
Gemini, and it’s used in Lagrange, Elpher with Gemini Write, and a bunch 
of command line tools.

I think the details of POST requests are tricky: another encoding, 
parameters in the headers, parameters in the body. Even a PUT might be 
simpler. But again, you have header parameters to consider. In a way, 
Titan is just a PUT request with a fixed number of headers (parameters). 
It’s the perfect trade off.

But as I wrote on https://alexschroeder.ch/wiki/2021-11-05_Capsules_for_Phoebe:

“after yet another discussion on the Gemini IRC channel where people said 
that the Titan protocol was a bad fit for Gemini, or that it was only fit 
for the particular wiki needs I had, or that writing stuff on Gemini via 
the web or uploading files via FTP or SCP was good enough. Well, I 
disagree. And not only do I disagree, but I wrote plenty of solutions:

Titan, obviously – I made it central to Phoebe; I wrote a command line 
client; I wrote an extension called Gemini Write for Elpher, an Emacs-based Gemini client
Spartan – out of curiosity, I implemented a Spartan plugin for Phoebe; I 
wrote a command line client
HTTPS – I implemented a web plugin for Phoebe, both for browsing and for editing
WebDAV – I implemented a WebDAV plugin for Phoebe and made sure it worked 
for both cadaver and Gnome Files
Hopefully it is understandable that I have little patience for people 
complaining about the existing solutions. There are so many of them, all I 
want to see is people supporting anything, anything at all, for their clients.

Luckily, thankfully, @jk has added Titan support to Lagrange.”

I would love for these endless circular discussions to end one day and for 
people to just write the client they want. Perhaps this is the main 
problem: people keep thinking about new ways to upload content because 
they don’t see many clients supporting it, thinking that the existing 
ideas must be too complex. But that is not true. I even implemented the 
Iapetus proposal for both client and server! All these proposals don’t 
need to be discussed, they just need to be implemented – and as I have 
shown, I think they can be implemented faster than they can be discussed 
on the mailing list, IRC, or anywhere else.

Just do it.

I ended up writing instructions on how to use it for my capsules (i.e. not 
wikis), in case you haven’t tried Lagrange:

https://transjovian.org:1965/phoebe/page/Capsules

https://transjovian.org:1965/phoebe/page/Capsule%20update%20using%20Lagrang
e%20on%20the%20phone

https://transjovian.org:1965/phoebe/page/Capsule%20update%20using%20Lagrang
e%20on%20the%20desktop

Here’s the Emacs Lisp code I used to add Titan to Elpher:

https://alexschroeder.ch/cgit/gemini-write/about/

Here’s a short video of how it can look:

https://alexschroeder.ch/videos/gemini_write_demo.mp4

Cheers

Alex

On Fri, Nov 12, 2021 at 10:24 AM Alex Schroeder <alex@alexschroeder.ch> wrote:
>
> “after yet another discussion on the Gemini IRC channel where people 
said that the Titan protocol was a bad fit for Gemini, or that it was only 
fit for the particular wiki needs I had, or that writing stuff on Gemini 
via the web or uploading files via FTP or SCP was good enough. Well, I 
disagree. And not only do I disagree, but I wrote plenty of solutions:
> (...)
> I would love for these endless circular discussions to end one day and 
for people to just write the client they want. Perhaps this is the main 
problem: people keep thinking about new ways to upload content because 
they don’t see many clients supporting it, thinking that the existing 
ideas must be too complex. But that is not true. I even implemented the 
Iapetus proposal for both client and server! All these proposals don’t 
need to be discussed, they just need to be implemented – and as I have 
shown, I think they can be implemented faster than they can be discussed 
on the mailing list, IRC, or anywhere else.
>
> Just do it.

Fair enough, can't really argue with that. Looks like I need to learn
more about existing solutions, especially Titan. Or implement
something else if I think something else would be better.

On Thu, 11 Nov 2021 23:03:02 -0500
Alex <alex@alexwennerberg.com> wrote:

> While both of these are technically feasible, they are a bit wonky, and 
for large files would involve many requests. What I’m wondering is whether 
these are ideas worth considering — ways to upload files without extending 
or replacing Gemini. Any thoughts?

Hi Alex,
I think it's fine if Gemini remains a tool primarily to download and
display text and other files. As such, there is no need to "replace"
Gemini because it's out of scope of the protocol. That said, and I've
think I've actually seen an example of this before, within the current
spec you could implement a line oriented editor, in the style of ed(1)
using 1x status responses. Documents could then not just be uploaded,
but authored directly within Gemini.

A sufficiently smart client™ could format a local document as a
sequence of inputs to such a line based editor. For that kind of use it
doesn't need to be complex. Just a way to open an editing session, a
way to open a new line, a way to append to the current line and a way
to terminate the editing session. E.g.

c: gemini://example.com/document/edit
s: 3x to gemini://example.com/document/edit/session/afdef947f9
s: Repeated 1x responses until terminated or session timeout

oHello! this is the first line, opened with "o"
+ and this is its continuation, added with "+"
oThis is the next line.
oThe next input after this terminates the session, with ".":
.

This will of course generate many requests, but at the cost of sensible
human use you could allow escape codes to indicate line feeding, so you
could use all the space allowed in a request

Another idea is to use email. Blog-by-email if you will, or
comment-by-email could be implemented as an automatic process. But
running an email server may not be a walk in the park. This seems more
sensible to me, but I'm not looking to run my own mailserver. Maybe a
tool could regularly connect to an existing IMAP server and work with
that.

-- 
Philip

> Another idea is to use email. Blog-by-email if you will, or
> comment-by-email could be implemented as an automatic process. But
> running an email server may not be a walk in the park. This seems more
> sensible to me, but I'm not looking to run my own mailserver. Maybe a
> tool could regularly connect to an existing IMAP server and work with
> that.

Ooh... I'd been trying to think of a way to implement uploading. This 
actually seems like a better idea than what I had come up with (posting a 
URL and having the Gemini server download it). It does present the problem 
of the possibility that the email would be sent in an insecure manner though.

--
Regards,
Jonathan Lamothe

Sent from my mobile device.

Oops... accidentally replied from the wrong email address (one that isn't 
on this list). Let me try again.

> Another idea is to use email. Blog-by-email if you will, or
> comment-by-email could be implemented as an automatic process. But
> running an email server may not be a walk in the park. This seems more
> sensible to me, but I'm not looking to run my own mailserver. Maybe a
> tool could regularly connect to an existing IMAP server and work with
> that.


Ooh... I'd been trying to think of a way to implement uploading. This 
actually seems like a better idea than what I had come up with (posting a 
URL and having the Gemini server download it). It does present the problem 
of the possibility that the email would be sent in an insecure manner though.

--
Regards,
Jonathan Lamothe

Sent from my mobile device.