Syntax highlighting

I'm interested in opinions on syntax highlighting for preformatted text. 
Gemini has a lot of potential for distraction-free developer reference (I 
just discovered that godocs.io works in Gemini!) and syntax highlighting would help.

In Markdown, you can add hints to preformatted text blocks like so:

 ```html
<p>This is HTML</p>
 ```

This tells the client that the preformatted block is HTML, so the client 
can present the code in whatever way makes the most sense. Usually this is 
syntax highlighting, but I could imagine some clients adding collapsable 
code blocks based on tags, braces, and/or indentation depending on the language.

The Gemtext spec would seem to have no problem with this, as the first 
line is considered "alt text" and it is up to the client to determine both 
what should be done with it and how it should affect the presentation. But 
I wanted to put this out there to get a broader perspective.

I've started on a patch for adding this to amfora, since the chroma 
library (also used by Hugo) can do syntax highlighting with ANSI escape 
codes, for use in terminals. It works, but it's got a couple of rendering 
bugs at the moment. Discussion and screenshot here: 
https://github.com/makeworld-the-better-one/amfora/issues/252

mntn@mntn.xyz writes:

> I'm interested in opinions on syntax highlighting for preformatted
> text. Gemini has a lot of potential for distraction-free developer
> reference (I just discovered that godocs.io works in Gemini!) and
> syntax highlighting would help.
> 
> [...]
> 
> The Gemtext spec would seem to have no problem with this, as the first
> line is considered "alt text" and it is up to the client to determine
> both what should be done with it and how it should affect the
> presentation. But I wanted to put this out there to get a broader
> perspective.

There was a thread on this topic some months back:

 gemini://gemi.dev/gemini-mailing-list/messages/005732.gmi


Alexis.

> On Sep 12, 2021, at 11:14 AM, mntn@mntn.xyz wrote:
> 
> I'm interested in opinions on syntax highlighting for preformatted text. 
Gemini has a lot of potential for distraction-free developer reference (I 
just discovered that godocs.io works in Gemini!) and syntax highlighting would help.
> 
> In Markdown, you can add hints to preformatted text blocks like so:
> 
> ```html
> <p>This is HTML</p>
> ```
> 
> This tells the client that the preformatted block is HTML, so the client 
can present the code in whatever way makes the most sense. Usually this is 
syntax highlighting, but I could imagine some clients adding collapsable 
code blocks based on tags, braces, and/or indentation depending on the language.
> 
> The Gemtext spec would seem to have no problem with this, as the first 
line is considered "alt text" and it is up to the client to determine both 
what should be done with it and how it should affect the presentation. But 
I wanted to put this out there to get a broader perspective.

Sounds like a neat idea. I can imagine _someone_ wouldn’t like it on the 
grounds that it’s on the noisy side color-wise, but a disabling toggle 
switch sounds like the sort of thing a buried preference would be for.

Hi mntn,

I have been wondering about the use of alttext details for formatting.

In the next couple of months I shall be experimenting with Parsing 
Expression Grammars. The idea is to build PEGs, using content from 
manpages and cheatsheets. I plan on doing it in the (regex and 
multidimensional array friendly) Lisp, TXR.

Hopefully it may be possible to identify when a command is being issued 
and deduce the format of the outputted code.

Of course, providing a switch 'sh' could open up things.

Even more so providing something with precision of software stacks may 
prove even more fruitful

 ``` guix environment --ad-hoc python2-numpy python-2.7 -- python
print("Hey, I wonder: Is this more interesting than using Orgmode?")
 ```

Some ideas for a tonk client:
* Reading that block and even generating an additional block with the 
(referenced) output below
* Splitting a block containing both input commands and the outputs into 
seperate (and referenced) blocks

Should I make any headway Ill be sure to inform the ML.

====================
Jonathan McHugh
indieterminacy@libre.brussels

September 12, 2021 8:14 PM, mntn@mntn.xyz wrote:

> I'm interested in opinions on syntax highlighting for preformatted text. 
Gemini has a lot of
> potential for distraction-free developer reference (I just discovered 
that godocs.io works in
> Gemini!) and syntax highlighting would help.
> 
> In Markdown, you can add hints to preformatted text blocks like so:
> 
> ```html
> <p>This is HTML</p>
> ```
> 
> This tells the client that the preformatted block is HTML, so the client 
can present the code in
> whatever way makes the most sense. Usually this is syntax highlighting, 
but I could imagine some
> clients adding collapsable code blocks based on tags, braces, and/or 
indentation depending on the
> language.
> 
> The Gemtext spec would seem to have no problem with this, as the first 
line is considered "alt
> text" and it is up to the client to determine both what should be done 
with it and how it should
> affect the presentation. But I wanted to put this out there to get a 
broader perspective.
> 
> I've started on a patch for adding this to amfora, since the chroma 
library (also used by Hugo) can
> do syntax highlighting with ANSI escape codes, for use in terminals. It 
works, but it's got a
> couple of rendering bugs at the moment. Discussion and screenshot here:
> https://github.com/makeworld-the-better-one/amfora/issues/252

Thanks, I read over it and the deeper linked conversations. It looks like 
people have been considering this from a number of angles. It does seem 
like a lot of people were suggesting changes to the spec; I don't want to 
add anything new! Gemini is fine as it is.

I am confident that the best option is just to use the name of the 
language as the alt text. Alt text of "javascript" is fairly clear as to 
what follows in the preformatted text block--so it still works as alt 
text--and a syntax highlighter could use that hint for parsing the text 
and highlighting the code. A properly designed syntax highlighter will 
give up and fail quietly/safely if it can't parse the code for whatever 
reason, say if it's actually a poem titled "javascript" instead of code.

Clients may want to ignore anything following a space or punctuation, so 
alt text could be even more descriptive, such as "javascript: excerpt from 
example.js" or "HTML example."

This would seem to be compatible with the spec as it stands, it doesn't 
munge the alt text, and it doesn't add any new requirements for clients.


September 12, 2021 5:57 PM, "Alexis" <flexibeast@gmail.com> wrote:

> mntn@mntn.xyz writes:
> 
>> I'm interested in opinions on syntax highlighting for > preformatted
>> text. Gemini has a lot of potential for distraction-free > developer
>> reference (I just discovered that godocs.io works in Gemini!) > and
>> syntax highlighting would help.
>> 
>> [...]
>> 
>> The Gemtext spec would seem to have no problem with this, as the > first
>> line is considered "alt text" and it is up to the client to > determine
>> both what should be done with it and how it should affect the
>> presentation. But I wanted to put this out there to get a > broader
>> perspective.
> 
> There was a thread on this topic some months back:
> 
> gemini://gemi.dev/gemini-mailing-list/messages/005732.gmi
> 
> Alexis.

On Mon, 13 Sept 2021 at 16:18, mntn <mntn@mntn.xyz> wrote:
> I am confident that the best option is just to use the name of the 
language as the alt text. Alt text of "javascript" is fairly clear as to 
what follows in the preformatted text block--so it still works as alt 
text--and a syntax highlighter could use that hint for parsing the text 
and highlighting the code. A properly designed syntax highlighter will 
give up and fail quietly/safely if it can't parse the code for whatever 
reason, say if it's actually a poem titled "javascript" instead of code.

Personally I think using the MIME type would be most sensible, it's
what's already done when a page is received, so it'd make development
easier for clients.
(I'm including non-""official"" MIME types. The ""official"" coverage
of source code files is a bit lacking)

> Clients may want to ignore anything following a space or punctuation, so 
alt text could be even more descriptive, such as "javascript: excerpt from 
example.js" or "HTML example."

Big +1 to this :)

These two examples given here don't seem to be useful, if the client
is adding syntax highlighting then it can probably generate an
alt-text saying "HTML source code" by itself.
Stating that it's an example or something is highly likely to be done
in the normal text anyways.

However, I imagine making a screen-reader read code would be absolute hell!
So it would be extremely useful for this to be a thing, so the code
can be roughly described in a screen-reader safe way.

-Oliver Simmons (GoodClover)

> Personally I think using the MIME type would be most sensible, it's
> what's already done when a page is received, so it'd make development
> easier for clients.
> (I'm including non-""official"" MIME types. The ""official"" coverage
> of source code files is a bit lacking)

You have a good point. I wonder "why not both" -- a client could make a 
best effort to detect the
language using several methods. For the example listed, "js" may also be 
appropriate alt text for
JavaScript. Again, if it can't be parsed as the detected language, it 
wouldn't be highlighted.

> However, I imagine making a screen-reader read code would be absolute hell!
> So it would be extremely useful for this to be a thing, so the code
> can be roughly described in a screen-reader safe way.

That's exactly what I was thinking. I think this would both benefit users 
with screen readers (so
they know what's coming) and people like myself who find it easier to 
parse code with some syntax
highlighting in place.

I can also imagine a highly advanced screen reader that might be able to 
parse code and present
some kind of intelligent navigation. (Next/previous statement or 
expression, enter or exit nested
code, etc.) I don't believe this exists today, but I can imagine it!

On Sun, 12 Sep 2021 18:14:07 +0000
mntn@mntn.xyz wrote:

> The Gemtext spec would seem to have no problem with this, as the
> first line is considered "alt text" and it is up to the client to
> determine both what should be done with it and how it should affect
> the presentation. But I wanted to put this out there to get a broader
> perspective.

That's not what alt text is. It's for screen readers. A bunch of people
are way into ASCII art so alt text is needed. They are image
descriptions.

I recommend the following instead:
Just name the language on the line before the <pre> block.
ie.:

lua:
 ```
local function whatever() end
 ```

This way you aren't making an accessibility feature worse by trying to
overload it to be some generic metadata container.

> On Sep 19, 2021, at 1:43 AM, raingloom <raingloom@riseup.net> wrote:
> 
> On Sun, 12 Sep 2021 18:14:07 +0000
> mntn@mntn.xyz wrote:
> 
>> The Gemtext spec would seem to have no problem with this, as the
>> first line is considered "alt text" and it is up to the client to
>> determine both what should be done with it and how it should affect
>> the presentation. But I wanted to put this out there to get a broader
>> perspective.
> 
> That's not what alt text is. It's for screen readers. A bunch of people
> are way into ASCII art so alt text is needed. They are image
> descriptions.
> 
> I recommend the following instead:
> Just name the language on the line before the <pre> block.
> ie.:
> 
> lua:
> ```
> local function whatever() end
> ```
> 
> This way you aren't making an accessibility feature worse by trying to
> overload it to be some generic metadata container.

Someone floated the idea of using the leading ``` line for alt text and 
the trailing ``` line for language colorization (or maybe the reverse). It 
sounded like a good idea to me, but the current maintainer of the Gemini 
spec (not unreasonably) has a try-it-first-and-report-back attitude to 
these sorts of things and, as far as I can tell, nobody’s modified a 
Gemini browser to try this sort of thing out.

I use the leading ``` for both types of preformatted blocks and I’d be 
happy to change my capsule to use this sort of bifurcated thing, but I 
wouldn’t know what to try out first. My hunch is that moving the 
language-colorization text to the final position is the winning play, but 
that’s the option that goes directly counter to preexisting (Markdown) 
practice/habit/muscle memory, so it’s liable to be the sort of thing 
that’s screwed up more often (people typing colorizer hints where alt text should go).

Nathan Galt <mailinglists@ngalt.com> writes:

>> On Sep 19, 2021, at 1:43 AM, raingloom <raingloom@riseup.net> wrote:
>> 
>> On Sun, 12 Sep 2021 18:14:07 +0000
>> mntn@mntn.xyz wrote:
>> 
>>> The Gemtext spec would seem to have no problem with this, as the
>>> first line is considered "alt text" and it is up to the client to
>>> determine both what should be done with it and how it should affect
>>> the presentation. But I wanted to put this out there to get a broader
>>> perspective.
>> 
>> That's not what alt text is. It's for screen readers. A bunch of people
>> are way into ASCII art so alt text is needed. They are image
>> descriptions.
>> 
>> I recommend the following instead:
>> Just name the language on the line before the <pre> block.
>> ie.:
>> 
>> lua:
>> ```
>> local function whatever() end
>> ```
>> 
>> This way you aren't making an accessibility feature worse by trying to
>> overload it to be some generic metadata container.
>
> Someone floated the idea of using the leading ``` line for alt text
> and the trailing ``` line for language colorization (or maybe the
> reverse). It sounded like a good idea to me, but the current
> maintainer of the Gemini spec (not unreasonably) has a
> try-it-first-and-report-back attitude to these sorts of things and, as
> far as I can tell, nobody’s modified a Gemini browser to try this sort
> of thing out.

Using the trailing ``` has a ugly drawback: we have to read all the code
block before being able to colorize it.  Gemini has a really nice
property of needing only the first few bytes of each line to decide its
"type", there's no going back and forward like it is for markdown (where
link references can be declared after the link itself) or HTML.  This
allows a really cool property: text/gemini files can be *streamed* on
the fly and easily parsed on the fly.  For instance: a irc-like
application can have a page that never stop loading where all the
message "appear" as soon as someone sends them, and a second page where
the users can send text.

That said, regarding the try-and-report part, if someone really wants to
see how this could work, I suggest try hacking something on top of
elpher: Emacs is really cool for this kind of things and already has
syntax highlighting for various programming languages.

Cheers,

Omar Polo <op@omarpolo.com> writes:

> Using the trailing ``` has a ugly drawback: we have to read all the
> code
> block before being able to colorize it.  Gemini has a really nice
> property of needing only the first few bytes of each line to decide its
> "type", there's no going back and forward like it is for markdown
> (where
> link references can be declared after the link itself) or HTML.

And having had to deal with parsing Markdown - due to Markdown libraries 
tending to assume that you want HTML output, and not making a parse tree 
available - i can confirm that this is a nice property indeed.

Fwiw, as someone with disabilities (albeit not vision-related), and who 
has watched accessibility affordances on the Web get (to put it politely) 
short shrift over the decades, i tend to agree with raingloom:

> That's not what alt text is. It's for screen readers. A bunch of people
> are way into ASCII art so alt text is needed. They are image
> descriptions.
> 
> I recommend the following instead:
> Just name the language on the line before the <pre> block.
> ie.:
> 
> lua:
> ```
> local function whatever() end
> ```
> 
> This way you aren't making an accessibility feature worse by trying to
> overload it to be some generic metadata container.


Alexis.