SSL Certificate Conversion
X.509 certificates can be stored in a variety of different forms. Worse, the certificates may be housed in a vendor or language specific certificate store that may require writing custom code to extract the certificates. Different forms or certificate stores can complicate migrations from one platform to another. It may be simpler to create new private keys and to make new certificates from those keys rather than trying to preserve the old key and certificate material, though users may then need to deal with a certificate change warning.
The following assumes a unix-shaped system (OpenBSD, mostly) with an OpenSSL-shaped SSL library (LibreSSL, mostly) with an effort to convert certificates from Agate to gemserv.
Locate the Certificates
Converting the certificates to a different form may be difficult if you have no idea where they are located. The location might be indicated in the documentation or configuration for the software; worst case, you may need to use process tracing tools (ktrace, strace, etc.) to see what files a process uses. Commercial software may even store keys and certificates in secure memory or off in some other device on the network. Good luck!
Agate Example
Running the binary directly from the build directory is not a best practice, but that's not important here.
Here one may not know where the ".certificates" directory is; process tracing may better reveal the location.
This is a relative path location, so will be in the current working directory, or relative to where any prior chdir(2) calls changed the working directory to. There may be a startup script or init system for agate that changes the working directory, in which case you may need to learn more about that init system (e.g. openrc, systemd, etc) and where the configuration or scripts for it are hidden. If the software was built as a port or package, the port system may have tools that tell you what files are part of a package. Good luck!
Type of Certificate
There are several forms certificates can be saved as; PEM and DER are common on unix systems, though there is also PKCS #12 and so forth. The type may be indicated by a file extension, or a tool like file(1) may be able to guess the file type. Vendors vary in where they hide certificates; the following assumes OpenBSD.
To continue the agate example, file(1) on OpenBSD cannot identify the type used by agate:
The reset(1) command may be handy if you cat(1) a binary file that messes up your terminal. From the extension however we might guess that *.der files are stored in DER format.
Conversion
SSL libraries typically ship with tools that can convert certificate formats, e.g. the openssl(1) tool that ships with OpenSSL and LibreSSL. If you have some other library, check its documentation for what it offers, or install a library you do know how to use.
OpenSSL has various subcommands for various key and certificate types, so it may take some tries to find the right commands.
So the cert.der file parses as ASN1 assuming the DER format, which is a good indication that it is actually DER. If it does not, one would need to determine what the code has done with the file—encryption? a custom format? actually instead using PEM or some other known format? etc.
And we probably need to convert the private key in addition to the certificate information:
The umask(1) command is to limit the permissions on the "key.pm" output file. Leaking the keys to other users or processes on the system might be a security risk. Be sure to change it back to the default, to close the shell instance used for the conversion. However, in this case, the "rsa" command failed. Maybe we do not actually have an RSA key here?
The "id-ecPublicKey" indicates a newer elliptic curve key, not a more typical RSA key. There is also an older DSA key type. LibreSSL (version 3.8.2 in OpenBSD 7.4) does not support converting EC key, so you may need to find some other SSL library to convert this key type:
So from two agate auto-generated files, we now have PEM versions of those files:
Usage with gemserv
Another problem is that we may not know what format(s) are supported in the software we intend to migrate to; documentation may be lacking, or there may be too much documentation to wade through. For example "gemserv" is somewhat lacking in documentation, though searching for "DER" or "PEM" in the source may give hints as to what is supported. It may also turn up a lot of noise to wade through.
Using strings(1) on an unknown binary may reveal the function calls used, or not, as well as something that logs function calls used, though those are different from the system functions that ktrace or strace log. It would help to know something of the API involved to help interpret any such output.
A less elegant approach is to try random formats until something works, or you run out of time. With practice you may hone in on things more likely to work. In particular the "pkcs8_private_keys" hints that gemserv may need a PKCS #8 format key file?
The backtrace (as is usual for a non-programmer, or at least one who does not use Rust) was not useful. Maybe gemserv needs the key in PKCS #8 format?
Well, that's no good. Maybe it's the elliptic curve key; what happens if we generate a more typical RSA key and use that?
The illegal instruction is probably "rust runs poorly on OpenBSD unless you fiddle with something, check the mailing list archives" though it does look that, probably, the server read the RSA key in PEM format. I would suspect that gemserv may not support the elliptic curve key format used by agate, so therefore one would need to generate a new keypair for use with gemserv, and retire the old key and certificate.
At least that's as far as I got; maybe someone else can go further with this.