##### singularity stuff

* built-in support for IB, Lustre, and schedulers (SLURM)
* built-in support for MPI and GPU resources; works as single command on system
* user within singularity container is same as user outside of container
* serve as stand-alone programs for ease of execution
* containers for different architectures require appropriate virtualization libraries

#### terms

* OCI 		= open container initiative
* SIF		= singularity image format
* instance	= >= 2.4, instances are daemonized, persistent containers used to run services


#### bind paths

* when using --writable, make sure that bind dst path exists in container, or expect entropy

* singularity exec|shell --bind src:dst	= binds src (outside container) to dst (inside container)
					can use commas to separate multiple binds: src1:dst1,src,src2:dst2
					if dst is omitted, it is the same as src

* SINGULARITY_BIND			= use environment variable instead of CLI



## supported container formats

* squashfs	= default container format, compressed read-only file system
		  used on CD's, USB's, and cell phone OS'
* ext3		= writable image file w/ ext3 FS which was the default format =< 2.4
* directory	= aka sandbox, standard UNIX directory containing root image
* tar.gz	= zlib compressed tar
* tar.bz2	= bzip2 compressed tar
* tar		= uncompressed tar

## supported URI's

* shub://	= singularity hub
* docker://	= singularity can pull Docker images from Docker registry; non persistent
		  runs, since they aren't saved upstream; subsequent pulls cannot be guaranteed
		  to be identical
* instance://	= singularity container running as service, called an instance, referenced
		  w/ this specific URI

## singularity development

* --sandbox	= build into writable directory; may need to fix permissions on non local systems while using root in yum cache
* --writable	= build into writable ext3 image


### development notes

* singularity binary should not be installed into a location with nosuid set
* non-system application installs should be done outside of /tmp within container
* pip installs should be done using -t as to not use ~/.local

## singularity commands
## prefix with singularity

* build 	= build container on endpoint or build environment
* exec		= execute command to container
* inspect	= see labels, run & test scripts, & environment vars
* pull		= pull image from Docker/Singularity hub (can use sha tags, too)
* run		= run image as executable
* shell		= shell into image


# image commands

* image.import	= import layers or other file content to image
* image.export	= export the contents of image to tar or stream
* image.create	= create new image, using old ext3 file system
* image.expand	= increase size of image w/ old ext3

# pull example

* singularity pull image.sif docker://debian:latest		= get latest base image


# build example (ensure root is used)

* export APPTAINER_TMPDIR=/path/to/largestorage			= Specify a different path other than tmpfs for building; avoids quota exceeded error messages

* singularity build --sandbox dir docker://debian:latest	= build sandbox from latest image
* singularity build --writable dir docker://debian:latest	= build writable image (save changes)
* singularity build --writable blah.img new.simg		= convert squashfs container to blah.img (ext3)



# execution examples

* exec blah.img	cmd						= run cmd within the container
* exec blah.img python script.py				= run host script.py using python from container
* exec blah.img python < /path/script				= same, but using redirection
* mpirun -np singularity exec img mpi_container (mpi options)	= run MPI container



###### definition files; must execute as root

* Bootstrap is _required_ for every type of build

** Header section

* Bootstrap	= determines the bootstrap agent that will be used to create base OS in container
		* Bootstrap: library ; will pull a container from the container library as base OS
		* Bootstrap: docker  ; will pull a container from the Docker hub as base OS

# Bootstrap agents
* library 					= uses "From:"
* docker
* shub
* localimage (images saved on local machine)
* yum (yum based systems)
* debootstrap (apt based systems)
* arch (arch linux)
* busybox (Busybox)
* zypper (Suse based systems)

** Sections ; each build file is comprised of multiple sections and build will fail if any command fails

%setup		= commands executed here are on the host system outside of container after base OS is installed
		$SINGULARITY_ROOTFS is used as reference to container FS in the %setup section
		PAY ATTENTION TO SECURITY HERE; use is discouraged over %files

%files		= allows copying of files on host system into container with greater safety
		<source> & <destination> pairs are used; latter can be omitted
		files are copied before %post section

%environment	= allows defining of environment variables to be set at runtime; not included during build time
		use %post section to define variables during build
		/.singularity.d/env/90-environment.sh contains environment variables set during build
		/.singularity.d/env/91-environment.sh contains environment variables added during %post

%post		= commands executed in container after base OS has been installed during build




#### miscellaneous stuff
* use --nv to bind to host's nvidia drivers so that container doesn't require its own drivers
* read-only file system may be due to not binding mountpoints available to the container