### losf tool (configuration management)

* Linux Open Server Framework

* config files contain stanzas for deployment, permissions, symlinks, etc.
* host-specific configuration files end in a hostname, access.conf.login3
* changes are stored in node:/tmp/losf (this is detailed in output on console)
* permissions on staged files are inherited
* non staged files have their permissions managed via config.$CLUSTER file under Permissions stanza
* partial config files are only updated via `update` and not just a config file sync
* Package installations using shared storage locations MUST be installed BEFORE the modulefile
** Lua modulefiles use a if exists localdir conditional and if it isn't present it will not honor the relocation

* @losf_synced_file_notice@	; Macro to use within text files stating file is managed via LosF
				  Default message in $LOSF/const_files/notify_header
* @losf_synced_file_location@	; Macro that points to file system  path of managed file
* /root/losf-noupdate		; place on nodes that should not receive updates
* root configuration directory	; /admin/build/admin/config
* per cluster config directory	; /admin/build/admin/config/const_files/$CLUSTER
* per role config directory	; /admin/build/admin/config/const_files/$CLUSTER/role (role is login, master, etc.)
* custom package install 	; /admin/build/admin/config/custom-packages/$CLUSTER/packages.config
		 		  run `/admin/losf/nodetype` to get node's current role
				  to add a custom package run `losf addrpm coolcustompackage` on a node within role; will propagate to rest of nodes in role via `update`
				  losf supports the use of aliases when installing, which are assigned via the [Custom Packages] stanza
				  aliases can be used for host groups or packages. format is class=@alias 
				  note - sometimes login=HASH entries will appear, these need to be removed
				  use of --install on console will produce MULTI in packages.config; allows multiple installs of package but different versions
				  default use is to UPGRADE RPM's with similarly named packages
				  NOTE - use of --upgrade with compilers will overwrite previous version(s) if INSTALL MULTI definitions are missing; add these manually if needed
				  NODEPS & IGNORESIZE are set by default
* cached package location	; /admin/build/admin/config/$CLUSTER/rpms/x86_64
				  * cached packages are located in /admin/build/admin/config/$CLUSTER/rpms/x86_64 (OS and custom)
				  ** this is important if version numbers didn't change and an attempt to reinstall fails with a cached package error message
* custom package location	; /admin/build/admin/rpms/$CLUSTER/$ARCH
* os package config		; /admin/build/admin/config/os-packages/$CLUSTER/packages.config
				  to add an os package run `losf addpkg [name]` on a node within role; will propagate out to rest of nodes in role via `update`
* common files applied to all	; /admin/build/admin/config/const_files/$CLUSTER/common
				  Standard (global) copy of file that isn't specific to a role
				  symlink from common/ to role/
* config.$CLUSTER		; specific configuration file $CLUSTER (lonestar, etc.)
				  host-specific configurations end with a role - [ConfigFiles/admin], [ConfigFiles/gpu], etc.
				  WARNING: trailing whitespace after a "= (yes|no) entry may render file syncing useless"
				  Provisioning settings are specific to integration with cobbler system add
				  assign_ips_from_file = assign IP addresses ips.$CLUSTER
				  [Kickstarts] = point to cobbler kickstart files in /var/lib/cobbler/templates
				  [Profiles] = point to cobbler profiles
				  [DNS-enable] = add DNS record for host
				  [Name-Servers] ] = add name server
				  [Name-Servers-Search] = add search domains
				  [Kernel-Boot-Options] = add kernel boot options
				  [Kernel-Boot-Options-Post] = add post kernel boot options
* config.machines		; node aliases based upon role (master = manager, etc.); regex based definitions of hostnames
				  aliases for koomie are found here
				  aliases require /etc/hosts entries
				  package host aliases are defined here
* ips.$CLUSTER			; IP mapping for cluster - must contain hostname, IP address, MAC address, device, and subnet mask
				  needs to be updated when machines are replaced/removed
* known stanzas			; Cluster-Names, Cobbler, ConfigFiles, DNS-Enable, Gateways, HostInterfaces, Kernel-Boot-Options, Kernel-Boot-Options-Post, Kickstarts, 
				  Name-Servers, Name-Servers-Search, PartialConfigFiles, Permissions, Profiles, Provisioning, Services, SoftLinks, VarSub, Warewulf

# commands
* losf				; root command for management
* losf update			; apply configuration to host(s)
				  use with -q for quiet output
* node_types			; run on host(s) to get their configured roles via machines.$CLUSTER
* losf addrpm ARGS		; add custom package to lsof
				  run on host type to apply to all hosts within type
				  lsof addrpm --alias ALIAS --install /path/to/rpm
				  lsof addrpm --alias ALIAS --relocate /original/path /installed/path --install /path/to/rpm
* losf addpkg			; add OS package to losf
				  can specify multiple packages
* koomie_cf 			; pdsh equivalent
				  -r racks (multiples permitted)
				  -n node_type
				  -t timeout (in seconds, default is 300)
				  -m maximum ssh sessions
				  example: koomie_cf -r 100-211 -t 7200 -m 500 'nice /admin/build/admin/losf/update -q'
* sync_config_files		; updates only configuration files
				  must be run on individual hosts
				  optionally supply file name for syncing single file

### O.S. updating overview

1.)  Prepare node for general dnf/yum update; edit yum.conf and remove exclusions
     * Does the kernel package need to be updated
     * Problems if staging compute/login
     * tool was created to just run `losf updatepkgs`
2.)  Perform dry run of dnf/yum update; this may involve enabling upstream repos and/or disabling cobbler repos
     * Point to base repos and not custom repos
     * Create new role, etc.
3.)  Once satisfied, use losf updatepkgs
4.)  Handle any upgraded custom RPM's as usual via losf (remove from packages.config if need be)