Use the write protection!
Join Crystal Kolipe once again for a walk through the process of adding a new feature to OpenBSD.
- It's part kernel and part userspace code in today's write-up, as we look at the softraid and SCSI subsystems.
- There's also some bug-fixing of code from 1995 thrown in for good measure.
- Let's get started!
Introduction
Removable storage with hardware write protection is a pretty useful safeguard against accidental overwrites and has saved me on several occasions.
At the same time, it's also fairly useful and indeed good industry practice to encrypt data that's being carried around on small devices that are easily lost.
Unfortunately, as of OpenBSD 7.7-release, trying to attach a softraid crypto volume that resides on a read only device fails:
An unexpected and somewhat unhelpful error message.
As well as being desirable to attach such read-only volumes, the error message displayed is unfortunate as it implies to the user that the metadata on disk is corrupted when it actually isn't.
Locating the problem
To diagnose this issue, we need to look at sr_meta_probe(), which can be found in sys/dev/softraid.c.
Here we can see that the call to VOP_OPEN() always tries to open the vnode read/write.
On a physical device which reports itself as read-only, this call to VOP_OPEN() will return EACCES. However it's also possible to see EROFS if we try to access, for example, a vnd device which is backed by a file on a read-only filesystem.
The first step towards making softraid volumes work read-only is to patch sr_meta_probe() to catch these situations and re-try calling VOP_OPEN() with just FREAD permissions:
Re-trying the call to VOP_OPEN() with just FREAD, if the existing call fails.
Success! This allows the volume to attach:
Promising results, except...
As might be expected, we see errors from the kernel when the softraid subsystem tries to update the metadata on the device:
Metadata on the underlying device can't be updated, since it's read-only.
Nevertheless however, the inability to update the metadata doesn't stop us making use of the softraid volume for read only access.
At least the essential functionality is already working.
Shortcomings and missing functionality
- Unfortunately, since the attached softraid volume doesn't report it's read-only capability, the system allows attempts to mount it read-write.
- This initially succeeds, but immediately causes a flood of I/O errors.
- To make things worse, on the test machine the volume could not be unmounted even when using the -f argument to umount.
SCSI device read-only flag reporting mechanism
Devices attaching as sd devices can report themselves as read-only. The sd driver discovers this by reading one page of the device's mode page data, (which for those readers unfamiliar with the SCSI protocols, is basically a table of drive capabilities, some of which can be changed).
Interestingly, the read only flag isn't actually part of the mode page data itself, but is instead returned as a bit in the device specific byte of the header for the mode page data.
Fun fact - visualising the flag from userland
We can see this from userland with a slightly modified version of the /sbin/scsi utility.
The existing code doesn't display the mode page header at all, and in fact won't even open a read-only device that's specified with the -f argument.
To do this we change the call to scsi_open() to use O_RDONLY if O_RDRW fails, and add the following printf() call to mode_edit():
Ready to apply patches are linked to at the end of this article for readers who don't want to write their own.
Now, querying the physical hardware device, we can see the top bit of the device specific header byte reflecting it's read-only status:
With the device write enabled:
With the device write protected:
When the device is configured as read-only, bit 7 of the third header byte is set.
Note: this simple demonstration with /sbin/scsi assumes that the device actually supports reading the mode page data in the first place, otherwise /sbin/scsi will just error out with a message similar to this:
Fun fact - long unfixed, (presumably undiscovered), bugs
The /sbin/scsi utility has various bugs in it's code to display the unformatted hex data.
In the examples above it displayed eleven bytes of data after the header, whereas the first header byte 0x0b actually indicates the amount of data to follow including the remaining three header bytes. Therefore, only eight bytes of the following data are relevant.
Additionally, although the code skips printing the block descriptors, (which, if present, immediately follow the header), the total count of bytes printed is not reduced by the number skipped.
Interestingly, these bugs have been present in the code since the initial revision imported to the OpenBSD CVS tree in 1996.
Earlier versions can be found in the FreeBSD project's historical codebase, (the /sbin/scsi utility was removed from FreeBSD some time ago). The earliest version of the /sbin/scsi code that I was able to find which includes code to parse the -m option is revision 1.3 from 1995-04-17. This version correctly skips the block descriptors, and stores the correct length value read from the second byte of the mode page data itself in variable page_length.
Unfortunately, the value of this variable is never used, and instead the hex printing loop uses the value of mode_data_length as it's loop test condition, so this version of the code already has the bugs described above.
Revision 1.6 from 1995-05-01 introduced the code as it is now, with the assignment to page_length removed, but with both the bugs described above still present.
Patches to address these issues are included in the patchset at the bottom of this article.
Understanding the sd driver
Knowing how the read-only status of a device is reported helps to demystify the relevant part of the source code for the sd driver, (and in fact the st driver which does almost the same thing). The code queries mode page zero, which is for non-standardised vendor specific use anyway, and promptly ignores the data returned if indeed the opcode even succeeds. Nevertheless, this process is enough to get the required header bytes.
Within the sd driver code, sdattach() calls sd_get_parms(), which calls through to scsi_do_mode_sense() in scsi_base.c, which in turn calls scsi_mode_sense() where ultimately the sense data is obtained by putting the opcode MODE_SENSE or MODE_SENSE_BIG in to scsi_xs_sync() and scsi_xs_put().
Returning to sd_get_parms(), if the relevant bit is set then the SDEV_READONLY flag is also set in the corresponding link->flags.
Looking back at sdattach(), (also in sys/scsi/sd.c), we can see that readonly is then printed in the device attach console output, if link->flags has SDEV_READONLY set.
Implementation note:
For those readers who are familiar with SCSI opcodes, as might be expected, MODE_SENSE and MODE_SENSE_BIG translate to the 6-byte and 10-byte versions of the MODE SENSE opcode respectively.
Implementing the same functionality on the virtual device
Obviously the write-protected hardware device on which our softraid chunk resides already does this reporting. What we want to do now is arrange for our newly created virtual sd device to do the same thing.
To signal the read-only status of the virtual device, we need to make it supply an appropriate response to the mode sense command sent by the sd driver when it queries mode page zero. We don't actually need to return any values, but we do need to send back the header, (which will also indicate that there are zero data values in the reply).
SCSI opcodes sent to softraid devices are handled by sr_scsi_cmd(). To avoid confusion here, note that we are now looking at code which implements the scsi target side of the communication, rather than the initiator part of the exchange, (in other words, this is the function that implements the part of the SCSI protocol that would traditionally be done by the firmware of a physical disk drive, providing the response data back to the host adaptor).
We can add stubs to the switch statement in sr_scsi_cmd() to visualise these opcodes being sent by sd_get_parms().
Note that scsi_do_mode_sense() always tries MODE_SENSE before trying MODE_SENSE_BIG, so as long as we return a valid response, (header), for MODE_SENSE, we can basically ignore MODE_SENSE_BIG.
Stub functions allow us to visualise the use of these SCSI opcodes.
Note that the mode page number being requested is in the lower six bits.
Before we add the code to the ioctl handler, though, we need to define a new capabilities flag in softraidvar.h:
The new flag defined above will be set in the following code if we open any of the underlying devices for the softraid volume read-only.
Of course, the choice of 0x00000020 as the bit value is arbitrary.
And we also need to add a line to our new code in sr_meta_probe() to set this flag, if we do indeed open the device as read-only:
The diagnostic printf() calls added previously can also be removed at this point.
Now that we have SR_CAP_READONLY set appropriately, our ioctl handler for MODE_SENSE becomes fairly simple:
With mode_sense_header[] also defined at the start of sr_scsi_cmd() as a local array:
With these kernel modifications in place, our softraid sd device correctly reports itself as read-only and is usable in read-only mode.
Attempts to mount it read-write or perform any write operation directly to the device will be denied by the kernel.
Top tip - read-only volumes in /etc/fstab
If a read-only volume, (any read-only volume, not just a softraid one), is mounted at boot time, then it needs to have the fs_passno field of fstab set to 0.
This is because the preen mode of fsck errors out when it detects a lack of write access to the device. Since a permanently read-only filesystem would be expected to be clean anyway, this shouldn't really pose a problem.
Userland utility support for read-only mode
Some changes are required to userland utilities in order to make full use of the new kernel features.
/sbin/bioctl fails to remove a read-only sd device:
The code attempts to open /dev/bio when the previous call to opendev() fails, regardless of the reason for the failure.
Since the opendev() call is hardcoded to open the device with O_RDWR, this obviously fails when it's read-only, and leads to the misleading error message
`Can't locate sdx device via /dev/bio'.
Another somewhat mis-leading error message. In fact, the problem is nothing to do with /dev/bio.
The solution is just to check for func == BIO_DELETERAID or func == BIO_INQ and call opendev() with O_RDONLY instead of O_RDWR in this case.
(func == BIOC_DELETERAID || func == BIOC_INQ ? O_RDONLY : O_RDWR)
Some functions don't actually require write access anyway.
/sbin/scsi won't open read-only devices
A similar issue exists in the /sbin/scsi utility, which returns the following error when trying to open a read-only device:
The /sbin/scsi utility always tries to open the device specified with the -f option as read-write.
This time the solution is even easier. If the call to scsi_open() fails with O_RDWR, we immediately retry with O_RDONLY:
Just one additional conditional statement is required to re-try the operation as read-only.
Other RAID disciplines
So far we've only considered the softraid crypto discipline, but the changes to the softraid code are to the common functions sr_meta_probe(), and sr_scsi_cmd(), which are shared with other disciplines and the read-only functionality works there too.
In the case of RAID disciplines with more than one device, (such as a mirror set), if any one of the underlying devices is opened read-only then the assembled softraid sd device will be flagged as read-only, even if other devices in the array are writable.
Downloads
A patchset is available for download which applies to OpenBSD 7.7-release.
Patchset against OpenBSD 7.7-release
The patchset is signed with our signify key, and includes the kernel patch, as well as patches for /sbin/scsi and /sbin/bioctl.
To apply the patchset, first place our signify key in /etc/signify if you don't already have it:
Then download the patchset:
Next, having ensured that you have the OpenBSD 7.7-release source tree in /usr/src, apply the patches to the base of the source tree:
Lastly, recompile the kernel and userland utilities:
The kernel compilation instructions given above are sufficient for a typical generic system, but for a more complete guide to kernel compiling on OpenBSD please refer to our our custom kernel compilation write-up.
Custom kernel compiling guide - part of our reckless guide to OpenBSD
Home page of the Exotic Silicon gemini capsule.
Your use of this gemini capsule is subject to the terms and conditions of use.
Copyright 2016 - 2026 Exotic Silicon. All rights reserved.