RAIDCTL(8)

NAME
raidctl - configuration utility for the RAIDframe disk driver

SYNOPSIS
-a component dev
-B dev
-c config_file dev
-C config_file dev
-f component dev
-F component dev
-g component dev
-i dev
-I serial_number dev
-r component dev
-R component dev
-s dev
-S dev
-u dev

DESCRIPTION
is the user-land control program for raid(4), the RAIDframe disk device. is primarily used to dynamically configure and unconfigure RAIDframe disk devices. For more information about the RAIDframe disk device, see raid(4).

This document assumes the reader has at least rudimentary knowledge of RAID and RAID concepts.

The command-line options for raidctl are as follows:

-a component dev
    Add component as a hot spare for the device dev.

-B dev
    Initiate a copyback of reconstructed data from a spare disk to it's original disk. This is performed after a component has failed, and the failed drive has been reconstructed onto a spare drive.

-c config_file dev
    Configure the RAIDframe device dev according to the configuration given in config_file. A description of the contents of config_file is given later.

-C config_file dev
    As for -c, but forces the configuration to take place. This is required the first time a RAID set is configured.

-f component dev
    This marks the specified component as having failed, but does not initiate a reconstruction of that component.

-F component dev
    Fails the specified component of the device, and immediately begin a reconstruction of the failed disk onto an available hot spare. This is one of the mechanisms used to start the reconstruction process if a component does have a hardware failure.

-g component dev
    Get the component label for the specified component.

-i dev
    Initialize (re-write) the parity on the device. This MUST be done before the RAID device is labeled and before filesystems are created on the RAID device, and is normally used after a system crash (and before a fsck(8)) to ensure the integrity of the parity.

-I serial_number dev
    Initialize the component labels on each component of the device. serial_number is used as one of the keys in determining whether a particular set of components belong to the same RAID set. While not strictly enforced, different serial numbers should be used for different RAID sets.

-r component dev
    Remove the spare disk specified by component from the set of available spare components.

-R component dev
    Fails the specified component, if necessary, and immediately begins a reconstruction back to component. This is another mechanism for starting the reconstruction process if a component has a hardware failure.

-s dev
    Display the status of the RAIDframe device for each of the components and spares.

-S dev
    Check the status of component reconstruction. The output indicates the amount of progress achieved in reconstructing a failed component.

-u dev
    Unconfigure the RAIDframe device.

The device used by raidctl is specified by dev. dev may be either the full name of the device, e.g. /dev/rraid0d, for the i386 architecture, and /dev/rraid0c for all others, or just simply raid0 (for /dev/rraid0d).

The format of the configuration file is complex, and only an abbreviated treatment is given here. In the configuration files, a ‘#’ indicates the beginning of a comment.

There are 4 required sections of a configuration file, and 2 optional components. Each section begins with a ‘START’, followed by the section name, and the confuration parameters associated with that section. The first section is the ‘array’ section, and it specifies the number of rows, columns, and spare disks in the RAID set. For example:

    START array
    1 3 0

indicates an array with 1 row, 3 columns, and 0 spare disks. Note that although multi-dimenstional arrays may be specified, they are NOT supported in the driver.

The second section, the ‘disks’ section, specifies the actual components of the device. For example:

    START disks
    /dev/sd0e
    /dev/sd1e
    /dev/sd2e

specifies the three component disks to be used in the RAID device. If any of the specified drives cannot be found when the RAID device is configured, then they will be marked as ‘failed’, and the system will operate in degraded mode. Note that it is imperative that the order of the components in the configuration file does not change between configurations of a RAID device. Changing the order of the components (at least at the time of this writing) will result in data loss.

The next section, which is the ‘spare’ section, is optional, and, if present, specifies the devices to be used as ‘hot spares’ -- devices which are on-line, but are not actively used by the RAID driver unless one of the main components fail. A simple ‘spare’ section might be:

    START spare
    /dev/sd3e

for a configuration with a single spare component. If no spare drives are to be used in the configuration, then the ‘spare’ section may be omitted.

The next section is the ‘layout’ section. This section describes the general layout parameters for the RAID device, and provides such information as sectors per stripe unit, stripe units per parity unit, stripe units per reconstruction unit, and the parity configuration to use. This section might look like:

    START layout
    # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level
    32 1 1 5

The sectors per stripe unit specifies, in blocks, the interleave factor; i.e. the number of contiguous sectors to be written to each component for a single stripe. Appropriate selection of this value (32 in this example) is the subject of much research in RAID architectures. The stripe units per parity unit and stripe units per reconstruction unit are normally each set to 1. While certain values above 1 are permitted, a discussion of valid values and the consequences of using anything other than 1 are outside the scope of this document. The last value in this section (5 in this example) indicates the parity configuration desired. Valid entries include:

0   RAID level 0. No parity, only simple striping.

1   RAID level 1. Mirroring.

4   RAID level 4. Striping across components, with parity stored on the last component.

5   RAID level 5. Striping across components, parity distributed across all components.

There are other valid entries here, including those for Even-Odd parity, RAID level 5 with rotated sparing, Chained declustering, and Interleaved declustering, but as of this writing the code for those parity operations has not been tested with NetBSD.

The next required section is the ‘queue’ section. This is most often specified as:

    START queue
    fifo 1

where the queueing method is specified as fifo (first-in, first-out), and the size of the per-component queue is limited to 1 requests. A value of 1 is quite conservative here, and values of 100 or more may been used to increase the driver performance. Other queuing methods may also be specified, but a discussion of them is beyond the scope of this document.

The final section, the ‘debug’ section, is optional. For more details on this the reader is referred to the RAIDframe documentation dissussed in the HISTORY section. See EXAMPLES for a more complete configuration file example.

EXAMPLES
The examples in this section will focus on a RAID 5 configuration. Other RAID configurations will behave similarly. It is highly recommended that before using the RAID driver for real filesystems that the system administrator(s) have used all of the options for, and that they understand how the component reconstruction process works. While this example is not created as a tutorial, the steps shown here can be easily dupilicated using four equal-sized partitions from any number of disks (including all four from a single disk).

The primary uses of is to configure and unconfigure raid(4) devices. To configure the device, a configuration file which looks something like:

    START array
    # numRow numCol numSpare
    1 3 1
    START disks
    /dev/sd1e
    /dev/sd2e
    /dev/sd3e
    START spare
    /dev/sd4e
    START layout
    # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_5
    32 1 1 5
    START queue
    fifo 100

is first created. In short, this configuration file specifies a RAID 5 configuration consisting of the components /dev/sd1e, /dev/sd2e, and /dev/sd3e, with /dev/sd4e available as a ‘hot spare’ in case one of the three main drives should fail. If the above configuration is in a file called ‘rfconfig’, raid device 0 in the normal case can be configured with:

    raidctl -c rfconfig raid0

The above is equivalent to the following:

    raidctl -c rfconfig /dev/rraid0d

on the i386 architecture. On all other architectures, /dev/rraid0c is used in place of /dev/rraid0d.

A RAID set will not configure with -c if the component labels are not correct. A ‘component label’ contains important information about the component, including a user-specified serial number, the row and column of that component in the RAID set, and whether the data (and parity) on the component is ‘clean’. See raid(4) for more information about component labels.

Since new RAID sets will not have correct component labels, the first configuration of a RAID set must use -C instead of -c:

    raidctl -C rfconfig raid0

The -C forces the configuration to succeed, even if any of the component labels are incorrect. This option should not be used lightly in situations other than initial configurations, as if the system is refusing to configure a RAID set, there is probably a very good reason for it.

When the RAID set is configured for the first time, it is necessary to initialize the component labels, and to initialize the parity on the RAID set. Initializing the component labels is done with:

    raidctl -I 112341 raid0

where ‘112341’ is a user-specified serial number for the RAID set. Using different serial numbers between RAID sets is strongly encouraged, as using the same serial number for all RAID sets will only serve to decrease the usefulness of the component label checking.

Initializing the parity on the RAID set is done via:

    raidctl -i raid0

Initializing the parity in this way may also be required after an unclean shutdown. Once the parity is known to be correct, it is then safe to perform disklabel(8), newfs(8), or fsck(8) on the device or its filesystems, and then to mount the filesystems for use.

To see how the RAID set is doing, the following command can be used to show the RAID set's status:

    raidctl -s raid0

The output will look something like:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: optimal
               /dev/sd3e: optimal
    Spares:
               /dev/sd4e: spare

This indicates that all is well with the RAID set.

To check the component label of /dev/sd1e, the following is used:

    raidctl -g /dev/sd1e raid0

The output of this command will look something like:

    Component label for /dev/sd2e:
    Version: 1
    Serial Number: 112341
    Mod counter: 6
    Row: 0
    Column: 1
    Num Rows: 1
    Num Columns: 3
    Clean: 0
    Status: optimal

For a component label to be considered valid, that particular component label must be in agreement with the other component labels in the set. For example, the serial number, 'modification counter', number of rows and number of columns must all be in agreement. If any of these are different, then the component is not considered to be part of the set.

If for some reason (perhaps to test reconstruction) it is necessary to pretend a drive has failed, the following will perform that function:

    raidctl -f /dev/sd2e raid0

The system will then be performing all operations in degraded mode, were missing data is re-computed from existing data and the parity. In this case, obtaining the status of raid0 will return:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: failed
               /dev/sd3e: optimal
    Spares:
               /dev/sd4e: spare

Note that with the use of -f a reconstruction has not been started. To both fail the disk and start a reconstruction, the -F option must be used:

    raidctl -F /dev/sd2e raid0

The -f option may be used first, and then the -F option used later, on the same disk, if desired. Immediately after the reconstruction is started, the status will report:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: reconstructing
               /dev/sd3e: optimal
    Spares:
               /dev/sd4e: used_spare

This indicates that a reconstruction is in progress. To find out how the reconstruction is progressing the -S option may be used. This will indicate the progress in terms of the percentage of the reconstruction that is completed. When the reconstruction is finished the -s option will show:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: spared
               /dev/sd3e: optimal
    Spares:
               /dev/sd4e: used_spare

At this point there are at least two options. First, if /dev/sd2e is known to be good (i.e. the failure was either caused by -f or -F, or the failed disk was replaced), then a copyback of the data can be initiated with the -B option. In this example, this would copy the entire contents of /dev/sd4e to /dev/sd2e. Once the copyback procedure is complete, the status of the device would be:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: optimal
               /dev/sd3e: optimal
    Spares:
               /dev/sd4e: spare

and the system is back to normal operation.

The second option after the reconstruction is to simply use /dev/sd4e in place of /dev/sd2e in the configuration file. For example, the configuration file (in part) might now look like:

    START array
    1 3 0
    START drives
    /dev/sd1e
    /dev/sd4e
    /dev/sd3e

This can be done as /dev/sd4e is completely interchangeable with /dev/sd2e at this point. Note that extreme care must be taken when changing the order of the drives in a configuration. This is one of the few instances where the devices and/or their orderings can be changed without loss of data! In general, the ordering of components in a configuration file should never be changed.

If a component fails and there are no hot spares available on-line, the status of the RAID set might look like:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: failed
               /dev/sd3e: optimal
    No spares.

In this case there are a number of options. The first option is to add a hot spare using:

    raidctl -a /dev/sd4e raid0

After the hot add, the status would then be:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: failed
               /dev/sd3e: optimal
    Spares:
               /dev/sd4e: spare

Reconstruction could then take place using -F as describe above.

A second option is to rebuild directly onto /dev/sd2e. Once the disk containing /dev/sd2e has been replaced, one can simply use:

    raidctl -R /dev/sd2e raid0

to rebuild the /dev/sd2e component. As the rebuilding is in progress, the status will be:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: reconstructing
               /dev/sd3e: optimal
    No spares.

and when completed, will be:

    Components:
               /dev/sd1e: optimal
               /dev/sd2e: optimal
               /dev/sd3e: optimal
    No spares.

The final operation performed by raidctl is to unconfigure a raid(4) device. This is accomplished via a simple:

    raidctl -u raid0

at which point the device is ready to be reconfigured.

WARNINGS
Certain RAID levels (1, 4, 5, 6, and others) can protect against some data loss due to component failure. However the loss of two components of a RAID 4 or 5 system, or the loss of a single component of a RAID 0 system will result in the entire filesystem being lost. RAID is NOT a substitute for good backup practices.

Recomputation of parity MUST be performed whenever there is a chance that it may have been compromised. This includes after system crashes, or before a RAID device has been used for the first time. Failure to keep parity correct will be catastrophic should a component ever fail -- it is better to use RAID 0 and get the additional space and speed, than it is to use parity, but not keep the parity correct. At least with RAID 0 there is no perception of increased data security.

FILES
/dev/{,r}raid*
    raid device special files.

SEE ALSO
raid(4), ccd(4), rc(8)

BUGS
Hot-spare removal is currently not available.

HISTORY
RAIDframe is a framework for rapid prototyping of RAID structures developed by the folks at the Parallel Data Laboratory at Carnegie Mellon University (CMU). A more complete description of the internals and functionality of RAIDframe is found in the paper "RAIDframe: A Rapid Prototyping Tool for RAID Systems", by William V. Courtright II, Garth Gibson, Mark Holland, LeAnn Neal Reilly, and Jim Zelenka, and published by the Parallel Data Laboratory of Carnegie Mellon University.

The raidctl command first appeared as a program in CMU's RAIDframe v1.1 distribution. This version of raidctl is a complete re-write, and first appeared in NetBSD 1.4.

COPYRIGHT

    The RAIDframe Copyright is as follows:
    Copyright (c) 1994-1996 Carnegie-Mellon University.
    All rights reserved.
    Permission to use, copy, modify and distribute this software and
    its documentation is hereby granted, provided that both the copyright
    notice and this permission notice appear in all copies of the
    software, derivative works or modified versions, and any portions
    thereof, and that both notices appear in supporting documentation.
    CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS"
    CONDITION.  CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND
    FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE.
    Carnegie Mellon requests users of this software to return to
     Software Distribution Coordinator  or  Software.Distribution@CS.CMU.EDU
     School of Computer Science
     Carnegie Mellon University
     Pittsburgh PA 15213-3890
    any improvements or extensions that they make and grant Carnegie the
    rights to redistribute these changes.