| ATA over Ethernet is a network protocol that provides simple access to | 
 | block storage on the LAN. | 
 |  | 
 |   http://support.coraid.com/documents/AoEr11.txt | 
 |  | 
 | The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ... | 
 |  | 
 |   http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html | 
 |  | 
 | It has many tips and hints!  Please see, especially, recommended | 
 | tunings for virtual memory: | 
 |  | 
 |   http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19 | 
 |  | 
 | The aoetools are userland programs that are designed to work with this | 
 | driver.  The aoetools are on sourceforge. | 
 |  | 
 |   http://aoetools.sourceforge.net/ | 
 |  | 
 | The scripts in this Documentation/aoe directory are intended to | 
 | document the use of the driver and are not necessary if you install | 
 | the aoetools. | 
 |  | 
 |  | 
 | CREATING DEVICE NODES | 
 |  | 
 |   Users of udev should find the block device nodes created | 
 |   automatically, but to create all the necessary device nodes, use the | 
 |   udev configuration rules provided in udev.txt (in this directory). | 
 |  | 
 |   There is a udev-install.sh script that shows how to install these | 
 |   rules on your system. | 
 |  | 
 |   There is also an autoload script that shows how to edit | 
 |   /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when | 
 |   necessary.  Preloading the aoe module is preferable to autoloading, | 
 |   however, because AoE discovery takes a few seconds.  It can be | 
 |   confusing when an AoE device is not present the first time the a | 
 |   command is run but appears a second later. | 
 |  | 
 | USING DEVICE NODES | 
 |  | 
 |   "cat /dev/etherd/err" blocks, waiting for error diagnostic output, | 
 |   like any retransmitted packets. | 
 |  | 
 |   "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to | 
 |   limit ATA over Ethernet traffic to eth2 and eth4.  AoE traffic from | 
 |   untrusted networks should be ignored as a matter of security.  See | 
 |   also the aoe_iflist driver option described below. | 
 |  | 
 |   "echo > /dev/etherd/discover" tells the driver to find out what AoE | 
 |   devices are available. | 
 |  | 
 |   In the future these character devices may disappear and be replaced | 
 |   by sysfs counterparts.  Using the commands in aoetools insulates | 
 |   users from these implementation details. | 
 |  | 
 |   The block devices are named like this: | 
 |  | 
 | 	e{shelf}.{slot} | 
 | 	e{shelf}.{slot}p{part} | 
 |  | 
 |   ... so that "e0.2" is the third blade from the left (slot 2) in the | 
 |   first shelf (shelf address zero).  That's the whole disk.  The first | 
 |   partition on that disk would be "e0.2p1". | 
 |  | 
 | USING SYSFS | 
 |  | 
 |   Each aoe block device in /sys/block has the extra attributes of | 
 |   state, mac, and netif.  The state attribute is "up" when the device | 
 |   is ready for I/O and "down" if detected but unusable.  The | 
 |   "down,closewait" state shows that the device is still open and | 
 |   cannot come up again until it has been closed. | 
 |  | 
 |   The mac attribute is the ethernet address of the remote AoE device. | 
 |   The netif attribute is the network interface on the localhost | 
 |   through which we are communicating with the remote AoE device. | 
 |  | 
 |   There is a script in this directory that formats this information in | 
 |   a convenient way.  Users with aoetools should use the aoe-stat | 
 |   command. | 
 |  | 
 |   root@makki root# sh Documentation/aoe/status.sh  | 
 |      e10.0            eth3              up | 
 |      e10.1            eth3              up | 
 |      e10.2            eth3              up | 
 |      e10.3            eth3              up | 
 |      e10.4            eth3              up | 
 |      e10.5            eth3              up | 
 |      e10.6            eth3              up | 
 |      e10.7            eth3              up | 
 |      e10.8            eth3              up | 
 |      e10.9            eth3              up | 
 |       e4.0            eth1              up | 
 |       e4.1            eth1              up | 
 |       e4.2            eth1              up | 
 |       e4.3            eth1              up | 
 |       e4.4            eth1              up | 
 |       e4.5            eth1              up | 
 |       e4.6            eth1              up | 
 |       e4.7            eth1              up | 
 |       e4.8            eth1              up | 
 |       e4.9            eth1              up | 
 |  | 
 |   Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver | 
 |   option discussed below) instead of /dev/etherd/interfaces to limit | 
 |   AoE traffic to the network interfaces in the given | 
 |   whitespace-separated list.  Unlike the old character device, the | 
 |   sysfs entry can be read from as well as written to. | 
 |  | 
 |   It's helpful to trigger discovery after setting the list of allowed | 
 |   interfaces.  The aoetools package provides an aoe-discover script | 
 |   for this purpose.  You can also directly use the | 
 |   /dev/etherd/discover special file described above. | 
 |  | 
 | DRIVER OPTIONS | 
 |  | 
 |   There is a boot option for the built-in aoe driver and a | 
 |   corresponding module parameter, aoe_iflist.  Without this option, | 
 |   all network interfaces may be used for ATA over Ethernet.  Here is a | 
 |   usage example for the module parameter. | 
 |  | 
 |     modprobe aoe_iflist="eth1 eth3" | 
 |  | 
 |   The aoe_deadsecs module parameter determines the maximum number of | 
 |   seconds that the driver will wait for an AoE device to provide a | 
 |   response to an AoE command.  After aoe_deadsecs seconds have | 
 |   elapsed, the AoE device will be marked as "down".  A value of zero | 
 |   is supported for testing purposes and makes the aoe driver keep | 
 |   trying AoE commands forever. | 
 |  | 
 |   The aoe_maxout module parameter has a default of 128.  This is the | 
 |   maximum number of unresponded packets that will be sent to an AoE | 
 |   target at one time. | 
 |  | 
 |   The aoe_dyndevs module parameter defaults to 1, meaning that the | 
 |   driver will assign a block device minor number to a discovered AoE | 
 |   target based on the order of its discovery.  With dynamic minor | 
 |   device numbers in use, a greater range of AoE shelf and slot | 
 |   addresses can be supported.  Users with udev will never have to | 
 |   think about minor numbers.  Using aoe_dyndevs=0 allows device nodes | 
 |   to be pre-created using a static minor-number scheme with the | 
 |   aoe-mkshelf script in the aoetools. |