Nexus
A Nexus, in the world of Storage Networking is a unique identifier. It identifies the components involved in a command or sequence of commands. It is normally made up of the following parts:
Initiator (I), Target (T), LUN (L), and Queue Tag (Q). This is an ITLQ nexus.
Establishing the Nexus
In parallel SCSI (and most storage systems), everything starts on the host side of the system. When a host wants move some data (do a READ or WRITE) it will need to know that a storage device exists, and then build and send a command to it. But there is another SCSI protocol that takes place to connect and issue the command. That protocol is below.
- Arbitration - initiator ID is placed on the bus (usually #7).
- Selection - target ID is place on the bus and target takes control. At this point, an I_T nexus exists.
- Message(s) – target asks for bytes to identify the LUN and Q-Tag (if needed). This is where the full I_T_L_Q nexus is established.
- Command - CDB is sent to the device, which processes the command and controls the flow.
- Data - this is the data movement phase (IN for READ, OUT for WRITE).
- Status - information is sent back to the host (command worked or had an error).
- Message - this tell the host that the command is done.
Once the command is finished, the nexus is “released” and can be used again. This is because the nexus is only needed while the command is in progress.
Disconnect and Reconnect
The protocol shown above shows a single connection, where everything is done during that connection. But where the nexus really comes into play is when there is a disconnect and reconnect after the command had been issued. This is one of the biggest benefits of “networked” storage. While the storage devices are disconnected, the host can connect to another device and start a different command with it.
The important part is that while a command is still in process (no status yet), the nexus remains and is known to the host. The storage device also can do “slow stuff” while disconnect, such as moving the heads, spinning the platters, that kind of thing. When the device has the data ready (READ), or has resources to accept more data (WRITE), then it will use the same nexus values (I_T_L_Q) and will reconnect to the original host to continue the command sequence. This can happen multiple times for a single command, and the nexus remains until the command is complete.
Tagged Command Queuing
Tagged Command Queuing (TCQ) is a function that allows multiple commands to be sent to a SCSI disk drive (includes Fibre Channel drives), and now some SATA drives. The “twist” when you do command queuing on SATA drives, is that they don’t used the same name – they call it Native Command Queuing (NCQ). I don’t think this is a good name because it might imply that command queuing is or was native in the SATA world. But I suppose the name was picked more for sales people and not technically oriented geeks like me (off soapbox now).
Why Command Queuing?
So you may be asking the question: “Why do we do queuing?” There are really a few main reasons which are:
Speed, efficiency, less stress on the physical parts, which results in longer device lifetime.
How Command Queuing Works
I will try to keep this as simple as possible by using an analogy or two. Try to think of a disk drive as an elevator, and the sectors (or blocks) where it stores the data are labeled from zero (0) to some maximum number (let’s say 100 to keep it easy). Each command that accesses data will include a “Where” field, which is normally called the Logical Block Address (LBA). Think of that as the “floor number” of this elevator.
Now imagine that the disk drive is the elevator sitting at Floor 0 (ground floor) and the operating system sends 5 commands to the drive, that all go to different floors and in a different order. For example
- Command 01 moves data to floor 90 (LBA=90)
- Command 02 moves data from floor 10 (LBA=10)
- Command 03 moves data to floor 80 (LBA=80)
- Command 04 moves data from floor 20 (LBA=20)
- Command 05 moves data to floor 70 (LBA=70)
The doors close and the elevator starts to do its thing (move the commands and data). Now if the elevator operator (the device firmware) had to do each of these commands in the order they were received, then that would take quite a bit of time and the elevator would have to move quite a distance between each command (first to floor 90, then to floor 10, then back to 80, and so on). As you can imagine, that would also add quite a bit of physical movement and is not very efficient.
But now think about how a real elevator works. These same five commands jump on the elevator at the ground floor and the doors close. What happens next? The elevator doesn’t care about the sequence the buttons for each floor were pressed, it just cares about which floor is closest to its current location, and what direction it is going. This simple “elevator algorithm” would result in the following command execution sequence:
- Command 02 moves data from floor 10 (LBA=10)
- Command 04 moves data from floor 20 (LBA=20)
- Command 05 moves data to floor 70 (LBA=70)
- Command 03 moves data to floor 80 (LBA=80)
- Command 01 moves data to floor 90 (LBA=90)
Note that the elevator only moved between floors (moved the head on the drive) in the most efficient manor, and did not bounce back and forth (seek) multiple times. The result is that the overall processing time will be reduced and there is less physical movement when compared to doing the commands in the original order that they were given.
So its all about speed and efficiency. If you have any questions, leave a comment.
Incoming search terms for the article:
Bits Bytes and Blocks
When discussing Storage Networking, or getting into the “nuts and bolts” of commands and moving data around, the topic of Bits, Bytes, and Blocks almost always seems to come up. It usually starts with a question like “where can I find the bit that turns on (or off) a particular function.”
For example, there is a bit called “IMMED” (Immediate) in several SCSI commands (CDBs) that tells the device to send back Status immediately. This is not really true, if you read the details about that particular bit, but the important part is where will you find this bit and how would you identify it in a precise manor. That is where the bits and bytes come in.
Bits and Bytes
The example above shows that this particular bit is Bit zero (0) of Byte one (1), in this particular command. Where the confusion comes in is usually based on what a person has worked with in the past and what they are used to.
In some environments, people will often start with bit 0 on the far left and move up as they move to the right. But since most of the storage industry started with bit 7 on the left of each byte, and count down as they move to the right, this is the way it is displayed and labeled in the SCSI Standards.
Also note that in SCSI land (the land of storage) when we start counting bytes, we start with zero (0) and count up from there (even if the bytes are shown going down the page). That is why this 6-byte command (or 6-byte CDB) starts with byte 0 and goes to by 5 – a total of six bytes. The key for most people is to be flexible, and get to know the system that you are working with.
Blocks
That brings us to Blocks, and the big question of “How big is a Block?” The real answer to that question is … It depends! What is depends on is the device type that you are talking to. Since a large amount of people in the storage industry deal with disk drives, the “typical” answer is – 512 bytes. I say “typical” because I have worked with systems that have larger block sizes, such as 520 or 528 byte blocks. These are usually hard drives that are installed in special RAID arrays and have been formatted for that system.
When you talk with different device types, the block size may be different. For example, many of the optical media devices I have worked with use a 2K byte block size. Again, you have to be flexible and have the ability to find out how the device you are talking to is used most of the time.
If you have any questions, just leave a comment here.