"ivy" is a Linux-based block storage synthetic workload generator and benchmarking system.

• This requires a 5.5 or newer Linux Kernel.
• Executables for ivy 3 using "aio" are also available.

• Low CPU overhead - Written in C++ using Linux native Asynchronous I/O interface to issue multiple I/Os in parallel from one workload thread.
• Vendor independent design - LUN attributes decoded by external vendor specific SCSI Inquiry-based LUN discovery tool become selectable in ivy.
  • The Hitachi LUN_discovery toolset is at https://github.com/Hitachi-Data-Systems/LUN_discovery
• Dynamic feedback control - PID loop feature enables for example "find IOPS at 1 ms response time".
• Test in minimum time - Will run a test as long as necessary to measure to specified plus/minus accuracy.
• Easy test setup - Select by vendor LUN attribute such as subsystem logical device, port, parity group, pool ID, etc. instead of by Linux /dev/sdxxx name.
• Sliced and diced data - [CreateRollup] "port"; will print csv files with data rolled up by port.
• Configuration validation - [CreateRollup "port" [quantity] 16 [MaxDroop] "20%"; will invalid results unless 16 ports are reporting and the IOPS for the slowest port is at most 20% below the IOPS for the fastest port.
• Workflow automation - .ivyscript programming language with "C/C++"-like syntax.

Authorized Hitachi internal users in our labs with the code and license key can use the "command device connector", which retrieves configuration and real-time performance monitor data from Hitachi subsystems.

Other vendors are encouraged to develop their own "connectors" for ivy.

With the Hitachi command device connector, ivy

• Augments the SCSI Inquiry attributes with logical device (LDEV) attributes obtained from the subsystem through the command device.
• Retrieves subsystem performance information synchronized and aligned with ivy host data.
• Filters real-time subsystem data by ivy "rollup instance" enabling fine-grained dynamic feedback control of IOPS to hit a target utilization of a component inside the subsystem.

Look in the doc subfolder and read the "Introduction to ivy" PowerPoint or .pdf presentation material, or else watch the "Introduction to ivy" video (link below).

See src subfolder. ivy is built using the CodeBlocks IDE.

Download the latest version version of the ivy & LUN_discovery binaries from https://hcpanywhere.hitachivantara.com/u/tERhdz1EYf-mRGF2/ivy.bin.zip?l

• When you unzip the ivy.bin.zip file, you will get a folder whose name includes the version number, so you won't overwrite any earlier downloads, and you can easily see if it's a new version.

If you are running a Linux kernel version lower than 5.5, here is the download link for ivy 3 executables: https://hcpanywhere.hitachivantara.com/u/oKCIrSmgiD2yoIis/ivy3.bin.zip?l

LUN_discovery on github

Download from https://hcpanywhere.hitachivantara.com/u/9K41Z32uNePmrjVm/ivy_sample_output.zip?l

See the doc subfolder.

(These videos are now a bit dated, sorry. Please take a look at the sample output - link just above - to see what these demos look like with the current ivy version. The PID loop feature has in particular been greatly simplified.)

Executive summary

• Standalone 6-minute executive summary clips from "introduction to ivy"

Video series

• Introduction to ivy
• Installing ivy
• Introduction to ivy output csv files
• demo0 DF - LUN discovery
• demo0 RAID - LUN discovery with command device connector
• demo1 fixed DF - testing for a fixed duration
• demo1 fixed RAID - real time subsystem data with command device connector
• demo2 [CreateRollup]
• demo3 layered workloads
• demo5 [EditRollup] - read vs. write example
• [Go] statement 1 of 3
• [Go] statement 2 of 3 - the focus rollup
• [Go] statement 3 of 3 - measure=on and the PID loop feature

Sample output from the demos is in the sample_output folder.

• Some of the demos also have a video - see list of videos above.
• Some of the demos come in two flavours, "DF", and "RAID".
• The "DF" version was done on an HDS AMS2100 subsystem.
• The "DF" demos show the ivy functionality that is available on any vendor's equipment.
• To adapt for another vendor's equipment, to specifically select some but not all LUNs we would need an equivalent to the the LUN_discovery tool to decode the attributes of that vendor's LUNs, such as the equivalent of Hitachi LDEVs (logical devices).
• The "RAID" demos were performed on a Hitachi RAID family subsystem with a command device configured to one of the test hosts, and where the Hitachi proprietary "ivy command device connector" software was installed on the test host together with a subsystem license key.
  • The command device connector software is proprietary and is not part of the ivy open source project. It is available only for authorized internal Hitachi lab use.
  • These RAID demos have more detailed subsystem configuration information retrieved from the command device, as well as subinterval-by-subinterval real-time performance data retrieved from the command device.
• The RAID demos are provided for two reasons
  1. To illustrate the full functionality of ivy.
  2. To show other vendors the functionality they can implement by developing their own connectors to their own subsystem products.
• Where there are both DF and RAID versions of the videos / demos, it is recommended to watch the DF version first and then the RAID version, as the RAID version generally builds on the DF version.
• Some demos which depend on having a command device connector such demos for performing dynamic feedback control on subsystem internal MP % busy or subsystem Parity Group % busy come only in one flavour. Other vendors by developing their own connectors could develop similar functionality.

ian.vogelesang@hitachivantara.com, Kumaran Subramaniam, stephen.morgan@hitachivantara.com

Apache 2.0