Tag Archives: Paging

Revisiting The zSeries Mainframe Storage Hierarchy

Posted on by

Recommendation: The next time you perform a zSeries Mainframe server upgrade, consider adding Flash Express cards, for an extra 1.4-5.6 TB of memory speed storage. Similarly, the next time you perform a zSeries Mainframe DASD subsystem upgrade, consider adding as much SSD (flash memory) capability that you can afford and justify. Both upgrades will deliver significant performance and business benefits, arguably for minimal cost, when considered as a several year TCO investment.

Conceptually the zSeries Mainframe storage hierarchy has comprised the same layers for many decades, while performance and capacity attributes have dramatically increased over time. Although System/390 introduced the concept of Expanded Storage (I.E. Hiperspace, Data Space) in 1990 and there have been various implementations of SSD (E.g. StorageTek 4080), the ability to transparently implement significant capacity memory layers has only recently become possible.

Let’s not forget, the closer data is to that most precious and expensive of resources, namely CPU, the faster it will process. When revisiting the traditional storage hierarchy, we can now consider two new layers, namely Flash Express and Solid State Drive (SSD):

zSeries Storage Hierarchy

I have previously written about the Flash Express layer. Flash Express is a new memory layer within the zSeries Mainframe storage hierarchy, which can be considered as either a Solid State Drive (SSD) or Storage Class Memory (SCM) technology. Flash Express is integrated on PCI Express attached RAID 10 Cards, packaged as a two card pair, each with a 1.4 TB capacity per mirrored card pair. A maximum of 4 card pairs can be configured, delivering up to 5.6 TB of memory capacity, assigned to LPAR resources, just like main memory.

The simplest function to benefit from Flash Express memory would be SVC dump processing, substantially reducing dump capture time.

Flash Express can also be deployed to replace z/OS disk paging, substantially reducing the response time associated (I.E. ~5-20 μs vs. ~10 ms). The benefit for z/OS paging is not the replacement of memory paging, but replacing disk paging with Flash Express storage. Flash Express is suitable for workloads that can tolerate paging, but will not benefit workloads that cannot tolerate paging activity. The fundamental z/OS design for Flash Express memory will not completely remove any virtual storage constraints created by a paging spike, although a modicum of scalability relief is expected due to the faster I/O associated with Flash Express memory.

In conjunction with Flash Express, there were advancements in the Real Storage Management (RSM) function, including pageable 1MB Large Page Support. Large Pages (1MB) deliver benefit, with increased performance, decreasing the number of Translation Lookaside Buffer (TLB) misses that an application incurs, reducing time when converting virtual addresses into physical addresses and reduced real storage usage to maintain DAT structures. The use of Large Pages typically deliver Internal Throughput Rate (ITR) performance benefits of ~1% for IMS, ~3% for DB2 and ~5% for Java workloads.

Although SSD (flash) storage might have been selectively deployed in the zSeries Mainframe Data Centre for the last 5 years or so, the ever increasing requirement for increased Quality of Service (QoS) in terms of data availability and ultra-fast transaction response times dictate the increased usage of SSD architectures. Entire DASD subsystems can be built upon SSD technologies, or more likely, hybrid subsystems, containing both SSD and traditional HDD technologies. This storage subsystem evolution allows organizations to gain significant competitive advantages, delivering new services for existing and more importantly, new customers alike.

Using SSD disk subsystems, overcomes the limitations of traditional spinning hard disk drives. However, not every enterprise application needs this ultra-high performance; since flash storage still costs more than spinning drives for the same capacity, organizations must be mindful of expenditure and now much flash memory (SSD) they deploy; as always, flexibility is key.

Complete or hybrid SSD I/O subsystems deliver performance and economic advantages for your mission critical business environment:

  • Green Data Centre: ~25-60% energy reduction (flash memory vs. spinning disk)
  • Data Centre Space: ~20-40% smaller footprint (memory cards vs. Hard Disk Drives)
  • Optimal Performance: Consistent ~1-3 ms access (Hard Disk Drives @ ~10 ms)

The utopia is for a self-tuning disk subsystem, automatically redirecting I/O between SSD and HDD, based on file performance and overridden, as and when required, by storage policies. Whether EMC, HDS (HP OEM) or IBM, this self-tuning ability is evolving, while each disk vendor has their own implementation. However, whatever your choice of disk subsystem, the ability to incorporate SSD into your storage hierarchy, either full or partial is evident.

In conclusion, ~25 years ago, the zSeries Mainframe user benefitted from faster performance via System/390 Expanded Storage and disk subsystems with cache and DASD Fast Write memory buffers. The cost of such memory storage was a major consideration then, but with good I/O tuning disciplines, the savvy zSeries Mainframe user benefitted from these technology advancements. Flash Express and SSD deliver the potential to deliver increased performance, for a relatively low cost, and now is the time to embrace these technologies. Ignore the storage hierarchy at your peril and as I previously documented, optimal I/O performance always delivers significant benefit.

Flash Express – Back To The Future

Posted on by

It’s just not science fiction films that get rebooted or reimagined, the same thing happens in the technology world, although not always as obviously.  Over the years we have seen Star Trek, Star Wars, and numerous superhero stories be updated for current day audiences, some better than others, and similarly, flash memory or Solid State Drives (SSD) is not a new concept in the Mainframe world.  For me, I wonder whether the Back To The Future films might be rebooted, and if they do, hopefully it’s done well!  Anyway, I digress, so back to the Mainframe flash memory observations… 

Flash Express is a new feature for the zEC12 server, designed to help drive System availability and performance to even higher levels. 

Flash Express cards are delivered as a RAID 10 mirrored pair for superior resiliency and reliability.  In the unlikely event of device failure, Flash Express cards can also be concurrently replaced.  The cards are designed for superior wear levelling and have a long expected lifetime.  From a security perspective, Flash Express stored data remains protected.  Data is encrypted on the Flash Express adapter with 128-bit AES encryption.  Encryption keys are stored on smart cards that are plugged into the SE.  Removing the smart cards renders the data on the card inaccessible. 

In the first instance, IBM seems to be targeting Systems Paging (I.E. Auxiliary Storage Manager – ASM) and System Dumps (E.g. SVC, Standalone) as an introduction to this new level of storage hierarchy. 

Flash Express reduces latency for critical system paging that might otherwise impact the availability and performance.  For system paging flexibility and efficiency, Flash Express is a higher performance option when compared with traditional auxiliary storage (I.E. Disk).  z/OS uses both Flash Express and page data sets for auxiliary storage by paging data to the preferred storage medium first, based on response times, data set characteristics, and other parameters.  Wherever possible, the system will page first to Flash Express resulting in faster performance.  Especially for data intensive applications the use of Pageable Large Pages with Flash Express enables the transfer of large amounts of data at faster speeds, which can result in improved performance for DB2 analytic workloads.

NB. Because Flash Express is not persistent across IPL events, it cannot be used for Virtual I/O or PLPA data used in warm starts.  VIO and PLPA datasets must still be defined on DASD.

During diagnostic collection, as in SVC or standalone dumps, IBM states that systems can become sluggish effectively rendering key systems unavailable.  When data is transferred into main memory as part of a dump event, the fast I/O rates and low latency associated with Flash Express provide decreased first failure data capture time, and faster page-ins of the critical pages needed for dump creation.  This allows the system to return to normal workload performance faster, without incurring extra delays.

So clearly, adding a flash memory layer to the Mainframe storage hierarchy can only deliver benefit, and over time, seemingly Flash Express can be used for other I/O intensive applications, further improving response times and decreasing associated CPU cycles.

No doubt the non-Mainframe technician might say “so what, we’ve been doing flash and SSD for nearly 20 years.  Typical Mainframe, always behind the rest of the IT world”!

Hmmm, perhaps not, hence the Back To The Future reference.  Depending on your viewpoint, and perhaps urban myth, from a release date viewpoint, but in the mid-to-late 1980’s, StorageTek introduced a device called the 4080, AKA Control Data Set Manager (CDSM) for their Host Software Component (HSC) product.  You will still see this 4080 device referenced in the HSC Systems Programmer’s Guide technical manual.  The 4080 is a Solid State Device, utilizing memory storage, emulating 3380 and 3390 device types, allowing z/OS (MVS) data sets to be created and accessed, as and if required.  Arguably if not certainly, this was the first instance of logical and physical device separation for Mainframe disks, and urban myth might dictate that this StorageTek 4080 technology was used or at least borrowed for the first EMC Symmetrix disk subsystem…

StorageTek typically packaged this 4080 device within a NearLine (ATL) solution sale and customers used this SSD device for HSC Control Data Sets, but also other high I/O files, such as the IMS Write-Ahead Data Set (WADS).  Therefore, Flash Express isn’t the first SSD solution for the Mainframe, and similarly, Flash Memory/Cards, USB Sticks, et al, aren’t the first SSD type technology in the IT world.

As somebody (Machiavelli) far wiser than I once said “whoever wishes to foresee the future must consult the past”.  The longevity of the Mainframe, nearly 50 Years of technical innovation, largely dictates that technological ideas are at least borrowed from the Mainframe, whether System Paging, Fibre Channels, Storage Area Networking (SAN), Virtual Storage, Flash/SSD Memory, naming but a few.

Should today’s zEC12 Mainframe customer deploy Flash Express?  Without doubt, but to deliver maximum ROI and benefit, it should be considered as more than a faster paging and dump solution, and we look forward to how IBM will further enhance this product offering.