Storage technologies evolve toward a data-processing platform

An IT industry analyst article published by SearchDataCenter.

Emerging technologies such as containers, HCI and big data have blurred the lines between compute and storage platforms, breaking down traditional IT silos.

Mike Matchett

With the rise of software-defined storage, in which storage services are implemented as a software layer, the whole idea of data storage is being re-imagined. And with the resulting increase in the convergence of compute with storage, the difference between a storage platform and a data-processing platform is further eroding.

Storage takes new forms

Let’s look at a few of the ways that storage is driving into new territory:

  • Now in containers! Almost all new storage operating systems, at least under the hood, are being written as containerized applications. In fact, we’ve heard rumors that some traditional storage systems are being converted to containerized form. This has a couple of important implications, including the ability to better handle massive scale-out, increased availability, cloud-deployment friendliness and easier support for converging computation within the storage.
  • Merged and converged. Hyper-convergence bakes software-defined storage into convenient, modular appliance units of infrastructure. Hyper-converged infrastructure products, such as those from Hewlett Packard Enterprise’s SimpliVity and Nutanix, can greatly reduce storage overhead and help build hybrid clouds. We also see innovative approaches merging storage and compute in new ways, using server-side flash (e.g., Datrium), rack-scale infrastructure pooling (e.g., Drivescale) or even integrating ARM processors on each disk drive (e.g., Igneous).
  • Bigger is better. If the rise of big data has taught us anything, it’s that keeping more data around is a prerequisite for having the opportunity to mine value from that data. Big data distributions today combine Hadoop and Spark ecosystems, various flavors of databases and scale-out system management into increasingly general-purpose data-processing platforms, all powered by underlying big data storage tools (e.g., Hadoop Distributed File System, Kudu, Alluxio).
  • Always faster. If big is good, big and fast are even better. We are seeing new kinds of automatically tiered and cached big data storage and data access layer products designed around creating integrated data pipelines. Many of these tools are really converged big data platforms built for analyzing big and streaming data at internet of things (IoT) scales.

The changing fundamentals

Powering many of these examples are interesting shifts in underlying technical capabilities. New data processing platforms are handling more metadata per unit of data than ever before. More metadata leads to new, highly efficient ways to innovate …(read the complete as-published article there)

Scale-out architecture and new data protection capabilities in 2016

An IT industry analyst article published by SearchDataCenter.

January was a time to make obvious predictions and short-lived resolutions. Now is the time for intelligent analysis of the shark-infested waters of high tech. The new year is an auspicious time for new startups to come out of the shadows. But what is just shiny and new, and what will really impact data centers?

From application-focused resource management to scale-out architecture, here are a few emerging trends  that will surely impact the data center.

…(read the complete as-published article there)

Siloing stifles data center growth

An IT industry analyst article published by SearchDataCenter.

It’s time to knock down those silos, one by one. IT is transforming from a siloed set of reactive cost centers into a service provider with a focus on helping the business compete.

In the old days of IT, admins built clear silos of domain expertise; IT infrastructure was complicated. Server admins monitored compute hosts, storage admins wrangled disks and network people untangled wires. Implementing parallel domains seemed like the best way to optimize IT. The theory was that you could run IT as efficiently as possible, allowing experts to learn specialized skills, deploy domain-specific hardware and manage complex resources.

Except that dealing with multiple IT domains was never optimal for anyone in the data center. When IT is organized into silos, anytime there is problem — troubleshooting application performance, competing for rack space, or allocating a limited budget — the resulting bickering, finger-pointing and political posturing wastes valuable time and money. And heterogeneous infrastructure is not very interoperable, despite standardized protocols and thorough vendor validation testing.

Navigating a byzantine organization just to try out new things can stifle business creativity and innovation, but things are beginning to change. There is a massive shift in IT organization and staffing…

…(read the complete as-published article there)

Converged Infrastructure in the Branch: Riverbed Granite Becomes SteelFusion

(Excerpt from original post on the Taneja Group News Blog)

With today’s rebranding of Riverbed Granite as SteelFusion, Riverbed is prodding all branch IT owners (and vested users) to step up and consider what branch IT should ideally look like. Instead of a disparate package of network optimization, remote servers and storage arrays, difficult if not foresworn data protection approaches, and independently maintained branch applications and IT support, simple converged SteelFusion edge appliances sit in the branch to provide local computing performance but work on “projected” data that is actually consolidated and protected back in the data center.

…(read the full post)

Converging Facilities and Systems Management – Controlling Energy in the Data Center

(Excerpt from original post on the Taneja Group News Blog)

Looking back over 2012, it has really been the year of convergence. IT resource domain silos have been breaking down in favor of more coherent, cohesive, and unified architectures that look more like construction building blocks and less like electronics research computer labs. However, the vast majority of today’s data centers still have a long-established hard line between data center facilities and IT operations.

Power and cooling have always been integral to the data center, but have been managed disparately from the occupying compute and storage infrastructures. However, there are emerging technologies driven by green initiatives and by cost efficiency projects (perhaps most importantly to service providers) that are going to become increasingly important to the enterprise data center. Many of these technologies will enable a level of convergence between facilities and systems management.

As an EE who has always fallen on the compute side of the fence, I’m entranced by the idea of integrating end-to-end power and cooling management with advanced analytical and predictive systems management solutions, especially proactive performance and capacity management. An interesting power systems company, TrendPoint, focuses on what facilities folks call branch circuit power and cooling monitoring, and recently we had a chance to drill down into how that provides value to the larger data center “enterprise”.  They produce clever metering solutions that track power utilization to each circuit/rack, and also metering for heating and cooling at a fine grain too (all as SNMP data sources!).

With detailed energy usage and demand information, you can recoup energy costs by billing back to each data center client.  Similiarly correlatable heat mapping can translate to spot cooling requirements. Energy costs now can be accounted for as part of the service level negotiation. This is extremely valuable to service providers and colo data center operators, but can be used to help drive costs down in enterprise data centers of any size.

If you can then utilize this energy/heat information dynamically on the IT management side, the values to be gained beyond simple economic accountability are tremendous. You can imagine migrating virtual machines around a data center (or even to other regions) to dynamically balance power consumption and also to take advantage of better cooling patterns in order to fully maximize vm densities. But most interestingly you increase service reliability on the compute/storage side by avoiding tripping breakers and shutting whole racks of equipment down on the power side, and by keeping all equipment running within heat tolerances to avoid introducing failures and heat-induced errors. (Gotta keep that flash cool!)  

I see 2013 as the year that more data center sensors, more metering, and more raw information will be collected and leveragable than ever before (an emerging big data problem). And I imagine we’ll see new active controls on power and cooling – a real software-defined data center facility (actually, it is not that much of a stretch – see Nest for a consumer market active thermostat). And software-defined managed resources of all kinds will require new advanced operational intelligence (a machine learning challenge). So keep an eye on enabling solutions like those from TrendPoint, they are poised to help change the way data centers work.

…(read the full post)

InfiniBand as Data Center Communication Virtualization

(Excerpt from original post on the Taneja Group News Blog)

Recently we posted a new market assessment of InfiniBand and its growing role in enterprise data centers, so I’ve been thinking a lot about low-latency switched fabrics and what they imply for IT organizations. I’d like to add a more philosophical thought about the optimized design of InfiniBand and its role as data center communication virtualization.

…(read the full post)