Virtualizing Hadoop Impacts Big Data Storage

An IT industry analyst article published by Enterprise Storage Forum.

by Mike Matchett, Sr. Analyst, Taneja Group
Hadoop is soon coming to enterprise IT in a big way. VMware’s new vSphere Big Data Extensions (BDE) commercializes its open source Project Serengeti to make it dead easy for enterprise admins to spin and up down virtual Hadoop clusters at will.

Now that VMware has made it clear that Hadoop is going to be fully supported as a virtualized workload in enterprise vSphere environments, here at Taneja Group we expect a rapid pickup in Hadoop adoption across organizations of all sizes.

However, Hadoop is all about mapping parallel compute jobs intelligently over massive amounts of distributed data. Cluster deployment and operation are becoming very easy for the virtual admin. But in a virtual environment where storage can be effectively abstracted from compute clients, there are some important complexities and opportunities to consider when designing the underlying storage architecture. Some specific concerns with running Hadoop in a virtual environment include considering how to configure virtual data nodes, how to best utilize local hypervisor server DAS, and when to think about leveraging external SAN/NAS.

The main idea behind virtualizing Hadoop is to take advantage of deploying Hadoop scale-out nodes as virtual machines instead of as racked commodity physical servers. Clusters can be provisioned on-demand and elastically expanded or shrunk. Multiple Hadoop virtual nodes can be hosted on each hypervisor physical server, and as virtual machines can be easily allocated more or less resource for a given application. Hypervisor level HA/FT capabilities can be brought to bear on production Hadoop apps. VMware’s BDE even includes QoS algorithms that help prioritize clusters dynamically, shrinking lower-priority cluster sizes as necessary to ensure high-priority cluster service.

…(read the complete as-published article there)

How Delicate is Your Virtual Egg Basket?

An IT industry analyst article published by Virtualization Review.

Server virtualization has taken us to places we’ve never been before, but there is some truth to that old adage about having seen it all before.

article_secure-cloudSome have said there is nothing new under the sun, that it all comes around in circles. We think server virtualization has taken us to places we’ve never been before, but there is some truth to that old adage about having seen it all before. As has happened in many previous technology adoption cycles, first we struggle with assuring sufficient “correctness” and availability, then we work hard to guarantee performance, and as a third act, we have to eventually harden the solution to both internal and external threats.

With virtualization, this cycle is perhaps more acute in that the whole point is to aggregate many clients and users into one cost-efficient shared resource pool. And the corresponding infrastructure convergence of formerly disparate IT silos concentrates the number of subject matter experts and admins while expanding the end-to-end scope and control of this talented remainder.

Security Is the Third Stage

Security should never be an afterthought, but in our rush to get out ahead of the competition — or even just survive economically to play another day — we stand something up as quick as we can just to see if it can be done. Then as we come to rely on it for day-to-day operations, we discover that it matters when it falls over or performs badly.

Enter big systems management corrections with add-on monitoring, automation, and optimization solutions. But security concerns may still seem a vague threat and an acceptable risk until we start really leveraging the new technology for our mission-critical applications — the most vulnerable “eggs” in our portfolio.

Many organizations have reached that third stage in virtualization whether they know it or not.

…(read the complete as-published article there)

Don’t Miss These VMworld 2013 Sessions

An IT industry analyst article published by Virtualization Review.

With 358 sessions, time is money. Here are five sessions where your time will be well spent.

article_dont-miss-these-vmworld-2013-sessionsTAKE 1 Directions in VMware EUC & the Multi-Device, Virtual Workspace (EUC4544)

Virtual desktop infrastructure (VDI) and related end-user computing capabilities are most definitely not dead. In fact, I think the technologies are finally starting to support practical and cost-effective implementations for every size business. VMware Horizon and Mirage likely have some hot things going, and it’s always interesting to see how PCoIP has evolved.

TAKE 2 Designing Your Next-Generation Datacenter for Network Virtualization (NET5184)

Can you spell VXLAN? If you come away from VMworld 2013 with a good understanding of software-defined networking (SDN) and network virtualization, you could be the geek hero of your IT shop.

…(read the complete as-published article there)