All articles

Exploring The VMware Software-defined Datacentre (SDDC) – Part 1 – NSX History And Future

Author:

Rob Sims

Hybrid Platforms

•  Nov 02, 2023

With VMware Explore fast approaching (November 6th - 9th, in Barcelona), along with the increased conversations around building Hybrid Clouds and the need to address skills shortages in our sector, I thought it would be good to explore the state of the current VMware offerings in this space.

With many components to the solution, we won’t cover everything in a single article! For the first two parts, I am going to dive into the virtual networking portfolio.

In Part 1 we will look at: 

  • Exploring the VMware Software-Defined Datacentre (SDDC) 
  • Early years of virtual networking 
  • 2012 and NSX was born 
  • Going beyond Layer 4! 
  • Help for the Operations team 

In Part 2 we will move on to: 

  • From NSX-V to NSX-T 
  • Quick Segway to SASE and SEE 
  • Other notable steps on the software-defined networking journey 
  • Developer Ready 
  • How does all this innovation come together today?  
  • Future of Software-Defined Networking 

Where did it come from, what innovation have we seen over the last decade and what could the future hold, as we move towards Explore in Barcelona?

Exploring The VMware Software-defined Datacentre (SDDC) 

Before we dig into networking in detail, let's take a quick look at how it fits into the wider SDDC vision. The VMware SDDC architecture is designed to provide organisations with a validated design for deploying and operating a Hybrid Cloud platform, driving towards full automation, reduced downtime, and increased operational effectiveness. 

In the diagram below we can see the high-level outcomes of the SDDC architecture, covering areas like Automation, Operations, Business Continuity and Security.  

Virtual 
Infrastructure 
physical 
Infrastructure 
Service Catalog 
Self-Service Portal 
Hypervisor 
Pools of Resources 
Virtualization Control 
Compute 
Storage 
Business 
Fault Tolerance 
Backup & 
Restore 
Security and 
identity and Acc* 
Management 
Industry 
Security 
Life Cycle 
Management

VMware SDDC Overview 

One of the key concepts in the SDDC is Automation; driving down the operational overhead whilst increasing the speed of operations for both IT teams and those consuming the resources, like developers. A key enable for this automation is virtualisation, something VMware understands in intimate detail. While we know compute and storage virtualisation (ESXi & VSAN respectively) are key components, I believe it is the networking (especially in a multi-cloud world) that truly unlocks the SDDC capabilities. This is why I chose to start with Networking: the lifeblood of every architecture.  

Early Years Of Virtual Networking 

In 2002, with the introduction of the first virtual switch (vSwitch) to the ESX hypervisor VMware started its journey on the virtual networking road. The concepts were new and for the first time, we had networking and infrastructure teams having to collaborate.  

One of the interesting concepts of the time was the Cisco Nexus 1000v, a virtual Cisco switch that ran inside the ESX hypervisor. The concept was to allow the networking teams to have control and visibility of every port on the network, something they lost when thousands of IP endpoints vanished behind the VMware vSwitch.  

It would take a decade though, for the next big leap in innovation when NSX entered the market. 

2012 And NSX Was Born

In July 2012, VMware announced the 1-billion-dollar acquisition of Nicira and started the revolution of networking! The headlines at the time read: 

"Acquisition expands VMware’s networking portfolio to revolutionize networking for the cloud and provide a full suite of capabilities for any cloud environment" 

Around 12 months later NSX joined the VMware SDDC architecture, and the future of virtual networking would never be the same. To provide context, solutions like Cisco ACI had not been launched and alternative solutions for micro-segmentation (one of the battlegrounds of the time) like Illumio had not yet come to market. This made NSX a game-changer in both the networking and virtualisation markets.  

Based on VXLAN encapsulation technology, NSX abstracted the switching, routing, and firewalling from the underlying hardware and managed it inside the hypervisor as a policy. Allowing for fast programable networks to be built, torn down, and rebuilt as use cases demanded.  Bringing the ability to create a Layer 2 network construct that could exist over a routed Layer 3 network NSX changed how we could consider network, datacentre and application construction. 

The other big advancement that NSX introduced was the distributed firewall (DFW). Allowing for Layer 4 controls to be placed at the virtual NIC of every Virtual Machine (VM) in the environment. Removing the traditional challenges of east-west security by scaling firewall throughput with every ESX node and moving the security boundary to each and every virtual workload. This concept is shown in the image below: 

Distributed Firewall

NSX Distributed Firewall 

This was the birth of the castle vs hotel analogies (if this is unfamiliar reach out and we can chat) and the market of micro-segmentation. It also brought up a whole world of other considerations, such as, “do we understand our application architectures in enough detail to deploy these new security principles?” 

Going Beyond Layer 4! 

Providing Layer 4 network segmentation was a great leap forward in the security paradigm for private clouds and traditional IT architectures it still had some gaps. One of the key issues was the ‘bad guys’ hiding inside known good ports and bypassing those L4 protections.  

In the early days of NSX VMware enabled Layer7 traffic inspection through an ecosystem of next-gen firewall partners and a concept known as service chaining. Service chaining allowed the NSX engine to 'forward' certain traffic to an external (to ESX) firewall for inspection at L7. While this allowed for greater levels of security and visibility it had some inherent challenges, mainly those of cost, complexity, and throughput. The concept was a great story but broke the concept of a distributed firewall, which was core to NSX performance and operational simplicity.  

The story around a service-defined firewall was not going to be halted by such trivial things as those early challenges. With the acquisition of Lastline in 2020, VMware was able to bring these capabilities into the core NSX product line and the three steps shown below became the first vision for the fully fledged Service-Defined Architecture.  

Enhancing Micro-segmentation beyond Layer 4 

Another advantage of having a software-defined network is we can offer extended services beyond core switching and routing, for example, load balancing. This allows consumers to consolidate technical debt into the NSX architecture. Early versions of NSX had a solid Load Balancer, but lacked some of the more valuable services a true Application Delivery Controller (ADC) would provide. As we moved to the NSX-T era (see below) the capabilities took off.  The following features are now integrated into the NSX Load Balancer: 

  • L4–L7 Load Balancing 
  • Predictive Autoscaling 
  • Automation and Programmability 
  • Integrations and Analytics 
  • Web Application Security 
  • Kubernetes Ingress Services 

Combining the core capabilities of NSX DFW, Lastline and the ADC features allows us to see the modern and complete vision of the Service-Defined Firewall. 

VMware Service-defined Firewall 

Help For The Operations Team

All this innovation in did not come without its challenges; for the first time we had to consider all the network flows in our data centre. Those flows going north/south had been firmly understood for years (protect the perimeter) but now we needed to understand the east/west traffic (between workloads inside the data centre) and that was an entirely new ball game! 

Applications had been developed and deployed for years, with no need to consider such flows and we all know how well such things got documented, or not. The early days of NSX required a degree in Wireshark and a lot of man hours spent untangling years of mess, it was a painful time! 

Luckily VMware introduced vRealise Network Insight (vRNI); a tool that could collate all the flow data from the virtual switch and provide a visual representation. This was a huge step forward in being able to understand and map those east-to-west flows. A challenge still existed though, the automation of rule creation and management was still manual. At the same time, we started to see the need to provide different information to the network team and the security team, with the latter taking more responsibility for the creation of a segmentation policy. This split of NetOps and SecOps requirements led to the introduction of NSX Intelligence.  

vRNI was set on a trajectory to augment and enable the NetOps teams that had direct responsibility for the network. The focus for the tool morphed to meet the needs of these teams and focus less on the direct micro-segmentation or security requirements. Providing the network professionals with in-depth data to ensure optimal packet delivery. 

NSX Intelligence took up the role of enabling the SecOps teams and one of the key functions it brought was automated rule creation (following a learning period and with admin approval). This advance brought the barrier to adoption right down, reducing the effort and time required to manage a segmented network.  

I am particularly interested to see what advancements we will see in NSX Intelligence at Explore as it’s an area that could continue to bring incremental value. 

Final Thoughts For Part 1 

From basic L4 segmentation to full L7 inspection with Intrusion detection and response; NSX has advanced a long way in a short period of time. Following the evolution of NSX over the last decade has been fascinating, I am looking forward to digging into it more in Part 2.  

Hope to see you in Barcelona at Explore if you have any questions, please reach out. If you would like to hear all the updates form Explore keep an eye out for our UK event on the November 22nd.

Contributors
  • Rob Sims

    Chief Technologist - Hybrid Platforms

Share
Subscribe to email updates

Related insights

Exploring The Vmware Software Defined Datacentre Sddc Part 4 Vsan New Architecture And Future Innovations Summary
  • Hybrid Platforms

Exploring the VMware Software-Defined Datacentre (SDDC) – Part 4 – vSAN: New Architecture and Future Innovations

Part 4 of the VMware Software-Defined Datacentre series, focusing on the VMware portfolio. In this part, we are continuing the vSAN discussion with a focus on the new architecture and future innovations.

Read article
EXPLORING THE VMWARE SOFTWARE DEFINED DATACENTRE SDDC PART 3 VSAN REDEFINING STORAGE
  • Hybrid Platforms

Exploring the VMware Software-Defined Datacentre (SDDC) – Part 3 – vSAN Redefining Storage

Read Part 3 of Robert Sims' blog on the VMware Software-Defined Datacentre, and learn how VMware vSAN is redefining data storage for VMware.

Read article
EXPLORING THE VMWARE SOFTWARE DEFINED DATACENTRE SDDC PART 2 NSX HISTORY AND FUTURE
  • Hybrid Platforms

Exploring the VMware Software Defined Datacentre (SDDC) – Part 2 – NSX History and Future

Part 2 Insights from Rob Sims, CDW's Chief Technologist - Hybrid Platforms, as he explores the VMware Software-Defined Datacentre (SDDC). Read about the history of NSX and what the future holds for the Software-Defined Datacentre.

Read article