Exploring the VMware Software-Defined Datacentre (SDDC)
Before we dig into networking in detail, let's take a quick look at how it fits into the wider SDDC vision. The VMware SDDC architecture is designed to provide organisations with a validated design for deploying and operating a Hybrid Cloud platform, driving towards full automation, reduced downtime, and increased operational effectiveness.
In the diagram below we can see the high-level outcomes of the SDDC architecture, covering areas like Automation, Operations, Business Continuity and Security.
VMware SDDC Overview
One of the key concepts in the SDDC is Automation; driving down the operational overhead whilst increasing the speed of operations for both IT teams and those consuming the resources, like developers. A key enable for this automation is virtualisation, something VMware understands in intimate detail. While we know compute and storage virtualisation (ESXi & VSAN respectively) are key components, I believe it is the networking (especially in a multi-cloud world) that truly unlocks the SDDC capabilities. This is why I chose to start with Networking: the lifeblood of every architecture.
Early years of virtual networking.
In 2002, with the introduction of the first virtual switch (vSwitch) to the ESX hypervisor VMware started its journey on the virtual networking road. The concepts were new and for the first time, we had networking and infrastructure teams having to collaborate.
One of the interesting concepts of the time was the Cisco Nexus 1000v, a virtual Cisco switch that ran inside the ESX hypervisor. The concept was to allow the networking teams to have control and visibility of every port on the network, something they lost when thousands of IP endpoints vanished behind the VMware vSwitch.
It would take a decade though, for the next big leap in innovation when NSX entered the market.
2012 and NSX was born.
In July 2012, VMware announced the 1-billion-dollar acquisition of Nicira and started the revolution of networking! The headlines at the time read:
"Acquisition expands VMware’s networking portfolio to revolutionize networking for the cloud and provide a full suite of capabilities for any cloud environment"
Around 12 months later NSX joined the VMware SDDC architecture, and the future of virtual networking would never be the same. To provide context, solutions like Cisco ACI had not been launched and alternative solutions for micro-segmentation (one of the battlegrounds of the time) like Illumio had not yet come to market. This made NSX a game-changer in both the networking and virtualisation markets.
Based on VXLAN encapsulation technology, NSX abstracted the switching, routing, and firewalling from the underlying hardware and managed it inside the hypervisor as a policy. Allowing for fast programable networks to be built, torn down, and rebuilt as use cases demanded. Bringing the ability to create a Layer 2 network construct that could exist over a routed Layer 3 network NSX changed how we could consider network, datacentre and application construction.
The other big advancement that NSX introduced was the distributed firewall (DFW). Allowing for Layer 4 controls to be placed at the virtual NIC of every Virtual Machine (VM) in the environment. Removing the traditional challenges of east-west security by scaling firewall throughput with every ESX node and moving the security boundary to each and every virtual workload. This concept is shown in the image below:
NSX Distributed Firewall
This was the birth of the castle vs hotel analogies (if this is unfamiliar reach out and we can chat) and the market of micro-segmentation. It also brought up a whole world of other considerations, such as, “do we understand our application architectures in enough detail to deploy these new security principles?”
Going beyond Layer 4!
Providing Layer 4 network segmentation was a great leap forward in the security paradigm for private clouds and traditional IT architectures it still had some gaps. One of the key issues was the ‘bad guys’ hiding inside known good ports and bypassing those L4 protections.
In the early days of NSX VMware enabled Layer7 traffic inspection through an ecosystem of next-gen firewall partners and a concept known as service chaining. Service chaining allowed the NSX engine to 'forward' certain traffic to an external (to ESX) firewall for inspection at L7. While this allowed for greater levels of security and visibility it had some inherent challenges, mainly those of cost, complexity, and throughput. The concept was a great story but broke the concept of a distributed firewall, which was core to NSX performance and operational simplicity.
The story around a service-defined firewall was not going to be halted by such trivial things as those early challenges. With the acquisition of Lastline in 2020, VMware was able to bring these capabilities into the core NSX product line and the three steps shown below became the first vision for the fully fledged Service-Defined Architecture. 
Enhancing Micro-segmentation beyond Layer 4
Another advantage of having a software-defined network is we can offer extended services beyond core switching and routing, for example, load balancing. This allows consumers to consolidate technical debt into the NSX architecture. Early versions of NSX had a solid Load Balancer, but lacked some of the more valuable services a true Application Delivery Controller (ADC) would provide. As we moved to the NSX-T era (see below) the capabilities took off. The following features are now integrated into the NSX Load Balancer:
- L4–L7 Load Balancing
- Predictive Autoscaling
- Automation and Programmability
- Integrations and Analytics
- Web Application Security
- Kubernetes Ingress Services
Combining the core capabilities of NSX DFW, Lastline and the ADC features allows us to see the modern and complete vision of the Service-Defined Firewall. 
VMware Service-defined Firewall
Help for the Operations team:
All this innovation in did not come without its challenges; for the first time we had to consider all the network flows in our data centre. Those flows going north/south had been firmly understood for years (protect the perimeter) but now we needed to understand the east/west traffic (between workloads inside the data centre) and that was an entirely new ball game!
Applications had been developed and deployed for years, with no need to consider such flows and we all know how well such things got documented, or not. The early days of NSX required a degree in Wireshark and a lot of man hours spent untangling years of mess, it was a painful time!
Luckily VMware introduced vRealise Network Insight (vRNI); a tool that could collate all the flow data from the virtual switch and provide a visual representation. This was a huge step forward in being able to understand and map those east-to-west flows. A challenge still existed though, the automation of rule creation and management was still manual. At the same time, we started to see the need to provide different information to the network team and the security team, with the latter taking more responsibility for the creation of a segmentation policy. This split of NetOps and SecOps requirements led to the introduction of NSX Intelligence.
vRNI was set on a trajectory to augment and enable the NetOps teams that had direct responsibility for the network. The focus for the tool morphed to meet the needs of these teams and focus less on the direct micro-segmentation or security requirements. Providing the network professionals with in-depth data to ensure optimal packet delivery.
NSX Intelligence took up the role of enabling the SecOps teams and one of the key functions it brought was automated rule creation (following a learning period and with admin approval). This advance brought the barrier to adoption right down, reducing the effort and time required to manage a segmented network.
I am particularly interested to see what advancements we will see in NSX Intelligence at Explore as it’s an area that could continue to bring incremental value.