By James Kehr, Networking Support Escalation Engineer
To understand Windows Network Virtualization (WNV) capture data you first need to understand what you’re looking at. Which is hard when you may not understand what goes on inside of Windows. This article will cover the basics of what goes on when WNV is in use, in a manner that will, hopefully, be easy to understand.
ETW
Starting with Windows 7 and Windows Server 2008 R2, an ETW (Event Tracing for Windows) provider was added that allows the network frames, a.k.a. packets, to be captured within Windows without installing a packet capture tool like Message Analyzer, Wireshark, or Network Monitor. This provider is called Microsoft-Windows-NDIS-PacketCapture. The ETWs for this provider can be written to an ETL file (Event Trace Log) and analyzed using a tool that can parse ETL files. Think of ETW logging as a low-level Event Viewer on steroids.
There are several ETW providers that are used for troubleshooting Windows networking. Some ETWs can only be viewed internally at Microsoft, while others can be viewed by the public. All the basic networking ETWs can be publicly parsed and read. You just need to learn how.
What is an NBL?
An NBL is a Net Buffer List. Think of an NBL as a packet inside of Windows. Because that’s what it is. NBLs are used for every virtualized network data transaction inside of Windows. Every. Single. One.
Container networking uses NBLs to route and deliver data from the wire to a Container. And from the Container back to the wire. Hyper-V and LBFO (NIC teaming) use NBLs to move data around inside of Windows, too.
These NBL “packets” virtually move across virtual switches and virtual NICs inside of Windows like an Ethernet frame/packet moves around a physical network. Except, instead of moving the actual NBL it’s the reference to the NBL that is passed along the virtual wire.
Sending data between two computers on a switch involves a packet being sent by a NIC on Server A to a switch, then to Server B, where the NIC passes the packet into the operating system. The exact same concept applies to WNV. A packet sent from VM A is passed to the vmSwitch (virtual machine switch) via an NBL. The NBL is passed across the vmSwitch and delivered to the vmNIC on VM B, where the packet is received by the local network stack.
Those who want to see the network data move around inside of Windows just needs to look at the NBLs. This is handy when troubleshooting problems with virtualized networking. It allows admins to see where potential problems are and, unlike packets, look for errors returned from within Windows by the ETW providers used by virtual networking.
Exceptions time! SR-IOV bypasses the entire Hyper-V network virtualization stack, or the parent partition for those who love technical terms. There are a few other technologies, like RDMA and Fibre Channel, which also bypass the Windows and/or Hyper-V network stacks. You will not see packets or NBLs in Windows when network stack offloading or bypass technologies are in use.
Compartments
There is a special type of virtual networking I have not mentioned yet: Hyper-V Network Virtualization (HNV), also known as Software Defined Networking (SDN). HNV/SDN adds something called a Compartment to virtual networking. The best way to think of a compartment is a VLAN, but, again, inside of Windows.
Compartments are the magic that allow HNV to have two sets of VMs, from two different companies, occupy the same Hyper-V host, yet use the same subnet. When HNV is in use Contoso can use the 10.0.0.0/8 subnet, Fabrikam can use 10.0.0.0/8, and the subnets will never overlap, collide, or cause routing issues. When configured properly.
I won’t go into too much detail about how all this works, HNV and SDN networking is a topic all its own, but it’s important to know because Containers on Windows can use Compartments, too. Depending on the configuration. Each Docker, or Container, network that is created gets a Compartment ID. The Compartment ID is what Windows uses to securely get the correct network data to the correct resource.
The compartment details are exposed on the host by using one of two commands. The trusty old “ipconfig” command can be used with a newer parameter, /allcompartments. Get-NetIpAddress with the IncludeAllCompartments parameter can be used on the PowerShell front.
ipconfig /allcompartments
=============================================================
Network Information for Compartment 2
=============================================================
Ethernet adapter vEthernet (Container NIC c337484d):
Connection-specific DNS Suffix . : Contoso.com
Link-local IPv6 Address . . . . . : fe80::297c:2597:40d2:59ab%16
IPv4 Address. . . . . . . . . . . : 172.23.181.37
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . : 172.23.176.1
Get-NetIPAddress -IncludeAllCompartments | where InterfaceAlias -match “container”
IPAddress : 172.23.181.37
InterfaceIndex : 16
InterfaceAlias : vEthernet (Container NIC c337484d)
AddressFamily : IPv4
Type : Unicast
PrefixLength : 20
PrefixOrigin : Manual
SuffixOrigin : Manual
AddressState : Preferred
ValidLifetime : Infinite ([TimeSpan]::MaxValue)
PreferredLifetime : Infinite ([TimeSpan]::MaxValue)
SkipAsSource : False
PolicyStore : ActiveStore
A Summary
So far, we’ve learned that packets on the wire are converted to NBLs inside of Windows. The NBLs are then passed between virtual components inside of Windows, behaving like a packet on the wire. The NBLs are then consumed by the endpoint network stack; such as, the guest VM, host, or Container (also the host, kind of). Compartments add isolation, security, and, potentially, the ability for multiple virtual networks to exist within the same host system.
ETW providers can be used to capture packets and events inside of Windows. These ETW events are stored in an ETL (Event Tracing Log) file that can be used to analyze network virtualization within Windows.
The next part covers capturing and reading WNV data, which includes Container networking. This is also handy if you want to learn how to read Hyper-V and LBFO networking traffic. That makes part 3 a triple win!
-James