Using NAT with Zones

This post originally appeared at http://blogs.sun.com/ford/entry/using_nat_in_the_global

Since sun.com is going away soon (thank a bunch Oracle…) I’m going to mirror it here because I found it useful.

Using NAT in the Global Zone

There have been requests to use IPFilter on Solaris 10 as a tool to allow use of networking by non-global zones without allocating an IP address for each zone. The general idea is to use network address translation (NAT) so that packets sent from a non-global zone configured with only a private IP address are modified before leaving the machine to use the global zone’s public IP address. Each zone will be assigned a private IP address that will never be used “on the wire” and has no meaning in the public internet.

I set out to see if this can be accomplished with the existing IPFilter implementation in Solaris 10. My experiments showed that it is possible with some unorthodox system configuration hacks. Special configuration of IP is needed to trick the system into sending this traffic through the right paths so that it is

• allowed by the zones networking restrictions,
• passes through IPFilter,
• is transmitted on the correct interface, and
• ultimately reaches the proper next-hop router.

My experiment was done using a Solaris 10 (FCS) system with only a DHCP-based connection to the net. My physical network interface was bge0.

This technique requires the use of a block of private IP addresses expressible as an address prefix. I refer to this block as the “imaginary network.” For my experiment I chose 10.10.10.0/24. Two special addresses in the block are allocated: one for the global zone (I chose 10.10.10.1) and one for the imaginary router (I chose 10.10.10.254). It is assumed that the all-zeroes and all-ones node addresses, 10.10.10.0 and 10.10.10.255, are unusable. All other addresses in the block are available for assignment to non-gobal zones (each zone will need one address).

The configuration steps were:

(0) Configure the system (global zone) with normal networking connectivity.

(1) Enable IPFilter.

  ed /etc/ipf/pfil.ap
	/bge/s/#//
	w
	q
	svcadm enable svc:/network/ipfilter
	shutdown -y -i6 -g0

(2) Configure NAT

	echo "map bge0 10.10.10.0/24 -> 0" | ipnat -f -

(3) Assign the global zone’s imaginary address to the physical interface:

ifconfig bge0 addif 10.10.10.1/24 up

(4) Add a default route to the imaginary router.

route  add default 10.10.10.254

(5) Create a fake ARP entry for the imaginary router using the real router’s MAC address.

	netstat -r | grep default
	arp therouter.my.org			# Look up the real router.
	arp -s 10.10.10.254 0:0:c:7:ac:6c	# Use the imaginary router's
						# IP address and the real
						# router's MAC address.

(6) Create one (or more) zone(s) with the same physical net interface as the global zone’s physical interface, and an address on the imaginary network.

zonecfg -z neutral
	...
	add net
	set address=10.10.10.2/24	# a different address for each zone
	set physical=bge0
	end
	commit
	exit

(7) Boot and use the zone(s).

I have only done light testing. Of course it only works with things for which NAT works. If you do something wrong, like forget to enable IPFilter or fail to create the fake ARP entry, you will probably be sending strange packets onto your LAN and network admins may hunt you down and give you dirty looks. The sample commands I list above were typed into this document and may have typos or be rough approximations. Some of the example commands are not persistent across reboot and require other steps to store them persistently. My experiment was on a single user system with simple network connectivity; in a more complex environment the steps to configure it would be very different and it may not work at all.

Conclusion: The technique I used is a hack. I think using it in a production system should be viewed with suspicion, not just because NAT is inherently dubious, but also because of the complex dependencies of the various bits of configuration that are required to make it work. But no doubt it can be useful as an interim technique for some users or applications and maybe a more polished form of the underlying technique can be added to the zones networking tool set eventually.