APIC-EM PoC about LXC network (grape-br0)

[Chin Leong Quek]

We are now at Customer site, working on an APIC-EM PoC for my FSI Banking customer. They hit some issue for setting up our standalone APIC-EM server. The APIC-EM is using 169.254.0.0/16 network as LXC network (grape-br0) by default. The grape-br0 is defined in /etc/default/lxc-net.

Customer has the following questions

1. Is the LXC network NATed to the eth NIC interface to customer IP Subnet address? For customer currently can see the 169.254.0.0/16 subnet address on their production network?
2. If the address is not NATed, can customer change the LXC network to a different IP Subnet as they have on their production network a similar 169.0.0.0 subnet?

Appreciate for your prompt reply.

[ngoldwat]

Hi,

What exactly is the issue that they ran into?  It is correct that the LXC's use a private internal network. The customer uses 169.x.x.x for their internal scope?

In order for a LXC to communicate with the outside world it connects to the Router Service. As such their network is not externally reachable.

Could you detail the issue a bit more.  As to motivation etc.

Thanks

[m.volodko]

Hi,

169.254/16 is local link address as defined in RFC. It can be used only for communication only within broadcast domain.

Customers cannot really use this subnet the same way as other IP addresses.

Misha

[Chin Leong Quek]

Hi Nicolas & Misha,

Wish both of you a Happy New Year! Thank you for both of your replies.

Coincidentally, my customer is using internally the same 169.254.xxx.xxx IP network for other purpose on their Production network. This clashes with APIC-EM LXC network IP range and is having Duplicate IP issue. Hence, Customer is asking

how can he change LXC network to other IP address range?

regards

Steven Quek

[ngoldwat]

Hi Steven,

Lets take a look at an example of ifconfig:

eth0	Link encap:Ethernet  HWaddr 00:50:56:84:9a:73
	inet addr:172.18.123.52  Bcast:172.18.123.255  Mask:255.255.255.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:957567775 errors:0 dropped:4522 overruns:0 frame:0
	TX packets:977463248 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:1000
	RX bytes:784743518825 (784.7 GB)  TX bytes:775960165949 (775.9 GB)

eth0:0	Link encap:Ethernet  HWaddr 00:50:56:84:9a:73
	inet addr:172.18.123.49  Bcast:172.18.123.255  Mask:255.255.255.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

grape-br0 Link encap:Ethernet  HWaddr fe:10:c2:93:eb:0b

	inet addr:169.254.1.1  Bcast:169.254.1.255  Mask:255.255.255.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:34040305 errors:0 dropped:0 overruns:0 frame:0
	TX packets:45462145 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:0
	RX bytes:6322852828 (6.3 GB)  TX bytes:12965249801 (12.9 GB)

Lets see what is reachable from outside:

$ ping -c 1 172.18.123.52

PING 172.18.123.52 (172.18.123.52): 56 data bytes

64 bytes from 172.18.123.52: icmp_seq=0 ttl=55 time=0.746 ms

--- 172.18.123.52 ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss

round-trip min/avg/max/stddev = 0.746/0.746/0.746/0.000 ms

$ ping -c 1 172.18.123.49

PING 172.18.123.49 (172.18.123.49): 56 data bytes

64 bytes from 172.18.123.49: icmp_seq=0 ttl=55 time=0.681 ms

--- 172.18.123.49 ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss

round-trip min/avg/max/stddev = 0.681/0.681/0.681/0.000 ms

$ ping -c 1 169.254.1.1

PING 169.254.1.1 (169.254.1.1): 56 data bytes

^C--- 169.254.1.1 ping statistics ---

1 packets transmitted, 0 packets received, 100% packet loss

So I am unsure as to your description of the issue you have.  169.254.1.1 is not routable.

Thanks

[Chin Leong Quek]

Hi Nicolas,

I understand the 169.254.xxx.xxx is non-routable in the Public network. Happens my customer uses 169.254.xxx.xxx in their Private Internal Production network for their servers hosts heart-beat purpose. This clashes with APIC-EM LXC network addresses range.  Later, we setup the APIC-EM in an isolated network and the issue is resolved.

Now customer would like to know anyway to change the IP network address & range in the APIC-EM LXC network?

regards

[m.volodko]

Hi,

Dont think change of IP subnet is supported...

From what I understand LXC address space is already isolated on APIC-EM. If your customer uses this subnet for servers heartbeat network only (isolated as well) there probably should be an issue. I could imagine it might have consequenses only when network devices use IPs from 169.254/16 subnet.

Misha

[Chin Leong Quek]

Hi Misha,

Thank you for your reply.

Hope this can be consider as a new feature on the future roadmap...

regards

Steven

[aradford]

Hi Steven,

you can click on the "I wish this page would..." at the bottom of the controller UI to request enhancements.

That will pop open an email to send to the product management team.

Someone from the product management team will get back to you on this.

Adam

[Chin Leong Quek]

Hi Adam,

Thank you very much for swift response.

regards

Steven