L3 switch not replying to ARP requests

[silviub]

Hello,

I have a problem that's really bothering: I have a Cisco WS-C3850 L3 switch and I want to use it (partly) as a router: The network:

I can ping from 10.9.0.3 to 10.9.0.1 and vice-versa.

The problem I'm facing: when trying to ping from the firewall / from the PC with the IP address 10.9.0.2 (C3850), I see the ARP requests going out but I see no ARP Reply. If I manually add the MAC address of 10.9.0.2 to the ARP table, the ping goes through. Why isn't the C3850 replying to ARPs?

P.S. I found out that the command

ip routing

seems to be causing this. Also, not really a fix, but this gets me ARP replies:

ip route 0.0.0.0 0.0.0.0 Te1/1/1

 but that obviously, throws a warning, rightfully so:

%Default route without gateway, if not a point-to-point interface, may impact performance

 What is going on?

Thank you in advance!

[MHM Cisco World]

%Default route without gateway, if not a point-to-point interface, may impact performance

message not prevent the defualt route to add to RIB 
what you need to check is 
is FW have route toward L3SW for VLAN 1500/1501/1502

MHM

[silviub]

Hello and thank you for replying,

I do know that the message won't prevent the route adding, but I'd rather have the route added via 

ip route 0.0.0.0 0.0.0.0 10.9.0.1

 as it should be done.

The firewall HAS the routes to 172.19.50.0/24, 172.19.51.0/24 and 172.19.52.0/24 but I don't understand how that matters. I'm not trying to get (yet) to the PCs in the those networks. 
What I'm trying is to ping from 10.9.0.3 to 10.9.0.2. That's all. And, for that, I don't get an arp reply from the C3850, for whatever reason (beats the hell out of me).

[MHM Cisco World]

if it L2SW can you
show interface trunk 
show vlan brief 
show vlan 
in this SW

[silviub]

Unfortunately, that's not a single switch. That switch represents a big portion of the network but, for simplicity, it has been represented as a single switch. What are you looking for, maybe I can get a way to make it happen?

[MHM Cisco World]

the trunk is allow VLAN 
the port connect to PC is in vlan 9 or not 
the L3SW and this SW use same native vlan ?
these what I looking for 

MHM

[silviub]

I have triple checked the entire chain, the VLAN is allowed on all the ports from the firewall to the L3SW.

The port connected to the PC is a trunk, actually, with a tagged interface for VLAN9.

The entire network uses the same native VLAN, VLAN 40.

Also, keep in mind: if I run

conf t
no ip routing

 on the L3SW, I am able to ping the 10.9.0.2. This, at least in my book, tells me that the entire chain from the firewall/PC to the L3SW is actually working fine. The issue is present only when I enable ip routing on that switch, and I don't get why....?

[MHM Cisco World]

I run lab and solve first part 

PC to L3SW not success 
the port from L2SW to PC need to be untag access not tag trunk, the PC send untag and hence the L2SW forward traffic to wrong VLAN.

for FW to L3SW  I need to know how you config the FW

MHM

[Georg Pauwen]

Hello,

when you configure the switch for routing, what is the default gateway the PC is using ? It should be 10.9.0.2 (the IP address of the Vlan 9 interface configured on the switch). 

[balaji.bandi]

You need to provide some information -

show run (post from 3850 Switch)

what Firewall you have ?

what is the device between Firwall and 3850 Switch ?  (or is this just representation only ?)

PC has IP, but what gateway PC has (can you post ipconfig /all from PC ? )

I can ping from 10.9.0.3 to 10.9.0.1 and vice-versa.

from what device you able to do this ? PC ?

Also you using extended VLAN, do you have STP for extended VLAN ?

[silviub]

@MHM Cisco World the port to the PC was already access, with VLAN 9 as access VLAN. As I said, pinging 10.9.0.1 from 10.9.0.3 works (so L2 is fine). Also, 10.9.0.2 can be pinged from 10.9.0.1 and 10.9.0.3 IF ip routing is not set. The moment I set ip routing, the L3SW stops responding to the APR requests.

@balaji.bandi the relevant configuration is in the picture. Nothing else is really configured on the switch but if you think it's useful, I can provide the full config, just that it's quite extensive. In short, the port (te1/1/1) connected to the L2 switch is mode trunk, with the allowed VLANs from the picture.

The firewall is a Palo alto but again, I feel like this is not really important. The firewall can be forgotten. I need to be able to reach the L3SW from the PC, while ip routing is enabled on the L3SW. Before I enable ip routing, I can get to the L3SW. The second I enable ip routing, that L3SW stops responding to ARP requests.

The device between the L3SW and the firewall is for representation only - there's quite an extensive network between the firewall and the L3SW.

Since the PC and the L3SW are in the same VLAN, I shouldn't need to have a gateway set, right? Devices in the same subnet, same VLAN should be able to communicate with each other without a gateway.

 I am able to ping FROM the PC to the L3SW IF ip routing is not enabled. The second I enable ip routing, the L3SW stops responding.

I don't use extended VLANs, I do use STP though. Still VLAN 9 is forwarding on all ports on the L3SW.

P.S. I replicated this setup in Cisco Packet Tracer but but I am unable to replicate this issue. If this is helpful, I can share the packet tracer file.

[MHM Cisco World]

debug arp <<- run this and ping from PC 
check the VLAN ARP use is it native vlan x or vlan 9

MHM

[silviub]

Well, this is quite awkward. Reloading the device actually seems to have fixed everything....?

[Richard Burts]

Thanks for the update. Glad to know that the problem is resolved. It is not unusual (especially when the problem seems strange) that a reload/restart will fix the issue.