Dropped PING

We recently had a situation where two servers on the same subnet would periodically loose contact with each other.  After going through the common troubleshooting steps:

• Verify IP / Subnet mask / Gateway
• Make sure that NIC drivers were correct
• Verify that the server port channels / interface configurations were correct...

We still had issues.  We after reviewing the arp cache on one of the servers, we were seeing two different MAC addresses.  Periodically, the correct server MAC would be populated, and periodically a different MAC would be show.

Ultimately, the issue was resolved after we discovered that a Cisco ASA on the same subnet had proxy arp enabled.  This was causing the arp cache of the server to periodically get populated with the wrong MAC (mac of the ASA vs the MAC of the target server.)