openvpn: Can't ping client when it's connected from inside LAN - ubuntu-16.04

Question

We have an openvpn server (I beleive on our router), and mobile clients that connect to the internet from far away locations, but also occasionally from inside our office. These systems are headless so configuring them differently before connecting to the in-office network is a nonstarter - we would like to SSH into them via their avahi hostnames regardless of where they physically are.
Right we can ping and SSH when they are connected to the internet outside of our network. When they are connected from inside our LAN, sometimes hostname.local resolves to 192.168.10.3 (and ping and SSH don't work) and sometimes to 192.168.1.211 (and ping and ssh do work).
When monitoring wireshark on the mobile client, ping requests to the 192.168.10.3 address do appear but are not answered.
How can we configure our clients so they can be reached when connecting from inside of our network?
output of ifconfig on client (connected to VPN via our office LAN):
eth0 Link encap:Ethernet HWaddr 00:04:4b:a7:fa:e5
inet addr:192.168.1.223 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::7a45:f5b1:1b87:c6f0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8964 errors:0 dropped:0 overruns:0 frame:0
TX packets:771 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1847719 (1.8 MB) TX bytes:160760 (160.7 KB)
Interrupt:42
tap0 Link encap:Ethernet HWaddr ce:d4:a6:18:48:21
inet addr:192.168.10.3 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::ccd4:a6ff:fe18:4821/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1381 errors:0 dropped:0 overruns:0 frame:0
TX packets:58 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:214474 (214.4 KB) TX bytes:7149 (7.1 KB)
output of route on client (connected to VPN via our office LAN):
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
default 192.168.10.1 0.0.0.0 UG 50 0 0 tap0
default 192.168.1.1 0.0.0.0 UG 100 0 0 eth0
link-local * 255.255.0.0 U 1000 0 0 eth1
192.168.1.0 * 255.255.255.0 U 100 0 0 eth0
192.168.2.0 * 255.255.255.0 U 0 0 0 eth1
192.168.10.0 * 255.255.255.0 U 50 0 0 tap0
Back-to-back pings from another machine on the same LAN to our mobile client. For whatever reason avahi .local names unpredictably resolve to the VPN IP or the other. Anyway, the ping to the VPN IP (second one) just hangs:
[15:51:25]~$ ping liber0.local
PING liber0.local (192.168.1.223) 56(84) bytes of data.
64 bytes from 192.168.1.223: icmp_seq=1 ttl=64 time=4.00 ms
64 bytes from 192.168.1.223: icmp_seq=2 ttl=64 time=6.09 ms
64 bytes from 192.168.1.223: icmp_seq=3 ttl=64 time=38.8 ms
^C
--- liber0.local ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 4.003/16.302/38.805/15.935 ms
[15:51:29]~$ ping liber0.local
PING liber0.local (192.168.10.3) 56(84) bytes of data.
^C
--- liber0.local ping statistics ---
27 packets transmitted, 0 received, 100% packet loss, time 26629ms
OpenVPN configuration file:
client
dev tap
proto udp
remote <redacted>
float
resolv-retry infinite
nobind
persist-key
persist-tun
verb 3
ca <redacted>.pem
cert <redacted>.pem
key <redacted>.key
cipher AES-256-CBC
auth SHA256

Answer 1

The key hint was that the ICMP packets made it to the VPN connected client, but were not answered. It turned out that the default rp_filter (reverse path filter) is the strictly checking and dropping packets. adding net.ipv4.conf.default.rp_filter = 2 to /etc/sysctl.conf sets rp_filter to loose reverse path checking, and everything works.