eBPF tools - skb_network_header crashes in a BPF Kernel trace function - ebpf

Question

I am looking to trace ip_forward_finish. The intent is to trace latency of all TCP connections going through a linux based gateway router. Hence thought of tracing ip_forward_finish kernel function. And capture the time-stamp of SYN, SYN-ACK and ACK messages at the router.
The issue is accessing iphdr inside the trace function makes the verifier complain with the following error:
bpf: Failed to load program: Permission denied
0: (79) r6 = *(u64 *)(r1 +96)
1: (b7) r1 = 0
2: (6b) *(u16 *)(r10 -24) = r1
3: (bf) r3 = r6
4: (07) r3 += 192
5: (bf) r1 = r10
6: (07) r1 += -24
7: (b7) r2 = 2
8: (85) call bpf_probe_read#4
9: (69) r1 = *(u16 *)(r10 -24)
10: (55) if r1 != 0x8 goto pc+7
R0=inv(id=0) R1=inv8 R6=inv(id=0) R10=fp0
11: (69) r1 = *(u16 *)(r6 +196)
R6 invalid mem access 'inv'
HINT: The invalid mem access 'inv' error can happen if you try to dereference
memory without first using bpf_probe_read() to copy it to the BPF stack.
Sometimes the bpf_probe_read is automatic by the bcc rewriter, other times
you'll need to be explicit.
The code fragment I originally had was as below and the crash occurs when an access to ip_Hdr->protocol is made. And I also checked that ip_Hdr is not null.
int trace_forward_finish(struct pt_regs *ctx,struct net *net,
struct sock *sk, struct sk_buff *skb)
{
if (skb->protocol != htons(ETH_P_IP))
return 0;
struct iphdr* ip_Hdr = (struct iphdr *) skb_network_header(skb);
if (ip_Hdr->protocol != IPPROTO_TCP)
return 0;
/// More code
}
Per the HINT in the message, I did try to change to bpf_probe_read but still the same outcome
int trace_forward_finish(struct pt_regs *ctx,struct net *net,
struct sock *sk, struct sk_buff *skb)
{
if (skb->protocol != htons(ETH_P_IP))
return 0;
struct iphdr ip_Hdr;
bpf_probe_read(&ip_Hdr, sizeof(ip_Hdr), (void*)ip_hdr(skb));
if (ip_Hdr.protocol != IPPROTO_TCP)
return 0;
return 0;
}
Any help would be appreciated.

Answer 1

bcc will try to transform your dereferences of kernel pointers into calls to bpf_probe_read. You can see that happening by passing debug=4 to the BPF() call.
In your case, I suspect that you would need to include function skb_network_header in your code so that bcc is able to rewrite it.
If that's not sufficient, then you might need a manual call to bpf_probe_read to retrieve structure struct iphdr from skb_network_header's pointer.