Debugging a kernel panic when using Cairo graphics with an 8 bit per pixel frame buffer - linux-device-driver

Question

I successfully got a 1 bit per pixel frame buffer working with Cairo Graphics and I’m running into issues converting the frame buffer to 8 bits per pixel.
In the frame buffer I increased the appropriate values by a factor of 8 and for a 128x64 display I get 8192 bytes of memory allocated in the driver.
Testing from the command line with cat /dev/urandom > /dev/fb0 works as expected.
I then use Cairo to draw a diagonal line just to test the boundaries and that’s when I start seeing issues, so I assume it's an issue with deferred IO in the frame buffer. Seems similar to Kernel panic using deferred_io on kmalloced buffer
I've hit a road block and need help moving forward.
Drawing a line from 0x,0y to 63x,0y has no problems.
cairo_move_to(cr,0,0);
cairo_line_to(cr,63,1);
cairo_stroke(cr);
Drawing from 0,0 to 64,0 I start seeing the following until 0,0 to 128,31:
[ 579.978373] ------------[ cut here ]------------
[ 579.992339] WARNING: CPU: 0 PID: 1920 at kernel/ptrace.c:190 SyS_ptrace+0x388/0x5c8()
[ 580.044483] Modules linked in: musb_dsps musb_hdrc udc_core musb_am335x
[ 580.176439] CPU: 0 PID: 1920 Comm: strace Not tainted 4.1.18-rt17 #121
[ 580.183027] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 580.189150] Backtrace:
[ 580.191692] [<c0013208>] (dump_backtrace) from [<c0013428>] (show_stack+0x18/0x1c)
[ 580.199324] r7:c06fc844 r6:000000be r5:00000009 r4:00000000
[ 580.205072] [<c0013410>] (show_stack) from [<c059c968>] (dump_stack+0x20/0x28)
[ 580.212365] [<c059c948>] (dump_stack) from [<c003a32c>] (warn_slowpath_common+0x80/0xb8)
[ 580.220503] [<c003a2ac>] (warn_slowpath_common) from [<c003a408>] (warn_slowpath_null+0x24/0x2c)
[ 580.229349] r8:00000000 r7:00000000 r6:00000018 r5:cc36668c r4:cc366680
[ 580.236184] [<c003a3e4>] (warn_slowpath_null) from [<c0042c84>] (SyS_ptrace+0x388/0x5c8)
[ 580.244348] [<c00428fc>] (SyS_ptrace) from [<c000f800>] (ret_fast_syscall+0x0/0x3c)
[ 580.252062] r9:cd6c4000 r8:c000f9a4 r7:0000001a r6:00000783 r5:b6f8b4d0 r4:00000017
[ 580.259896] ---[ end trace 0000000000000002 ]---
Drawing at 128,32 and above causes a kernel panic:
[ 984.692851] flags: 0x14(referenced|dirty)
[ 984.697350] page dumped because: non-NULL mapping
[ 984.703013] Modules linked in: musb_dsps musb_hdrc udc_core musb_am335x
[ 984.709706] CPU: 0 PID: 2295 Comm: test Tainted: G W 4.1.18-rt17 #121
[ 984.717864] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 984.725323] Backtrace:
[ 984.727874] [<c0013208>] (dump_backtrace) from [<c0013428>] (show_stack+0x18/0x1c)
[ 984.736361] r7:00000000 r6:c07055d0 r5:cff87cb4 r4:c0880c80
[ 984.742767] [<c0013410>] (show_stack) from [<c059c968>] (dump_stack+0x20/0x28)
[ 984.750440] [<c059c948>] (dump_stack) from [<c00e0620>] (bad_page+0xc8/0x12c)
[ 984.761358] [<c00e0558>] (bad_page) from [<c00e08b8>] (free_pages_prepare+0x234/0x2f0)
[ 984.770049] r9:cc215dc8 r8:cff87cb4 r7:00000001 r6:00000001 r5:cff87cb4 r4:cfdad000
[ 984.777961] [<c00e0684>] (free_pages_prepare) from [<c00e2944>] (free_hot_cold_page+0x34/0x298)
[ 984.787421] r10:c08616f0 r9:cc215dc8 r8:00080000 r7:00000000 r6:00000014 r5:cff87cb4
[ 984.795881] r4:cfdad000
[ 984.798476] [<c00e2910>] (free_hot_cold_page) from [<c00e2bf4>] (free_hot_cold_page_list+0x4c/0xec)
[ 984.809020] r10:c08616f0 r9:cc215dc8 r8:cc214000 r7:cc215db4 r6:00000000 r5:cff87cb4
[ 984.817006] r4:cff87cb4
[ 984.819598] [<c00e2ba8>] (free_hot_cold_page_list) from [<c00ea080>] (release_pages+0x24c/0x264)
[ 984.829523] r10:cc215dc8 r9:00000002 r8:cc36d004 r7:00000002 r6:00000000 r5:00000002
[ 984.837500] r4:cff87cb4 r3:00000000
[ 984.847293] [<c00e9e34>] (release_pages) from [<c0116f74>] (free_pages_and_swap_cache+0xa0/0xa4)
[ 984.858132] r10:b6f50000 r9:cc215eb8 r8:cc36d000 r7:00000002 r6:cc36d004 r5:cff87cb4
[ 984.866577] r4:00000002
[ 984.869406] [<c0116ed4>] (free_pages_and_swap_cache) from [<c0105c40>] (unmap_single_vma+0x3f8/0x59c)
[ 984.880475] r9:cc215eb8 r8:b6f50000 r7:cff87cb4 r6:b6f4f000 r5:8d30534f r4:00000000
[ 984.888944] [<c0105848>] (unmap_single_vma) from [<c0106bc0>] (unmap_vmas+0x5c/0x70)
[ 984.898548] r10:b6f4e000 r9:cc4ab318 r8:00000000 r7:b6f4e000 r6:cc215eb8 r5:b6f50000
[ 984.906901] r4:cd6f2370
[ 984.909555] [<c0106b64>] (unmap_vmas) from [<c0109f0c>] (unmap_region+0xb4/0x1ac)
[ 984.919229] r8:cd6f2370 r7:cc215ee0 r6:b6f50000 r5:cd4af8f0 r4:cc51b700
[ 984.926694] [<c0109e58>] (unmap_region) from [<c010bddc>] (do_munmap+0x28c/0x3fc)
[ 984.935467] r10:cd6f2370 r9:cc4ab318 r8:cc51b704 r7:b6f4e000 r6:b6f50000 r5:cd6f2370
[ 984.945040] r4:cc51b700
[ 984.948196] [<c010bb50>] (do_munmap) from [<c010bf90>] (vm_munmap+0x44/0x58)
[ 984.956689] r10:00000100 r9:cc214000 r8:c000f9a4 r7:00002000 r6:b6f4e000 r5:cc51b700
[ 984.965245] r4:cc51b748
[ 984.967906] [<c010bf4c>] (vm_munmap) from [<c010cf2c>] (SyS_munmap+0x24/0x28)
[ 984.976574] r7:0000005b r6:00029a30 r5:00002000 r4:b6f4e000
[ 984.982811] [<c010cf08>] (SyS_munmap) from [<c000f974>] (__sys_trace_return+0x0/0x2c)
[ 984.990844] r5:00000001 r4:00029858
[ 984.998016] Disabling lock debugging due to kernel taint
[ 985.602383] Unable to handle kernel paging request at virtual address a2763fc4
[ 985.609738] pgd = c0004000
[ 985.613681] [a2763fc4] *pgd=00000000
[ 985.617326] Internal error: Oops: 5 [#1] PREEMPT ARM
[ 985.617347] Modules linked in: musb_dsps musb_hdrc udc_core musb_am335x
[ 985.617363] CPU: 0 PID: 1766 Comm: kworker/0:0 Tainted: G B W 4.1.18-rt17 #121
[ 985.617367] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 985.617397] Workqueue: events fb_deferred_io_work
[ 985.617403] task: cc364000 ti: cc44a000 task.ti: cc44a000
[ 985.617420] PC is at __wake_up_bit+0x10/0x48
[ 985.617436] LR is at unlock_page+0x38/0x3c
[ 985.617445] pc : [<c0065b9c>] lr : [<c00dba2c>] psr: a00f0013
[ 985.617445] sp : cc44be98 ip : cc44beb8 fp : cc44beb4
[ 985.617449] r10: cfd86500 r9 : 0cfd8650 r8 : 00000000
[ 985.617455] r7 : cd24dc00 r6 : cd2b9150 r5 : cd2b9154 r4 : cc215db4
[ 985.617460] r3 : 289d8fec r2 : 00000000 r1 : cc215db4 r0 : a2763fb0
[ 985.617469] Flags: NzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
[ 985.617475] Control: 10c5387d Table: 8d4f0019 DAC: 00000015
[ 985.617480] Process kworker/0:0 (pid: 1766, stack limit = 0xcc44a210)
[ 985.617486] Stack: (0xcc44be98 to 0xcc44c000)
[ 985.617495] be80: c07f0000 cff87cb4
[ 985.617506] bea0: 00000000 cc44bea0 cc44becc cc44beb8 c00dba2c c0065b98 cc215db4 cd2b9154
[ 985.617518] bec0: cc44beec cc44bed0 c0305528 c00dba00 cd24de28 cd496380 c0860f5c c07ff8b8
[ 985.617529] bee0: cc44bf2c cc44bef0 c004fdac c03054f0 cc44a000 cc364000 cc44bf14 00000000
[ 985.617540] bf00: c0050a34 c07ff8b8 cd496398 c07ff8dc 00000008 c0860c6d c07ff8b8 cd496380
[ 985.617550] bf20: cc44bf64 cc44bf30 c0050120 c004fc9c 00000000 c07ff8dc 00000000 00000000
[ 985.617561] bf40: cd713fc0 cd496380 c00500e8 00000000 00000000 00000000 cc44bfac cc44bf68
[ 985.617572] bf60: c0054cd4 c00500f4 00000000 00000000 00000000 cd496380 00000000 cc44bf7c
[ 985.617582] bf80: cc44bf7c 00000000 cc44bf88 cc44bf88 cd713fc0 c0054c08 00000000 00000000
[ 985.617593] bfa0: 00000000 cc44bfb0 c000f8a8 c0054c14 00000000 00000000 00000000 00000000
[ 985.617603] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 985.617613] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[ 985.617617] Backtrace:
[ 985.617634] [<c0065b8c>] (__wake_up_bit) from [<c00dba2c>] (unlock_page+0x38/0x3c)
[ 985.617648] [<c00db9f4>] (unlock_page) from [<c0305528>] (fb_deferred_io_work+0x44/0xe4)
[ 985.617656] r5:cd2b9154 r4:cc215db4
[ 985.617680] [<c03054e4>] (fb_deferred_io_work) from [<c004fdac>] (process_one_work+0x11c/0x458)
[ 985.617692] r7:c07ff8b8 r6:c0860f5c r5:cd496380 r4:cd24de28
[ 985.617707] [<c004fc90>] (process_one_work) from [<c0050120>] (worker_thread+0x38/0x52c)
[ 985.617723] r10:cd496380 r9:c07ff8b8 r8:c0860c6d r7:00000008 r6:c07ff8dc r5:cd496398
[ 985.617727] r4:c07ff8b8
[ 985.617744] [<c00500e8>] (worker_thread) from [<c0054cd4>] (kthread+0xcc/0xe4)
[ 985.617760] r10:00000000 r9:00000000 r8:00000000 r7:c00500e8 r6:cd496380 r5:cd713fc0
[ 985.617764] r4:00000000
[ 985.617783] [<c0054c08>] (kthread) from [<c000f8a8>] (ret_from_fork+0x14/0x2c)
[ 985.617795] r7:00000000 r6:00000000 r5:c0054c08 r4:cd713fc0
[ 985.617806] Code: e1a0c00d e92dd800 e24cb004 e24dd010 (e590e014)
[ 985.927655] ---[ end trace 0000000000000006 ]---
Here are the relative parts of the frame buffer driver:
static const struct fb_var_screeninfo oledfb_var = {
.bits_per_pixel = 8,
.grayscale = 1,
};
vmem_size = par->width * par->height * oledfb_var.bits_per_pixel / 8;
vmem = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
get_order(vmem_size));
info->fix.line_length = par->width * oledfb_var.bits_per_pixel / 8;
info->var.red.length = oledfb_var.bits_per_pixel;
info->var.red.offset = 0;
info->var.red.msb_right = 1;
info->var.green.length = oledfb_var.bits_per_pixel;
info->var.green.offset = 0;
info->var.green.msb_right = 1;
info->var.blue.length = oledfb_var.bits_per_pixel;
info->var.blue.offset = 0;
info->var.blue.msb_right = 1;
info->var.transp.length = oledfb_var.bits_per_pixel;
info->var.transp.offset = 0;
info->var.transp.msb_right = 1;
info->screen_base = (u8 __force __iomem *)vmem;
info->fix.smem_start = __pa(vmem);
info->fix.smem_len = vmem_size;
And here is how I’m initializing my memory map and surface in cairo:
// Figure out the size of the screen in bytes
device->fb_screensize = device->fb_vinfo.xres * device->fb_vinfo.yres
* device->fb_vinfo.bits_per_pixel / 8;
printf(" Size (bytes): %d\n", (int)device->fb_screensize);
printf("Mapping memory\n");
// Map the device to memory
device->fb_data = (char *)mmap(0, device->fb_screensize,
PROT_READ | PROT_WRITE, MAP_SHARED,
device->fb_fd, 0);
if ((int)device->fb_data == -1) {
perror("Error: failed to map framebuffer device to memory");
exit(4);
}
// Get fixed screen information
if (ioctl(device->fb_fd, FBIOGET_FSCREENINFO, &device->fb_finfo) == -1) {
perror("Error reading fixed information");
exit(2);
}
printf(" Stride_for_width: %d\n", cairo_format_stride_for_width(CAIRO_FORMAT_A8, device->fb_vinfo.xres));
surface = cairo_image_surface_create_for_data((unsigned char *)device->fb_data,
CAIRO_FORMAT_A8,
device->fb_vinfo.xres,
device->fb_vinfo.yres,
cairo_format_stride_for_width(CAIRO_FORMAT_A8, device->fb_vinfo.xres));
cairo_surface_set_user_data(surface, NULL, device, &cairo_linuxfb_surface_destroy);

Answer 1

I changed
vmem = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, get_order(vmem_size));
...
__free_pages(__va(info->fix.smem_start), get_order(info->fix.smem_len));
to
vmem = kzalloc(vmem_size, GFP_KERNEL);
...
kfree(info->screen_base);
and that seems to have fixed the problem. I based my driver off of https://github.com/torvalds/linux/blob/master/drivers/video/fbdev/ssd1307fb.c since the hardware is similar. I don't know enough to understand why one way works over the other but the __ in front of the function call raised a red flag.