I have a f1-micro instance running on Google Compute Engine running CentOS.
Some time ago I got a mail stating that the zone I am using (us-central1-b) is going through a hardware upgrade to new Hashwell servers, and all instances need to be restarted. So, I did restart. I shut down my services, and performed a restart via the developer dev. console.
But the instance never came up - the boot on the new hardware failed, and now I can not log in to the instance with ssh. I will add the serial console content below this.
My main issue is that there are some content I would really like to get my hands on (nope - didn't do a full backup - shoot!).
How do I restore the instance, when I can not log in using ssh?
Changing serial settings was 0/0 now 3/0
Start bios (version 1.7.2-20150226_170051-google)
Unable to unlock ram - bridge not found
Ram Size=0x26600000 (0x0000000000000000 high)
Relocating low data from 0x000e5810 to 0x000ef780 (size 2161)
Relocating init from 0x000e6081 to 0x265d3540 (size 51612)
CPU Mhz=2301
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 4 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: map device bdf=00:03.0 bar 0, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:04.0 bar 0, addr 0000c040, size 00000040 [io]
PCI: map device bdf=00:03.0 bar 1, addr febfe000, size 00001000 [mem]
PCI: map device bdf=00:04.0 bar 1, addr febff000, size 00001000 [mem]
PCI: init bdf=00:01.0 id=8086:7110
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0xb008, freq 3579 kHz
PCI: init bdf=00:03.0 id=1af4:1004
PCI: init bdf=00:04.0 id=1af4:1000
Found 1 cpu(s) max supported 1 cpu(s)
MP table addr=0x000fdaf0 MPC table addr=0x000fdb00 size=240
SMBIOS ptr=0x000fdad0 table=0x000fd990 size=314
Memory hotplug not enabled. [MHPE=0xffffffff]
ACPI DSDT=0x265fe070
ACPI tables: RSDP=0x000fd960 RSDT=0x265fe030
Scan for VGA option rom
Machine UUID 2b33f2bc-6042-1f12-c61f-0edd33ff965d
Found 0 serial ports
found virtio-scsi at 0:3
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#0,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#1,0
virtio-scsi vendor='Google' product='PersistentDisk' rev='1' type=0 removable=0
virtio-scsi blksize=512 sectors=20971520
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#2,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#3,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#4,0
....
....
....
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#248,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#249,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#250,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#251,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#252,0
Searching bootorder for: /pci#i0cf8/*#3/*#0/*#253,0
KBD: int09 handler: AL=0
PS2 keyboard initialized
All threads complete.
Scan for option roms
Searching bootorder for: HALT
drive 0x000fd920: PCHS=0/0/0 translation=lba LCHS=1024/255/63 s=20971520
Space available for UMB: 000c0000-000eb800
Returned 122880 bytes of ZoneHigh
e820 map has 6 items:
0: 0000000000000000 - 000000000009fc00 = 1 RAM
1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 00000000265fe000 = 1 RAM
4: 00000000265fe000 - 0000000026600000 = 2 RESERVED
5: 00000000fffbc000 - 0000000100000000 = 2 RESERVED
Unable to lock ram - bridge not found
KBD: int09 handler: AL=0
Changing serial settings was 3/2 now 3/0
enter handle_19:
NULL
Booting from Hard Disk 0...
Booting from 0000:7c00
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.32-431.17.1.el6.x86_64 (mockbuild#c6b8.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Wed May 7 23:32:49 UTC 2014
Command line: ro root=UUID=a8cf6ab7-92fb-42c6-b95f-d437f94aaf98 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM console=ttyS0,38400n8 rd_NO_DM
KERNEL supported cpus:
Intel GenuineIntel
AMD AuthenticAMD
Centaur CentaurHauls
Disabled fast string operations
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000265fe000 (usable)
BIOS-e820: 00000000265fe000 - 0000000026600000 (reserved)
BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
DMI 2.4 present.
SMBIOS version 2.4 # 0xFDAD0
Hypervisor detected: KVM
last_pfn = 0x265fe max_arch_pfn = 0x400000000
x86 PAT enabled: cpu 0, old 0x70106, new 0x7010600070106
Using GB pages for direct mapping
init_memory_mapping: 0000000000000000-00000000265fe000
RAMDISK: 25f82000 - 265ed02e
ACPI: RSDP 00000000000fd960 00014 (v00 Google)
ACPI: RSDT 00000000265fe030 00034 (v01 Google GOOGRSDT 00000001 GOOG 00000001)
ACPI: FACP 00000000265fff00 000F4 (v02 Google GOOGFACP 00000001 GOOG 00000001)
ACPI: DSDT 00000000265fe070 015B6 (v01 GOOG GODSDT 00000001 INTL 20140214)
ACPI: FACS 00000000265ffec0 00040
ACPI: SSDT 00000000265ff780 0073D (v01 Google GOOGSSDT 00000001 GOOG 00000001)
ACPI: APIC 00000000265ff660 0006E (v01 Google GOOGAPIC 00000001 GOOG 00000001)
ACPI: WAET 00000000265ff630 00028 (v01 Google GOOGWAET 00000001 GOOG 00000001)
Setting APIC routing to flat.
No NUMA configuration found
Faking a node at 0000000000000000-00000000265fe000
Bootmem setup node 0 0000000000000000-00000000265fe000
NODE_DATA [0000000000009000 - 000000000003cfff]
bootmap [000000000003d000 - 0000000000041cbf] pages 5
(7 early reservations) ==> bootmem [0000000000 - 00265fe000]
#0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
#1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000]
#2 [0001000000 - 0002020aa4] TEXT DATA BSS ==> [0001000000 - 0002020aa4]
#3 [0025f82000 - 00265ed02e] RAMDISK ==> [0025f82000 - 00265ed02e]
#4 [000009fc00 - 0000100000] BIOS reserved ==> [000009fc00 - 0000100000]
#5 [0002021000 - 00020210a5] BRK ==> [0002021000 - 00020210a5]
#6 [0000008000 - 0000009000] PGTABLE ==> [0000008000 - 0000009000]
found SMP MP-table at [ffff8800000fdaf0] fdaf0
crashkernel=auto resulted in zero bytes of reserved memory.
kvm-clock: Using msrs 4b564d01 and 4b564d00
kvm-clock: cpu 0, msr 0:1c247c1, boot clock
Zone PFN ranges:
DMA 0x00000001 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal 0x00100000 -> 0x00100000
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
0: 0x00000001 -> 0x0000009f
0: 0x00000100 -> 0x000265fe
ACPI: PM-Timer IO Port: 0xb008
Setting APIC routing to flat.
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 1 CPUs, 0 hotplug CPUs
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 26600000 (gap: 26600000:d99bc000)
Booting paravirtualized kernel on KVM
NR_CPUS:4096 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 31 pages/cpu #ffff880002200000 s94872 r8192 d23912 u2097152
pcpu-alloc: s94872 r8192 d23912 u2097152 alloc=1*2097152
pcpu-alloc: [0] 0
kvm-clock: cpu 0, msr 0:22167c1, primary cpu clock
Built 1 zonelists in Node order, mobility grouping on. Total pages: 154835
Policy zone: DMA32
Kernel command line: ro root=UUID=a8cf6ab7-92fb-42c6-b95f-d437f94aaf98 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd_NO_LVM console=ttyS0,38400n8 rd_NO_DM
PID hash table entries: 4096 (order: 3, 32768 bytes)
xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
Checking aperture...
No AGP bridge found
Memory: 594376k/628728k available (5326k kernel code, 392k absent, 33960k reserved, 7013k data, 1280k init)
Hierarchical RCU implementation.
NR_IRQS:33024 nr_irqs:256
Console: colour *CGA 80x25
console [ttyS0] enabled
allocated 2621440 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Detected 2299.982 MHz processor.
Calibrating delay loop (skipped) preset value.. 4599.96 BogoMIPS (lpj=2299982)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux: Initializing.
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys net_prio
Disabled fast string operations
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 32 MCE banks
alternatives: switching to unfair spinlock
SMP alternatives: switching to UP code
Freeing SMP alternatives: 36k freed
ACPI: Core revision 20090903
ftrace: converting mcount calls to 0f 1f 44 00 00
ftrace: allocating 21778 entries in 86 pages
Enabling x2apic
Enabled x2apic
APIC routing finalized to physical x2apic.
..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1
CPU0: Intel(R) Xeon(R) CPU # 2.30GHz stepping 00
Performance Events: unsupported p6 CPU model 63 no PMU driver, software events only.
NMI watchdog disabled (cpu0): hardware events not enabled
Brought up 1 CPUs
Total of 1 processors activated (4599.96 BogoMIPS).
devtmpfs: initialized
regulator: core version 0.5
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: Interpreter enabled
ACPI: (supports S0 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: host bridge window [io 0x0000-0x0cf7]
pci_root PNP0A03:00: host bridge window [io 0x0d00-0xffff]
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A03:00: host bridge window [mem 0x26600000-0xfebfffff]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7]
pci_bus 0000:00: root bus resource [io 0x0d00-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0x26600000-0xfebfffff]
pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4 ACPI
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
Switching to clocksource kvm-clock
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 8 devices
ACPI: ACPI bus type pnp unregistered
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 6572k freed
audit: initializing netlink socket (disabled)
type=2000 audit(1430203988.308:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 1173
alg: No test for stdrng (krng)
ksign: Installing public key data
Loading keyring
- Added public key A760F53BCC8FA210
- User ID: CentOS (Kernel Module GPG key)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
ipmi message handler version 39.2
IPMI System Interface driver.
ipmi_si: Adding default-specified kcs state machine
ipmi_si: Trying default-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified smic state machine
ipmi_si: Trying default-specified smic state machine at i/o address 0xca9, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified bt state machine
ipmi_si: Trying default-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Unable to find any System Interface(s)
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button [PWRF]
input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
ACPI: Sleep Button [SLPF]
[Firmware Bug]: No valid trip found
GHES: HEST is not enabled!
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
crash memory driver: version 1.1
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
�serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
serial8250: ttyS2 at I/O 0x3e8 (irq = 6) is a 16550A
Refined TSC clocksource calibration: 2300.003 MHz.
serial8250: ttyS3 at I/O 0x2e8 (irq = 7) is a 16550A
00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:05: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:06: ttyS2 at I/O 0x3e8 (irq = 6) is a 16550A
00:07: ttyS3 at I/O 0x2e8 (irq = 7) is a 16550A
brd: module loaded
loop: module loaded
input: Macintosh mouse button emulation as /devices/virtual/input/input2
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
i8042.c: Warning: Keylock active.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:01: RTC can wake from S4
rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one day, 114 bytes nvram
cpuidle: using governor ladder
cpuidle: using governor menu
EFI Variables Facility v0.08 2004-May-17
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
GRE over IPv4 demultiplexor driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
registered taskstats version 1
rtc_cmos 00:01: setting system clock to 2015-04-28 06:53:10 UTC (1430203990)
Initalizing network drop monitor service
Freeing unused kernel memory: 1280k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 800k freed
Freeing unused kernel memory: 1584k freed
dracut: dracut-004-336.el6_5.2
dracut: rd_NO_LUKS: removing cryptoluks activation
dracut: rd_NO_LVM: removing LVM activation
udev: starting version 147
dracut: Starting plymouth daemon
dracut: rd_NO_MD: removing MD RAID activation
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3
%GFATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
input: ImPS/2 Generic Wheel Mouse as /devices/platform/i8042/serio1/input/input4
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
dracut Warning: No root device "block:/dev/disk/by-uuid/a8cf6ab7-92fb-42c6-b95f-d437f94aaf98" found
%GFATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.32-431.17.1.el6.x86_64/modules.dep: No such file or directory
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
dracut Warning: Signal caught!
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: init Not tainted 2.6.32-431.17.1.el6.x86_64 #1
Call Trace:
[<ffffffff815274ef>] ? panic+0xa7/0x16f
[<ffffffff81077292>] ? do_exit+0x862/0x870
[<ffffffff8118a525>] ? fput+0x25/0x30
[<ffffffff810772f8>] ? do_group_exit+0x58/0xd0
[<ffffffff81077387>] ? sys_exit_group+0x17/0x20
[<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
I suggest at first to follow standard troubleshooting steps to validate the disk's filesystem as documented.
Then you can try to snapshot the disk and create a new instance with boot disk from snapshot to avoid exclude a disk UUID conflict.
Finally you can try to attach the disk to recover the files to a brand new instance as I suggested in my previous comment.
If you can not validate the filesystem as per troubleshooting guide the data is most probably lost and nothing more can be done to recover it.
Please note as a good practice to take care of either periodic snapshot, GCS differentiual backup or other replication method.
Thank you.
Sincerely,
P.
Related
I'm already running up to 4 DPDK processes next to each other without any issues and I can also restart secondary processes successfully.
I read here in end of the symmetric multi-process section, that you can destroy the primary process and restart it as a secondary.
But when I'm trying to restart the primary process, I run into some problems.
For example:
Running 2 processes. Each of them will stream data from its own dedicated port to the 0. queue of the virtual function. The goal is now to restart the first process as secondary.
After the init of the EAL , mbufs, and rings, I call rte_eal_remote_launch() for each process with its own dedicated lcore which launches a function that does some packet processing.
Start primary:
$ sudo mp_dpdk_app -l 0-4 -n 2 --proc-type=primary -- -p 3 --num-procs=2 --proc-id=0
Output:
EAL init start.
EAL: Detected CPU lcores: 64
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:19:10.0 (socket 0)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:19:10.1 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
EAL Process Type: PRIMARY
Start the secondary:
$ sudo mp_dpdk_app -l 0-4 -n 2 --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1
Output:
EAL init start.
EAL: Detected CPU lcores: 64
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_13330_2fd6664d78de
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:19:10.0 (socket 0)
eth_ixgbevf_dev_init(): No TX queues configured yet. Using default TX function.
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:19:10.1 (socket 0)
eth_ixgbevf_dev_init(): No TX queues configured yet. Using default TX function.
EAL Process Type: SECONDARY
Kill primary and restart with:
$ sudo mp_dpdk_app -l 0-4 -n 2 --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=0
But the init fails with the following output:
EAL init start.
EAL: Detected CPU lcores: 64
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_13473_2fda4aa02c52
EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to Connection refused
EAL: Fail to send request /var/run/dpdk/rte/mp_socket:bus_vdev_mp
vdev_scan(): Failed to request vdev from primary
EAL: Selected IOVA mode 'PA'
EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to Connection refused
EAL: Fail to send request /var/run/dpdk/rte/mp_socket:eal_vfio_mp_sync
EAL: Cannot request default VFIO container fd
EAL: VFIO support could not be initialized
EAL: Requested device 0000:19:10.0 cannot be used
EAL: Requested device 0000:19:10.1 cannot be used
EAL: Error - exiting with code: 1
Cause: :: no Ethernet ports found
I noticed that a new mp socket is created (mp_socket_13473_2fda4aa02c52).
But somehow the EAL tries then to connect to the rte/mp_socket, which was created by the primary process at the beginning and don't use the new one.
If I exit the primary process with rte_eal_cleanup() , the /rte/mp_socket is removed and I still can't start a new secondary process due to the error /rte/mp_process does not exist
My hardware setup:
Network devices using DPDK-compatible driver
============================================
0000:19:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf
0000:19:10.1 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf
The processes don't have to communicate in-between each other.
Can anybody give me a clue about this issue?
Any help will be appreciated.
The short answer to the question In DPDK multiprocess scenario (Primary & 1 or multiple secondary processes), if the DPDK primary process is stopped|killed can restarting the process allows it to communicate with the secondary process? is No, it can not.
let me explain this for clarity below
Please make use of DPDK documentation on multiprocess, which clarifies it is primary process is one which initializes the huge pages and creates the MP_HANDLE to communicate with the secondary process. In contrast, secondary or multiple secondaries attach with primary using MP_HANDLE.
Primary process makes use of PCI, virtual, Hugepage, cores and creates unique file-id in the default path as /var/run/dpdk/{file-prefix}-appname.
there ae configuration or conf settings saved in this path, which helps the secondary to make use of the hugepage address to MMAP in shared mode.
So when a primary comes up with all the required resource secondary dpdk process make uses pre-initialized environment to run certain subset of DPDK API
So when a primary process is stopped|killed, the resource mapping and configuration are not cleaned by default, which allows the existing secondary process to continue working. In case of EAL-ARGS --no-shconf immediate clean up of the folder /var/run/dpdk/{file-prefix}-dpdk is triggered (also does not support secondary process). Thereby starting|restarting a new primary will fail, as the PCI resources, hugepages, cores and file path (/var/run/dpdk/{file-prefix}-appname) already is been used.
hence the expectation of restarting primary can it connect to running secondary is not true. There are internal check rte_eal_init routine with the standard DPDK releases.
On one of my raspberry pis (Raspbian OS) with a USB serial device connected, there is periodic disconnection of the serial device while still being registered as a USB device.
lsusb returns a valid Bus Device entry, but there is no corresponding /dev/serial/ entry.
The only way to fix this issue is rebooting the pi, which I'd like to avoid.
I'd love some ideas on how I can debug USB serial connections on a Raspberry Pi Linux platform, or methods to reset the serial connection without requiring a reboot.
We tried:
Tracking udevadm monitor to see USB related events. This is what is seen where we catch a serial disconnection / reconnection happening:
KERNEL[65879.184887] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0/tty/ttyACM1 (tty)
KERNEL[65879.185179] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0 (usb)
KERNEL[65879.185389] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.1 (usb)
KERNEL[65879.185728] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.2 (usb)
KERNEL[65879.186275] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3 (usb)
UDEV [65879.193792] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.1 (usb)
UDEV [65879.197016] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0/tty/ttyACM1 (tty)
UDEV [65879.197414] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.2 (usb)
UDEV [65879.203665] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0 (usb)
UDEV [65879.205795] remove /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3 (usb)
==> /var/log/messages <==
Jan 20 18:36:42 rpi4161a-172-101 kernel: [65879.307684] usb 1-1.3: USB disconnect, device number 12
==> /var/log/syslog <==
Jan 20 18:36:42 rpi4161a-172-101 kernel: [65879.307684] usb 1-1.3: USB disconnect, device number 12
KERNEL[65880.121912] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3 (usb)
KERNEL[65880.122424] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0 (usb)
KERNEL[65880.125157] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0/tty/ttyACM0 (tty)
KERNEL[65880.125702] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.1 (usb)
KERNEL[65880.126324] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.2 (usb)
UDEV [65880.158159] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3 (usb)
UDEV [65880.167588] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0 (usb)
UDEV [65880.168004] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.1 (usb)
UDEV [65880.169128] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.2 (usb)
UDEV [65880.177534] add /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0/tty/ttyACM0 (tty)
==> /var/log/messages <==
Jan 20 18:36:43 rpi4161a-172-101 kernel: [65880.244987] usb 1-1.3: New USB device found, idVendor=****, idProduct=****, bcdDevice= 1.00
Jan 20 18:36:43 rpi4161a-172-101 kernel: [65880.245001] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jan 20 18:36:43 rpi4161a-172-101 kernel: [65880.245010] usb 1-1.3: Product: J-Link Pro OB
Jan 20 18:36:43 rpi4161a-172-101 kernel: [65880.245020] usb 1-1.3: Manufacturer: ****
Jan 20 18:36:43 rpi4161a-172-101 kernel: [65880.245028] usb 1-1.3: SerialNumber: ****
Jan 20 18:36:43 rpi4161a-172-101 kernel: [65880.248227] cdc_acm 1-1.3:1.0: ttyACM0: USB ACM device
Jan 20 18:36:43 rpi4161a-172-101 mtp-probe: checking bus 1, device 13: "/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3"
Jan 20 18:36:43 rpi4161a-172-101 mtp-probe: bus: 1, device: 13 was not an MTP device
Checking if the raspberry pi was overheating, using vcgencmd. It returns 8000 Soft temperature limit has occurred flag sometimes, but not always.
Tried the following method to "reset" the serial connection, which had no effect
echo -n '1-1.3:1.0' | sudo tee -a unbind
echo -n '1-1.3:1.0' | sudo tee -a bind
Followed some of the solutions here: https://raspberrypi.stackexchange.com/questions/9264/how-do-i-reset-a-usb-device-using-a-script, including the solution to reset the USB bus, with no luck.
USB Serial malfunctioning may be caused by many different factors, including software, hardware or noisy environment.
The following diagram roughly describes the related components,
-------------------- ----------------
| Userspace (Apps) | | Power Supply |
-------------------- ----------------
^
|
-------------------- ------------- ----------
| Kernel (Drivers) | <-- | USB Cable | <-- | Device |
-------------------- ------------- ----------
Narrow down to the faulty part
As first step, elimination method can help us to narrow down to the faulty part, basically you replace the suspect part with another good component, and observe if it changes result. for example,
use the latest official raspbian image to eliminate userspace issue
choose a different kernel/firmware image to eliminate kernel/driver issue
change usb cable
change usb device
use a decent power supply
Troubleshooting on software issue
Assume we could narrow down the scope to software, I would use the following procedures to do further check,
Make sure the usb device is recognized by kernel, check with lsusb, dmsg, udevadm.
Have a correct name (dev path), check udev rules;
Run udevadm monitor to observe any unexpected disconnection events, make sure it works (stable connection) on a fresh OS image, and install only necessary packages step by step.
Other tips
Check if your usb device is a "smart" device? meaning the device may have its own logic to disconnect or change mode, so that affect your host side connection.
To see what happened on the USB bus it is better to use usbmon (see https://wiki.wireshark.org/CaptureSetup/USB).
I think you should carefully check usb descriptors: lsusb -v and check how much current it attempts to take (check configuration descriptor: https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-configuration-descriptors).
I think reconnects could occur due to insufficient current to you usb device especially if device supplied only from bus. If current > than usb host controller can provide, usb host-controller limits current this could lead to device reset. To check this hypothesis connect your device to Raspberry using active hub with external power supply.
I have a Segger J-Link which I am trying to use on a Macbook running MacOS Catalina 10.15.4, with openocd and GDB against an ESP32 board. The problem is that I can not seen the device:
$ ls /dev/cu.*
/dev/cu.Bluetooth-Incoming-Port /dev/cu.JimsiPhone-WirelessiAP /dev/cu.SLAB_USBtoUART /dev/cu.usbserial-0001
None of these is the J-Link. If I run lsusb I can see it:
$ lsusb
Bus 020 Device 002: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP210x UART Bridge / myAVR mySmartUSB light
Bus 020 Device 003: ID 1366:0101 SEGGER J-Link ARM
I have installed the "J-Link Software and Documentation Pack" downloaded from Segger. I have checked the Mac "Security and privacy" settings and it does not report that it blocked any drivers or programs from being installed or run.
On the J-Link, the green LED is on, with a very brief flash about twice per second.
I'm sure I have a piece missing, and would appreciate some help.
UPDATE: I have been following instructions here:
OpenOCD Instructions
It all works until I get to step 6, and I follow these instructions:
Serial driver instructions
The problem is, a path for the driver never shows up, as I described above. I don't think I can run OpenOCD if I can't make it talk to my J-link.
When I run openocd-esp32, I get (full paste from openocd-esp32 output is below):
Error: No J-Link device found.
The contents of esp32-wroom-32.cfg is:
echo "WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release."
set ESP32_FLASH_VOLTAGE 3.3
source [find target/esp32.cfg]
Here is the full paste:
Jims-MacBook-Pro-486:~ jim$ openocd -f interface/jlink.cfg -f board/esp-wroom-32.cfg -c "program_esp32 build/hello-world.bin 0x10000 verify exit"
Open On-Chip Debugger v0.10.0-esp32-20200420 (2020-04-20-16:15)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release.
Info : Configured 2 cores
Error: No J-Link device found.
** OpenOCD init failed **
shutdown command invoked
Assertion failed: (jtag_trst == 0), function jtag_checks, file src/jtag/core.c, line 343.
Abort trap: 6
Running JLinkExe does find the J-Link:
Jims-MacBook-Pro-486:~ jim$ JLinkExe
SEGGER J-Link Commander V6.80b (Compiled Jun 5 2020 17:42:04)
DLL version V6.80b, compiled Jun 5 2020 17:41:46
Connecting to J-Link via USB...Updating firmware: J-Link V11 compiled Apr 23 2020 16:49:23
Replacing firmware: J-Link V11 compiled Aug 14 2019 16:21:09
Waiting for new firmware to boot
New firmware booted successfully
O.K.
Firmware: J-Link V11 compiled Apr 23 2020 16:49:23
Hardware version: V11.00
S/N: 51000936
License(s): GDB
VTref=0.000V
Type "connect" to establish a target connection, '?' for help
J-Link>
After doing the above I now get a different error message when running openocd-esp32 (perhaps because of the J-Link FW upgrade?). Initially it complained that there was not an adapter speed set, so I modified interface/jlink.cfg and added:
adapter_khz 3000
I now get a different error:
Error: JTAG scan chain interrogation failed: all ones
Which I have been Googling, and which could mean a bad board or still another configuration issue. There is no SD card in the SD card socket and no other SPI devices on the board, although the ESP32-WROVER-32U has SPI flash on it.
Here is the complete output from openocd-esp32:
Jims-MacBook-Pro-486:~ jim$ openocd -f interface/jlink.cfg -f board/esp-wroom-32.cfg -c "program_esp32 build/hello-world.bin 0x10000 verify exit"
Open On-Chip Debugger v0.10.0-esp32-20200420 (2020-04-20-16:15)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
adapter speed: 3000 kHz
WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release.
Info : Configured 2 cores
Info : J-Link V11 compiled Apr 23 2020 16:49:23
Info : Hardware version: 11.00
Info : VTarget = 0.000 V
Info : clock speed 3000 kHz
Error: JTAG scan chain interrogation failed: all ones
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway...
Error: esp32.cpu0: IR capture error; saw 0x1f not 0x01
Warn : Bypassing JTAG setup events due to errors
Info : Listening on port 3333 for gdb connections
Error: JTAG scan chain interrogation failed: all ones
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway...
Error: esp32.cpu0: IR capture error; saw 0x1f not 0x01
Warn : Bypassing JTAG setup events due to errors
Info : cpu0: Debug controller 0 was reset.
Info : cpu0: Core 0 was reset.
Error: esp32_soc_reset: Couldn't halt target before SoC reset
embedded:startup.tcl:449: Error: ** Unable to reset target **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/jim/.espressif/tools/openocd-esp32/v0.10.0-esp32-20200420/openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/jim/.espressif/tools/openocd-esp32/v0.10.0-esp32-20200420/openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 75
at file "embedded:startup.tcl", line 449
Warn : Flash driver of esp32.flash does not support free_driver_priv()
Warn : Flash driver of esp32.irom does not support free_driver_priv()
Warn : Flash driver of esp32.drom does not support free_driver_priv()
Success! This circuit used the Segger 10 pin needle connector. On that connector pin 1 is VTREF and on my board it was left floating, when it should have been connected to V3.3. I connected it and:
Jims-MacBook-Pro-486:~ jim$ openocd -f interface/jlink.cfg -f board/esp32-wrover.cfg
Open On-Chip Debugger v0.10.0-esp32-20200420 (2020-04-20-16:15)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
adapter speed: 1000 kHz
WARNING: boards/esp32-wrover.cfg is deprecated, and may be removed in a future release.
If your board is ESP32-WROVER-KIT, use board/esp32-wrover-kit-1.8v.cfg instead.
Info : Configured 2 cores
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : J-Link V11 compiled Apr 23 2020 16:49:23
Info : Hardware version: 11.00
Info : VTarget = 3.290 V
Info : clock speed 1000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : esp32: Debug controller 0 was reset.
Info : esp32: Core 0 was reset.
Info : esp32: Debug controller 1 was reset.
Info : esp32: Core 1 was reset.
Info : Listening on port 3333 for gdb connections
I would suggest to validate your setup step by step - I personally do not like big bang integrations.
Verify that your the Segger software can see your JLink probe - the good thing is that lsusb can see it. JLink Commander should provide some useful information.
Launch openocd without any executable-related arguments: openocd -f interface/jlink.cfg -f board/esp-wroom-32.cfg
Verify that basic commands are working, i.e. resetting the CPU, displaying registers, reading/writing memory.
If you you are still experimenting issues, double-check your wiring - see here for more information.
I have built the very latest u-boot release and placed the image on a Raspberry PI. After boot, I get this error message:
U-Boot 2013.10 (Oct 24 2013 - 09:35:44)
DRAM: 448 MiB
WARNING: Caches not enabled
MMC: bcm2835_sdhci: 0
Using default environment
In: serial
Out: lcd
Err: lcd
Hit any key to stop autoboot: 0
mmc0 is current device
** No partition table - mmc 0 **
U-Boot>
Obviously, u-boot failed to mount the file system.
Can this be related to some mmc related setting as access speed?
Thanks in advance
I got two identical servers, in both is installed postgresql server version 9.0.4 with the same configuration. If I launch a .sql file that performs about 5k inserts, on the first one it takes a couple of seconds, on the second one it takes 1 minute and 30 seconds.
If I set synchronous_commit, speed dramatically reduces (as expected), and the performances of the two servers are comparable. But if I set synchronous_commit to on, on one server the insert script execution time increases of less than one second, on the other one it increases too much, as I said in the first period.
Any idea about this difference in performances? Am I missing some configuration?
Update: tried a simple disk test: time sh -c "dd if=/dev/zero of=ddfile bs=8k count=200000 && sync"
fast server output:
1638400000 bytes (1.6 GB) copied, 1.73537 seconds, 944 MB/s
real 0m32.009s
user 0m0.018s
sys 0m2.298s
slow server output:
1638400000 bytes (1.6 GB) copied, 4.85727 s, 337 MB/s
real 0m35.045s
user 0m0.019s
sys 0m2.221s
Common features (both servers):
SATA, RAID1, controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller, distribution: linux centOS. mount -v output:
/dev/md2 on / type ext3 (rw)
proc on /proc type proc (rw)
none on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/md1 on /boot type ext3 (rw)
fast server: kernel 2.6.18-238.9.1.el5 #1 SMP
Disk /dev/sda: 750.1 GB, 750156374016 bytes
255 heads, 63 sectors/track, 91201 cylinders, total 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot Start End Blocks Id System
/dev/sda1 3906 4209029 2102562 fd Linux raid autodetect
/dev/sda2 4209030 4739174 265072+ fd Linux raid autodetect
/dev/sda3 4739175 1465144064 730202445 fd Linux raid autodetect
slow server: kernel 2.6.32-71.29.1.el6.x86_64 #1 SMP
Disk /dev/sda: 750.2 GB, 750156374016 bytes
64 heads, 32 sectors/track, 715404 cylinders, total 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006ffc4
Device Boot Start End Blocks Id System
/dev/sda1 2048 4194303 2096128 fd Linux raid autodetect
/dev/sda2 4194304 5242879 524288 fd Linux raid autodetect
/dev/sda3 5242880 1465147391 729952256 fd Linux raid autodetect
Could it be useful to address the performance issue?
I suppose your slow server with newer kernel has working barriers. This is good, as otherwise you can loose data in case of a power failure. But it is of course slower than running with write cache enabled and without barriers, aka running with scissors.
You can check if barriers are enabled using mount -v — search for barrier=1 in output. You can disable barriers for your filesystem (mount -o remount,barrier=0 /) to speed up, but then you risk data corruption.
Try to do your 5k inserts in one transaction — Postgres won't have to write to disk on every row inserted. The theoretical limit for number of transactions per second wound be comparable to disk rotational speed (7200rpm disk ≈ 7200/60 tps = 120 tps) as a disk can only write to a sector once per rotation.
To me this sounds like in the "fast" server there is a write cache enbled for the harddisk(s), whereas in the slow server the harddisk(s) are really writing the data when PG writes it (by calling fsync)