Binutils LD creates huge files - ld

I'm trying to create as small ELF as possible. I created a test file like this (NASM syntax):
SECTION .text
dd 0xdeadbeef
With this linker script:
SECTIONS {
.text : {
*(.text)
}
}
Then I checked sizes of flat binary and ELFs built two ways:
nasm -f bin -o test test.asm
It's flat binary, so 4 bytes.
nasm -f elf -o test.o test.asm
i686-elf-ld -Tlinker.ld test.o -o test
I'd expect something like 500 bytes max, but the resulting file is 4396 bytes long! There is an option however, named --strip-all, that could make this file smaller.
i686-elf-ld -Tlinker.ld test.o -o test --strip-all
4244 bytes. Still huge.
Why is LD generating so big files? Is there a way to make it smaller?

The linker is page aligning your text section to the nearest page boundary so that demand paging can be used.
$ objdump --headers -f test
test: file format elf32-i386
architecture: i386, flags 0x00000102:
EXEC_P, D_PAGED
start address 0x00000000
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000004 00000000 00000000 00001000 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
Notice the "Align" column of the text section is set to 4KB. Because of the alignment is set to 4Kb and demand paging is in use (D_PAGED), the .text section is located 4Kb into the file. Your text section is only 4 bytes long.
Link with -n to disable demand paging:
$ ld -Tlinker.ld test.o -o test --strip-all -n
$ objdump --headers -f test
test: file format elf32-i386
architecture: i386, flags 0x00000002:
EXEC_P
start address 0x00000000
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000004 00000000 00000000 00000060 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
$ ls -l test
-rwxrwxr-x 1 mikel mikel 240 Apr 15 12:31 test

Related

rsync tries to copy although file is copied

I'm trying to understand the following, maybe one of you guys can help me out by explaining on what exactly happens here:
My goal is to write a script, which copies files from the find-command, parallels rsync-commands and backup those, by re-creating the folder-structure from the source on the destination as well.
Somehow my initial script does not work like that, and i don't really have an idea on how to fix it to behave that way.
Initially, i've copied the files with "ls -1 /foldername" as sourceFolder, which is not best-practice. So i've tried to change it to using "find" as described beyond.
find "$sourceFolder" -maxdepth 2 -mindepth 2 -print0 | xargs --verbose -0 -I {} -P $maxThreads -n 1 rsync -valhP {}/ "$destFolder"/ --checksum --human-readable --stats --dry-run >> "$logDir"/result.log
--
If i run this script, it literaly would recopy the file(s), although those exist in the destination folder i would expect it to be.
sending incremental file list
.d..tp..... ./
>f+++++++++ macs.txt
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 242 bytes
Total transferred file size: 242 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.484 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 83
Total bytes received: 22
So as it turned out, sourceFolder and destFolder have to match, and i can't run anything like sourceFolder/folder1/folder1a destFolder/.
Can anyone help me out on what's wrong and why it behaves like that?
Thanks a Lot,
M.

Boot sector linked with ld has QEMU print "Hard Disk Boot Failed: not a bootable disk"

A few friends and I are working on a very simple kernel project. At the moment, we have a bootloader (using GNU assembly) and a simple kernel (C with no stdlib). I've been tasked with setting up a QEMU simulation to test the OS, but ran into several issues along the way.
I've gotten QEMU to boot, and I have created a bootable disk image.
Makefile:
%.o: %.S
as -o $# -c $<
bootsect: boot.o
ld -o ./bin/bootsect.bin $^ -nostdlib -Ttext 0x7C00
img: bootsect
dd if=/dev/zero of=boot.img bs=512 count=2880
dd if=./bin/bootsect.bin of=boot.img conv=notrunc bs=512 count=1
I tried to run it with qemu-system-i386 -drive format=raw,file=boot.img
I have also tried various ways of booting the image, but always end up with the same error:
Booting from Hard Disk:
Boot failed: not a bootable disk
Here is the boot loader code if needed:
.code16
.org 0
.text
.global _start
_start:
cli
xorw %ax, %ax
movw %ax, %es
movw %ax, %ds
movw %ax, %es
movw %ax, %ss
movw $0x3000, %sp
sti
movw $hello, %si
call print
print:
xorb %bh, %bh
movb $0x0E, %ah
lodsb
cmpb $0, %al
je 1f
int $0x10;
jmp print
1:
ret
hello:
.asciz "HELLO WORLD\n"
.org 510
.word 0xAA55
Where is the problem?
The linker is creating an ELF executable, not a raw binary. You need to extract the raw code contained in the .text section into a stand-alone file, without any wrapping object code format. Only then you can splice that file into the disk image as the boot sector.
There are two ways you can go about doing this. One is to add an extra step to the Makefile that invokes objcopy:
bootsect: bin/bootsect.bin
bin/bootsect.elf: boot.o
ld -o $# $^ -nostdlib -Ttext=0x7c00
%.bin: %.elf
objcopy -Obinary -j.text $< $#
The other is to use a linker script. Put the following in a file named bootsect.ld:
OUTPUT_FORMAT("binary")
SECTIONS {
.image 0x7c00 : {
*(.text)
}
}
And in the Makefile:
bin/bootsect.bin: boot.o bootsect.ld
ld -o $# boot.o -nostdlib -Tbootsect.ld
To check that this works, run file bin/bootsect.bin. It should output ‘DOS/MBR boot sector’.

Raspberry Pi 4 saving corrupt h264 files?

I've been able to record .h264 files without a problem, both using raspivid and through Python, but whatever I try, I can't convert them to a .mp4, .mkv or .avi file.
I've tried converting using MP4box (suggested in several places) and mkvmerge, but to no avail.
When I use MP4box (MP4box -add <source.h264> <dest.mp4>), I get :
AVC-H264 import - frame size 1920 x 1080 at 25.000 FPS
WARNING: NAL Unit type 0 not handled - adding5/100)
AVC Import results: 44 samples - Slices: 3 I 41 P 0 B - 0 SEI - 3 IDR
Saving video.mp4: 0.500 secs Interleaving
Then I try to play this file using VLC, it doesn't give an error, but shows garbled data.
To be clear, the camera works fine (shows perfect video using Python code and raspistill outputs a perfect jpg file). I tried a different camera as well.
An attempt with mkvmerge :
pi#raspberrypi:~/cam $ raspivid -o test.h264 -fps 30 -t 15000 -w 1920 -h 1080
pi#raspberrypi:~/cam $ mkvmerge --default-duration 0:30p -o video.mkv test.h264
mkvmerge v31.0.0 ('Dolores In A Shoestand') 32-bit
'test.h264': Using the demultiplexer for the format 'AVC/h.264'.
'test.h264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
The file 'video.mkv' has been opened for writing.
Error: 'test.h264' track 0: mkvmerge encountered broken or unparsable data in this AVC/h.264 video track. Either your file is damaged (which mkvmerge cannot cope with yet) or this is a bug in mkvmerge itself. The error message was:
Success
pi#raspberrypi:~/cam $
This is on a Raspberry Pi 4.
A sample file can be found here : https://filebin.net/c40usz0crhgggadf
Created with : raspivid -t 30000 -w 640 -h 480 -fps 25 -b 1200000 -p 0,0,640,480 -o pivideo.h264
I'm going to respond to this myself. After doing an 'apt-get upgrade', the problem is gone.
If anyone wants to figure this out to find the root cause, this is what apt-get did :
The following packages will be upgraded:
bind9-host bluez-firmware firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek libbind9-161 libdns-export1104 libdns1104 libgs9 libgs9-common libisc-export1100 libisc1100 libisccc161
libisccfg163 liblwres161 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc libraspberrypi0 pi-bluetooth raspberrypi-bootloader raspberrypi-kernel raspberrypi-kernel-headers raspi-config rpd-plym-splash
The following packages will be DOWNGRADED:
python3-pgzero
27 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.

cdb/windbg output too large

We are using cdb (command line version of winDBG) to resolve a cab.
For getting the output in a file we are using the -logo output to specify the output file.
For a certain cab we are getting "CvRegToMachine(x86) conversion failure for 0x7536" more than a million times.
Basically we get a huge resolved code log, nearly 1GB, and all of it filled with the above string on each line.
We are using the following cdb command
cdb -z "abc.cab" -y "SymbolsPath" -G -logo "outputfile" -lines -c ".kframes 100;!analyze -v;!load msec.dll;!exploitable -v;vertarget;lmv;q"
Does anyone have any clue about what could be wrong here?
you cannot use -c and -G at the same time -c needs the first break to read the initial
command and act upon it if you need to run the code use g; at the end of -c commands
also many times -c commands need to be provided first and the debugee needs
to be at the end of commandline
cdb -c "<some cmd;someother cmd;g>" -z foo.cab
.load not !load should be used to load third party extensions
the string emitted "CvReg.........." seems to be related to Either SYMFLAG_NULL or SYMFLAG_REGISTER in the Flags member of SYMBOLINFO Struct .
a sample trial didnt cross that code path in my machine so either the corrupt dmp or more information regarding the dmp file may be needed to find the reason for the spew
creating dump
C:\>dir /b *.cab
File Not Found
C:\>cdb -c ".dump /ma /b foo.cab;q" calc | grep -i -E "dmp|dump|wr"
0:000> cdb: Reading initial command '.dump /ma /b foo.cab;q'
Creating C:\DOCUME~1\Admin\LOCALS~1\Temp\foo.cab.dmp - mini user dump
Dump successfully written
Adding C:\DOCUME~1\Admin\LOCALS~1\Temp\foo.cab.dmp - added
Wrote foo.cab
C:\>dir /b *.cab
foo.cab
**loading dump as debugees(cdb) debuggee (foo.cab) and looking around **
C:\>cdb cdb -z foo.cab
0:000> s -u dbgeng l?39b000 "CvReg"
020341f8 0043 0076 0052 0065 0067 0054 006f 004d C.v.R.e.g.T.o.M.
0:000> # *(*20341f8 dbgeng l?39b000
dbgeng!MachineInfo::CvRegToMachine+0xfe:
021bf8ae 68f8410302 push offset dbgeng!`string' (020341f8)
0:000> # call*dbgeng!MachineInfo::CvRegToMachine dbgeng l?39b000
dbgeng!ImageInfo::CvRegToMachine+0x22:
021b62f2 e8b9940000 call dbgeng!MachineInfo::CvRegToMachine (021bf7b0)
0:000> # call*CvRegToMachine dbgeng l?39b000
dbgeng!ImageInfo::CvRegToMachine+0x22:
021b62f2 e8b9940000 call dbgeng!MachineInfo::CvRegToMachine (021bf7b0)
dbgeng!TypeInfoValueFromSymInfo+0x4b:
022541ab e82021f6ff call dbgeng!ImageInfo::CvRegToMachine (021b62d0)
dbgeng!TypedData::SetToSymbol+0x25f:
02285edf e8ec03f3ff call dbgeng!ImageInfo::CvRegToMachine (021b62d0)
dbgeng!TypedData::SetToSymbol+0x2da:
02285f5a e87103f3ff call dbgeng!ImageInfo::CvRegToMachine (021b62d0)
0:000> ln 0x21bf8ae
(021bf7b0) dbgeng!MachineInfo::CvRegToMachine+0xfe | (021bf8d0) dbgeng!Ma
chineInfo::GetContextState
doing and uf on this function yields the check where SymbolInfo->Flags is
checked and decided my sample dump above doesnt enter the path

windbg conflicting information

I have WinDBG 6.12.0002.633 x86 and I'm using it to view a post-mortem kdmp from a Windows Mobile 6 ARMV4I application.
When I go to analyze the callstack I get a lot of unknowns. In the analysis, I can see in the *FAULTING_IP* section that the fault is in the tcpstk module. (for which I also have symbols. But, in the *STACK_TEXT* section, the tcpstk addresses appear as just addresses, no symbols.
Also, in the *MODULE_NAME* section, I get another unknown even though it just said the faulting module was in tcpstk.
The result of the !analyze -v command is:
1:128:armce> !analyze -v
***snip!***
FAULTING_IP:
tcpstk!_DerefIF+38 [\private\winceos\comm\tcpipw\ip\iproute.c # 1032]
01b0d6f0 ???????? ???
***snip!***
IP_ON_HEAP: 07b00090
The fault address in not in any loaded module, please check your build's rebase
log at <releasedir>\bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.
FRAME_ONE_INVALID: 1
STACK_TEXT:
761efa6c 07b00090 : 7b858453 00000003 00000000 00000000 : 0x7b0d6f0
761efa7c 07b0020c : 7b858453 506f010a 00000000 00000000 : 0x7b00090
761efacc 78012d38 : 7b858453 506f010a 00000000 00000000 : 0x7b0020c
761efaf4 78013cdc module_78010000!AdapterBindingManager::NetUp+0xb4 [bar.cpp # 268]
761efb34 78014b78 module_78010000!AdapterBindingManager::EnterState+0x5e4 [bar.cpp # 1327]
761efda4 78015c08 module_78010000!AdapterBindingManager::ProcessEvent+0x8e4 [bar.cpp # 1298]
761efdd8 03f668dc module_78010000!MediaSense+0x25c [foo.cpp # 673]
761efe94 00000000 coredll_3f49000!ThreadBaseFunc+0x98 [\private\winceos\coreos\core\dll\apis.c # 633]
MODULE_NAME: Unknown_Module
IMAGE_NAME: Unknown_Image
DEBUG_FLR_IMAGE_TIMESTAMP: 0
STACK_COMMAND: ~128s ; kb
FAILURE_BUCKET_ID: INVALID_POINTER_WRITE_c0000005_Unknown_Image!Unknown
If I switch to the kp command, I suddenly can see that part of the callstack
1:128:armce> kp
Child-SP RetAddr Call Site
761efa6c 01b0d6e0 tcpstk!_DerefIF(struct Interface * IF = 0x7b858453)+0x38 [\private\winceos\comm\tcpipw\ip\iproute.c # 1032]
761efa6c 00000000 tcpstk!_DerefIF(struct Interface * IF = 0x7b858453)+0x28 [\private\winceos\comm\tcpipw\ip\iproute.c # 1026]
Why isn't the !analyze -v command able to show the fully decoded callstack? Why does it show so many unknowns?
I think that WinDBG cannot debug ARM I have not seen any documentation that states it is capable of debugging ARM, only x86 and x64 applications.
There is a Windbg provided in the ARM toolkit that is the windowed version of armsd which is not related to the microsoft WindDbg.