I used Yocto to build a filesystem, using a .bbappend of core-image-minimal. Two questions:
how can i figure out which package is taking huge storage space on the rootfs?
I can't think of a way other than to look into the ${D} of every package and see how big its components are. There's gotta be a more systematic, and intelligent way to do that.
From what i can decipher from the manifest, there is nothing related to the size of the package that is being included.
Also, removing some of the packages I added using the IMAGE_INSTALL object, seems to remove the package but the end result of the built image doesn't show a change in its size!!
I compared the size of a particular .so on the build machine and on the installation device (a vm) and found that the size on the installation device was 20-30% of the original size seen on the build machine. Any explanation?
Thanks!
1) One way is to enable buildhistory, by adding the following to local.con
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"
This will create a directory (git repo) buildhistory in your $BUILDDIR. There you'll be able to find e.g.
images/$MACHINE/eglibc/$IMAGE/installed-package-sizes.txt
That file will give you the sizes of all installed packages.
There are a lot more things you can learn from buildhistory, see buildhistory introduction
2) Where did you compare the particular .so-file? If it was from the package's ${B} (i.e. where the library is built), it's not surprising, as the installed .so-file will be stripped. The debug information is installed into -deb.rpm (as the debug info is usually useless on the target and the smaller size is of much higher importance).
With some looking inside the scripts/ subdir and some googling about some of the existing scripts, it turns out that the good people of Yocto do have these scripts properly functioning out of the box:
scripts/tiny/dirsize.py and ksize.py.
dirsize.py will give you a breakdown of pkg sizes for your rootfs; while ksize.py will give you the equivalent info for the kernel.
Related
I have an .osm.pbf file which I want to use to generate vector tiles with (.mbtiles).
Im currently on a windows machine utilising docker, I have tried to use the tool tilemaker (https://github.com/systemed/tilemaker) though I cannot get it to work on my files and get issues like so
"
terminate called after throwing an instance of 'std::runtime_error'
what(): Exception during zlib decompression: (-5)
"
I was just wondering if anyone else was able to generate these tiles from said file type, if so could you provide a low level detailed guide on how you did so, as I am new to vector tiles and am getting confused within some circumstances.
For anyone interested I use this code to run the docker:
docker run tilemaker tilemaker --input=sud-latest.osm.pbf --output=sud.mbtiles
I have to put tilemaker twice as otherwise it says it cannot open the .osm.pbf otherwise
I made a tutorial on how to generate tiles using maptiler:
https://blog.kleunen.nl/blog/tilemaker-generate-map
It is focused on linux, but you can run it on windows as well. You can find a pre-built version of maptiler on the CI:
https://github.com/systemed/tilemaker/pull/208/checks?check_run_id=2143761163
Probably soon they will also become available on the github page.
Once you have the prebuilt executable and the resources (config and process lua), you can simply do:
tilemaker.exe --input=sud-latest.osm.pbf --output=sud.mbtiles --process resources/process-openmaptiles.lua --config resources/config-openmaptiles.json
The output works best from zoom level 8 - 14, borders are still missing, so lower zoom levels look pretty empty.
You can use ogr2ogr (see other answer here) to translate osm.pbf into geojson, and then Mapbox's tippecanoe tool to convert the geojson to mbtiles.
possible solutions :
1.Might be RAM issue try to run small size osm.pbf file with tilemaker
2.Run tilemaker.exe from executable file (by making build from github tilemaker clone) ---> it may solve most of issues
I have customized buildroot with the new board ( derived from raspberry pi zero ). So my changes are (in-tree):
.config
board/passkeeper/genimage-passkeeper.cfg
board/passkeeper/post-build.sh
board/passkeeper/post-image.sh
board/passkeeper/rootfs_overlay/etc/init.d/S41passkeeper
board/passkeeper/rootfs_overlay/etc/mdev.conf
board/passkeeper/rootfs_overlay/etc/udhcpd.conf
configs/passkeeper_defconfig
output/build/linux-custom/.config
Now, reading the documentation - I am a bit confused on how to put all these things into the separate folder via BR2_EXTERNAL. Also I'm not sure how do I move the linux configuration from output/build/linux-custom/.config
make linux-update-defconfig BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE=/tmp/passkeeper/linux/linux-config
results in
Unable to perform linux-update-defconfig when using a defconfig rule
Can somebody please provide step-by-step guide on that?
[You are asking two questions. I will answer only the question about saving the linux .config file; the other question is too generic.]
You need to set the appropriate options in menuconfig, not just override on the command line, otherwise they are inconsistent.
The process complete for creating a linux defconfig based on a pre-existing in-tree defconfig is the following. You have already done steps 1, 2 and 3.
In the Buildroot configuration, select BR2_LINUX_KERNEL_USE_DEFCONFIG or BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG.
Run make linux-menuconfig and adapt the linux configuration to your needs.
Build and test, iterate over 2 until you have the configuration you want.
In the Buildroot configuration, switch to BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG and set BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE to the place where you want to save it (typically board/passkeeper/linux.config or $(BR2_EXTERNAL_PASSKEEPER)/board/passkeeper/linux.config if you are using an external).
Run make linux-update-defconfig. It is essential you do this before doing anything else, otherwise Buildroot will complain that the file doesn't exist.
I'm trying out yocto (2.0, jethro) and I want to build an image starting from core-image-minimal. This works fine.
Every website out there mention modifying the file build/config/local.conf with (some of) my customization. For example, the target machine (through MACHINE) or some global settings (through EXTRA_IMAGE_FEATURES).
I also need to modify some specific packages and the way to do it is to create a custom layer. So far so good.
What I don't understand is how to "save" all my configuration to version control. I want everything I change to be locate in files that I can commit so that anybody else can reproduce the exact same build (or even contribute to that project). Putting almost everything in build/config/local.conf goes against that goal; the file is under a "build" directory and so I can't just clone a git repo and start the building...
Is it really the way the yocto project works? Or am I missing a different configuration file where I need to put these settings? I though I could place all these in a custom layer but it does not seem to work...
Any idea or suggestion?
Thanks!
Thanks Ross, that clarified it!
Here's some notes about my file organization which I couldn't format into a comment to your answer.
Thanks. So all my custom configurations went into meta-mylayer/conf/distro/mylayer.conf
Almost all my customization went into a layer meta-mylayer, except:
DISTRO which is set in build/conf/local.conf. This is how you tell yocto what you want to build.
MACHINE which is also set in build/conf/local.conf. The reason is that the same image/distro combination could be built for different machines and thus this can't be hard-coded for every images.
Layers are manually added to build/conf/layers.conf. That's the last bit I wish I could moved to my DISTRO or something. For now the folders are git submodules and they are added using bitbake-layers add-layer.
In general everything in your local.conf that is "your project" should be moved to your own distro configuration (MACHINE, image features, package lists). Stuff like where DL_DIR is can be moved to a common site.conf if you wish. Eventually you should end up with a local.conf which just sets DISTRO and some other personal variables.
I downloaded MikeOS and I ran his OS through Oracle VirtualBox and I thought it would be fun to tweak the OS a bit by adding programs/features etc.
So I modified some code, and I know the code is right because there were no compiling errors.
Anyway once the OS was modified, I downloaded MagicISO so I can make my own version of an ISO file to put the modified OS on.
However each time I try to boot up my ISO VirtualBox says "No Bootable Medium found!".
Am I turning it into an ISO wrong? Here is how MikeOS is formatted:
http://prntscr.com/37vchg
What I did was, I modified the "programs" folder with my new code then I copied everything in the folder shown in the screenshot above.
I pasted it into a new folder on the desktop called "MikeOS(modified)". I then right clicked and turned that folder into an ISO. Did I do something wrong?
Thanks!
PS: I can't add pictures because I need 20 Reputation.
"So I modified some code, and I know the code is right because there were no compiling errors."
Not sure what you mean by your code being "right".
"No compiling errors" just means your syntax was correct and all libraries, etc, were available. It doesn't mean your code is logically correct.
"However each time I try to boot up my ISO VirtualBox says "No Bootable Medium found! Am I turning it into an ISO wrong?"
Yes, you're doing it wrong.
Any bootable disk, be it floppy, hard drive or CD, requires a boot block to be present on the bootable media. The boot block is hidden from normal user inspection.
When you rip a CD (or whatever) all you're doing is grabbing data files off the media which may include jpgs, exes, dlls, sys files, and/or whatever else you might have, but doesn't touch the boot block.
In your case, you had all your data files, some of which you modified, and then burned them to a CD. It may LOOK just like the CD you ripped, but YOUR CD doesn't have the boot block, so it won't boot - as you have now discovered.
To fix your problem you need to get a boot block image (or as magic ISO calls them, a "boot image file") and add it to your CD composition and THEN burn your CD. Then your CD will be "bootable".
There's a few ways of doing that, and I won't go into that, but look under the "tools" tab of magiciso to get you started.
I'm interested in Qooxdoo as a possible web development framework. I have downloaded the SDK and installed it in a central location on my PC as I expect to use it on multiple projects. I used the create-application.py script to make a new test application and added all the generated files to my version control system.
I would like to be able to collaborate on this with other developers on other PCs. They are likely to have the SDK installed in a different location. The auto-generated files in Qooxdoo seem to include the SDK path in both config.json and generator.py: if the SDK path moves, the generator.py script stops working. generator.py doesn't seem to be too much of a problem as it looks in config.json for an updated path, but I'm not sure how best to handle config.json.
The only options I've thought of so far are:
Exclude it from the VCS, but there doesn't seem to be a script to regenerate it automatically, so this could be dangerous.
Add it to the VCS but have each developer modify the path line and accept that it might need to be adjusted whenever changes are merged.
Change config.json to be a path and a single 'include' line that points to a second file that contains all the non-SDK-path related information.
Use a relative path to the SDK and keep a separate, closely located copy of the SDK for every project that uses it.
Approach 1 would be ideal if the generation script existed; approach 2 is really nasty; I couldn't get approach 3 to work and approach 4 is a bit messy as it means multiple copies of the SDK littered about the place.
The Android SDK seems to deal with this very well (using approach 1), with the SDK path in its own file with a script that automatically generates that file. As far as I can tell, Qooxdoo puts lots of other important information in config.json and the only way to automatically generate that file is to create a new project.
Is there a better/recommended way to deal with this?
As an alternative to using symlinks, you can override the QOOXDOO_PATH macro on the command line:
./generate.py source -m QOOXDOO_PATH:<local_path_to_qooxdoo>
(Depending on the shell you are using you might have to apply some proper quoting of the -m argument). This way, every programmer can use his locally installed qooxdoo SDK. You can even drop the QOOXDOO_PATH entry from config.json to enforce this.
We work with a symbolic link pointing to the sdk ... config.json contains just the path of the link.