How to copy whole directories containing subdirectories to /boot (i.e bootfs) in Yocto while inheriting core-image class? - yocto

I have a directory which again contains subdirectories, which are built has part of other recipe and moved to DEPLOY_DIR_IMAGE using deploy bb class. So now I want to copy it to main image boot partition.
If it was a single file then appending required filename to IMAGE_EFI_BOOT_FILES variable, then yocto copies it to /boot. But same doesn't work for directories containing subdirectories please provide style to include even the subdirectories. Thank you
PS: I have tried appending IMAGE_EFI_BOOT_FILES += "parent_dir/*" didnt work.

It is obvious that IMAGE_EFI_BOOT_FILES is acting like the well known IMAGE_BOOT_FILES and other variables that are responsible for having the files necessary to be shipped in the boot partition. And that needs files and not directories.
So, if you do not need to specify all the files by hand, but instead you want to pass the directory, I suggest you use a python method to collect the files for you and append them to the variable.
See the following example I developed and tested:
def get_files(d, dir):
import os
dir_path = dir
if not os.path.exists(os.path.dirname(dir)):
dir_path = d.getVar(dir)
return ' '.join(f for f in os.listdir(d.getVar(dir)) if os.path.isfile(f))
IMAGE_EFI_BOOT_FILES += "${#get_files(d, 'DEPLOY_DIR_IMAGE')}"
The method will test if the argument is a real path then it will directly check for files, if not it will assume that it is a bitbake variable and it will get its content, so if DEPLOY_DIR_IMAGE is, for example, /home/user/dir, passing DEPLOY_DIR_IMAGE or /home/usr/dir will give the same result.
IMPORTANT
It is obvious also that IMAGE_EFI_BOOT_FILES is used in a .conf file such as local.conf or a custom machine configuration file. So adding that python function in .conf file will not work. I suggest creating a class for it and inherit it globally in your .conf file:
meta-custom/classes/utils.bbclass
local.conf:
INHERIT += "utils"
IMAGE_EFI_BOOT_FILES += "${#get_files(d, 'DEPLOY_DIR_IMAGE')}"
Try this and let me know in the comments.
EDIT
I have just realized that bitbake already imports os within python expressions expansions, so you can do it in one line without any need for a separate python function:
PATH = "/path/to/directory/" or
PATH = "A variable containing the path"
IMAGE_EFI_BOOT_FILES += "${#' '.join('%s' % f for f in os.listdir('${PATH}') if os.path.isfile(f))}"

Note: I am looking for Yocto built-in which can achieve solution for above mentioned , would like to share other way to resolve the functionality for community's benefit.
Add following in bb file if you are using one or refer to talel-belhadjsalem answer to use utils.bbclass.
def add_directory_bootfs(d, dirname, bootfs_dir):
file_list = list()
boot_files_list = list()
deploy_dir_image = d.getVar('DEPLOY_DIR_IMAGE')
for (dirpath, dirnames, filenames) in os.walk(os.path.join(deploy_dir_image, dirname)):
file_list += [os.path.join(dirpath, file) for file in filenames]
for file in file_list:
file_rel_path = os.path.relpath(file, os.path.join(deploy_dir_image, dirname))
boot_file_entry = os.path.join(dirname, file_rel_path) + ';' + os.path.join(bootfs_dir, dirname, file_rel_path)
boot_files_list.append(boot_file_entry)
return ' '.join(boot_files_list)
IMAGE_EFI_BOOT_FILES += "${#add_directory_bootfs(d, 'relative_path_to_dir_in_deploy_dir_image', 'EFI/BOOT')}"

Related

"failed to load any lstm-specific dictionaries for lang " tesseract 4.1

I tried to train the tesseract 4.1 using OCRD project but after training completed I copied the lang.traineddata but getting above error.
The tesseractWiki page is very confusing to understand asking to use combine_lang_model after making lstmf file. So Actually I have the lstmf file. I created these file by using tif/box pair.
Please help me for further step.
Related discussions:Failed to load any lstm-specific dictionaries for lang xxx
Suppose your training folder like this:
OCRD/makefile
OCRD/data/foo-ground-truth.
You could try as following steps:
Find the WORDLIST_FILE/NUMBERS_FILE/PUNC_FILE in the makefile, and change them to:
WORDLIST_FILE := data/$(MODEL_NAME).wordlist
NUMBERS_FILE := data/$(MODEL_NAME).numbers
PUNC_FILE := data/$(MODEL_NAME).punc
Suppose your base traineddata is eng.traineddata.
2.1 Download the .wordlist/.numbers/.punc files from the langdata_lstm.
2.2 Place them in OCRD/data
2.3 if the MODEL_NAME = foo, rename them as: foo.wordlist, foo.numbers, foo.punc
if you don't have the base traineddata, you could try this too. But if your base traineddata is afr, you should download the files from langdata_lstm/afr.
make training again
The cause of this error:
In OCRD, the default path of the above three files is $ (OUTPUT_DIR) = data / $ (MODEL_NAME), and all files in this path are automatically generated during the training process.
If the variable START_MODEL is not assigned, the makefile will not generate any related files under this path;
If the variable START_MODEL has been assigned, the foo.lstm-number-dawg、foo.lstm-punc-dawg、foo.lstm-word-dawg and so on will be produced in data / $ (MODEL_NAME). But they are not the right one. So there may be a bug in OCRD.

how to get ${THISDIR} inside do_unpack_append in .bbappend file

I'm attempting to replace a file from another layer with a .bbappend file. My goal is to overwrite a specific configuration file with a customized one during the unpack stage.
In my .bbappend I'm attempting to append the do_unpack to copy a file from the same directory as the .bbappend file into the working directory ${WORKDIR} The problem is: When inside do_unpack_append, ${THISDIR} is returning the directory of the original .bb recipe, rather than the directory of .bbappend
Here's an example:
The original recipe resides in: meta-origLayer/recipe.bb
My *.bbappend resides in: meta-newLayer/recipe.bbappend
recipe.bbappend:
`FILESEXTRAPATHS_prepend := "${THISDIR}:"`
do_unpack_append(){
bb.build.exec_func('replace_file', d)
}
replace_file(){
cp -f ${THISDIR}/fileToBeReplaced ${WORKDIR}/fileToBeReplaced
echo ${THISDIR} > ${WORKDIR}/shouldContain_meta-newLayer
}
There are two issues with recipe.bbappend:
I would expect the file shouldContain_meta-newLayer to contain meta-newLayer, but instead it contains meta-origLayer.
I'd primarily like to understand why ${THISDIR} behaves differently when placed inside do_unpack_append() from when it is used for prepending FILESEXTRAPATHS
When running bitbake, the recipe fails, producing the following error:
cp: cannot stat '/fileToBeReplaced': No such file or directory
This error occurs because fileToBeReplaced resides in a subdirectory of meta-origLayer (i.e. meta-origLayer/machine1/fileToBeReplaced) and the .bbappend expects to find the file in /fileToBeReplaced
My Question. . .
I have assumed ${THISDIR} would behave consistently within the same .bbappend, but it doesn't appear to. What is the best way to reference meta-newLayer/fileToBeReplaced from within do_unpack_append()?
This *.bbappend correctly overwrites fileToBeReplaced in the working directory during the unpack task:
FILESEXTRAPATHS_prepend := "${THISDIR}:"
SRC_URI_append += " file://fileToBeReplaced "
SAVED_DIR := "${THISDIR}"
do_unpack_append(){
bb.build.exec_func('replace_file', d)
}
replace_file(){
cp -f ${SAVED_DIR}/fileToBeReplaced ${WORKDIR}/fileToBeReplaced
}
Thanks for the explanation between bbappend parsing and execution johannes-schaub-ltb

Stripping base path off the unpacked source tree with bitbake SRC_URI file:// fetcher

The manual here says that there is a basepath option to SRC_URI that should "strip the specified directories from the source path when unpacking".
I'm trying to fetch the sources from a local directory, say /src/someproject.
For that purpose I configured my recipe as follows:
SRC_URI="file:///src/someproject;subdir=source;basepath=/src/someproject"
The intention was to have the sources taken from /src/someproject directory and put into build/tmp/work/target/someproject/1.0-r1/source/. Instead, I'm getting the sources under build/tmp/work/target/someproject/1.0-r1/source/src/someproject.
Is there a way to get rid of /src/someproject subdirectory inside source ?
The documentation you point to is for yocto 1.6, release April 2014. The basepath parameter appears to have been removed in later releases, with no fanfare and no apparent replacement.
Instead you can do something like:
SRCDIR = "/path/to/your/files"
SRC_URI = "file://${SRCDIR}/contents/;subdir=src"
S = "${WORKDIR}/src"
Then you can access your files under ${S}/${SRCDIR}.
If you find using the ${SRCDIR} part too cumbersome, you can hook onto the do_patch target, which need the prefunc mechanism and not _prepend if you want to use shell scripting, as do_patch is otherwise written in python:
relocate_source() {
mv ${S}/${SRCDIR}/* ${S}/
rmdir ${S}/${SRCDIR}
}
do_patch[prefunc] += "relocate_source"
This will reorganise your source before applying any patch you can add to SRC_URI.
Also note that a file:// URI does not get cached in ${DL_DIR}, so there is no name-conflict to handle (the way we would need to use downloadfilename= in an http:// URI).

How to do File creation and manipulation in functional style?

I need to write a program where I run a set of instructions and create a file in a directory. Once the file is created, when the same code block is run again, it should not run the same set of instructions since it has already been executed before, here the file is used as a guard.
var Directory: String = "Dir1"
var dir: File = new File("Directory");
dir.mkdir();
var FileName: String = Directory + File.separator + "samplefile" + ".log"
val FileObj: File = new File(FileName)
if(!FileObj.exists())
// blahblah
else
{
// set of instructions to create the file
}
When the programs runs initially, the file won't be present, so it should run the set of instructions in else and also create the file, and after the first run, the second run it should exit since the file exists.
The problem is that I do not understand new File, and when the file is created? Should I use file.CreateNewFile? Also, how to write this in functional style using case?
It's important to understand that a java.io.File is not a physical file on the file system, but a representation of a pathname -- per the javadoc: "An abstract representation of file and directory pathnames". So new File(...) has nothing to do with creating an actual file - you are just defining a pathname, which may or may not correspond to an existing file.
To create an empty file, you can use:
val file = new File("filepath/filename")
file.createNewFile();
If running on JRE 7 or higher, you can use the new java.nio.file API:
val path = Paths.get("filepath/filename")
Files.createFile(path)
If you're not happy with the default IO APIs, you an consider a number of alternative. Scala-specific ones that I know of are:
scala-io
rapture.io
Or you can use libraries from the Java world, such as Google Guava or Apache Commons IO.
Edit: One thing I did not consider initially: I understood "creating a file" as "creating an empty file"; but if you intend to write something immediately in the file, you generally don't need to create an empty file first.

Copy all files with given extension to output directory using CMake

I've seen that I can use this command in order to copy a directory using cmake:
file(COPY "myDir" DESTINATION "myDestination")
(from this post)
My problem is that I don't want to copy all of myDir, but only the .h files that are in there. I've tried with
file(COPY "myDir/*.h" DESTINATION "myDestination")
but I obtain the following error:
CMake Error at CMakeLists.txt:23 (file):
file COPY cannot find
"/full/path/to/myDIR/*.h".
How can I filter the files that I want to copy to a destination folder?
I've found the solution by myself:
file(GLOB MY_PUBLIC_HEADERS
"myDir/*.h"
)
file(COPY ${MY_PUBLIC_HEADERS} DESTINATION myDestination)
this also works for me:
install(DIRECTORY "myDir/"
DESTINATION "myDestination"
FILES_MATCHING PATTERN "*.h" )
The alternative approach provided by jepessen does not take into account the fact that sometimes the number of files to be copied is too high. I encountered the issue when doing such thing (more than 110 files)
Due to a limitation on Windows on the number of characters (2047 or 8191) in a single command line, this approach may randomly fail depending on the number of headers that are in the folder. More info here https://support.microsoft.com/en-gb/help/830473/command-prompt-cmd-exe-command-line-string-limitation
Here is my solution:
file(GLOB MY_HEADERS myDir/*.h)
foreach(CurrentHeaderFile IN LISTS MY_HEADERS)
add_custom_command(
TARGET MyTarget PRE_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${CurrentHeaderFile} ${myDestination}
COMMENT "Copying header: ${CurrentHeaderFile}")
endforeach()
This works like a charm on MacOS. However, if you have another target that depends on MyTarget and needs to use these headers, you may have some compile errors due to not found includes on Windows. Therefore you may want to prefer the following option that defines an intermediate target.
function (CopyFile ORIGINAL_TARGET FILE_PATH COPY_OUTPUT_DIRECTORY)
# Copy to the disk at build time so that when the header file changes, it is detected by the build system.
set(input ${FILE_PATH})
get_filename_component(file_name ${FILE_PATH} NAME)
set(output ${COPY_OUTPUT_DIRECTORY}/${file_name})
set(copyTarget ${ORIGINAL_TARGET}-${file_name})
add_custom_target(${copyTarget} DEPENDS ${output})
add_dependencies(${ORIGINAL_TARGET} ${copyTarget})
add_custom_command(
DEPENDS ${input}
OUTPUT ${output}
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${input} ${output}
COMMENT "Copying file to ${output}."
)
endfunction ()
foreach(HeaderFile IN LISTS MY_HEADERS)
CopyFile(MyTarget ${HeaderFile} ${myDestination})
endforeach()
The downside indeed is that you end up with multiple target (one per copied file) but they should all end up together (alphabetically) since they start with the same prefix ORIGINAL_TARGET -> "MyTarget"