Can't test if directory already exists from within Fastlane with Ruby - fastlane

I can’t test whether a directory exists from within a Fastlane action nor lane. None of the following work for save_path = "./dir_name" nor save_path = "dir_name" in the current directory (from which fastlane is being run):
!Dir.empty?(save_path)
!Dir[save_path].empty?
Dir.exist?(save_path)
File.exist?(save_path)
File.directory?(save_path)
I even tried to expand the relative path:
File.exists? File.expand_path(save_path)
I have referred to the following:
Check if directory is empty in Ruby
How to check if a given directory exists in Ruby
https://blog.bigbinary.com/2017/02/28/dir-emtpy-included-in-ruby-2-4.html
https://ruby-doc.org/core-2.2.0/Dir.html
How does one test to see if a directory exists from within Fastlane? Thank you for reading and for your help!

The correct format to check this:
Dir.exist? "#{save_path}"
Will return true if it exists or false if not.

LOL Well, I discovered my issue. I didn't realize that I had to include the entire path to each of these statements, since I'm using Dir.foreach to iterate over part of the filesystem; the updated syntax is:
Dir.foreach(save_path) do |dir|
next if dir == '.' or dir == '..'
if (File.directory?("#{save_path}#{dir}"))
begin
FileUtils.mkdir_p("./#{save_path}#{dir}/images")
rescue StandardError => e
UI.message("Failed to make directory ./#{save_path}#{dir}/images: #{e.to_s}")
return
end
end
end
This is versus File.directory?("#{dir}") in that if statement. I thought that the Ruby Dir and FileUtil libraries would look relative to the current directory for the remainder of the path.

Related

How to copy whole directories containing subdirectories to /boot (i.e bootfs) in Yocto while inheriting core-image class?

I have a directory which again contains subdirectories, which are built has part of other recipe and moved to DEPLOY_DIR_IMAGE using deploy bb class. So now I want to copy it to main image boot partition.
If it was a single file then appending required filename to IMAGE_EFI_BOOT_FILES variable, then yocto copies it to /boot. But same doesn't work for directories containing subdirectories please provide style to include even the subdirectories. Thank you
PS: I have tried appending IMAGE_EFI_BOOT_FILES += "parent_dir/*" didnt work.
It is obvious that IMAGE_EFI_BOOT_FILES is acting like the well known IMAGE_BOOT_FILES and other variables that are responsible for having the files necessary to be shipped in the boot partition. And that needs files and not directories.
So, if you do not need to specify all the files by hand, but instead you want to pass the directory, I suggest you use a python method to collect the files for you and append them to the variable.
See the following example I developed and tested:
def get_files(d, dir):
import os
dir_path = dir
if not os.path.exists(os.path.dirname(dir)):
dir_path = d.getVar(dir)
return ' '.join(f for f in os.listdir(d.getVar(dir)) if os.path.isfile(f))
IMAGE_EFI_BOOT_FILES += "${#get_files(d, 'DEPLOY_DIR_IMAGE')}"
The method will test if the argument is a real path then it will directly check for files, if not it will assume that it is a bitbake variable and it will get its content, so if DEPLOY_DIR_IMAGE is, for example, /home/user/dir, passing DEPLOY_DIR_IMAGE or /home/usr/dir will give the same result.
IMPORTANT
It is obvious also that IMAGE_EFI_BOOT_FILES is used in a .conf file such as local.conf or a custom machine configuration file. So adding that python function in .conf file will not work. I suggest creating a class for it and inherit it globally in your .conf file:
meta-custom/classes/utils.bbclass
local.conf:
INHERIT += "utils"
IMAGE_EFI_BOOT_FILES += "${#get_files(d, 'DEPLOY_DIR_IMAGE')}"
Try this and let me know in the comments.
EDIT
I have just realized that bitbake already imports os within python expressions expansions, so you can do it in one line without any need for a separate python function:
PATH = "/path/to/directory/" or
PATH = "A variable containing the path"
IMAGE_EFI_BOOT_FILES += "${#' '.join('%s' % f for f in os.listdir('${PATH}') if os.path.isfile(f))}"
Note: I am looking for Yocto built-in which can achieve solution for above mentioned , would like to share other way to resolve the functionality for community's benefit.
Add following in bb file if you are using one or refer to talel-belhadjsalem answer to use utils.bbclass.
def add_directory_bootfs(d, dirname, bootfs_dir):
file_list = list()
boot_files_list = list()
deploy_dir_image = d.getVar('DEPLOY_DIR_IMAGE')
for (dirpath, dirnames, filenames) in os.walk(os.path.join(deploy_dir_image, dirname)):
file_list += [os.path.join(dirpath, file) for file in filenames]
for file in file_list:
file_rel_path = os.path.relpath(file, os.path.join(deploy_dir_image, dirname))
boot_file_entry = os.path.join(dirname, file_rel_path) + ';' + os.path.join(bootfs_dir, dirname, file_rel_path)
boot_files_list.append(boot_file_entry)
return ' '.join(boot_files_list)
IMAGE_EFI_BOOT_FILES += "${#add_directory_bootfs(d, 'relative_path_to_dir_in_deploy_dir_image', 'EFI/BOOT')}"

Deploying app, troubles to reffer to datasets. Streamlit

Hello i have one more problem with deploying my app by Streamlit. It works localy but when I want to upload it on git hub it doesnt work..Have no idea whats wrong. It seems that there is problem with path to the file:
"File "/app/streamlit/bobrza.py", line 14, in <module>
bobrza_locations = pd.read_csv(location)"
Here is link to my github repo. Will be very very grateful for help. Thank in advance.
https://github.com/Bordonous/streamlit
The problem is you are hard coding the path of the bobrza1.csv and route.csv to the path on your computer so when running the code on a different environment the path in not legal.
The solution is to make location independent from running environment, for this we will use the following:
__file__ variable - the path to the current python module (the .py file).
os.path.dirname() - a function to get directory name from path.
os.path.abspath() - a function to get a normalized absolutized version of path.
os.path.join() - a function to join one or more path components.
Now you need to change your location and location2 variables in the code to the following:
# get the absolute path to the directory contain the .csv file
dir_name = os.path.abspath(os.path.dirname(__file__))
# join the bobrza1.csv to directory to get file path
location = os.path.join(dir_name, 'bobrza1.csv')
# join the route.csv to directory to get file path
location2 = os.path.join(dir_name, 'route.csv')
Resulting in an independent path of the bobrza1.csv and route.csv.

What does this setup.m file do and what is it for?

What does this setup.m file do?
function setup
%SETUP Adds directories for Metrics to your MATLAB path
%
% Author: Ben Hamner (ben#benhamner.com)
myDir = fileparts(mfilename('fullpath'));
paths = genpath(myDir);
paths = strread(paths,'%s','delimiter',':');
pathsToAdd = [];
for i=1:length(paths)
thisPath = paths{i};
thisPathSplit = strread(thisPath,'%s','delimiter','/');
addThisPath = 1;
% Do not add any directories or files starting with a . or a ~
for j=1:length(thisPathSplit)
thisStr = thisPathSplit{j};
if (~isempty(thisStr)) && ((thisStr(1) == '.') || (thisStr(1) == '~'))
addThisPath = 0;
end
end
if addThisPath ==1
if ~isempty(pathsToAdd)
thisPath = [':' thisPath];
end
pathsToAdd = [pathsToAdd thisPath];
end
end
addpath(pathsToAdd);
savepath;
I understand from the description that it adds directories to Matlab's search path. But which one and why? My Matlab scripts are often scattered with addpath('data') lines. Does this mean I don't have to do that anymore? Your comments are much appreciated.
The file that you are linking is a setup file for the Metrics package - it adds paths to various folders so that you can use Metrics package without setting up the paths manually.
More specifically the setup.m function adds all paths at the level and below where it is located. If you copy this file to any directory and run it - it will add this directory and all its subdirs and subdirs of subdirs etc. (excluding folders starting with . or ~)
But I have a hunch that what you are looking for is this:
http://www.mathworks.com/help/matlab/ref/startup.html
http://www.mathworks.com/help/matlab/ref/matlabrc.html
This is adding the directory that setup.m is located in as well as every sub directory within that directory. fileparts(mfilename('fullpath')) gets the directory that the file exists in, and genpath(myDir); gets all of the subdirectories. Note that this leaves out any directory starting with a '.' or a '~'

opkg install error - wfopen no such file or directory

I have followed instructions to create an .ipk file, the Packages.gz and host them on a web server as a repo. I have set the opkg.conf in my other VM to point to this repo. The other VM is able to update and list the contents of repositories successfully.
But, when I try to install, I get this message. Can you please describe why I am getting this and what needs to be changed?
Collected errors:
* wfopen: /etc/repo/d1/something.py: No such file or directory
* wfopen: /etc/repo/d1/something-else.py: No such file or directory
While creating the .ipk, I had created a folder named data that had a file structure as /etc/repo/d1/ with the file something.py stored at d1 location. I zipped that folder to data.tar.gz. And, then together with control.tar.gz and 'debian-binary`, I created the .ipk.
I followed instructions from here:
http://bitsum.com/creating_ipk_packages.htm
http://www.jumpnowtek.com/yocto/Managing-a-private-opkg-repository.html
http://www.jumpnowtek.com/yocto/Using-your-build-workstation-as-a-remote-package-repository.html
It is very likely that the directory called /etc/repo/d1/ does not exist on the target system. If you create the folder manually, and try installing again, it probably will not fail. I'm not sure how to force opkg to create the empty directory by itself :/
Update:
You can solve this problem using a preinst script. Just create the missing directories on it, like this:
#!/bin/sh
mkdir -p /etc/repo/d1/
# always return 0 if success
exit 0

Copy all files with given extension to output directory using CMake

I've seen that I can use this command in order to copy a directory using cmake:
file(COPY "myDir" DESTINATION "myDestination")
(from this post)
My problem is that I don't want to copy all of myDir, but only the .h files that are in there. I've tried with
file(COPY "myDir/*.h" DESTINATION "myDestination")
but I obtain the following error:
CMake Error at CMakeLists.txt:23 (file):
file COPY cannot find
"/full/path/to/myDIR/*.h".
How can I filter the files that I want to copy to a destination folder?
I've found the solution by myself:
file(GLOB MY_PUBLIC_HEADERS
"myDir/*.h"
)
file(COPY ${MY_PUBLIC_HEADERS} DESTINATION myDestination)
this also works for me:
install(DIRECTORY "myDir/"
DESTINATION "myDestination"
FILES_MATCHING PATTERN "*.h" )
The alternative approach provided by jepessen does not take into account the fact that sometimes the number of files to be copied is too high. I encountered the issue when doing such thing (more than 110 files)
Due to a limitation on Windows on the number of characters (2047 or 8191) in a single command line, this approach may randomly fail depending on the number of headers that are in the folder. More info here https://support.microsoft.com/en-gb/help/830473/command-prompt-cmd-exe-command-line-string-limitation
Here is my solution:
file(GLOB MY_HEADERS myDir/*.h)
foreach(CurrentHeaderFile IN LISTS MY_HEADERS)
add_custom_command(
TARGET MyTarget PRE_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${CurrentHeaderFile} ${myDestination}
COMMENT "Copying header: ${CurrentHeaderFile}")
endforeach()
This works like a charm on MacOS. However, if you have another target that depends on MyTarget and needs to use these headers, you may have some compile errors due to not found includes on Windows. Therefore you may want to prefer the following option that defines an intermediate target.
function (CopyFile ORIGINAL_TARGET FILE_PATH COPY_OUTPUT_DIRECTORY)
# Copy to the disk at build time so that when the header file changes, it is detected by the build system.
set(input ${FILE_PATH})
get_filename_component(file_name ${FILE_PATH} NAME)
set(output ${COPY_OUTPUT_DIRECTORY}/${file_name})
set(copyTarget ${ORIGINAL_TARGET}-${file_name})
add_custom_target(${copyTarget} DEPENDS ${output})
add_dependencies(${ORIGINAL_TARGET} ${copyTarget})
add_custom_command(
DEPENDS ${input}
OUTPUT ${output}
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${input} ${output}
COMMENT "Copying file to ${output}."
)
endfunction ()
foreach(HeaderFile IN LISTS MY_HEADERS)
CopyFile(MyTarget ${HeaderFile} ${myDestination})
endforeach()
The downside indeed is that you end up with multiple target (one per copied file) but they should all end up together (alphabetically) since they start with the same prefix ORIGINAL_TARGET -> "MyTarget"