How to remove last 2 positions in split and get remaining first value.? - azure-data-factory

Let's say string is a variable file name like few examples below:
file1_name_cr_001.csv
file2_name1_name2.nn.123.456_updt_000.csv
filename_2012.444.1234_utc_del_004.csv
The length of last 8 string values will always remain fixed i.e. (_001.csv,_000.csv,_004.csv). We need to only extract values = cr, updt, del
How can we get the value as single value before _cr,_updt,_del.?
any suggetions.?
output should get like this:
file1_name/cr/001
file2_name1_name2.nn.123.456/updt/000
filename_2012.444.1234_utc/del/004

I have reproduced the above and got the below results.
First, I took a sample file name in set variable.
Then, I got the string from start to length-8.
#substring(variables('sample'),0,sub(length(variables('sample')),8))
For end folder:
#replace(split(substring(variables('sample'),sub(length(variables('sample')),8), 8),'.')[0],'_','')
For Start folder:
#substring(variables('before_8'), 0, lastIndexOf(variables('before_8'), '_'))
For middle folder:
#split(variables('before_8'), '_')[sub(length(split(variables('before_8'), '_')), 1)]
Result folder structure:
#concat(variables('start'),'/',variables('middle'),'/',variables('end'))
Result:
Give this variable in copy activity source folder path and it will generate the folder structure for you.
For multiple file names, first store all file names in an array then use a ForEach and inside ForEach do the same operations as above.

Related

saving multiple fields of structure as separate mat files and creating directory non-existing directories

There are two parts of my query
1) How to save different fields of structures as separate files(each file containing only named field of structure )?
2) Forcing save command to create directories in the save path when intermediate directories do not exist?
For first part:
data.a.name='a';
data.a.age=5;
data.b.name='b';
data.b.age=6;
data.c.name='c';
data.c.age=7;
fields=fieldnames(data);
for i=1:length(fields)
save(['E:\data\' fields{i} '.mat'],'-struct','data');
end
I want to save each field of struct data as a separate .mat file. So that after executing the loop, I should have 3 files inside E:\data viz. a.mat,b.mat and c.mat and a.mat contains only data of field 'a', b.mat contains only data of field 'b' and so on.
When I exeucte the above code, I get three files in my directory but each file contains identical content of all three variables a, b and c, instead of individual variables in each file.
Following command does not work:
for i=1:length(fields)
save(['E:\data\' fields{i} '.mat'],'-struct',['data.' fields{i} ]);
end
Error using save
The argument to -STRUCT must be the name of a scalar structure variable.
Is there some way to use save command to achieve my purpose without having to create temporary vaiables for saving each field?
For Second Part:
I have large number of files which need to stored in a directory structure. I want following to work.
test='abcdefgh';
save(['E:\data\' test(1:2) '\' test(3:4) '\' test(5:6) '\result.mat'])
But it showing following error
Error using save
Cannot create 'result.mat' because 'E:\data\ab\cd\ef' does not exist.
If any intermediate directory are not present, then they should be created by save command. I can get this part to work by checking if directory is present or not using exist command and then create directory using mkdir. I am wondering if there is some way to force save command to do the work using some argument I am not aware of.
Your field input argument to save is wrong. Per the documentation, the format is:
'-struct',structName,field1,...,fieldN
So the appropriate save syntax is:
data.a.name='a';
data.a.age=5;
data.b.name='b';
data.b.age=6;
data.c.name='c';
data.c.age=7;
fields = fieldnames(data);
for ii = 1:length(fields)
save(['E:\data\' fields{ii} '.mat'], '-struct', 'data', fields{ii});
end
And no, you cannot force save to generate the intermediate directories. Check for the existence of the save path first and create it if necessary.

MATLAB: Save figure with default name

I am running a matlab-script that produces a figure. To save this figure I use:
print(h_f,'-dpng','-r600','filename.png')
What this means is that if I don't change filename for each time I run the script, the figure filename.png will be overwritten.
Is there a way to save a figure to a default name, e.g. untitled.png, and then when the script is run twice it will make a new figure untitled(1).png instead of overwriting the original one?
You could create a new filename based on the number of existing files
defaultName = 'untitled';
fileName = sprintf('%s_%d.png', defaultName, ...
length(dir([defaultName '_*.png'])));
print(h_f,'-dpng','-r600', fileName)
Add a folder path to your dir search path if the files aren't located in your current working directory.
This will create a 0-index file name list
untitled_0.png
untitled_1.png
untitled_2.png
untitled_3.png
...
You could also use tempname to generate a long random name for each iteration. Unique for most cases, see section Limitations.
print(h_f,'-dpng','-r600', [tempname(pwd) '.png'])
The input argument (pwd in the example) is needed if you do not want to save the files in your TEMPDIR
You can try something like this:
for jj=1:N
name_image=sscanf('filename','%s') ;
ext=sscanf('.png','%s') ;
%%do your stuff
filename=strcat(name_image,num2str(jj),ext);
print(h_f,'-dpng','-r600',filename)
end
If you want to execute your script multiple time (because you don't want to use a "for") just declare a variable (for example jjthat will be incremented at the end of the script:
jj=jj+1;
Be careful to don't delete this variable and, when you start again your script, you will use the next value of jj to compose the name of the new image.
This is just an idea

Get a list of all subdirectories in Matlab

I'm trying get an absolute path of all subfolders in project_dirs.
project_dirs='D:\MPhil\Model_Building\Models\TGFB\Vilar2006\SBML_sh_ver\vilar2006_SBSH_test7\Python_project3_IQM_project';
all_project_dirs=dir(project_dirs)
for i=all_project_dirs,
full_dir=fullfile(project_dirs,i.name)
The above code gives a single string of all the subfolder directories concatenated together. How do I modify my code to get a cell array of these absolute paths?
There's a function for that: genpath(). It will give you all directories recursively in a string, split by :. Use strsplit() to parse the result.
You can do this:
all_project_dirs = {all_project_dirs([all_project_dirs.isdir]).name};
How it works:
This selects, among the elements of all_project_dirs, those that are directories;
From them it gets the name field;
The values of that field are contatenated into a cell array.
You may want to remove the first two directory names, which are always '.' and '..':
all_project_dirs = all_project_dirs(3:end);
To obtain full paths, you can use strcat:
all_project_dirs = strcat(project_dirs, filesep, all_project_dirs);
or, as suggested by Jørgen, use fullfile:
all_project_dirs = fullfile(project_dirs, all_project_dirs);

Store user input as wildcard

I am having some trouble with a data processing function in MATLAB. The function takes the name of the file to be processed as an input, finds the desired files, and reads in the data.
However, several of the desired files are variants, such as Data_00.dat, Data.dat, or Data_1_March.dat. Within my function, I would like to search for all files containing Data and condense them into one usable file for processing.
To solve this, I would like desiredfile to be converted into a wildcard.
Here is the statement I would like to use.
selectedfiles = dir *desiredfile*.dat % Search for file names containing desiredfile
This returns all files containing the variable name desiredfile, rather than the user input.
The only solution that I can think of is writing a separate function that manually condenses all the variants into one file before my function is run, but I am trying to keep the number of files used down and would like to avoid this.
You could concatenate strings for that. Considering desiredFile as a variable.
desiredFile = input('Files: ');
selectedfiles = dir(['*' desiredfile '*.dat']) % Search for file names containing desiredfile
Enclosing strings between square brackets [string1 string2 ... stringN]concatenates them. Matlab's dir function receives a string.
I believe you can achieve that using the dir command.
dataSets = dir('/path/to/dir/containing/Data*.dat');
dataSets = {dataSets.name};
Now simply loop over them, more information here.
To quote the matlab help:
dir lists the files and folders in the MATLAB® current folder. Results appear in the order returned by the operating system.
dir name lists the files and folders that match the string name. When name is a folder, dir lists the contents of the folder. Specify name using absolute or relative path names. You can use wildcards (*).

reading multiple image from multiple folders in matlab

I have a folder(Enroll) which contain 100 or more sub folders and each of them contain one image. I want to read this image and do some processing on this image. I have problem with how to read them from the different folders ?
note *
( the sub folders name is number like : 1, 2,.. " this number arrived from user " )
(the image name is number but different and not sequential like : 433535.bmp , 126554,bmp ,...)
foldername=1; // name of the sub folder arrived from user
d4= dir('C:\Users\Sarah\Desktop\Log\Log\Enroll\',foldername,'\*.bmp');// here problem when i put foldername variable
foldername2=d4(1).name;
w=imread(fullfile('C:\Users\Sarah\Desktop\Log\Log\Enroll\',foldername,'\*.bmp', foldername2));
help me please :(
foldername is not a string. therefore, you need to make it a string. I believe that what you want is
d4= dir(['C:\Users\Sarah\Desktop\Log\Log\Enroll\' num2str(foldername) '\*.bmp']);
Note:
1- you need to convert from number to string whatever number you have. If foldername is a string, then num2str is not needed.
2- You need to concatenate arrays, it doesn't happen automatically. therefore you need to ad the brackets [].