I want to iterate over all json files in a specific subdirectory.
#!/bin/sh
source_dir="nodes"
for file_path in $source_dir/*.json;
do
file_name=$(basename $file_path .${file_path##*.})
echo $file_name
done
My code is working as expected if there is at least one json file in the directory.
If there is no json file in the directory, the loop will still be executed. The file_name is then "*".
How do I have to change the for loop so that it is only executed if there is at least one json file in the directory?
you can wrap you loop in an if clause to check if the pattern matches anything.
Check this SO question on how to do this: Check if a file exists with wildcard in shell script
Related
I have multiple files in my folder , i want to pattern match if any file is present , if that file is present then store the variable with whole file path.
how to achieve this in pyspark
Since you want to store the whole path in a variable, you can achieve this with a combination of dbutils and Regular expression pattern matching.
We can use dbutils.fs.ls(path) to return the list of files present in a folder (storage account or DBFS). Assign its return value to a variable called files.
#my sample path- mounted storage account folder.
files = dbutils.fs.ls("/mnt/repro")
Loop through this list. Now using Python's re.match() you can check if the current item's file name matches your pattern. If it matches, append its path to your result variable (list).
from re import match
matched_files=[]
for file in files:
#print(file)
if(match("sample.*csv", file.name)): #"sample.*csv" is pattern to be matched
matched_files.append(file.path)
#print("Matched files: ",matched_files)
Sample output:
I have a perl script that reads a .txt and a .bam file, and creates an output called output.txt.
I have a lot of files that are all in different folders, but are only slightly different in the filename and directory path.
All of my txt files are in different subfolders called PointMutation, with the full path being
/Volumes/Lab/Data/Darwin/Patient/[Plate 1/P1H10]/PointMutation
The text(s) in the bracket is the part that changes, But the Patient subfolder contains all of my txt files.
My .bam file is located in a subfolder named DNA with a full path of
/Volumes/Lab/Data/Darwin/Patient/[Plate 1/P1H10]/SequencingData/DNA
Currently how I run this script is go on the terminal
cd /Volumes/Lab/Data/Darwin/Patient/[Plate 1/P1H10]/PointMutation
perl ~/Desktop/Scripts/Perl.pl "/Volumes/Lab/Data/Darwin/Patient/[Plate
1/P1H10]/PointMutation/txtfile.txt" "/Volumes/Lab/Data/Darwin/Patient/[Plate
1/P1H10]/SequencingData/DNA/bamfile.bam"
With only 1 or two files, that is fairly easy, but I would like to automate it once the files get much larger. Also once I run these once, I don't want to do it again, but I will get more information from the same patient, is there a way to block a folder from being read?
I would do something like:
for my $dir (glob "/Volumes/Lab/Data/Darwin/Patient/*/"){
# skip if not a directory
if (! -d $dir) {
next;
}
my $txt = "$dir/PointMutation/txtfile.txt";
my $bam = "$dir/SequencingData/DNA/bamfile.bam";
# ... you magical stuff here
}
This is assuming that all directories under /Volumes/Lab/Data/Darwin/Patient/ follow the convention.
That said, more long term/robust way of organizing analyses with lots of different files all over the place is either 1) organize all files necessary for each analysis under one directory, or 2) to create meta files (i'd use JSON/yaml) which contain the necessary file names.
I have a foreach loop that gets a list of objects in Folder4 after trimming the full path where the objects reside.
Here is sample code:
$row.Path = $path.InnerText.Replace("/Folder1/Folder2/folder3/folder4/","")
Sample Output:
usp_StoredProcedurename.prc,
fn_FunctionName.udf
File.sql
The last thing I need to do is to remove any extension, ie .prc, .pdf, .udf, .sql, etc
Here is the coplete for each:
You are probably looking for the static GetFileNameWithoutExtension method. To use it, you have to pass a single file or path to it:
[System.Io.Path]::GetFileNameWithoutExtension("usp_StoredProcedurename.prc")
Depending on the actual output of $row.Path you could split the path and join them back later if you want.
Alternative, you could use a regex to remove the file extensions for alle files within your string at once:
$row.Path -replace '\..*'
Be aware that regex will remove everything after a dot.
I am writing one perl script which is having some if else conditions. There is another .txt file in which, I want to place that conditional statements which exists in if else (in perl file) after a certain string. i did some search for this but most of the programs are based on merging two files. But in my case one file is perl file itself in which conditional statements exist and other is text file in which I want to append that conditional statements after a certain string. My files look like-
File 1
If (n==1 && m==1){
print (".include xyz.txt")}
else if(n==1 && m==0){
print (".include abc.txt")}.....
File 2
lines....
lines....
*matching string
Here I want to append #.include xyz.txt
lines....
lines....
Can both files run simultaneously and my conditional statements can be added in another file? Or first I have to take output from file 1 in other output file then to append it in second file. Please help me out. Thanks
Using perl from command line,
perl -MFcntl=:seek -pe 'seek(ARGV,0,SEEK_END) if /match/ and !$c++' fil1 fil2
It skips to fil2 file when it finds string match within fil1, and !$c++ ensures that skipping occurs only once.
I am trying to create a script that manipulates a configuration file . So I need to take back up of the existing configuration file , in case there is any problem during manipulation the contents of the back up file should replace the contents of the configuration file . also when rollback is given as argument to the script the contents of the backup file should replace configuration file .
Typically I'd create a file with a name based on the original file's name:
my $file = 'input.txt';
my $new_file = "$file.new";
Start reading lines from the input file, and manipulate them as necessary before writing them to the new file.
When you reach the end-of-file of the input file, close them both. Rename the input file to "$file.old", then rename the new file to the old name $file.
You want to keep the original file intact as long as possible so it remains available in case something fails during processing.
If you have to roll back, reverse the rename process if processing completed. If processing didn't complete just delete the new file.