I was trying to experiment with subprocess.Popen() constructor by using file object for the stdout. When I run the below code, the file gets created (with the contents) but contents are not displayed on the console. I cannot understand what I am missing. Can anyone please point out my mistake(if any)?
file_name = "abc.txt"
fo = open(file_name, 'w')
cmd = 'dir'
child = subprocess.Popen(cmd, stdout=fo, stderr=fo, shell=True)
fo.close()
with open(file_name, 'r') as fo: print(fo.read())
You have to wait for subprocess finish before reading the content.
#!/usr/bin/env python
import subprocess
file_name = "abc.txt"
fo = open(file_name, 'w')
cmd = 'dir'
child = subprocess.Popen(cmd, stdout=fo, stderr=fo, shell=True)
child.wait()
fo.flush()
fo.close()
with open(file_name) as fo:
print(fo.read())
# print (child.poll())
Related
I'm not sure why subprocess can't find my file. I keep getting this error
WindowsError: [Error 2] The system cannot find the file specified
Here is my code:
import subprocess
from subprocess import PIPE
file= r'C:\Users\NIA\Desktop\2.py'
x = 'python' + file + str(7) + str(8)
process = subprocess.Popen([x], stdout=PIPE, stderr=PIPE)
stdout, stderr = process.communicate()
if stderr:
print stderr
else:
print stdout
I am writing a function of a class to convert a sam file to bam file, then sort it and index it in python (I know it is easy to fulfill in bash, but it does not hurt to learn more).
A sam file has already been generated in the defined directory (cwd) by another function of the class already.
Here is the code:
def sam2bam(self):
"""
return a sorted and index bam file in the working directory
"""
if str(self.fq)[7:9] == '24':
# create bam
comnd = "samtools view -bS " + str(self.fq)[0:10] + ".sam > " + str(self.fq)[0:10] + ".bam"
print("Converting sam to bam ... (executed command: {})\n".format(comnd))
comnd_input = shlex.split(comnd)
with open((str(self.fq)[0:10] + '.bam'), 'w') as f:
Popen(comnd_input, stdout = f, stderr = PIPE, cwd = 'G24/' + str(self.fq))
# sort bam
comnd2 = "samtools sort " + str(self.fq)[0:10] + ".bam -o " + str(self.fq)[0:10] + ".sorted.bam"
print('Sorting bam ... (executed command: {}\n)'.format(comnd2))
comnd_input2 = shlex.split(comnd2)
with open((str(self.fq)[0:10] + 'sorted.bam'), 'w') as f_sort:
Popen(comnd_input2, stdout = f_sort, stderr = PIPE, cwd = 'G24/' + str(self.fq))
# index bam
comnd3 = "samtools index " + str(self.fq)[0:10] + ".sorted.bam"
print('Indexing bam ... (executed command: {}\n)'.format(comnd3))
comnd_input3 = shlex.split(comnd3)
with open((str(self.fq)[0:10] + 'sorted.bam'), 'w') as f_index:
Popen(comnd_input3, stdout = f_index, stderr = PIPE, cwd = 'G24/' + str(self.fq))
But nothing came out
I tried:
Popen(comnd_input, stdout = PIPE, stderr = PIPE, cwd = 'G24/' + str(self.fq))
yet nothing either. Tried save it as a variable and do .communicate(), nothing.
So I don't know what wrong I did?
thanks,
Xp
refer to here.
It is because of the '>' redirecting. I modified the code to:
Popen(comnd, stdout = PIPE, stderr = PIPE, shell = True, cwd = 'G14/' + str(self.fq))
And it worked.
I have multiple text files named split01.txt, split02.txt etc... with the data in the format below: (This is what I have)
/tmp/audio_files/n000001.wav;
/tmp/audio_files/n000002.wav;
/tmp/audio_files/n000003.wav;
/tmp/audio_files/p000004.wav;
/tmp/audio_files/p000005.wav;
I would like to create another file with the data taken from the split01.txt, split02.txt etc... file in the format below: (this is the format I would like to see)
[playlist]
NumberOfEntries=5
File000001=n000001.wav
Title000001=n000001.wav
File000002=n000002.wav
Title000002=n000002.wav
File000003=n000003.wav
Title000003=n000003.wav
File000004=p000004.wav
Title000004=p000004.wav
File000005=p000005.wav
Title000005=p000005.wav
Version=2
Can this be done in one instance? The reason I ask is that I'm going to be running/calling the command (awk,grep,sed,etc...) from inside of octave/matlab after the initial process has completed creating the audio files.
example: of what I mean in one instance below: (matlab/octave code)
system(strcat({'split --lines=3600 -d '},dirpathwaveformstmp,fileallplaylistStr,{' '},dirpathwaveformstmp,'allsplit'))
This splits a single file into multiple files with the names allsplit01 allsplit02 etc.. and each file only has a max of 3600 lines.
For those who asked this is creating playlist files for audio files I create with octave/matlab.
Any suggestions?
Here's one way you could do it with awk:
parse.awk
BEGIN {
print "[playlist]"
print "NumberOfEntries=" len "\n"
i = 1
}
{
gsub(".*/|;", "")
printf "File%06d=%s\n" , i, $0
printf "Title%06d=%s\n\n", i, $0
i++
}
END {
print "Version 2"
}
Run it like this:
awk -v len=$(wc -l < infile) -f parse.awk infile
Output:
[playlist]
NumberOfEntries=5
File000001=n000001.wav
Title000001=n000001.wav
File000002=n000002.wav
Title000002=n000002.wav
File000003=n000003.wav
Title000003=n000003.wav
File000004=p000004.wav
Title000004=p000004.wav
File000005=p000005.wav
Title000005=p000005.wav
Version 2
If you're writing your program in Octave, why don't you do it in Octave as well? The language is not limited to numerical analysis. What you're trying to do can be done quite easily with Octave functions.
filepath = "path for input file"
playlistpath = "path for output file"
## read file and prepare cell array for printing
files = strsplit (fileread (filepath)', "\n");
if (isempty (files{end}))
files(end) = [];
endif
[~, names, exts] = cellfun (#fileparts, files, "UniformOutput", false);
files = strcat (names, exts);
files(2,:) = files(1,:);
files(4,:) = files(1,:);
files(1,:) = num2cell (1:columns(files))(:);
files(3,:) = num2cell (1:columns(files))(:);
## write playlist
[fid, msg] = fopen (playlistpath, "w");
if (fid < 0)
error ("Unable to fopen %s for writing: %s", playlistpath, msg);
endif
fprintf (fid, "[playlist]\n");
fprintf (fid, "NumberOfEntries=%i\n", columns (files));
fprintf (fid, "\n");
fprintf (fid, "File%06d=%s\nTitle%06d=%s\n\n", files{:});
fprintf (fid, "Version 2");
if (fclose (fid))
error ("Unable to fclose file %s with FID %i", playlistpath, fid);
endif
I'm writing a Perl script to run an external program on every file in a directory. This program converts files from one format to another. Here's the deal...
When I run the program from the command line, everything works as it should:
computer.name % /path/program /inpath/input.in /outpath/output.out
converting: /inpath/input.in to /outpath/output.out
computer.name %
Here's the code I wrote to convert all files in a directory (listed in "file_list.txt"):
#!/usr/bin/perl -w
use warnings;
use diagnostics;
use FileHandle;
use File::Copy;
# Set simulation parameters and directories
#test_dates = ("20110414");
$listfile = "file_list.txt";
$execname = "/path/program";
foreach $date (#test_dates)
{
# Set/make directories
$obs_file_dir = "inpath";
$pred_file_dir = "outpath";
mkdir "$pred_file_dir", 0755 unless -d "$pred_file_dir";
# Read input file names to array
$obs_file_list = $obs_file_dir . $listfile;
open(DIR, $obs_file_list) or die "Could not open file!";
#obs_files = <DIR>;
close(DIR);
# Convert and save files
foreach $file (#obs_files)
{
$file =~ s/(\*)//g;
$infile = $obs_file_dir . $file;
$outfile = $pred_file_dir . $file;
$outfile =~ s/in/out/g;
print $infile . "\n";
#arg_list = ($execname, $infile, $outfile);
system(#arg_list);
}
}
The output shows me the following error for every file in the list:
computer.name % perl_script_name.pl
/inpath/input.in
converting: /inpath/input.in to /outpath/output.out
unable to find /inpath/input.in
stat status=-1
error while processing the product
I verified every file is in the proper place and have no idea why I am getting this error. Why can't the files be found? When I manually pass the arguments using the command line, no problem. When I pass the arguments through a variable via a system call, they can't be found even though the path and file names are correct.
Your advice is greatly appreciated!
Your list of files (#obs_files) comes from reading in a file via #obs_files = <DIR>;
When you do that, each element of array will be a line from a file (e.g. directory listing), with the line being terminated by a newline character.
Before using it, you need to remove the newline character via chomp($file).
Please note that s/(\*)//g; does NOT remove that trailing newline!
I want to run and executable ./runnable on argument input.afa. The standard input to this executable is through a file finalfile. I was earlier trying to do the same using a bash script, but that does not seem to work out. So I was wondering whether Perl provides such functionality. I know I can run the executable with its argument using backticks or system() call. Any suggestions on how to give standard input through file.
_ UPDATE _
As I said I had written a bash script for the same. I not sure how to go about doing it in Perl. The bash script I wrote was:
#!/bin/bash
OUTFILE=outfile
(
while read line
do
./runnable input.afa
echo $line
done<finalfile
) >$OUTFILE
The data in standard input file is as follows, where each line correspond to one time input. So if there are 10 lines then the executable should run 10 times.
__DATA__
2,9,2,9,10,0,38
2,9,2,10,11,0,0
2,9,2,11,12,0,0
2,9,2,12,13,0,0
2,9,2,13,0,1,4
2,9,2,13,3,2,2
2,9,2,12,14,1,2
If I understood your question correctly, then you are perhaps looking for something like this:
# The command to run.
my $command = "./runnable input.afa";
# $command will be run for each line in $command_stdin
my $command_stdin = "finalfile";
# Open the file pointed to by $command_stdin
open my $inputfh, '<', $command_stdin or die "$command_input: $!";
# For each line
while (my $input = <$inputfh>) {
chomp($input); # optional, removes line separator
# Run the command that is pointed to by $command,
# and open $write_stdin as the write end of the command's
# stdin.
open my $write_stdin, '|-', $command or die "$command: $!";
# Write the arguments to the command's stdin.
print $write_stdin $input;
}
More info about opening commands in the documentation.
Perl code:
$stdout_result = `exescript argument1 argument2 < stdinfile`;
Where stdinfile holds the data you want to be passed through stdin.
edit
The clever method would be to open stdinfile, tie it via select to stdin, and then execute repeatedly. The easy method would be to put the data you want to pass through in a temp file.
Example:
open $fh, "<", "datafile" or die($!);
#data = <$fh>; #sucks all the lines in datafile into the array #data
close $fh;
foreach $datum (#data) #foreach singluar datum in the array
{
#create a temp file
open $fh, ">", "tempfile" or die($!);
print $fh $datum;
close $fh;
$result = `exe arg1 arg2 arg3 < tempfile`; #run the command. Presumably you'd want to store it somewhere as well...
#store $result
}
unlink("tempfile"); #remove the tempfile