Perl Copy to more than one Location - perl

Hello I'm attempting to copy a file to more then one location. I have written a script that will copy files using File::Find and File::Copy together to one location but I can not get the script to copy to a second or third location. I've tried to add a second variable target2 to the script so I can also copy the JPG files to this second target location. When I try to do this I get an error message. I want the copy to run on a loop that will copy the files X amount of seconds so I added the sleep function in as well. Can anyone explain to me why I can't copy to more than one location or help me find a way to do this? Thank you.
while (1)
{ sleep (10);
find(
sub {
if (-f &&/\.jpg$/i) {
print "$File::Find::name -> $target, $target2";
copy($File::Find::name, $target ,$target2)
or die(q{copy failed:} . $!);
}
},
#source
);
}
Error message: Bad Buffer size for copy: 0

copy function can't copy to 2 destinations. Call it twice:
copy($File::Find::name, $target);
copy($File::Find::name, $target2);
Look here to see the parameters explanation: http://perldoc.perl.org/File/Copy.html#SYNOPSIS
As the docs say:
An optional third parameter can be used to specify the buffer size
used for copying. This is the number of bytes from the first file,
that will be held in memory at any given time, before being written to
the second file.
So, when you specified the third parameter, Perl understood, that you want to set the buffer size manually. But you gave a string instead of a number, so it converted the string to a number: 0, and gave you an error:
Bad Buffer size for copy: 0

Related

perl: utf8 <something> does not map to Unicode while <something> doesn't seem to be present present

I'm using MARC::Lint to lint some MARC records, but every now and them I'm getting an error (on about 1% of the files):
utf8 "\xCA" does not map to Unicode at /usr/lib/x86_64-linux-gnu/perl/5.26/Encode.pm line 212.
The problem is that I've tried different methods but cannot find "\xCA" in the file...
My script is:
#!perl -w
use MARC::File::USMARC;
use MARC::Lint;
use utf8;
use open OUT => ':utf8';
my $lint = new MARC::Lint;
my $filename = shift;
my $file = MARC::File::USMARC->in( $filename );
while ( my $marc = $file->next() ) {
$lint->check_record( $marc );
# Print the errors that were found
print join( "\n", $lint->warnings ), "\n";
} # while
and the file can be downloaded here: http://eroux.fr/I14376.mrc
Is "\xCA" hidden somewhere? Or is this a bug in MARC::Lint?
The problem has nothing to do with MARC::Lint. Remove the lint check, and you'll still get the error.
The problem is a bad data file.
The file contains a "directory" of where the information is located in the file. The following is a human-readable rendition of the directory for the file you provided:
tagno|offset|len # Offsets are from the start of the data portion.
001|00000|0017 # Length include the single-byte field terminator.
006|00017|0019 # Offset and lengths are in bytes.
007|00036|0015
008|00051|0041
035|00092|0021
035|00113|0021
040|00134|0018
050|00152|0022
066|00174|0009
245|00183|0101
246|00284|0135
264|00419|0086
300|00505|0034
336|00539|0026
337|00565|0026
338|00591|0036
546|00627|0016
500|00643|0112
505|00755|9999 <--
506|29349|0051
520|29400|0087
533|29487|0115
542|29602|0070
588|29672|0070
653|29742|0013
710|29755|0038
720|29793|0130
776|29923|0066
856|29989|0061
880|30050|0181
880|30231|0262
Notice the length of the field with tag 505, 9999. This is the maximum value supported (because the length is stored as four decimal digits). The catch is that value of that field is far larger than 9,999 bytes; it's actually 28,594 bytes in size.
What happens is that the module extracts 9,999 bytes rather than 28,594. This happens to cut a UTF-8 sequence in half. (The specific sequence is CA BA, the encoding of ΚΌ.) Later, when the module attempts to decode that text, an error is thrown. (CA must be followed by another byte to be valid.)
Are these records you are generating? If so, you need to make sure that no field requires more than 9,999 bytes.
Still, the module should handle this better. It could read until it finds a end-of-field marker instead of using the length when it finds no end-of-field marker where it expects one and/or it could handle decoding errors in a non-fatal manner. It already has a mechanism to report these problems ($marc->warnings).
In fact, if it hadn't died (say if the cut happened to occur in between characters instead of in the middle of one), $marc->warnings would have returned the following message:
field does not end in end of field character in tag 505 in record 1

Multiple file upload with mojolicious fails on large number of files

I've hit a wall and my google skills have this time failed me. I'm in the process of learning mojolicious to create a useful front end for a series of Perl scripts that I frequently use. I've got a long way through it but I'm stumped at (multiple) file uploads when the total number of files reaches 950.
Previously, I encountered the problem where- in multiple file uploads- files would begin to be uploaded, but once the filesize reached 16 mb the upload stopped. I fixed this by setting $ENV{MOJO_MAX_MESSAGE_SIZE} = 50000000000. However, this problem is different. To illustrate, this is part of my script where I try to grab the uploaded files:
my $files = $self->req->every_upload('localfiles');
for my $file ( #{$files} ) {
my $fileName = $file->filename =~ s/[^\w\d\.]+/_/gr;
$file->move_to("temporary_uploads/$fileName");
$self->app->log->debug("$fileName uploaded\n");
push #fileNames, $fileName;
};
say "FILES: ".scalar(#fileNames);
I apologise that it may be ugly. If I attempt to upload 949 files, my array #fileNames is populated correctly, but if I try to upload 950 files, my array ends up empty, and it seems as though $files is empty also. If anyone has any ideas or pointers to guide me to the solution I would be extremely grateful!
If I attempt to upload 949 files, my array #fileNames is populated correctly, but if I try to upload 950 files, my array ends up empty, and it seems as though $files is empty also.
That means the process is running out of file descriptors. In particular, the default for the Linux kernel is 1024:
For example, the kernel default for maximum number of file descriptors (ulimit -n) was 1024/1024 (soft, hard), and has been raised to 1024/4096 in Linux 2.6.39.

Why I cannot open a txt file

I am really a new person in MATLAB and I need use it to finish my homework.
First, I try to open a txt file to get data.
So, I do like this:
folder='C:\Users\yshi20\Desktop\COSC6335\proj_1';
file='transactionDB.txt';
myData=fullfile(folder,file);
[Datafile, message] = fopen('transactionDB.txt', 'r');
But the datafile value always show -1 which means it failed to open.
So, I use this to check why I cannot open it:
if Datafile < 0
disp(message);
c = [];
else
Data = fread(Datafile, 5, 'uint8=>char')'
end
But the result says: No such file or directory.
But I checked many times, and I am sure the file name is correct and the location folder is correct, so, how to solve the problem?
You're using the wrong variable. You need to use myData in fopen.
[Datafile, message] = fopen(myData, 'r');
myData stores the complete path to your file whereas in your original code, you're using relative referencing which means that it's going to look for the file in the current working directory. A -1 code means that it can't find the file... and rightfully so given your error. It can't find the file in the current working directory. As such, make sure you change your fopen statement so that the correct path to your file is specified.

Matlab publish - Want to use a custom file name to publish several pdf files

I have several data log files (here: 34) for those I have to calculate some certain values. I wrote a seperate function to publish the results of the calculation in a pdf file. But I only can publish one file after another, so it takes a while to publish all 34 files.
Now I want to automize that with a loop - importing the data, calculate the values and publish the results for every log file in a new pdf file. I want 34 pdf files for every log file at the end.
My problem is, that I couldn't find a way to rename the pdf files during publishing. The pdf file is always named after the script which is calculating the values. Obviously the pdf is overwritten within a loop. So at the end everything is calculated, but I only have the pdf from the last calculated log file.
There was this hacky solution to change the Matlab publish script, but since I don't have admin rights I can't use that:
"This is really hacky, but I would modify publish to accept a new option prefix. Replace line 93
[scriptDir,prefix] = fileparts(fullPathToScript);
with
if ~isfield(options, 'prefix')
[scriptDir,prefix] = fileparts(fullPathToScript);
else
[scriptDir,~] = fileparts(fullPathToScript);
prefix = options.prefix; end
Now you can set options.prefix to whatever filename you want. If you want to be really hardcore, make the appropriate modifications to supplyDefaultOptions and checkOptionFields as well."
Any suggestions?
Thanks in advance,
Martin
Here's one idea using movefile to rename the resultant published PDF on each iteration:
for i = 1:34
file = publish(files(i)); % Replace with your own command(s)
[pathStr,fileName,ext] = fileparts(file);
newFile = [pathStr filesep() fileName '_' int2str(i) ext]; % Example: append _# to each
[success,msg,msgid] = movefile(file,newFile);
if ~success
error(msgid,msg);
end
end
Also used are fileparts and filesep. See this question for other ways to rename and move files.

How to "jump" to a line of a file, rather than read file line by line, using Perl

I am opening a file containing a single but very long column. I want to retrieve from it just a short segment, starting at a specified line and ending at another specified line. Currently, my script is reading the file line by line until the desired lines are found. I am using:
my ( $from, $to ) = ( some line number, some larger line number );
my $count = 1;
my #seq = ();
while ( <SEQUENCE> ) {
print "$_ for $count\n";
$count++;
while ( $count >= $from && $count <= $to ) {
push( #seq, $_ );
last;
}
}
print "seq is: #seq\n";
Input looks like:
A
G
T
C
A
G
T
C
.
.
.
How might I "jump" to where I want to be?
You'll need to use seek to move to the correct portion of the file. ref: http://perldoc.perl.org/functions/seek.html
This works on bytes, not on lines, so generally if you need to use line seeking its not an option. However, since you're working on a fixed length line (2 or 3 bytes depending on your platform's EOL encoding) you can multiply the line length by the line you want (0 indexed) and you'll be at the correct location for reading.
If you happen to know that all the lines are of exactly the same length (accounting for line ending characters, generally 1 byte on Unix/Linux and 2 on Windows), you can use seek to go directly to a specified point in the file
The seek function lets you specify a file position in bytes/characters, not in lines. In the general case, the only way to go to a specified line number is to read from the beginning and skip that many lines (minus one).
Unless you have an index mapping line numbers to byte offsets; then you can look up the specified line number in the index and use seek to jump to that location. To do this, you have to build the index separately (a process that will require reading through the entire file) and make sure the index is always up to date. If the file changes frequently, this is likely to be impractical.
I'm not aware of any existing tools for building and using such an index, but I wouldn't be surprised if they exist. But it should be easy enough to roll your own.
But unless scanning the file to find the line number you want is a significant performance bottleneck, I wouldn't bother with the extra complexity.