This post explains how to put file write lock on regular files.
But is it possible to put a lock on a yaml file, so only one script at a time can write to it?
I am using YAML::Syck, if that makes a difference.
You can lock a file using flock. You'll need to open the file first as flock operates on filehandles.
Alternatively you can use LockFile::Simple from CPAN.
As others have said, use flock. Also, see perlfaq5 under "How can I lock a file".
It's important to note that it's advisory locking anyway, so there is no need to actually lock the .yaml file. You can just lock a lock file (maybe yourfile.yaml.lock) and unlock it when done.
This also gives you the capability to say "this block of operations requires exclusive access", rather than just limit your exclusivity to a single file.
Related
I'm using ExifTool to add keywords to a file. I do this in batch, spawning ExifTool as a child process from my app.
exiftool -keywords+="courgettes" -keywords+="edible gardening" -keywords+="vegetables" -keywords+="Glasshouse" -keywords+="Greenhouse" -overwrite_original "/pictures/garden/P8151927.JPG"
However, it seems that with -overwrite_original the process isn't 100% transactional and atomic; occasionally it fails for some reason or other, but instead of leaving the original file untouched, it leaves it renamed to P8151927.JPG_exiftool_tmp. This means it looks like the original image has gone missing.
Is there any way to guarantee that if the ExifTool update fails, the original won't be touched? My understanding was that it would create a temp file with the new/changed keywords, and then carry out a transactional delete/rename which would either work or not, but it seems the delete can succeed, but the rename from the temp file can then fail.
Is there any way to work around this - any additional options I can use to make it 100% resilient? If not, my only alternative is to copy the file to a temp location, run exiftool on it, and then handle the rename/replace myself. Alternatively, I could check the return code, and if non-zero, I can rename the .JPG_exiftool_tmp file back to the original, I suppose.
Any better ideas?
I'm looking for solution to delete or (preferably directly) overwrite source of an exe file while it is running.
To explain further before you get it all wrong, I'll give an example:
I have an exe file on drive D:\ which I run (with previously posted question's answer, giving params to "Start in" folder on C:\Program Files\MyProgram\" so it finds its dlls.
Now after the file is running, I'd like to rewrite the file's byte stream (just like opening it in hex editor...), or at least delete it so I can copy over new exe file directly using same name.
So far the solution I'm using is that I trigger format D: command for the whole drive D:\ (which, in my case is ramdisk and thumb-drive, as I only have this exe on it, I copy it there as necessary), since that removes the file and let's me copy new file there.
Trying to use del myProgram.exe even with -force flag triggers error that access to the file is denied. Same goes if I try to overwrite the contents of the file.
Is there any alternative to do that without using the format command, as that requires to have partition drive only for the purpose?
Update: Note: MoveFileEx and similar techniques that require termination of the process or system restart/reboot are not qualified as a solution. This should be done while the process is running without further actions that can compromise the process's run state.
On a side note, when formatting the drive using the Powershell's format command, the file is gone, although if viewing the partition using Hex viewer tool, there is full binary (hex) content of the exe visible there and an be restored using just as simple as copy-paste technique. This is one of the points as to where overwriting the file contents would be preferable than deleting the file directly.
Please note: This is a knowledge and skills based question, and would therefore appreciate sparing the moral and security-concerning comments about such actions and behaviour.
For deleting/replacing/overwriting a file at least two conditions must be met:
The user performing the operation must have the required permissions to do so. This can be verified for instance via Get-Acl or icacls.
Windows must not have an open handle to the file. This can be checked for instance with tools like Process Explorer or handle. These tools can also be used to forcibly close open handles, although that's not recommended as it may cause data loss and/or damage to the files in question. I'm not sure, though, if it's actually possible to close handles to an executable without terminating the process.
Note that antivirus software is likely to interfere with this kind of operation.
The basic problem here is that Windows loads from the .EXE upon demand, it's not all read in at once.
If you destroy the original file what happens when it tries to load in a page that no longer exists?
If I had to write something of this sort I would copy the .exe to a temporary location (beware that running code from the temp directory may be prohibited), run the new .exe, terminate the old one and then do what I want to it.
I would like to open file in shared mode for editing (other processes must have access to that file for writing as well) under Windows OS. Is it possible in Perl?
For example, in WinApi there is a possibility to specify flag FILE_SHARE_WRITE in CreateFile() function.
Thank you!
Multiple processes writing to the same file is a bad situation. An example would be a database scenario where a table must be locked by a processes in order to changes to be written to the table reliably. You could write something that does nothing else but listen to the other processes to write to the file.
perlmonks suggest using flock
Here's the link. Hope this helps!
I have two scripts. Which opens a file by
IO::Handle open for appending (">>filename"). then I call $io->autoflush(1);
The question is will it work fine if I do it in two scripts at the same time? Or would some lines be lost while appending?
You'll want to use syswrite, like the Log4Perl docs suggest for this sort of situation. syswrite blocks other writers while writing, and shares the end of file marker with other processes when appending.
That will not work, as append mode is more like shortcut to "open the file, don't truncate it and after opening, seek to the end of file". So yes, you will lose lines.
I have a very common situation. I have a file, and I need to entirely overwrite that file with new contents. However, the original file is accessed on every page load (this is a web app), so it can't be missing for very long. A few ms is OK (though not ideal), a second is not OK.
Right now I do this by writing a temp file to the same directory and then renaming that temp file to the name of the new file. I'm just using the normal File::Temp and "rename" to do this, in Perl. I was wondering--is there some other recommended/better way to do this? Preferably one that doesn't require a CPAN module, as this is the only place in my system that I need to do this, and I don't want a whole new dependency just for this.
Oh, and all of this has to work on Windows, Linux, BSD, OS X, Solaris, and most other common platforms.
Here is the code in question, for those interested.
Your method seems just fine. It's quick, it's atomic, it uses core modules only, and File::Temp is a safe way to deal with temporary files. What more do you need?
I'd do it the same way you're doing it. At least on Unix-type OSes, a file rename is guaranteed to be atomic so you won't have any instants where either the original or the new files isn't there.
Rename is sufficient. However:
Is your temporary file at risk for race conditions? The filename should be randomized so nobody can cause problems by inserting their own file. Use an interface to mkstemp() if possible.