I'm trying overwrite a csv file that already exists in Matlab on Windows. The problem is sometimes I have the file open in Excel. When this is the case, the write operation fails.
Is there anyway to overwrite the file in Matlab? I do NOT want to use ActiveX to connect to the Excel session and edit the file that way.
Seeing this post about FileShare settings made me think it might be possible, but none of the fopen parameters seem to be able to do the trick. Excel may be locking the file with exclusive write access, in which case there is no getting around it. Does anyone know how to check this?
Example of problem:
% make csv file
x = magic(4);
csvwrite('foo.csv', x);
% open foo.csv in EXCEL
% try writing again
csvwrite('foo.csv', x); % cannot write a new file
[fid, msg] = fopen('foo.csv' ,'w'); % cannot open handle for writing
As a side note, I used to be able to overwrite a file open in Excel when the file existed on a Linux box, and I had the file open over the network on a Windows box.
Those FileShare settings are for files opened from the Win32 API's CreateFile family of functions, not the C style fopen family, which Matlab exposes. The fopen options won't get you there. See http://support.microsoft.com/kb/99173 for a quick rundown of the differences. (If you really wanted to use CreateFile or other Win32 I/O, e.g. to check to see if Excel has the file locked or lock it yourself, you could call it from Matlab through .NET using System.IO.File.)
Regardless, by default, Excel opens the file in write mode, and gets an exclusive (write) lock. So you couldn't open it for writing anyway, those file sharing settings would only let you open the file for reading. If you want to be able to overwrite an Excel file while it's open in Excel, you need to have Excel open it in read-only mode like katrasnikj suggests. This causes Excel to read it in to memory once and then release the filehandle.
Try turning on the Read-Only file attribute on these files after you write them by shelling out to the attrib command. This will cause Excel to open them read-only by default. Then clear the read-only attribute right before opening it from Matlab for rewriting.
if exist(file, 'file')
[status,result] = system(sprintf('attrib -R "%s"', file));
end
[fid,msg] = fopen(file, 'w');
% ... write the file and close it ...
[status,result] = system(sprintf('attrib +R "%s"', file));
There's still a race condition between reader and writer, but if you write the file fast, it'll be a short window. Better would be to write out the csv to a temp file in the same directory, and then just turn off the read-only attribute long enough to swap the new file in to place with a movefile or java.io.File.renameTo. Still a race, but probably good enough to use in practice.
You could also change the permissions on the directory the files are in so your writer process has Modify permissions but the users running Excel only have Read access. Then the Excels will always open read-only and you don't have to fiddle with file attribs.
Related
I'm looking for solution to delete or (preferably directly) overwrite source of an exe file while it is running.
To explain further before you get it all wrong, I'll give an example:
I have an exe file on drive D:\ which I run (with previously posted question's answer, giving params to "Start in" folder on C:\Program Files\MyProgram\" so it finds its dlls.
Now after the file is running, I'd like to rewrite the file's byte stream (just like opening it in hex editor...), or at least delete it so I can copy over new exe file directly using same name.
So far the solution I'm using is that I trigger format D: command for the whole drive D:\ (which, in my case is ramdisk and thumb-drive, as I only have this exe on it, I copy it there as necessary), since that removes the file and let's me copy new file there.
Trying to use del myProgram.exe even with -force flag triggers error that access to the file is denied. Same goes if I try to overwrite the contents of the file.
Is there any alternative to do that without using the format command, as that requires to have partition drive only for the purpose?
Update: Note: MoveFileEx and similar techniques that require termination of the process or system restart/reboot are not qualified as a solution. This should be done while the process is running without further actions that can compromise the process's run state.
On a side note, when formatting the drive using the Powershell's format command, the file is gone, although if viewing the partition using Hex viewer tool, there is full binary (hex) content of the exe visible there and an be restored using just as simple as copy-paste technique. This is one of the points as to where overwriting the file contents would be preferable than deleting the file directly.
Please note: This is a knowledge and skills based question, and would therefore appreciate sparing the moral and security-concerning comments about such actions and behaviour.
For deleting/replacing/overwriting a file at least two conditions must be met:
The user performing the operation must have the required permissions to do so. This can be verified for instance via Get-Acl or icacls.
Windows must not have an open handle to the file. This can be checked for instance with tools like Process Explorer or handle. These tools can also be used to forcibly close open handles, although that's not recommended as it may cause data loss and/or damage to the files in question. I'm not sure, though, if it's actually possible to close handles to an executable without terminating the process.
Note that antivirus software is likely to interfere with this kind of operation.
The basic problem here is that Windows loads from the .EXE upon demand, it's not all read in at once.
If you destroy the original file what happens when it tries to load in a page that no longer exists?
If I had to write something of this sort I would copy the .exe to a temporary location (beware that running code from the temp directory may be prohibited), run the new .exe, terminate the old one and then do what I want to it.
Alright, here's what I'm dealing with (you can skip to TLDR if all you need to see is what I want to run):
I'm having an issue with file formatting for a nasty conglomeration of several ancient programs I've strung together. I have some data in .CSV format, and I need to put it into .SPC format. I've tried a set of proprietary MATLAB programs called 'GS tools' for fast and easy conversion, but fast and easy doesn't look like its gonna happen here since there are discrepancies in how .spc files are organized now and how they were organized back when my ancient programs were written.
If I could find the source code for the old programs I could probably alter the GS tools code to write my .spc files appropriately, but all I can find are broken links circa 2002 and earlier. Seeing as I don't know what my programs are looking for, I have no choice but to try resaving my data with other programs until one of them produces something workable.
I found my Cinderella program: if I open the data I have in a program called Spekwin and save the file with a .spc extension... viola! Everything else runs on those files. The problem is that I have hundreds of these files and I'd like to automate the conversion process.
I either need to extract the writing rubric Spekwin uses for .spc files (I believe that info is stored in a dll file within the program, but I'm not sure if that actually makes sense) and use it as a rule to write a file from my input data, or I need a piece of code that will open a file with Spekwin, tell Spekwin to save that file under the .spc extension, and terminate Spekwin.
TLDR: Need a command that tells the computer to open a file with a certain program, save that file under a different extension through that program (essentially open*.csv>save as>*.spc), then terminate the program.
OR--I need a way to tell MATLAB to write a file according to rules specified by a .dll, but I'm not sure I fully understand what that entails.
Of course I'm open to suggestions on other ways to handle this.
I have a MATLAB script that I could have sworn worked fine the last time I used it (a year ago). Now, I get this error:
Invalid file identifier. Use fopen to generate a valid file identifier.
If I understand correctly, it is failing to find or open(?) a file specified elsewhere in the script. Is this right? If so, what could cause it?
fid (file identifier) is the output of fopen. It's an integer, but not related to the file permanently. You need to use fopen to get the fid. It seems to me that you are using incorrect fid (file identifier) in some file-related I/O command, such as fread, fscanf or fclose. Unsuccessful fopen gives fid of -1. For any valid normal file successful fopen will give fid that is 3 or greater integer.
However, without any code it's impossible to say where or what the bug or error is. You could use MATLAB debugger to single-step the code from relevant fopen (set breakpoint there and run your program) until the relevant fclose and see if fid (or whatever variable name you use for file identifier) or any data structure for your file identifiers (if have more than one file identifier in your code) changes in any point between relevant fopen and fclose.
I solved this problem for my self by adding permission option to fopen.
As you see in http://www.mathworks.se/help/matlab/ref/fopen.html , fopen syntax is:
fileID = fopen(filename,permission)
Possible permissions, for example are:
'r' (default) | 'w' | 'a' | 'r+' | 'w+' | 'a+' | ...
'r' – Open file for reading.
'w' – Open or create new file for writing. Discard existing contents, if any.
'a' –
Open or create new file for writing. Append data to the end of the file.
'r+' – Open file for reading and writing.
'w+' – Open or create new file for reading and writing. Discard existing contents, if any.
'a+' – Open or create new file for reading and writing. Append data to the end of the file.
...
If I use fopen without permission option, or if I use 'r' (default) option, fopen will return -1, which is error. I success with this:
fid=fopen('tmp.txt', 'w');
fid=fopen('tmp.txt', 'a');
I had this problem. It turned out that the file I was trying to write was too large (I didn't have enough free space to accommodate it). However, the program didn't fail until the call to fclose. Very confusing!
Try freeing up some space, or writing a very small file, to test this diagnosis.
I encountered the same problem when trying to open ASF toolbox demos. Running Matlab as an administrator(right-click to open) seemed to solve this issue for me.
fopen can fail because MATLAB doesn't have the permissions to read/write the file you've specified.
Try opening a file in a location where you/MATLAB have all the rights (depending on your OS).
I have used fopen with permission and the same error came out. However, I started MATLAB as admin and that took care of the problem.
I had the file opened in excel and as a result fopen returned a -1.
Took me forever to find such a trivial problem.
It also happens when trying to create a file in a non-existent directory. Try mkdir('folderName') within MATLAB or just create the directory beforehand.
The path with a forward slash at the beginning can cause the same error.
filename = '/data/myfile.txt';
throws this error, while
filename = 'data/myfile.txt';
does not produce an error.
For my situation, I have checked everything, but missed an easy step.
Please select "browse your folder" and browse for your current document location before you run your 'fopen' code.
It also occurs when a script is trying to read beyond the end of the file.
I have a script in MATLAB that outputs various files, including NIFTI (MRI images) files and text log files. Sometimes the output files that are created have their file permissions set to no read or write for anyone, including the file owner, and this appears to occur randomly.
This normally isn't a problem unless it occurs with the logs, as it leads MATLAB to endlessly recurse as it tries to write the error to the logs. Unfortunately, I haven't been able to find anyone who has experienced similar behavior from any program, including MATLAB.
The script, input files, and output files are all located on a NAS drive connected to a server running Ubuntu 14.04, so I'm wondering if this is a problem with the script (probably not because it happens intermittently regardless of input), the matlab environment, or the NAS drive.
I'm not sure why yoru fileattributes/permissions are changing, but I know the solution. You want file attributes if you know chmod from unix this should be familiar, if not you will still be fine
something like this will make your files writeable for all user groups on a unix (ubuntu) system
fileattrib('/home/work/results/my_file.log','+w','a')
edit
since these files are non-existant this may work better. Simply use fopen with the 'w+' flag. Which means write mode for appending. The + will also create the file if it doesn't exist(and you have permission to create files in the specified directory)
fid = fopen('my_new_log.txt','w+');
fprintf(fid,'some strings for my file');
fclose(fid);
it is also important you make sure to close the file after you are done.
I have a MATLAB script that I could have sworn worked fine the last time I used it (a year ago). Now, I get this error:
Invalid file identifier. Use fopen to generate a valid file identifier.
If I understand correctly, it is failing to find or open(?) a file specified elsewhere in the script. Is this right? If so, what could cause it?
fid (file identifier) is the output of fopen. It's an integer, but not related to the file permanently. You need to use fopen to get the fid. It seems to me that you are using incorrect fid (file identifier) in some file-related I/O command, such as fread, fscanf or fclose. Unsuccessful fopen gives fid of -1. For any valid normal file successful fopen will give fid that is 3 or greater integer.
However, without any code it's impossible to say where or what the bug or error is. You could use MATLAB debugger to single-step the code from relevant fopen (set breakpoint there and run your program) until the relevant fclose and see if fid (or whatever variable name you use for file identifier) or any data structure for your file identifiers (if have more than one file identifier in your code) changes in any point between relevant fopen and fclose.
I solved this problem for my self by adding permission option to fopen.
As you see in http://www.mathworks.se/help/matlab/ref/fopen.html , fopen syntax is:
fileID = fopen(filename,permission)
Possible permissions, for example are:
'r' (default) | 'w' | 'a' | 'r+' | 'w+' | 'a+' | ...
'r' – Open file for reading.
'w' – Open or create new file for writing. Discard existing contents, if any.
'a' –
Open or create new file for writing. Append data to the end of the file.
'r+' – Open file for reading and writing.
'w+' – Open or create new file for reading and writing. Discard existing contents, if any.
'a+' – Open or create new file for reading and writing. Append data to the end of the file.
...
If I use fopen without permission option, or if I use 'r' (default) option, fopen will return -1, which is error. I success with this:
fid=fopen('tmp.txt', 'w');
fid=fopen('tmp.txt', 'a');
I had this problem. It turned out that the file I was trying to write was too large (I didn't have enough free space to accommodate it). However, the program didn't fail until the call to fclose. Very confusing!
Try freeing up some space, or writing a very small file, to test this diagnosis.
I encountered the same problem when trying to open ASF toolbox demos. Running Matlab as an administrator(right-click to open) seemed to solve this issue for me.
fopen can fail because MATLAB doesn't have the permissions to read/write the file you've specified.
Try opening a file in a location where you/MATLAB have all the rights (depending on your OS).
I have used fopen with permission and the same error came out. However, I started MATLAB as admin and that took care of the problem.
I had the file opened in excel and as a result fopen returned a -1.
Took me forever to find such a trivial problem.
It also happens when trying to create a file in a non-existent directory. Try mkdir('folderName') within MATLAB or just create the directory beforehand.
The path with a forward slash at the beginning can cause the same error.
filename = '/data/myfile.txt';
throws this error, while
filename = 'data/myfile.txt';
does not produce an error.
For my situation, I have checked everything, but missed an easy step.
Please select "browse your folder" and browse for your current document location before you run your 'fopen' code.
It also occurs when a script is trying to read beyond the end of the file.