I'm working on a MATLAB program in which I read a text file using textscan and then store the data in corresponding arrays. It does that every run, and it takes about two minutes each time. I would like to know if there is a way to save the arrays after I loaded the data, and have the program remember them so I can read the data only once and save time while running. I looked into the load function, but I'm not sure if that's what I need.
Related
I'm trying to compare two files based off of variables capture from the first input file. If I have $inputvariable and I want to quickly check it against the contents of the second input file, how do I perform that? I've tried using Get-Content, however the second file has roughly 500,000 entries. The first file has around 1000, so even with breaking, it still checks every line from the first input file rather slowly. Essentially, is there a way to index the second input file as if it were entered into a database and quickly search?
I have developed a process to semi-automate the Simulink Load Flow Tool to allow the Load Flow to continuously be computed for different Load values. Each time the Load Flow is computed I require the data to be copied from the Powergui Load Flow Tool (Clipboard) to an array, at the moment I am doing this manually and cannot figure out a way to automate. To give reference to the data I require to copy from the clipboard I have attached the following image:Powergui Load Flow Tool, at this stage I only need to copy the data from the 2nd column.
Perhaps a more generic question is how to copy data from a Simulink Simulation GUI to an array in Matlab?
Any help would be greatly appreciated!
Thanks
You can use power_loadflow command to perform load flow and store results.
For example, if you run the simulation of Matlab in-built 5 bus system,
LF = power_loadflow('-v2','power_LFnetwork_5bus','solve')
LF is a structure where all results are stored, so you may save this variable in Matlab workspace. You can also store the results in Excel file as follows,
LF = power_loadflow('-v2','power_LFnetwork_5bus','solve','ExcelReport',fname)
Edit
To export to excel: load flow data is anyway stored in the variable LF. You can put the following command in your loop to copy the results to your excel file.
xlswrite('results_bus',cellstr(num2str([LF.bus.Vbus]')))
I used cellstrand num2str to save the complex results to excel. LF.bus.Vbus is the vector where bus voltage result is stored.
I would like to force overwrite logdata to a CSV file. It might well be that another user is reading that file at the moment.
What give me the possibility take no care about this kind of locking and write in the file? Of course, the appended data should displayed after writing when the user close and open file again.
Maybe a useful information:
Writing data would occur many times for a very short term.
I am doing calculations on an image and intend to save a variable to a .mat file. When accessing the variable, would it be faster for me to load from the file or store the variable to the GUI appdata?
Also, I noticed that when I originally didn't save the variable (89x512x512 double array), it ran much faster. Is saving to a file generally time expensive?
You already have that array in memory - so storing it via setappdata/getappdata is certainly the faster alternative and, given the moderate size doesn't have any real drawback.
So, no reason to store it to a file, imho.
Any yes, writing stuff to a file is comparingly slow and i.e. takes a certain minimum amount of time not matter how tiny your data is.
I have 2 independent Matlab workers, with FIRST getting/saving data and SECOND reading it (and doing some calculations etc).
FIRST saves data as .mat file on the hard-disk while SECOND reads it from there. It takes ~20 seconds to SAVE this data as .mat and 8millisec to DELETE it. Before SAVING data, FIRST deletes the old file and then saves a newer version.
How can the SECOND verify that data exists and is not corrupt? I can use exists but that doesn't tell me if the data is corrupt or not. For eg, if SECOND tries to read data exactly when FIRST is saving it, exists passes but LOAD gives you an error saying - Data Corrupt etc.
Thanks.
You can't, without some synchronization mechanism - by the time SECOND completes its check and starts to read the file, FIRST might have started writing it again. You need some sort of lock or mutex.
Two options for base Matlab.
If this is on a local filesystem, you could use a separate lock file sitting next to the data file to manage concurrent access to the data file. Use Java's NIO FileChannel and FileLock objects from within Matlab to lock the first byte of the lock file and use that as a semaphore to control access to the data file, so the reader waits until the writer is finished and vice versa. (If this is on a network filesystem, don't try this - file locking may seem to work but usually is not officially supported and in my experience is unreliable.)
Or you could just put a try/catch around your load() call and have it pause a few seconds and retry if you get a corrupt file error. The .mat file format is such that you won't get a partial read if the writer is still writing it; you'll get that corrupt file error. So you could use this as a lazy sort of collision detection and backoff. This is what I usually do.
To reduce the window of contention, consider having FIRST write to a temporary file in the same directory, and then use a rename to move it to its final destination. That way the file is only unavailable during a quick filesystem move operation, not the 20 seconds of data writing. If you have multiple writers, stick the PID and hostname in the temp file name to avoid collisions.
Sounds like a classic resource sharing problem between 2 threads (R-W)
In short, you should find a method of inter-workers safe communication. Check this out.
Also, try to type
showdemo('paralleldemo_communic_prof')
in Matlab