The problem that I am attempting to solve is as follows:
I have a .log file that is updated every x seconds (an interval that I can change), with updated status information from a piece of test equipment. At each interval, another line is added to the .log file, with the updated information. My goal is to have the most recent status information (the last two lines of the .log file) easily viewable in Matlab.
Here is an example of what each update looks like, in case that is relevant (a single line of text):
What I have tried:
I used the readtable command (shown below) to view the information in the .log file, but that gives me the entire .log file every time the function is called, when I only want/need the last two lines.
data = readtable('FileName.log','FileType','text')
I know that this would be simpler if I was working with a .csv or .xlsx file, but the test equipment only updates the .log file, so I cannot just change the file type, as it would not get updates then.
Any advise would be appreciated.
If the .log file is in plain text format (as I assume based on your code snippet), you can get the last 2 lines of the file by using the following system command in MATLAB:
[status,output] = system(['tail -n 2 ', path]);
Please do keep in mind that this requires the tail command to be available, which is not available in windows by default - however you can get around this by installing a package containing the tail command, for example Cygwin.
Related
I want to test an application with a file containing 10000 lines of records (plus header and footer lines). I have a test file with 10 lines now, so I want to duplicate these line 1000 times. I don't want to create a C# code in my app to generate that file (is only for test), so I am looking for a different and simple way to do that.
What kind of tool can I use to do that? CMD? Visual Studio/VS Code extension? Any thought?
If your data is textual, load the 10 records from your test file into an editor. Select all, copy, insert at the end of file. Repeat until the file is of length 10000+
This procedure requires ceil(log_2(1000)) cycles, 10 in your case, in general ceil(log_2(<target_number_of_lines>/<base_number_of_lines>)).
Alternative (large files)
Modern editors should not have performance problems here. However, the principle can be applied using a cat cli command. Assuming that you copy the original file into a file named dup0.txt proceed as follows:
cat dup0.txt dup0.txt >dup1.txt
cat dup1.txt dup1.txt >dup0.txt
leaving you with the quadrupled number of lines in dup0.txt.
We have a producer process that write files into a specific folder, which run continuously, we have to read files one by one using talend, there is 2 issues:
The 1st: tWaitForFile read only files which exist before its starting, so files which have created after the component starting are not visible for it.
The 2nd: There is no way to know if the file is released by the producer process, it may be read while it is not completely written, the parameter _wait_release_ of tWaitForFile does not work on Linux system !
So how can make Talend read complete written files from a directory that have an increasing files number ?
I'm not sure what you mean by your first issue. tWaitForFile has options to trigger when files are created, modified or deleted in a folder.
As for the second issue, your best bet here is for the file producer to be creating an OK or control file which is a 0 byte touch when it has finished writing the file you want.
In this case you simply look for the appearance of the OK file and then pick up the relevant completed file. If you name the 2 files the same but with a different file extension (the OK file is typically called ".OK" then this should be easy enough to look for. So you would set your tWaitForFile to look for "*.OK" files and then connect this to an iterate to a tFileInputDelimited (in the case you want to pick up a delimited text file) and then declare the file name as ((String)globalMap.get("tWaitForFile_1_CREATED_FILE")).substring(0,((String)globalMap.get("tWaitForFile_1_CREATED_FILE")).length()-3) + ".txt"
I've included some screenshots to help you below:
I tried to read .jpg files from a folder in matlab using dir command. But I am not getting them from the first image stored in the folder. Instead it started from 10th image. I want to know how to read the files serially starting from the beginning.
I am almost certain that if you are using a simple enough command, it will give you all files that are there. However, this may not seem to be the case because of this little line in the description:
Results appear in the order returned by the operating system.
This may mean, that you will first see files like 1, 100, 1000,1999 and only later a file with numer 2. Of course you can sort the results after you have collected them, and then process them in your desired order.
For completenes, one would like to have something like:
dir *.jpg
or if you want to be sure to catch everything that even remotely resembles .jpg:
dir *.*j*p*g*
I use Doxygen in command line mode(via a batch file scheduled to run at certain time) for generating code documentation.
I supply it with a Doxyfile that has all the settings.
And it overwrites the previously generated documentation for same code.
My problem is, the input location name may change at some time.
So, in the log, doxygen shows that it cannot find ONE of the input locations.
What my aim is that the documentation should be generated/overwritten ONLY IF there is NO ERROR.
There are three possible solutions in my mind(in the order of decreasing convenience):
Interrupt docs generation in case of ANY error.
If its not possible to interrupt the docs generation, I can generate docs in some other place and then replace older ones using simple move command in batch file IF Doxygen has not thrown any errors. For this, I will somehow have to get hold of Doxygen errors in batch file runtime.
Save docs to different folder, save log to txt, scan txt in batch for errors(This, I can do but prefer avoiding)
So, is there any way I can get hold of this error while running Doxygen via batch file?
To generate an ouput file (.rwo format) containing the desired result, we have to run the executable of the report program which uses a parameter file (.rwd format). The output file is ReportBHP1.rwo, this file will contain the variable exported.
Now to implement this in Matlab, below is a small script giving a gist of what I am trying to achieve. It call CMG (software) for each realization and extracts the data.
for nr=1:NREAL
dlmwrite(‘PERM.INC’,PERM(:,nr),’delimiter’,’\n’); % Writes the permeability file for each realization
system('mx200810.exe -f ReservoirModel_CMGBulider.dat') % Calls CMG
system('report.exe /f ReportBHP1.rwd /o ReportBHP1.rwo') % Calls Parameter file and generates output file
[TIME(:,j),BHP1(:,j)]=textread('ReportBHP1.rwo','%f\t%f','headerlines',5); % reading the time and BHP from output file generated for well 1
end
All of my software's required .exe and .dll files are in the same folder where I have my ReservoirModel_CMGBulider.dat file. So I am running the .m file through the same folder where I have all these files.
I am able to generate the output (.rwo) file. However there is an error in the simulation report of the software which is displayed in Matlab's command window as follow:
"...STOP: Unable to open the following file as data file:
'ReservoirModel_CMGBuilder.dat'
Check path name for spaces, special character or a total length greater than 256 characters
Cannot find data file named 'ReservoirModel_CMGBuilder.dat'
Date and Time of End of Run: .....
ans = 0"
I think its reading the .dat file only one time and then overwriting the first realization result in rest of the realizations. As a consequence of this the result of all the realizations is coming out to be same as the first realization. I need help in correcting this error.
First, if my previous answer was OK, could you accept it? Thanks!
Second, have you tried using full path names for all the files as before?