I have a two column data with mmyyyy and SPI (Standardized Precipitation Index) variables. The first two samples have no data (NAN). The file is:
011982 NAN
021982 NAN
031982 -1.348
.
.
.
122013 1.098
I load the time and SPI data into MATLAB, then I would like to plot it but it is not working.
I would like to plot line graph but I really have no idea how to plot time in x-axis and I would like my x-axis to show only the year.
Using the new datetime data type in MATLAB (added in R2014b), this should be easy.
Here is an example. First we load the data into a MATLAB table:
% import data from file
fid = fopen('file.dat', 'rt');
C = textscan(fid, '%{MMyyyy}D %f');
fclose(fid);
% create table
t = table(C{:}, 'VariableNames',{'date','SPI'});
You get something like this:
>> t(1:10,:)
ans =
date SPI
______ ________
011982 NaN
021982 NaN
031982 2.022
041982 1.5689
051982 0.75813
061982 -0.74338
071982 -1.7323
081982 -2.4466
091982 -0.86604
101982 0.085698
Next to plot the data with date and time, it's as easy as calling plot:
plot(t.date, t.SPI)
xlabel('Date'), ylabel('Standardized Precipitation Index')
By default, plot chooses tick mark locations based on the range of data. When you zoom in and out of a plot, the tick labels automatically adjust to the new axis limits.
But if you want, you can also specify a custom format for the datetime tick labels. Note that when you do this, the plot always formats the tick labels according to the specified value, they won't adjust on zoom:
plot(t.date, t.SPI, 'DatetimeTickFormat','yyyy')
I'm adding another answer that works in older MATLAB versions without table or datetime data types.
Like before, we first import the data from file, but this time we read the dates as strings then convert them to serial date numbers using datenum function (defined as the number of days since "January 0, 0000"):
% import data from file
fid = fopen('file.dat', 'rt');
C = textscan(fid, '%s %f');
fclose(fid);
% create matrix
t = [datenum(C{1},'mmyyyy') C{2}];
The data looks like this:
>> format long
>> t(1:10,:)
ans =
1.0e+05 *
7.239120000000000 NaN
7.239430000000000 NaN
7.239710000000000 0.000005606888474
7.240020000000000 0.000009156147863
7.240320000000000 0.000004504804864
7.240630000000000 0.000008359005819
7.240930000000000 0.000007436313932
7.241240000000000 0.000002800134237
7.241550000000000 0.000005261613664
7.241850000000000 0.000001809901372
Next we plot the data like before, but instead we use the datetick function to format the x-axis as dates ('yyyy' for years):
plot(t(:,1), t(:,2))
datetick('x', 'yyyy')
xlabel('Date'), ylabel('Standardized Precipitation Index')
Unfortunately the tick labels will not automatically update when you zoom in and out... The good news, there are solutions on the File Exchange that solve this issue, for example datetickzoom and datetick2.
Related
I need to read data from a file and plot a graph with its data. The problem is:
(1) I can't change the format of data in the file
(2) The format contains information and characters that I don't know how to deal with.
Here is a part of the data file, it's in a txt format:
Estation;Date;Time;Temp1;Temp2;Pressure;
83743;01/01/2016;0000;31.9;25.3;1005.1;
83743;01/01/2016;1200;31.3;26.7;1005.7;
83743;01/01/2016;1800;33.1;25.4;1004.3;
83743;02/01/2016;0000;26.1;24.2;1008.6;
What I'm trying to do is to plot the Date and Time against Temp1 and Temp2, not worrying about Pressure. The first column can be neglected as well. How can I extract the Date, Time and Temps into and matrix so I can plot them? All I did so far was this:
fileID = fopen('teste.txt','r');
[A] = fscanf(fileID, ['%d' ';']);
fclose(fileID);
disp(A);
Which just reads the first value, 83743.
To build on m7913d's answer:
fileID = fopen('MyFile.txt','r');
A = fscanf(fileID, ['%s' ';']); % read the header line
B = fscanf(fileID, '%d;%d/%d/%d;%d;%f;%f;%f;', [8,inf]); % read all the data into B (the date is parsed into three columns)
fclose(fileID);
B = B.'; % transpose B
% C is just for verification, can be omitted
C = datetime([B(:,4:-1:2) B(:,5)/100zeros(numel(B(:,1)),2)],'InputFormat','yyyy-MM-dd HH:mm:ss');
D = datenum(C); % Get the date in a MATLAB usable format
Titles = strsplit(A,';'); % Get column names
figure;
hold on % hold the figure for multiple plots
plot(D,B(:,6),'r')
plot(D,B(:,7),'b')
datetick('x') % Set a date tick as axis
legend(Titles{4},Titles{5}); % uses titles for legend
note the parsing of the date into C: first is the date as given by you in dd-MM-yyyy format, which I flip to the official standard of yyyy-MM-dd, then your hour, which needs to be divided by 100, then a 0 for both minutes and seconds. You might need to rip those apart when you don't have exactly hourly data. Finally transform to a regular datenum, which MATLAB can use for processing.
Which results in:
You might want to play around with the datetick format, as it's got lots of options which might appeal to you.
fileID = fopen('input.txt','r');
[A] = fscanf(fileID, ['%s' ';']); % read the header line
[B] = fscanf(fileID, '%d;%d/%d/%d;%d;%f;%f;%f;', [8,inf]); % read all the data into B (the date is parsed into three columns)
fclose(fileID);
disp(B');
Note that %d reads an integer (not a double) and %f reads a floating point number.
See fscanf for more details.
I have some data from a tide gauge I have it at the moment as a .csv file. I would like to load this data into MATLAB as I need to edit it. There are 2 columns that I am interested in, the first is a date & time column in the format [dd/mm/yyyy hh:mm] and the second is a column for tidal elevation. The tidal elevation data are primarily numbers to 3 d.p. however some of the data have letters which are used as flags. I can't use csvread because the date & time format so I changing it to a number in excel (I would prefer to keep it in date time format) but I then couldn't use csvread because it didn't like the letter flags. I tried using readtable which worked (for dates as numbers) however my tidal elevation data is stuck as in a cell as cell2mat doesn't work because I read the elevation data in in string format because of the letters.
I would basically like to know is there an easier way to get the data loaded into MATLAB as what I am doing is a real mess at the moment.
Sample Data:
28/01/1994 22:15 3.312
28/01/1994 22:30 3.057
28/01/1994 22:45 2.793
28/01/1994 23:00 2.541T
28/01/1994 23:15 2.303T
28/01/1994 23:30 2.083
28/01/1994 23:45 1.882
What I've tried:
filename = 'C:\User\Documents\Tide_Data\Fish_all.csv';
fileID = fopen(filename);
data = textread(filename,'%{dd/MM/yyyy HH:mm}D %s');
Badly formed format string, so I changed the date to a number in excel.
data = csvread(filename);
Can't read the letter T so outputs an error.
I had more code which got further before I reached a dead end but I can't reproduce it
I would suggest to read the file using textscan and then convert the date & time string to datenum and convert the last column to double if the letter exist or not.
C = textscan(fileID,'%s %s %s');
% allocate
result = zeros(7,2);
for ii = 1:7
% current date string
dateX = [C{1,1}{ii,1} C{1,2}{ii,1}];
% current number
numStr = C{1,3}{ii,1};
if sum(numStr == 'T') > 0
% remove char
numStr = regexprep(numStr,'T','');
end
% collect
result(ii,:) = [datenum(dateX, 'dd/mm/yyyyHH:MM'), str2double(numStr)];
end
You can then convert date numbers to any string format using datestr
I would like to plot a time series in Matlab of a data set I have in Excel.
The Excel file looks as follows:
Data: | Value:
2005-04-01 | 5.20
2006-12-02 | 3.12
...
How could I load this into Matlab and plot the time series of it?
There's 2 easy way of plotting dates, but I'll give you the script to read from the xls file first.
% Read from Excel
[N,T] = xlsread( filepath );
You then need to extract/convert the dates as follows. Dates are the 1st column of the text.
d = datetime( T(:,1) );
Then you can plot the variables as follows
figure;
plot( d, N(:,1) );
A sample plot is here
Alternatively, you can use datenum instead of datetime if you want the date as an integer instead of a datetime object using the following line.
d = datenum( T(:,1) );
use xlsread to load in data as string, now converting the date and values can be done in a number of ways, the most strait forward way for your dates is perhaps using the str2num which allows you to read in the N th character as a number, for example:
string = "2005-04-01|5.20"
year = str2num(string(1:4))
month = str2num(string(6:7)) //%where the number is the N th character in string, and has to be numerical or the str2num will return error
//%Note that this method does not read non-numerical strings, in your case "-" "|" will not be read.
I believe this is one of the faster methods, since it mainly involve a type conversion on a standard format of data. Any other suggestions?
PS: in your time series plot, I would suggest converting your dates into a long number, i.e. 20050401, 20061202, etc; which would be most efficient. (you can do this by years*1000+months*100+days.
I am trying to export data from Matlab in format that would be understood by another application... For that I need to change the NaN, Inf and -Inf strings (that Matlab prints by default for such values) to //m, //inf+ and //Inf-.
In general I DO KNOW how to accomplish this. I am asking how (and whether it is possible) to exploit one particular thing in Matlab. The actual question is located in the last paragraph.
There are two approaches that I have attempted (code bellow).
Use sprintf() on data and strrep() the output. This is done in line-by-line fashion in order to save memory. This solution takes almost 10 times more time than simple fprintf(). The advantage is that it has low memory overhead.
Same as option 1., but the translation is done on the whole data at once. This solution is way faster, but vulnerable to out of memory exception. My problem with this approach is that I do not want to unnecessarily duplicate the data.
Code:
rows = 50000
cols = 40
data = rand(rows, cols); % generate random matrix
data([1 3 8]) = NaN; % insert some NaN values
data([5 6 14]) = Inf; % insert some Inf values
data([4 2 12]) = -Inf; % insert some -Inf values
fid = fopen('data.txt', 'w'); %output file
%% 0) Write data using default fprintf
format = repmat('%g ', 1, cols);
tic
fprintf(fid, [format '\n'], data');
toc
%% 1) Using strrep, writing line by line
fprintf(fid, '\n');
tic
for i = 1:rows
fprintf(fid, '%s\n', strrep(strrep(strrep(sprintf(format, data(i, :)), 'NaN', '//m'), '-Inf', '//inf-'), 'Inf', '//inf+'));
end
toc
%% 2) Using strrep, writing all at once
fprintf(fid, '\n');
format = [format '\n'];
tic
fprintf(fid, '%s\n', strrep(strrep(strrep(sprintf(format, data'), 'NaN', '//m'), '-Inf', '//inf-'), 'Inf', '//inf+'));
toc
Output:
Elapsed time is 1.651089 seconds. % Regular fprintf()
Elapsed time is 11.529552 seconds. % Option 1
Elapsed time is 2.305582 seconds. % Option 2
Now to the question...
I am not satisfied with the memory overhead and time lost using my solutions in comparison with simple fprintf().
My rationale is that the 'NaN', 'Inf' and '-Inf' strings are simple data saved in some variable inside the *printf() or *2str() implementation. Is there any way to change their value at runtime?
For example in C# I would change the System.Globalization.CultureInfo.NumberFormat.NaNSymbol, etc. as explaind here.
In the limited case mentioned in comments that a number of (unknown, changing per data set) columns may be entirely NaN (or Inf, etc), but that there are not unwanted NaN values otherwise, another possibility is to check the first row of data, assemble a format string which writes the \\m strings directly, and use that while telling fprintf to ignore the columns that contain NaN or other unwanted values.
y = ~isnan(data(1,:)); % find all non-NaN
format = sprintf('%d ',y); % print a 1/0 string
format = strrep(format,'1','%g');
format = strrep(format,'0','//m');
fid = fopen('data.txt', 'w');
fprintf(fid, [format '\n'], data(:,y)'); %pass only the non-NaN data
fclose(fid);
By my check with two columns of NaN this fprintf is pretty much the same as your "regular" fprintf and quicker than the loop - not taking into account the initialisation step of producing format. It would be fiddlier to set it up to automatically produce the format string if you also have to take +/- Inf into account, but certainly possible. There is probably a cleaner way of producing format as well.
How it works:
You can pass in a subset of your data, and you can also insert any text you like into a format string, so if every row has the same desired "text" in the same spot (in this case NaN columns and our desired replacement for "NaN"), we can put the text we want in that spot and then just not pass those parts of the data to fprintf in the first place. A simpler example for trying out on the command line:
x = magic(5);
x(:,3)=NaN
sprintf('%d %d ihatethrees %d %d \n',x(:,[1,2,4,5])');
I have some problems in reading data from text file and plotting it.The text file contains
Date; Time; Temp °C
05.08.2011; 11:00:47;23.75
05.08.2011; 11:01:21;23.69
05.08.2011; 11:01:56;25.69
05.08.2011; 11:02:16;23.63
05.08.2011; 11:02:50;23.63
05.08.2011; 11:03:24;23.63
I want to plot the Temperature values with elapsed minutes.
firstly i used
[a,b]=textread('file1.txt','%s %s','headerlines',1)
to read the data in a string and I get
'17:09:16;21.75'
After that I used
a= strread('17:08:00;21.81','%s','delimiter', ';')
to get
'17:08:00'
'21.81'
But after this I am not been able to figure out how to move forward to deal with both these strings, especially time.
I want to plot temperature with time on xaxis..but not this time the elapsed time..in this case 2 mins 37 secs.
Help needed
Thanks Aabaz.thats really a big favor..I dun why I could figure it out ..I spent so much time on it
I have some 50 files comprising this data..If i want to loop it under this code , how can accomplish it, cz i have names of the file under ROM IDs..alike 1AHJDDHUD1224.txt.
How wud pass the file names in the loop.Do I have to change the names of the files then pass them under loop.I dun knw
I have one more question that if I wanted the values to be plotted after every 60 seconds..alike as soon the data is available in text files graph is plotted , and then graph is updated after every 60 sec until some more values are available in text file
Consider the following code. It will cycle through all .DAT files in a specific directory, read the data files, then plots with a the x-axis formatted as date/time:
%# get a list of files
BASE_DIR = 'C:\Users\Amro\Desktop';
files = dir( fullfile(BASE_DIR,'*.dat') );
files = {files.name};
%# read all files first
dt = cell(numel(files),1);
temps = cell(numel(files),1);
for i=1:numel(files)
%# read data file
fname = fullfile(BASE_DIR,files{i});
fid = fopen(fname);
C = textscan(fid, '%s %s %f', 'delimiter',';', 'HeaderLines',1);
fclose(fid);
%# datetime and temperature
dt{i} = datenum( strcat(C{1},{' '},C{2}) );
temps{i} = C{3};
end
Now we can plot the data (say we had 16 files, thus layout subplots as 4-by-4)
figure
for i=1:16
subplot(4,4,i), plot(dt{i}, temps{i}, '.-')
xlabel('DateTime'), ylabel('Temp °C')
datetick('x','HH:MM:SS')
end
You can merge the time strings with sprintf and translate them to seconds with datenum. Then the rest will be easy. Here is how it could work:
fid=fopen('data','r');
header=fgetl(fid);
data=textscan(fid,'%s','delimiter',';');
fclose(fid);
data=data{:};
day=data(1:3:end);
hour=data(2:3:end);
temp=str2double(data(3:3:end));
time=cellfun(#(x) sprintf('%s %s',day{strcmpi(hour,x)},x),hour,'uniformoutput',0);
% timev=datevec(time,'mm.dd.yyyy HH:MM:SS');
timen=datenum(time,'mm.dd.yyyy HH:MM:SS');
seconds=timen*86400;
plot(seconds-seconds(1),temp);
You may want to check the date format as I did not know which format you were using, so I guessed it was mm.dd.yyyy HH:MM:SS (see Matlab date specifiers)