When writing numbers into .csv in Matlab, it seems to modify the data.
This is alarming to me, something I have never seen before.
>> csvwrite('FirstCol.csv',[201210;201211])
>> twodates =csvread('FirstCol.csv')
twodates =
201210
201210
Now compare with xlswrite
>> xlswrite('FirstCol.xls',[201210;201211])
>> aa=xlsread('FirstCol.xls')
aa =
201210
201211
Could the reason be some automatic formatting underneath date-similar numbers? (My explanation is just mystcism)
From the csvwrite documentation
csvwrite writes a maximum of five significant digits. If you need greater precision, use dlmwrite with a precision argument.
So doing:
csvwrite('FirstCol.csv',[201210;201211])
csvread('FirstCol.csv')
you do indeed lose the final digit.
But by using dlmwrite, you can do
dlmwrite('FirstCol.csv',[201210;201211],'precision',6)
dlmread('FirstCol3.csv')
which does indeed result in the correct output.
I am using a Mac and I can't use xlswrite, but obviously that is a reasonable method as well.
Related
I have a fixed width file format (original was input for a Fortran routine). Several lines of the file look like the below:
1078.0711005.481 932.978 861.159 788.103 716.076
How this actually should read:
1078.071 1005.481 932.978 861.159 788.103 716.076
I have tried various methods, textscan, fgetl, fscanf etc, however the problem I have is, as seen above, sometimes because of the fixed width of the original files there is no whitespace between some of the numbers. I cant seem to find a way to read them directly and I cant change the original format.
The best I have come up with so far is to use fgetl which reads the whole line in, then I reshape the result into an 8,6 array
A=fgetl
A=reshape(A,8,6)
which generates the following result
11
009877
703681
852186
......
049110
787507
118936
So now I have the above and thought I might be able to concatenate the rows of that array together to form each number, although that is seeming difficult as well having tried strcat, vertcat etc.
All of that seems a long way round so was hoping for some better suggestions.
Thanks.
If you can rely on three decimal numbers you can use a simple regular expression to generate the missing blanks:
s = '1078.0711005.481 932.978 861.159 788.103 716.076';
s = regexprep(s, '(\.\d\d\d)', '$1 ');
c = textscan(s, '%f');
Now c{1} contains your numbers. This will also work if s is in fact the whole file instead of one line.
You haven't mentioned which class of output you needed, but I guess you need to read doubles from the file to do some calculations. I assume you are able to read your file since you have results of reshape() function already. However, using reshape() function will not be efficient for your case since your variables are not fixed sized (i.e 1078.071 and 932.978).
If I did't misunderstand your problem:
Your data is squashed in some parts (i.e 1078.0711005.481 instead
of 1078.071 1005.481).
Fractional part of variables have 3 digits.
First of all we need to get rid of spaces from the string array:
A = A(~ismember(A,' '));
Then using the information that fractional parts are 3 digits:
iter = length(strfind(A, '.'));
for k=1:iter
[stat,ind] = ismember('.', A);
B(k)=str2double(A(1:ind+3));
A = A(ind+4:end);
end
B will be an array of doubles as a result.
I need to write 4D matrix (M-(16x,101x,101x,6x) to a file with high precision ('precision'-'%1.40f') in MATLAB.
I've found save('filename.mat', 'M' ); for multidimensional matrix but precision cannot be set (only -double). On the other hand I've found dlmwrite('filename.txt', M, 'delimiter', '\t', 'precision', '%1.40f'); to set the precision but only limited to 2-D array.
Can somebody suggest a way to tackle with my problem?
What is the point in storing 40 digits of fractional part if double precision number in MATLAB keeps only 16 of them?
Try this code:
t=pi
whos
fprintf('%1.40f\n',t)
The output is
Name Size Bytes Class Attributes
t 1x1 8 double
3.1415926535897931000000000000000000000000
The command save('filename.mat', 'M' ); will store numbers in their binary representation (8 bytes per double-precision number). This is unbeatable in terms of space-saving comparing with plain-text representation.
As for the 4D shape the way j_kubik suggested seems simple enough.
I always thought that save will store exactly the same numbers you already have, with the precision that is already used to store them in matlab - you are not losing anything. The only problems might be disk space consumption (too precise numbers?) and closed format of .mat files (cannot be read by outside programs). If I wanted to just store the data and read them with matlab later on, I would definitely go with save.
save can also print ascii data, but it is (as dlmwrite) limited to 2D arrays, so using dlmwrite will be better in your case.
Another solution:
tmpM = [size(M), 0, reshape(M, [], 1)];
dlmwrite('filename.txt', tmpM, 'delimiter', '\t', 'precision', '%1.40f');
reading will be a bit more difficult, but only a bit ;)
Then you can just write your own function to write stuff to a file using fopen & fprintf (just as dlmwrite does) - there you can control every aspect of your file format (including precision).
Something I would have done if I really cared about precision, file-size and execution time (this is probably not the way for you) would be to write a mex function that takes a matrix parameter and stores it in a binary file by just copying raw data buffer from matlab. It would also need some indication of array dimensions, and would probably be the quickest (not sure if save doesn't already do something similar).
I stumbled over the following strange thing while using matlab's symbolic toolbox
/ >> syms e
/>> y=11111111111111111^e
y =
11111111111111112^e
Seems like there is a limitation when working with large numbers. Can this be solved without changing to a completely different system, like sage?
I think the problem is that Matlab parses the number into a double before it converts it to
a symbolic expression. As a double has a 52-bit mantissa, you have approximately 16 significant digits but your number is longer.
As an alternative, you could try to create the number directly from a string:
y=sym('11111111111111111')^e
Unfortunately, I do not have Matlab available right now, so this answer is untested.
It's not a bug, it's a feature. And it's called "round-off error"
matlab uses a double format to store normal variable, just like the double in C programming language, and many other languages like C++.
Actually, the "bug" has nothing to do with the "^x",as we can see:
>> clear
>> syms y
>> format bank
>> y=11111111111111111
y =
11111111111111112.00
Even a simple assign triggers the "bug".
And we can see how a double variable is really stored in memory in VS, using debug mode:
As you can see in the screenshot, both a and b are stored as "2ea37c58cccccccc" in the memory, which means the computer can't tell one from the other.
And that's the reason of the "bug" you found.
To avoid this, you can use symbolic constant instead:
>> y=sym('11111111111111111')
y =
11111111111111111
In this way, the computer will store the "y" in memory in a different format, which will avoid round-off error and cost more memory.
I'm trying to understand how a system I'm working on creates some hash code starting from a numeric code. I'm collecting some of these pairs “small_number, big_number”, but I cannot figure out how the system encodes small_number to obtain big_number. Decoding is possible, because system can obtain small_number from big_number.
Numbers look like these:
197 >> 29337857947107767585
1078 >> 84080031635590040762
1083 >> 32373003898332638476
1409 >> 79402294967209014727
1498 >> 25254493910542918727
2945 >> 85687067382221703925
2946 >> 88767616208189692328
I have no clue at all. Can you point to some reading too?
Thank you
If the system can reverse the function, the function isn't a hashing function at all, but a cipher.
It looks like the output is always a 20 digit number - did you try testing it on non-numeric input, or strings larger than 20 digits?
In either case, its likely that the system uses a well known encryption algorithm (my guess is AES or DES), so without the key, it's infeasible for you to guess at the function.
Worse, if the system is not directly taking the input but adding some other information, you might have the right algorithm and key but still not realize it.
I want to save a matrix to a text file, so I can read it by another program. Right now I use:
save('output.txt', 'A','-ascii');
But this saves my file as
6.7206983e+000 2.5896414e-001
6.5710723e+000 4.9800797e-00
6.3466334e+000 6.9721116e-001
5.9975062e+000 1.3346614e+000
6.0224439e+000 1.8127490e+000
6.3466334e+000 2.0517928e+000
6.3965087e+000 1.9721116e+000
But I would like to have them saved without the "e-notation" and not with all the digits. Is there an easy way to do this?
Edit: Thank you! That works just fine. Sorry, but I think I messed up your edit by using the rollback.
I would use the fprintf function, which will allow you to define for yourself what format to output the data in. For example:
fid = fopen('output.txt', 'wt');
fprintf(fid,'%0.6f %0.6f\n', A.');
fclose(fid);
This will output the matrix A with 6 digits of precision after the decimal point. Note that you also have to use the functions fopen and fclose.
Ditto gnovice's solution, if you need performance & custom formatting.
dlmwrite gives you some control (global, not per-field basis) of formatting. But it suffers from lower performance. I ran a test a few years ago and dlmwrite was something like 5-10x slower than the fopen/fprintf/fclose solution. (edit: I'm referring to large matrices, like a 15x10000 matrix)