I try to run a demo for speaker verification using MSR Identity toolkit. However it left error after training UBM step. The error is as follow. It looks like fopen return -1 and cause error to fread. I can't understand why it can't read the filenames. I can't attach the code since it involves many functions. I just hope someone that familiar with this toolkit can help me.
Error using fread
Invalid file identifier. Use fopen to generate a valid file identifier.
Error in htkread (line 7)
nframes = fread(fid, 1, 'int32'); % number of frames
Error in mapAdapt>load_data (line 107)
data{ix} = htkread(filenames{ix});
Error in mapAdapt (line 52)
dataList = load_data(dataList);
Error in demo_gmm_ubm (line 69)
gmm_models{spk} = mapAdapt(spk_files, ubm, map_tau, config);
Part of the code where lead to the error as follow:
function data = load_data(datalist)
% load all data into memory
if ~iscellstr(datalist)
fid = fopen(datalist, 'rt');
filenames = textscan(fid, '%s');
fclose(fid);
filenames = filenames{1};
else
filenames = datalist;
end
nfiles = size(filenames, 1);
data = cell(nfiles, 1);
for ix = 1 : nfiles,
data{ix} = htkread(filenames{ix});
end
function [data, frate, feakind] = htkread(filename)
% read features with HTK format (uncompressed)
fid = fopen(filename, 'r','b'); %ERROR HERE
nframes = fread(fid, 1, 'int32'); % number of frames
frate = fread(fid, 1, 'int32'); % frame rate in nano-seconds unit
nbytes = fread(fid, 1, 'short'); % number of bytes per feature value
feakind = fread(fid, 1, 'short'); % 9 is USER
ndim = nbytes / 4; % feature dimension (4 bytes per value)
data = fread(fid, [ndim, nframes], 'float');
fclose(fid);
datalist contains:
'features\fadg0_sa2.htk'
'features\fadg0_si1279.htk'
'features\fadg0_si1909.htk'
'features\fadg0_si649.htk'
'features\fadg0_sx109.htk'
'features\fadg0_sx19.htk'
'features\fadg0_sx199.htk'
'features\fadg0_sx289.htk'
'features\fadg0_sx379.htk'
I use MSR Identity Toolkit without problem, fortunately.
What I have in htkread.m is as follows:
...
fid = fopen(filename, 'rb', 'ieee-be');
nframes = fread(fid, 1, 'int32'); % number of frames
...
Maybe the error you encountered comes from:
big-endian/little-endian issue
the *.htk feature you have is missing
the *.htk is with other format
regards,
tommy
Is your fid returning a negative value? If yes, try specifying entire path where dataList= 'ubm.lst'; is in the code
Related
Right now I can only write a fixed length data to the text file ,but I need to change to a variable length data.
My code:
fileID = fopen(logfilePathLocal,'at+');
formatSpec = '%s,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d\n';
fprintf(fileID,formatSpec,data{1,:});
fclose(fileID);
You can use %f instead of %d, and specify width and precision, e.g. width 3 and precision 5 would be %3.5f.
For more information on specifier syntax, see the fprintf reference in the matlab documentation.
EDIT: If what you mean instead is that you don't know "how many %d my format-string will end up having", you can construct the format-string manually first (e.g. by concatenaton, or by using sprintf), and then use the resulting string in fprintf, e.g.
N = 5; % number of data, subject to change on each run.
% construct format-string
s = '%s';
for i = 1:N-1; s = [s, '%d, ']; end
s = [s, '%d\n']; % final data point
% use it with fprintf to print your data
fprintf (s, 'The data is: ', data{1,:});
Following the answer from here, you can use the combination of eval and an array of characters to produce the desired format specification alongside the data entry you're looking to print into a file.
clear
clc
fid = fopen('file.txt','at+');
% Generate some random data in a cell
data = {rand(5,1) rand(3,1)};
% Calculate required dimensions for fprintf
% and store the results in file.txt
n = size(data,2);
m = [];
for i=1:n
m = [m size(data{i},1) - 1];
end
% Finally, we will have n = 2, m = [4 2]
% Produce the final dynamic code snippet
fmt = [];
for i=1:n
for j=1:m(i)
fmt = [fmt '%d,'];
end
fmt = [fmt '%d\n'];
eval(['fprintf(fid,fmt,data{' num2str(i) '});']);
fmt = [];
end
An example output which is borrowed from file.txt is:
9.063082e-01,8.796537e-01,8.177606e-01,2.607280e-01,5.943563e-01
2.251259e-02,4.252593e-01,3.127189e-01
I have to process thousands of binary files (each of 16MB) by reading pairs of them and creating a bit-level data structure (usually a 1x134217728 array) in order to process them on bit level.
Currently I am doing this the following way:
conv = #(c) uint8(bitget(c,1:32));
measurement = NaN(1,(sizeOfMeasurements*8)) %(1,134217728)
fid = fopen(fileName, 'rb');
byteContent = fread(fid,'uint32');
fclose(fid);
bitRepresentation1 = arrayfun(conv, byteContent, 'UniformOutput', false);
measurement=[bitRepresentation1{:}];
Thus, I replaced fopen with memmapfile as below:
m = memmapfile(fileName,'Format',{'uint32', [4194304 1], 'byteContent'});
byteContent = m.data.byteContent;
byteContent = double(byteContent);
I printed timing information (using tic/toc) for the individual instructions and it turns out that the bottleneck is:
bitRepresentation1 = arrayfun(conv, byteContent, 'UniformOutput', false); % see first line of code for conv
Are there more efficient ways of transforming byteContent into an array that stores a bit per index (i.e. that is a bit representation of byteContent)?
Let looping over all numbers be handled by bitget. You loop over the bits:
fid = fopen(fileName, 'rb');
bitContent = fread(fid,'*ubit64');
fclose(fid);
conv = #(ii) uint8(bitget(bitContent, ii));
bitRepresentation = arrayfun(conv, 1:64, 'UniformOutput', false);
measurement = [bitRepresentation{:}]';
measurement = measurement(:).';
EDIT you can also try a direct loop:
fid = fopen(fileName, 'rb');
bitContent = fread(fid,'*ubit64');
fclose(fid);
sz = 64 * size(bitContent,1);
measurement3 = zeros(1, sz, 'uint8');
weave = 1:64:sz;
for ii = 1:64
measurement3(weave + ii - 1) = uint8(bitget(bitContent, ii)); end
but on my system, that is (surprisingly) slower than arrayfun...but, my MATLAB version is from the stone age, your mileage may be different. Give it a try
Several things that seem to provide further improvement on Rody's suggestion:
(minor:) Using a local function instead of a function handle for conv.
(major:) Converting the result of conv to logical using ~~ instead of uint8.
(major:) cell2mat instead of [bitRepresentation{:}]'.
The result:
function q40863898(filename)
fid = fopen(filename, 'rb');
bitContent = fread(fid,'*ubit64');
fclose(fid);
bitRepresentation = arrayfun(#convert, 1:64, 'UniformOutput', false);
measurement = reshape(cell2mat(bitRepresentation).',[],1).';
function out = convert(ii)
out = ~~(bitget(bitContent, ii, 'uint64'));
end
end
Benchmark result (on MATLAB R2016b, Win10 x64, 14MB file):
Rody's vectorized method: 0.87783
Rody's loop method: 2.37
Dev-iL's method: 0.68387
Benchmark code:
function q40863898(filename)
%% Common code:
fid = fopen(filename, 'rb');
bitContent = fread(fid,'*ubit64');
fclose(fid);
%% Verification:
ref = Rody1();
res = {Rody2(), uint8(Devil1())};
assert(isequal(ref,res{1}));
assert(isequal(ref,res{2}));
%% Benchmark:
disp(['Rody''s vectorized method: ' num2str(timeit(#Rody1))]);
disp(['Rody''s loop method: ' num2str(timeit(#Rody2))]);
disp(['Dev-iL''s method: ' num2str(timeit(#Devil1))]);
%% Functions:
function measurement = Rody1()
conv = #(ii) uint8(bitget(bitContent, ii));
bitRepresentation = arrayfun(conv, 1:64, 'UniformOutput', false);
measurement = [bitRepresentation{:}]';
measurement = measurement(:).';
end
function measurement = Rody2()
sz = 64 * size(bitContent,1);
measurement = zeros(1, sz, 'uint8');
weave = 1:64:sz;
for ii = 1:64
measurement(weave + ii - 1) = uint8(bitget(bitContent, ii));
end
end
function measurement = Devil1()
bitRepresentation = arrayfun(#convert, 1:64, 'UniformOutput', false);
measurement = reshape(cell2mat(bitRepresentation).',[],1).';
function out = convert(ii)
out = ~~(bitget(bitContent, ii, 'uint64'));
end
end
end
Hello when l run the code above l got this error . l don't where is the error ,my codes (.m) and training examples are in the same working directory.
Error using fread
Invalid file identifier. Use fopen to generate a valid file identifier.
Error in MNISTParser (line 3)
magic_number = fread(fp,1,'uint32',0,'b');
Error in MNIST_gen (line 2)
train_labels = MNISTParser('./MNIST/train-labels-idx1-ubyte');
MNIST_gen.m
clear
**train_labels = MNISTParser('./MNIST/train-labels-idx1-ubyte');**
train_images = MNISTParser('./MNIST/train-images-idx3-ubyte');
test_labels = MNISTParser('./MNIST/t10k-labels-idx1-ubyte');
test_images = MNISTParser('./MNIST/t10k-images-idx3-ubyte');
test_item_number = length(test_labels);
train_item_number = length(train_labels);
image_scale = size(test_images,2);
test_images_unfold = reshape(test_images,test_item_number,image_scale^2)';
test_labels_unfold = full(ind2vec(test_labels'+1));
train_images_unfold = reshape(train_images,train_item_number,image_scale^2)';
train_labels_unfold = full(ind2vec(train_labels'+1));
save MNIST.mat;
colormap(gray);
axis off
axis image
%show an image of a digit in test samples
for i=1:1
j=randi(length(test_labels),1);
image(reshape(255-test_images(j,:,:),28,28));
title(sprintf('%d',test_labels(j)));
pause(1);
image(reshape(test_images_unfold(:,j),28,28));
title(vec2ind(test_labels_unfold(:,j))-1);
pause(1);
end
%show an image of a digit in train samples
for i=1:1
j=randi(length(train_labels),1);
image(reshape(255-train_images(j,:,:),28,28));
title(sprintf('%d',train_labels(j)));
pause(1);
image(reshape(train_images_unfold(:,j),28,28));
title(vec2ind(train_labels_unfold(:,j))-1);
pause(1);
end
MNISTParser.m
function res = MNISTParser(filename)
fp = fopen(filename,'r');
**magic_number = fread(fp,1,'uint32',0,'b');**
items_number = fread(fp,1,'uint32',0,'b');
if 2049==magic_number
res = fread(fp,items_number,'uint8',0,'b');
else
if 2051==magic_number
img_rows = fread(fp,1,'uint32',0,'b');
img_cols = fread(fp,1,'uint32',0,'b');
res = zeros(items_number,img_rows,img_cols);
for i=1:items_number
res(i,:,:) = fread(fp,[img_cols,img_rows],'uint8',0,'b')';
end
else
error('wrong magic number');
end
end
fclose(fp);
end
Thanks for helps
l find the mistake. it is related to the extension of the files.
here is the instruction that has helped me to find out my mistake [fp,msg] = fopen(filename,'r');
if fp<1, error([msg ' File: ' filename]);
Is there a faster way to replace the third line of file by another one without using cell ?
I've used this code but it slows my programs, especially that my txt file is composed by more than 1000 lines
% Read txt into cell A
fid7 = fopen([handles.filenameproba],'r');
i = 1;
tline = fgetl(fid7);
A{i} = tline;
while ischar(tline)
i = i+1;
tline = fgetl(fid7);
A{i} = tline;
end
fclose(fid7);
% Change cell A
newval =...
A{3} = sprintf('StartExperiment:%s',num2str(newval);
% Write cell A into txt
fid7 = fopen([handles.filenameproba], 'w');
for i = 1:numel(A)
if A{i+1} == -1
fprintf(fid7,'%s', A{i});
break
else
fprintf(fid7,'%s\n', A{i});
end
end
fclose(fid7);
Thanks !
If performance is your primary concern, try this importdata approach to see if it's any faster -
f=importdata(handles.filenameproba,'')
f(3)={sprintf('StartExperiment:%s',num2str(newval))}
%%// Save the modified text file
fid1 = fopen(handles.filenameproba,'w');
for k = 1:numel(f)
fprintf(fid1,'%s\n',f{k});
end
fclose(fid1);
Cells are not your problem. Your problem is reading and writing one line at a time. Additionally, you are re-sizing your cell array at every iteration.
For the following problem, I created a test file with 10000 lines in it.
fileId = fopen('test.txt', 'w');
for i = 1:10000
fprintf(fileId, 'This is the %dth line.\n', i);
end
fclose(fileId);
I'm calling your method ranellMethod.
>> timeit(#ranellMethod)
ans =
0.5160
A better way to do it is to limit the number of read/write operations you have to do. Assuming your file is small enough, you can read the entire contents into memory at once. Perform your operations, and write everything at once.
function entireFileMethod()
fileName = 'test.txt';
fileId = fopen(fileName, 'r');
try
text = fread(fileId, '*char')'; %'
catch Me
fclose(fileId);
Me.rethrow();
end
fclose(fileId);
newLineInds = find(text == char(10));
newLine = sprintf('This is the new line. %1.*f', randi(10), rand);
newText = [text(1:newLineInds(2)), newLine, text(newLineInds(3):end)];
fileId = fopen(fileName, 'w');
try
fprintf(fileId, '%s', newText);
catch Me
fclose(fileId);
Me.rethrow();
end
fclose(fileId);
end
This function has one read operation, and one write operation:
>> timeit(#entireFileMethod)
ans =
0.0043
See Fastest Matlab file reading? for more detailed information about file IO in MATLAB
hi i am new in matlab gui i have text file which i convert into binary then pass these binnary to mfsk function but it give me following error plese help me as soon as possible i post my full code what i can do please......
Error using .*
Integers can only be combined with integers of the same class, or scalar doubles.
Error in GUI2>menu_mscheme_mfsk_2_Callback (line 217)
d_xover_den =((4*pi./lambda).^2*(1+alpha)*SNR_unc./(2*Gr*Gt).*b*B*noise*NF.*Ton*(1- 1/CG));
Error in gui_mainfcn (line 96)
feval(varargin{:});
Error in GUI2 (line 43)
gui_mainfcn(gui_State, varargin{:});
my text code is
filename= uigetfile('*.txt','File select text');
txt = fopen(filename);
txtbits = fread(txt, inf, '*ubit1', 'b');
openvar('txtbits');
fclose(txt);
fileID = fopen(filename);
C = textscan(fileID,'%s');
i = C{1}{1};
dec2bin(i);
fclose(fileID);
celldisp(C);
set(handles.text3,'string',filename);
set(handles.text4,'string',txtbits);
handles.binary=txtbits;
guidata(hObject, handles);
my mfsk code is
decimal=handles.binary
N=decimal;
charact = {'b-+','b-o','b-v','b-^','b-*'};
j=0;
for N=50:20:150
j=j+1;
b=2;
L = decimal
freq=linspace(0.3,7,20)*1e+9;
lambda=1e+8./freq;
B = 1e+6; % Bandwidth
n = 5; % Path loss
noise = 4e-21;
P_ckt=0.02; % watt
% Ton=0.1; % sec
Ton=L*2^b./(b*B);
SNR_unc = 10;
K=24;
N_o_K = 30/24;
alpha = 1.9;
n=6; % path loss component
E_comp = 5; % To CONFORM the value
Gt = 1;
Gr = 1;
CG = 10; % Code Gain
NF = 1; % noise-factor
d_xover_num = (P_ckt*Ton*(N/K-1)+L*E_comp*N/K);
d_xover_den = ((4*pi./lambda).^2*(1+alpha)*SNR_unc./(2*Gr*Gt).*b*B*noise*NF.*Ton*(1- 1/CG));
d_xover = (d_xover_num./d_xover_den).^(1/n);
% d_xover = abs(d_xover);
axes(handles.abc)
plot(freq,d_xover,charact{j});
hold on
xlabel('Carrier Frequency'), ylabel('Cross-over distance (m)')
grid on
hold on
end
legend ('N=50','N=70','N=90','N=110','N=130')
You may want to check decimal. The handle are read using fread as txtbits = fread(txt, inf, '*ubit1', 'b'), which have the precision *ubit1. From the documentation you would see that this reads the text and returns the data as unsigned integers. This is then not compitable with the vector frequency, which is of type double. You need to convert the data to the same format.
However, I would in the future recommend to use the debugger and try to debug the code yourself first.