Why does arithenco/arithdeco not work with an input sequence including zeros and is there a way to work around this?
Example) Input message would be [2,1,2,1,2,0,0] and the decoded sequence is [3,2,3,2,3,1,2]. Whereas if I have an input message of [2,1,2,1,2,1,2], my decoded sequence is identical to the input as [2,1,2,1,2,1,2].
Below is my testing code. I believe you'll need the Signal Processing ToolBox for the arithenco/arithdeco functions.
clc;
clear all;
close all;
message = [2,1,2,1,2,0,1];
%% Encoder
table_size = height(message);
rows = table_size(1);
encodedY = [];
for row = 1:rows
coeff_row = message(row,:);
% Calculate unique values
source = unique(coeff_row);
for i=1:length(source)
counts(i)=length(strfind(coeff_row,source(i)));
end
seq=zeros(1,length(coeff_row));
for i=1:length(coeff_row)
seq(i)=strfind(source,coeff_row(i));
end
% Get arithmetic code for the block
code = arithenco(seq, counts);
% Store the block's code
encodedY = [encodedY; {code}];
end
%% Decoder
decodedY = [];
for row = 1 : size(encodedY)
temp = cell2mat(encodedY(row,:));
dseq = arithdeco(temp,counts,length(seq));
decodedY = [decodedY;dseq];
end
%%
Related
I am making a Matlab program that uses data from an excel file designated from an open file dialog.
[filename, pathname] = uigetfile({'*.xlsx','Excel Files(*.xlsx)'; '*.txt','Txt Files(*.txt)'}, 'Pick a file');
FilePath = append(pathname,filename);
opts = detectImportOptions(FilePath, "ReadVariableNames", false);
opts = setvartype(opts, 1, 'char');
data = readtable(FilePath, opts);
table = data(:,1);
Now the code is like this.
enter image description here
After that, as you see the date is saved as string.
But what I really want to find is time difference (duration) in milliseconds.
The raw data looks like this:
enter image description here
A column and C column has the same time, so I want to only use A column data.
Please help a newbie with this!! I appreciate!
Problem resolved.
I share my code to help other people suffering the same problem.
Please leave comments if things can be more simplified.
Importing an Excel file to analyze
% Clean the memory and the code previously running
clc;
clear all;
close all;
% Sampling frequency of the acquired data
fs = 1e2; % Sampling Frequency - this can be found on LabView code.
Ts = 1/fs; % Sampling Interval
%Importing data from an excel file
[filename, pathname] = uigetfile({'*.xlsx','Excel Files(*.xlsx)'; '*.txt','Txt Files(*.txt)'}, 'Pick a file');
FilePath = append(pathname,filename);
[fPath ,fName, fExt] = fileparts(FilePath);
a. To find "Time duration" from the file
opts = spreadsheetImportOptions("NumVariables", 1);
% Specify sheet and range
opts.Sheet = "sheet1";
opts.DataRange = "A2";
% Specify column names and types
opts.VariableNames = "Time";
opts.VariableTypes = "datetime";
% Specify file level properties
opts.ImportErrorRule = "omitrow";
opts.MissingRule = "omitrow";
% Specify variable properties
opts = setvaropts(opts, "Time", "InputFormat", 'mm:ss.SSS');
tTable = readtable(FilePath, opts, "UseExcel", false);
tArray = table2array(tTable);
% Calculating time duration
tArray = tArray - tArray(1);
% Coverting to seconds
time = milliseconds(tArray)*1e-3;
% Clear temporary variables
clear opts;
% Discarding data if time difference is too big
ii = size(time(:,1));
k = 0;
disp('Now removing error data elements');
for i = 1:1:ii(1)-1
a = time(i);
b = time(i+1);
if (b-a)>0.5 && k==0
k=i+1;
fprintf('Elements from %d seconds will be removed. (%dth element)\n', time(k),k);
for j = ii(1):-1:k
if rem(j,10)==0
fprintf('%dth element is removed... \n',j);
end
time(j) = [];
end
break % Break the loop after removing error data
end
end
disp('Time table is set.');
clearvars -except filename FilePath fs pathname time k ii Ts fName
I created a similar excel file and I found the time columns are read as a char. So, what I did is get the index of those columns and convert it to datatime. After that, looks that it is working. Hopefully, this works.
% Define the excel file name (can be the path)
ExcelFile = 'demo_stackflow.xlsx';
% Get the options for that sheet and preserving the variable name
opts = detectImportOptions(ExcelFile,'Sheet','Sheet1',...
"VariableNamingRule","preserve")
% Get the idx where the variable is a char (In this case col 1 and 3)
CharVars = opts.VariableNames(contains(opts.VariableTypes,'char'));
% Convert the char columns to datetime
opts = setvaropts(opts,CharVars,'Type',"datetime");
% Get the data
data = readtable(ExcelFile, opts)
% Print the 1st column to see if the data type of the column is
% datetime
T = data(:,1)
Good evening, May I please get advice with the following Matlab code? Here it is:
%% CLEAR ALL
close all
clear all
clc
%% LOAD MODEL AND LHC FILE
tic %start the clock
idx=1;
model = 'PG_PN_basic_rev1'; %This is the simulink file you wish to run.
load_system(model);
load 'LHC_input.mat' %Call in the file created by LHC_Final.m
LHC = (LHC1_input);
k_dc = LHC((1:5),1);
k_r = LHC((1:5),2);
a_1 = LHC((1:5),3);
b_1 = LHC((1:5),4);
Kg_PG = LHC((1:5),5);
Kg_PN = LHC((1:5),6);
for i = length(k_dc):-1:1
in(i) = Simulink.SimulationInput('PG_PN_basic_rev1');
in(i) = in(i).setVariable('k_dc',k_dc(i));
for j = length(k_r):-1:1
in(j) = in(j).setVariable('k_r',k_r(j));
for k = length(a_1):-1:1
in(k) = in(k).setVariable('a_1',a_1(k));
for l = length(b_1):-1:1
in(l) = in(l).setVariable('b_1',b_1(l));
for m = length(Kg_PG):-1:1
in(m) = in(m).setVariable('Kg_PG',Kg_PG(m));
for n = length(Kg_PN):-1:1
in(n) = in(n).setVariable('Kg_PN',Kg_PN(n));
end
end
end
end
end
end
out = parsim(in, 'ShowProgress', 'on');
% eval(['PN_batch', int2str(idx),' =PN;']);
% data = eval(['PN_batch', int2str(idx)]);
% a{idx} = data;
% idx=idx+1;
% run = idx
timeElapsed = toc %How long did you code run for?
I wish to be able to generate an output file per parsim run (PN_batch1, PN_batch2,...etc.). However, the data often falls under just 1 output, and isn't divided up into readable workspace objects that I can read later with another script. Any advice would be greatly appreciated. Thank you.
out is a vector of length equal to the number of simulations with the data of a simulation stored in each entry. If you have to workspace blocks in your model, you can access that data per simulation using out(10).NameOftoWorkspaceData, in case you want to get the data of the 10th simulation. More info on the out variable can be found here on the Mathworks site.
Tip: run the model and check out the variable out, then you can explore its structure
In short, I'm having a headache in multiple languages to read a txt file (linked below). My most familiar language is MATLAB so for that reason I'm using that in this example. I've found a way to read this file in ~ 5 minutes, but given I'll have tons and tons of data from my instrument shortly as it measures all day every 30 seconds this just isn't feasible.
I'm looking for a way to quickly read these irregular text files so that going forward I can knock these out with less of a time burden.
You can find my exact data at this link:
http://lb3.pandonia.net/BostonMA/Pandora107s1/L0/Pandora107s1_BostonMA_20190814_L0.txt.bz2
I've been using the "readtable" function in matlab and I have achieved a final product I want but I'm looking to increase the speed
Below is my code!
clearvars -except pan day1; % Clearing all variables except for the day and instrument variables.
close all;
clc;
pan_mat = [107 139 155 153]; % Matrix of pandora numbers for file-choosing
reasons.
pan = pan_mat(pan); % pandora number I'm choosing
pan = num2str(pan); % Turning Pandora number into a string.
%pan = '107'
pandora = strcat('C:\Users\tadams15\Desktop\Folders\Counts\Pandora_Dta\',pan)
% string that designates file location
%date = '90919'
month = '09'; % Month
day2 = strcat('0',num2str(day1)) % Creating a day name for the figure I ultimately produce
cd(pandora)
d2 = strcat('2019',num2str(month),num2str(day2)); % The final date variable
for the figure I produce
%file_pan = 'Pandora107s1_BostonMA_20190909_L0';
file_pan = strcat('Pandora',pan,'s1_BostonMA_',d2,'_L0'); % File name string
%Try reading it in line by line?
% Load in as a string and then convert the lines you want as numbers into
% number.
delimiterIn = '\t';
headerlinesIn = 41;
A = readtable(file_pan,'HeaderLines', 41, 'Delimiter', '\t'); %Reading the
file as a table
A = table2cell(A); % Converting file to a cell
A = regexp(A, ' ', 'split'); % converting cell to a structure matrix.
%%
A= array2table(A); % Converting Structure matrix back to table
row_num = 0;
pan_mat_2 = zeros(2359,4126);
datetime_mat = zeros(2359,2);
blank = 0;
%% Converting data to proper matrices
[length width] = size(A);
% The matrix below is going through "A" and writing from it to a new
% matrix, "pan_mat_2" which is my final product as well as singling out the
% rows that contain non-number variables I'd like to keep and adding them
% later.
tic
%flag1
for i = 1:length; % Make second number the length of the table, A
blank = 0;
b = table2array(A{i,1});
[rows, columns] = size(b);
if columns > 4120 && columns < 4140
row_num = row_num + 1;
blank = regexp(b(2), 'T', 'split');
blank2 = regexp(blank{1,1}(2), 'Z', 'split');
datetime_mat(row_num,1) = str2double(blank{1,1}(1));
datetime_mat(row_num,2) = str2double(blank2{1,1}(1));
for j = 1:4126;
pan_mat_2(row_num,j) = str2double(b(j));
end
end
end
toc
%flag2
In short, I'm already getting the result I want but the part of the code where I'm writing to a new array "flag 1" to "flag 2" is taking roughly 222 seconds while the entire code only takes about 248 seconds. I'd like to find a better way to create the data there than to write it to a new array and take a whole bunch of time.
Any suggestions?
Note:
There are a quite a few improvments you can make for speed but there are also corrections. You preallocate you final output variable with hard coded values:
pan_mat_2 = zeros(2359,4126);
But later you populate it in a loop which run for i = 1:length.
length is the full number of lines picked from the file. In your example file there are only 784 lines. So even if all your line were valid (ok to be parsed), you would only ever fill the first 784 lines of the total 2359 lines you allocated in your pan_mat_2. In practice, this file has only 400 valid data lines, so your pan_mat_2 could definitely be smaller.
I know you couldn't know you had only 400 line parsed before you parsed them, but you knew from the beginning that you had only 784 line to parse (you had the info in the variable length). So in case like these pre-allocate to 784 and only later discard the empty lines.
Fortunately, the solution I propose does not need to pre-allocate larger then discard. The matrices will end up the right size from the start.
The code:
%%
file_pan = 'Pandora107s1_BostonMA_20190814_L0.txt' ;
delimiterIn = '\t';
headerlinesIn = 41;
A = readtable(file_pan,'HeaderLines', 41, 'Delimiter', '\t'); %Reading the file as a table
A = table2cell(A); % Converting file to a cell
A = regexp(A, ' ', 'split'); % converting cell to a structure matrix.
%% Remove lines which won't be parsed
% Count the number of elements in each line
nelem = cell2mat( cellfun( #size , A ,'UniformOutput',0) ) ;
nelem(:,1) = [] ;
% find which lines does not have enough elements to be parsed
idxLine2Remove = ~(nelem > 4120 & nelem < 4140) ;
% remove them from the data set
A(idxLine2Remove) = [] ;
%% Remove nesting in cell array
nLinesToParse = size(A,1) ;
A = reshape( [A{:}] , [], nLinesToParse ).' ;
% now you have a cell array of size [400x4126] cells
%% Now separate the columns with different data type
% Column 1 => [String] identifier
% Column 2 => Timestamp
% Column 3 to 4125 => Numeric values
% Column 4126 => empty cell created during the 'split' operation above
% because of a trailing space character.
LineIDs = A(:,1) ;
TimeStamps = A(:,2) ;
Data = A(:,3:end-1) ; % fetch to "end-1" to discard last empty column
%% now extract the values
% You could do that directly:
% pan_mat = str2double(Data) ;
% but this takes a long time. A much computationnaly faster way (even if it
% uses more complex code) would be:
dat = strjoin(Data) ; % create a single long string made of all the strings in all the cells
nums = textscan( dat , '%f' , Inf ) ; % call textscan on it (way faster than str2double() )
pan_mat = reshape( cell2mat( nums ) , nLinesToParse ,[] ) ; % reshape to original dimensions
%% timestamps
% convert to character array
strTimeStamps = char(TimeStamps) ;
% convert to matlab own datetime numbering. This will be a lot faster if
% you have operations to do on the time stamps later
ts = datenum(strTimeStamps,'yyyymmddTHHMMSSZ') ;
%% If you really want them the way you had it in your example
strTimeStamps(:,9) = ' ' ; % replace 'T' with ' '
strTimeStamps(:,end) = ' ' ; % replace 'Z' characters with ' '
%then same again, merge into a long string, parse then reshape accordingly
strdate = reshape(strTimeStamps.',1,[]) ;
tmp = textscan( strdate , '%d' , Inf ) ;
datetime_mat = reshape( double(cell2mat(tmp)),2,[]).' ;
The performance:
As you can see on my machine your original code takes ~102 seconds to execute, with 80% of that (81s) spent on calling the function str2double() 3,302,400 times!
My solution, run on the same input file, takes ~5.5 seconds, with half of the time spent on calling strjoin() 3 times.
When you read the code above, try to understand how I limited the repetition of function call in lengthy loops by trying to keep everything as vectorised as possible.
Using the profiler, you can see that you call str2double 3302400 times in a run which takes about 80% of the total time on my pc. Now thats suboptimal, as each time you only translate 1 value and as far as your code goes you dont need the values as string again. I added this under you original code:
row_num = 0;
pan_mat_2_b = cell(2359,4126);
datetime_mat_b = cell(2359,2);%not zeros
blank = 0;
tic
%flag1
for i = 1:length % Make second number the length of the table, A
blank = 0;
b = table2array(A{i,1});
[rows, columns] = size(b);
if columns > 4120 && columns < 4140
row_num = row_num + 1;
blank = regexp(b(2), 'T', 'split');
blank2 = regexp(blank{1,1}(2), 'Z', 'split');
%datetime_mat(row_num,1) = str2double(blank{1,1}(1));
%datetime_mat(row_num,2) = str2double(blank2{1,1}(1));
datetime_mat_b(row_num,1) = blank{1,1}(1);
datetime_mat_b(row_num,2) = blank2{1,1}(1);
pan_mat_2_b(row_num,:) = b;
% for j = 1:4126
% pan_mat_2(row_num,j) = str2double(b(j));
% end
end
end
datetime_mat_b = datetime_mat_b(~all(cellfun('isempty',datetime_mat_b),2),:);
pan_mat_2_b=pan_mat_2_b(~all(cellfun('isempty',pan_mat_2_b),2),:);
datetime_mat_b=str2double(string(datetime_mat_b));
pan_mat_2_b=str2double(pan_mat_2_b);
toc
Still not great, but better. If you want to speed this up further i recommend you take a closer look at the readtable part. As you can save up quite some time if you start with reading in the format as doubles right from the beginning
I wrote a code for generating random number of rods on Matlab within a specified domain and then saving the output in a text file. I would like to ask for help on adding the following options to the code;
(i) if the randomly generated rod exceeds the specified domain size, the length of that rod should be shortened so that to keep it in that particular domain.
(ii) i would like to avoid the overlapping of the newly generated number (rod) with that of the previous one, in case of overlap generate another place for the new rod.
I can't figure out how shall I do it. It would be of much help if someone may help me write code for these two options.
Thank you
% myrandom.m
% Units are mm.
% domain size
bx = 160;
by = 40;
bz = 40;
lf = 12; % rod length
nf = 500; % Number of rods
rns = rand(nf,3); % Start
rne = rand(nf,3)-0.5; % End
% Start Points
for i = 1:nf
rns(i,1) = rns(i,1)*bx;
rns(i,2) = rns(i,2)*by;
rns(i,3) = rns(i,3)*bz;
end
% Unit Deltas
delta = zeros(nf,1);
for i = 1:nf
temp = rne(i,:);
delta(i) = norm(temp);
end
% Length Deltas
rne = lf*rne./delta;
% End Points
rne = rns + rne;
fileID = fopen('scfibers.txt','w');
for i = 1:nf
fprintf(fileID,'%12.8f %12.8f %12.8f\r\n',rns(i,1),rns(i,2),rns(i,3));
fprintf(fileID,'%12.8f %12.8f %12.8f\r\n\r\n',rne(i,1),rne(i,2),rne(i,3));
end
fclose(fileID);
I would start from writing a function that creates the random rods:
function [rns,rne] = myrandom(domain,len,N)
rns = rand(N,3).*domain; % Start --> rns = bsxfun(#times,rand(N,3),domain)
rne = rand(N,3)-0.5; % End
% Unit Deltas
delta = zeros(N,1);
for k = 1:N
delta(k) = norm(rne(k,:));
end
% Length Deltas
rne = len*rne./delta; % --> rne = len*bsxfun(#rdivide,rne,delta)
% End Points
rne = rns + rne;
% remove rods the exceed the domain:
notValid = any(rne>domain,2); % --> notValid = any(bsxfun(#gt,rne,domain),2);
rns(notValid,:)=[];
rne(notValid,:)=[];
end
This function gets the domain as [bx by bz] and also the length of the rods as len, and N the number of rods to generate. Note that using elementwise multiplication (.*) I have eliminated the first for loop.
In case you use MATLAB version prior to 2016b, you need to use bsxfun:
In MATLAB® R2016b and later, the built-in binary functions listed in this table independently support implicit expansion.
The affected lines are marked with --> in the code (with the alternative).
The last three lines in the function remove from the result all the rodes that exceed the domain size (I hope I got you correctly on this).
Next, I call this function within a script:
% domain size
bx = 160;
by = 40;
bz = 40;
domain = [bx by bz];
lf = 12; % rod length
nf = 500; % Number of rods
[rns,rne] = myrandom(domain,lf,nf);
u = unique([rns rne],'rows');
remain = nf-size(u,1);
while remain>0
[rns_temp,rne_temp] = myrandom(domain,lf,remain);
rns = [rns;rns_temp];
rne = [rne;rne_temp];
u = unique([rns rne],'rows');
remain = nf-size(u,1);
end
After the basic definitions, the function is called and returns rne and rns, which are probably smaller than nf. Then we check for duplicates, and store all unique rods in u. We calculate the rods remain to compute, and we use a while loop to generate new rods as needed. In each iteration of the loop, we add the newly created rods to those we have in rne and rns, and check how many unique vectors we have now, and if there are enough we quit the loop (then you can add printing to the file).
Note that:
I was not sure what you mean by "in case of overlap generate another place for the new rod" - do you want to have more than nf rods if some are duplicates, that from which nf are unique (what the code above does)? or you want to remove the duplicates and remain only with nf unique rods? In the case of the latter option, I would insert the unique function part into the function that creates the rods myrandom.
The wile loop as written above is not efficient since no preallocating of memory is done. I'm not sure that this is possible if you just want to create more rods but keep the duplicates, but if not (the second option in 1 above) and if you are going to use this allot, then preallocating is very recommended.
My code is below. In the code, I am evaluating only the data in the 'fb2010' file. I want to add other files" 'fb2020', 'fb2030', and 'fb2040' and evaluate their data by the same code. My question is how to apply a for loop and include the other data files. I tried, but I got confused by the for loop.
load('fb2010'); % loading the data
x = fb2010(3:1:1502,:);
% y_filt = filter(b,a,x); % filtering the received signal
y_filt= filter(b,a,x,[],2);
%%%%%%% fourier transform
nfft = length(y_filt);
res = fft(y_filt,nfft,2)/nfft;
res2 = res(:,1:nfft/2+1); %%%% taking single sided spectrum
res3 = fft(res2,[],2);
for i = 3:1:1500 %%%% dividing each row by first row.
resd(i,:) = res3(i,:)./res3(1,:);
end
I'm assuming that your files are MAT-files, not ASCII. You can do this by having load return a struct and using dynamic field referencing:
n = 4;
for i = 1:n
vname = ['fb20' int2str(i) '0']; % Create file/variable name based on index
s = load(vname); % Load data as struct (overwriting previous s)
x = s.(vname)(3:1:1502,:); % Access struct with dynamic field reference
% Rest of your code
...
end
If you're using a plain ASCII file, load won't produce a struct. However, such files are much simpler (see documentation for load/save). The following code would probably work:
n = 4;
for i = 1:n
vname = ['fb20' int2str(i) '0']; % Create file/variable name based on index
s = load(vname); % Load data as matrix (overwriting previous s)
x = s(3:1:1502,:); % Directly index matrix
% Rest of your code
...
end
It would be a good idea to add the file extension to your load command to make your code more readable.