I have two txt files that are 1.txt and 2.txt. It stores data such as
1.txt:
P_e = [-0.1 0.71 0.88;-0.09 0.59 0.839;-0.08 0.55 0.816;-0.07 0.546 0.811;-0.06 0.46 0.769]
and data in 2.txt is
P_e = [-0.1 0.5 0.6;-0.09 0.1 0.2;-0.08 0.3 0.4;-0.07 0 01;-0.06 0 0]
I want to cacluate sum of P_e variables column by column,except column 1 in 1.txt and 2.txt and store it into P_e_sum variable.
-0.1 0.71+0.5 0.88+0.6
-0.09 0.59+0.1 0.839+0.2
-0.08 0.55+0.3 0.816+0.4
-0.07 0.546+0 0.811+0
-0.06 0.46+0 0.769+0
So the result is
Pe_sum=[ -0.1 1.21 1.48;
-0.09 0.69 1.039;
-0.08 0.85 1.216
-0.07 0.546 0.811;
-0.06 0.46 0.769]
Could you help me to implement it by matlab? Thank you so much
Its going to be hard to read those in as text. But the contents of those text files are basically formatted as matlab scripts anyway. Rename them to .m then you can just do this:
run ('1.m')
p1 = P_e;
run ('2.m')
p2 = P_e;
pSum = [p1(:, 1), (p1(:, 2:end) + p2(:, 2:end))];
Will do what you want.
As a side note, consider not naming these files just 1.txt Matlab doesn't agree with numeric first file names.
Related
Given a set of values fitting into a category, I'd like to
a) plot the data values as dots (y axis) according to category (x axis)
b) match dot color to category
c) add a line ranging from minimum to maximum of each set
What I did, was using this code:
set terminal png
set output 'animals.png'
set ytics nomirror
unset key
set xrange [-0.5:5.5]
plot for [i=2:5] 'cat.dat' using i:xtic(1)
show xrange
That successfully labels by category on the x-axis, but colors are set according to column (not row) and I would not know how to add the range bars (note: not errorbars or percentiles, but the full min->max range)- especially since the data is called columnwise but would then need to be analysed rowwise. AFAIK gnuplot does columns only, though.
Any ideas?
Output with above code:
Example data (tab-delimited):
cat 0.26 0.4 0.23 0.16
dog 0.317 0.264 0.25 0.26
bat 0.33 0.42 0.32 0.48
rat 0.59 0.62 0.57 0.56
foo 0.59 0.67 0.71 0.70
bar 0.664 0.75 0.68 0.6
As you noticed, gnuplot doesn't like rows and unfortunately does not (yet?) offer a transpose function. In your solution, you are using Unix system calls/tools and sed, which are not necessarily platform independent. Furthermore, you are plotting points and separate arrows to connect, I guess you can simplify this by linespoints if you don't insist on a horizontal bar at the minimum and maximum values.
Let me show some "simplified" platform-independent gnuplot-only code.
General Procedure:
load file to datablock
transpose datablock
plot columns with linespoints
Datafile TAB-separated without header: Animals.dat
cat 0.26 0.4 0.23 0.16
dog 0.317 0.264 0.25 0.26
bat 0.33 0.42 0.32 0.48
rat 0.59 0.62 0.57 0.56
foo 0.59 0.67 0.71 0.70
bar 0.664 0.75 0.68 0.6
The code below requires a FileToDatablock routine and a DatablockTranspose routine.
Procedure to load file into datablock: FileToDatablock.gpp
### Load datafile "as is" into datablock for different platforms
# ARG1 = input filename
# ARG2 = output datablock
if (GPVAL_SYSNAME[:7] eq "Windows") { # "Windows_NT-6.1" is shown on a Win7 system
load '< echo '.ARG2.' ^<^<EOD & type "'.ARG1.'"'
}
if (GPVAL_SYSNAME eq "Linux") { # that's shown on a Raspberry
load '< echo "\$Data << EOD" & cat "'.ARG1.'"'
}
if (GPVAL_SYSNAME eq "Darwin") { # this was shown on a MacOS Sierra 10.12.6
load '< echo "\$Data << EOD" & cat "'.ARG1.'"' # identical to Linux
}
### end of code
gnuplot procedure for transposing a datablock: DatablockTranspose.gpp
### transpose datablock (requires whitespace as separator)
# ARG1 = Input datablock
# ARG2 = Output datablock
set print #ARG2
do for [DBT_i=1:words(#ARG1[1])] {
DBT_Line = ""
do for [DBT_j=1:|#ARG1|] {
DBT_Line = DBT_Line.word(#ARG1[DBT_j],DBT_i).\
(DBT_j < |#ARG1| ? "\t" : "")
}
print DBT_Line
}
set print
undefine DBT_*
### end of code
The actual code:
### plotting rows
reset session
# load file to datablock
call "FileToDatablock" "Animals.dat" "$Data"
# transpose datablock by gnuplot procedure
call "DatablockTranspose.gpp" "$Data" "$DataTransposed"
set palette defined ( 0 'purple', 1 'blue', 2 'green', \
3 'yellow', 4 'orange', 5 'red' , 6 'black' )
unset colorbox
set xrange[0.5:|$Data|+0.5]
plot for [i=1:|$Data|] $DataTransposed u (i):i:(i):xtic(columnhead(i)) w lp pt 7 ps 1.5 lc palette not
### end of code
The result:
This takes a few more steps, above all, each category is given a unique index number, and the data is transposed:
(I'll refer to GNU unix shell commands here)
$cat -n data_orig.dat | datamash transpose > data_trans.dat
$cat data_trans.dat #added spaces for readability
1 2 3 4 5 6
cat dog bat rat foo bar
0.26 0.317 0.33 0.59 0.59 0.664
0.4 0.264 0.42 0.62 0.67 0.75
0.23 0.25 0.32 0.57 0.71 0.68
0.16 0.26 0.48 0.56 0.70 0.6
Now the data can be properly analyzed in columns and colors be defined according to the index number.
The bars are made with arrows, where minimum and maximum are taken from the statistical analysis of each column.
The xticlabels are read into a 1D word array (this is an internal gnuplot function) with a system call and the array indices are made to match the unique indices of the data columns.
The script with very detailed explanations to better support new gnuplot users:
#output and style settings: make png-file, name it 'animals.png',
# yaxis tics on both sides, no legend
set terminal png
set output 'animals.png'
set ytics mirror
unset key
#data indices are integers from 1 to 6, a bit of space for the looks
set xrange [0.5:6.5]
#define color scheme for each data series
set palette defined ( 0 'purple', 1 'blue', 2 'green', \
3 'yellow', 4 'orange', 5 'red' , 6 'black' )
#hide color gradient bar of palette
unset colorbox
#define array names using word function:
# read in 2nd line of data by system call and run through words
# each space-delimited word is now an array element of names
names(n) = word( system("sed -n '2p' cat.dat_t" ) , n )
#create min->max bars
#loop over all data sets to create bars
do for [i=1:6] {
#among others this gives minimum and maximum values of the data set
#using i -> only handle column i in statistics
#every ::3 start with row 3 for statistical analysis
stats 'data_trans.dat' using i every ::3
#use min/max values for arrow y positions, index i for x positions
#heads = arrow head on both sides
#size 0.1,90 = 0.1 line lengths for arrow head
# and 90° arrow head line angles = T bar style
#lc palette cb i = use line color (lc) from palette value matching
# color bar (cb) value of index i
set arrow from i,STATS_min to i,STATS_max heads size 0.1,90 lc palette cb i
}
#plotting:
# for [i=1:6] loop over all 6 columns, use i as loop variable
# every ::3 start with row 3 for data plotting
# using (i):i:(i):xtic(names(i))
# syntax of using
# x-value:y-value:z-value:label_x_axis [:label_y_axis:label_z_axis]
# (i) -> literal value of i for x and z, z is used as color definition
# i -> y-values from column i
# xtic(names(i)) get element i of array names for xtic label
# lc palette -> coloring according to defined palette
# pt 7 ps 1.5 -> point style and size definition
plot for [i=1:6] 'data_trans.dat' every ::3 using (i):i:(i):xtic(names(i)) lc palette pt 7 ps 1.5
References:
coloring based on x-values
array from word function
Result:
EDIT:
As shown in #theozh 's answer, linespoints are far more practicable for showing the range. This allows skipping the whole bar/arrow creation block by just adding w lp in the plotting command line.
I have several large-scale-model, that each model shows a Generation (171A00,181B30,...) and has its own parameters (approximately 200 parameters that includes values, matrix, ...) .
I have changed them to struct in order to separate parameters of each generation from other ones ( because most parameters have identical name) . However, the values of parameters are different in each project. There are also global parameters as variables in base workspace.
Moreover, geneartions work as enable subsystems.
I need to change these parameters with new ones from another project.
I'd like to know, if it is possible to change parameters programmatically in stand-alone mode. (i.e.) I transmit model on USB and connect USB in Target PC and then change parameters with a script in MATLAB.
p.s: this solution does not work : set_param('vdp/Mu','Gain','10')
It is really difficult to find where a parameter is used. ( as a gain, as a break Point in lookup-table, or …) That’s why this solution is not possible.
For example:
Project 1:
P181. AV_BILIM_POT_EXTENSION_Thermal = 4000;
Or
P181.BmsCurLimn_BatSocMaxSp_P = [0 0.02 0.232 0.492 0.692 0.742 0.793 0.845 0.897 0.999 1];
P171. AV_BILIM_POT_EXTENSION_Thermal = 2000 (dummy)
P171. BmsCurLimn_BatSocMaxSp_P = dummy
Project 2:
P181. AV_BILIM_POT_EXTENSION_Thermal = 3000;
P181.BmsCurLimn_BatSocMaxSp_P = [0 0.04 0.45 0.52 0.653 0.732 0.756 0.823 0.867 0.965 1];
P171. AV_BILIM_POT_EXTENSION_Thermal = 2000 (dummy)
P171. BmsCurLimn_BatSocMaxSp_P = dummy
Project 3:
P181. AV_BILIM_POT_EXTENSION_Thermal = dummy
P181.BmsCurLimn_BatSocMaxSp_P = dummy
P171. AV_BILIM_POT_EXTENSION_Thermal = 1000
P171. BmsCurLimn_BatSocMaxSp_P = [0 0.01 0.23 0.42 0.553 0.832 0.856 0.923 1]
Project 4:
This project could have a real value for both generation.(not dummy)
Thanks,
How can I add tabs (spaces) to strings for plots in Octave see code below. It doesn't create a tab (There should be a tab between Signal and Max Freq in the plot)
Also it produces warning messages
warning: text_renderer: skipping missing glyph for character '9'
warning: called from
annotation>update_textbox at line 1080 column 11
annotation at line 248 column 7
clf
plot(0:0)
var=456
t1='Signal ';
t2=[char(9), 'Max Freq'];
t3=[char(10), 'nextline',num2str(var)];
str=strcat(t1,t2,t3);
annotation('textbox',...
[0.15 0.65 0.3 0.15],...
'String',{str},...
'FontSize',14,...
'FontName','Arial',...
'LineStyle','--',...
'EdgeColor',[1 1 0],...
'LineWidth',2,...
'BackgroundColor',[0.9 0.9 0.9],...
'Color',[0.84 0.16 0]);
Ps: I'm using Octave 4.2.2 on Ubuntu 18.04 64bit
I added t4 for blanks...doesn't look very nice. Also note I am using Matlab, not Octave so I didn't get your error. Not sure about that.
clf
plot(0:0)
var=456
t1='Signal ';
t4 = blanks(5);
t2=[char(9),t4, 'Max Freq'];
t3=[char(10), 'nextline',num2str(var)];
str=strcat(t1,t2,t3);
annotation('textbox',...
[0.15 0.65 0.3 0.15],...
'String',{str},...
'FontSize',14,...
'FontName','Arial',...
'LineStyle','--',...
'EdgeColor',[1 1 0],...
'LineWidth',2,...
'BackgroundColor',[0.9 0.9 0.9],...
'Color',[0.84 0.16 0]);
I reproduced the results of a hierarchical model using the rethinking package with just rstan() and I am just curious why n_eff is not closer.
Here is the model with random intercepts for 2 groups (intercept_x2) using the rethinking package:
Code:
response = c(rnorm(500,0,1),rnorm(500,200,10))
predicotr1_continuous = rnorm(1000)
predictor2_categorical = factor(c(rep("A",500),rep("B",500) ))
data = data.frame(y = response, x1 = predicotr1_continuous, x2 = predictor2_categorical)
head(data)
library(rethinking)
m22 <- map2stan(
alist(
y ~ dnorm( mu , sigma ) ,
mu <- intercept + intercept_x2[x2] + beta*x1 ,
intercept ~ dnorm(0,10),
intercept_x2[x2] ~ dnorm(0, sigma_2),
beta ~ dnorm(0,10),
sigma ~ dnorm(0, 10),
sigma_2 ~ dnorm(0,10)
) ,
data=data , chains=1 , iter=5000 , warmup=500 )
precis(m22, depth = 2)
Mean StdDev lower 0.89 upper 0.89 n_eff Rhat
intercept 9.96 9.59 -5.14 25.84 1368 1
intercept_x2[1] -9.94 9.59 -25.55 5.43 1371 1
intercept_x2[2] 189.68 9.59 173.28 204.26 1368 1
beta 0.06 0.22 -0.27 0.42 3458 1
sigma 6.94 0.16 6.70 7.20 2927 1
sigma_2 43.16 5.01 35.33 51.19 2757 1
Now here is the same model in rstan():
# create a numeric vector to indicate the categorical groups
data$GROUP_ID = match( data$x2, levels( data$x2 ) )
library(rstan)
standat <- list(
N = nrow(data),
y = data$y,
x1 = data$x1,
GROUP_ID = data$GROUP_ID,
nGROUPS = 2
)
stanmodelcode = '
data {
int<lower=1> N;
int nGROUPS;
real y[N];
real x1[N];
int<lower=1, upper=nGROUPS> GROUP_ID[N];
}
transformed data{
}
parameters {
real intercept;
vector[nGROUPS] intercept_x2;
real beta;
real<lower=0> sigma;
real<lower=0> sigma_2;
}
transformed parameters { // none needed
}
model {
real mu;
// priors
intercept~ normal(0,10);
intercept_x2 ~ normal(0,sigma_2);
beta ~ normal(0,10);
sigma ~ normal(0,10);
sigma_2 ~ normal(0,10);
// likelihood
for(i in 1:N){
mu = intercept + intercept_x2[ GROUP_ID[i] ] + beta*x1[i];
y[i] ~ normal(mu, sigma);
}
}
'
fit22 = stan(model_code=stanmodelcode, data=standat, iter=5000, warmup=500, chains = 1)
fit22
Inference for Stan model: b212ebc67c08c77926c59693aa719288.
1 chains, each with iter=5000; warmup=500; thin=1;
post-warmup draws per chain=4500, total post-warmup draws=4500.
mean se_mean sd 2.5% 25% 50% 75% 97.5% n_eff Rhat
intercept 10.14 0.30 9.72 -8.42 3.56 10.21 16.71 29.19 1060 1
intercept_x2[1] -10.12 0.30 9.73 -29.09 -16.70 -10.25 -3.50 8.36 1059 1
intercept_x2[2] 189.50 0.30 9.72 170.40 182.98 189.42 196.09 208.05 1063 1
beta 0.05 0.00 0.21 -0.37 -0.10 0.05 0.20 0.47 3114 1
sigma 6.94 0.00 0.15 6.65 6.84 6.94 7.05 7.25 3432 1
sigma_2 43.14 0.09 4.88 34.38 39.71 42.84 46.36 53.26 3248 1
lp__ -2459.75 0.05 1.71 -2463.99 -2460.68 -2459.45 -2458.49 -2457.40 1334 1
Samples were drawn using NUTS(diag_e) at Thu Aug 31 15:53:09 2017.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at
convergence, Rhat=1).
My Questions:
the n_eff is larger using rethinking(). There is simulation differences but do you think something else is going on here?
Besides the n_eff being different the percentiles of the posterior distributions are different. I was thinking rethinking() and rstan() should return similar results with 5000 iterations since rethinking is just calling rstan. Are differences like that normal or something different between the 2 implementations?
I created data$GROUP_ID to indicate the categorical groupings. Is this the correct way to incorporate categorical variables into a hierarchical model in rstan()? I have 2 groups and if I had 50 groups I use the same data$GROUP_ID vector but is that the standard way?
Thank you.
I have a code which is producing output files containing information about some mesh which I need to analyse using MATLAB.
The output files look like this.
Vertex 1 1.3 -2.1 0 {z=(1.3e+0 -2.1e+0) mu=(1.4e-3 2.0e-3) uv=(-0.6 0.4)}
Vertex 2 1.4 -2.1 0 {z=(1.4e+0 -2.1e+0) mu=(2.8e-3 1.5e-3) uv=(-0.6 0.4)}
Vertex 3 -1.9 1.9 0 {z=(-1.9e+0 1.9e+0) mu=(-8.9e-2 1.4e-1) uv=( 0.7 -0.2)}
.
.
.
I would like my MATLAB code to read in this data file and form a matrix containing all the numbers
in the order specified.
So e.g I would want the above 3 lines to be processed into the matrix
1 1.3 -2.1 0 1.3e+0 -2.1e+0 1.4e-3 2.0e-3 -0.6 0.4
2 1.4 -2.1 0 1.4e+0 -2.1e+0 2.8e-3 1.5e-3 -0.6 0.4
3 -1.9 1.9 0 -1.9e+0 1.9e+0 -8.9e-2 1.4e-1 0.7 -0.2
Is there some convenient MATLAB facility/command to do this?
I think you could use textscan for this:
Example date.txt:
Vertex 1 1.3 -2.1 0 {z=(1.3e+0 -2.1e+0) mu=(1.4e-3 2.0e-3) uv=(-0.6 0.4)}
Vertex 2 1.4 -2.1 0 {z=(1.4e+0 -2.1e+0) mu=(2.8e-3 1.5e-3) uv=(-0.6 0.4)}
Vertex 3 -1.9 1.9 0 {z=(-1.9e+0 1.9e+0) mu=(-8.9e-2 1.4e-1) uv=( 0.7 -0.2)}
Code:
fileID = fopen('data.txt');
C = textscan(fileID,'Vertex %f %f %f %f {z=(%f %f) mu=(%f %f) uv=(%f %f)}');
fclose(fileID);
mtxC = [C{:}];
Result:
mtxC =
1.0000 1.3000 -2.1000 0 1.3000 -2.1000 0.0014 0.0020 -0.6000 0.4000
2.0000 1.4000 -2.1000 0 1.4000 -2.1000 0.0028 0.0015 -0.6000 0.4000
3.0000 -1.9000 1.9000 0 -1.9000 1.9000 -0.0890 0.1400 0.7000 -0.2000
MATLAB Option (partly tested)
I had to do something similar with a CMM once and it was easy to do in Python (see below). You could use the MATLAB command regexp(text, expression) to match a regular expression that gets what you want. This will return string data though, which you can save to a data file and then load that data file, or convert to numbers using str2double.
To use this, you first have to get your data file into MATLAB as series of strings. You can do this with fgetl.
in_fid = fopen('my_input_file.txt', 'r');
out_fid = fopen('my_output_file.txt', 'w');
data = [];
line = fgetl(in_fid);
while ischar(line)
match = regexp(line, '[+-]?\d+\.?\d*e?[+-]?\d*', 'match'); % find all matches
% Write to text file
fprintf(out_fid, '%s\t', match); % write values to file with tabs between
fprintf(out_fid, '\n'); % write a new line to the file
% Or save to an array locally
data = [data; str2double(match)];
line = fgetl(in_fid); % grab the next line
end
fclose('all');
% If you wrote to a text file, retrieve the data
data = dlmread('my_output_file.txt', 'delimiter', '\t'); % not sure about this...
Note that this will not match numbers that begin with a decimal point with no preceding digit, i.e. .2. Also note that this will match numbers that match the pattern in any file that you feed it, so it is generalized. For how to match floating point numbers, see this site (I changed it a bit though to add the e portion for scientific notation).
I was able to test the regexp and str2double operations on a remote machine, and it looks like building your data array directly works. I was unable to test the file I/O portion, so there may be some bugs there still.
Python Option (my favorite)
I suggest using regular expressions in Python for this sort of thing. I had to do something similar with a CMM once and it was easy to do in Python with something like:
import re
# Make pattern to match scientific notation numbers
pattern = re.compile(r"[+-]?\d+\.?\d*e?[+-]?\d*")
with open("your_input_file.txt", "r") as in_file:
with open("your_output_file.txt", "w") as out_file:
for line in in_file:
match = pattern.findall(line) # find all matches in the line
out_file.write("\t".join(match) + "\n") # write the results to a line in your output
For a good introduction to regex in Python, see Dive Into Python 3, which I recommend just about everybody reads. I tested this on your example file and it gives me:
1 1.3 -2.1 0 1.3e+0 -2.1e+0 1.4e-3 2.0e-3 -0.6 0.4
2 1.4 -2.1 0 1.4e+0 -2.1e+0 2.8e-3 1.5e-3 -0.6 0.4
3 -1.9 1.9 0 -1.9e+0 1.9e+0 -8.9e-2 1.4e-1 0.7 -0.2
in your_output_file.txt, so I think it works! The last step then is to just dlmread('your_output_file.txt', 'delimeter', '\t') in MATLAB and you should be good to go.
If you want to get fancy, you could upgrade your Python script so that it can be called from the command line with your input and output files as arguments (look into the sys.argv method), but this gets a bit more complicated and it is easy enough to just open the script and change the filename manually. Unless you need to do this all the time on differently-named files, in which case arguments are a good route. There is a good example of this here.