Matplotlib: Formatting date in day.month.year style - date

I want to plot some lines with the date on the x axis, but all examples I could find use the American style like 12-31-2012. But I want 31.12.2012, but it doesn't seem to work by just changing the date formatter from
dateFormatter = dates.DateFormatter('%Y-%m-%d')
to
dateFormatter = dates.DateFormatter('%d.%m.%y')
My date list works like this: I want to define a "firstDay" manually, and then generate X succeeding days. That works as I can see when I print that result list.
But when I want to plot that list (converted by num2date) I have totally different dates.
E.g. I set my firstDay to 734517.0 which is January the 15th in 2012. Then I print my dates on the axis and I get as a first date 01.01.87 ??
Here is my full code:
import numpy as np
import matplotlib.pyplot as plot
import matplotlib.ticker as mticker
from matplotlib import dates
import datetime
fig = plot.figure(1)
DAU = ( 2, 20, 25, 60, 190, 210, 18, 196, 212, 200, 160, 150, 185, 175, 316, 320, 294, 260, 180, 145, 135, 97, 84, 80, 60, 45, 37, 20, 20, 24, 39, 73, 99)
WAU = ( 50, 160, 412, 403, 308, 379, 345, 299, 258, 367, 319, 381, 461, 412, 470, 470, 468, 380, 290, 268, 300, 312, 360, 350, 316, 307, 289, 321, 360, 378, 344, 340, 346)
MAU = (760, 620, 487, 751, 612, 601, 546, 409, 457, 518, 534, 576, 599, 637, 670, 686, 574, 568, 816, 578, 615, 520, 499, 503, 529, 571, 461, 614, 685, 702, 687, 649, 489)
firstDay = 734517.0 #15. Januar 2012
#create an array with len(DAU) entries from given starting day...
dayArray = []
for i in xrange(len(DAU)):
dayArray.append(firstDay + i)
#...and fill them with the converted dates
dayLabels = [dates.num2date(dayArray[j]) for j in xrange(len(DAU))]
for k in xrange(len(DAU)):
print dayLabels[k]
spacing = np.arange(len(DAU)) + 1
line1 = plot.plot(spacing, DAU, 'o-', color = '#336699')
line2 = plot.plot(spacing, WAU, 'o-', color = '#993333')
line3 = plot.plot(spacing, MAU, 'o-', color = '#89a54e')
ax = plot.subplot(111)
plot.ylabel('', weight = 'bold')
plot.title('', weight = 'bold')
ticks, labels = plot.xticks(spacing, dayLabels)
plot.setp(labels, rotation = 90, fontsize = 11)
dateFormatter = dates.DateFormatter('%d.%m.%y')
ax.xaxis.set_major_formatter(dateFormatter)
#ax.fmt_xdata = dates.DateFormatter('%Y-%m-%d')
#fig.autofmt_xdate()
yMax = max(np.max(DAU), np.max(WAU), np.max(MAU))
yLimit = 100 - (yMax % 100) + yMax
plot.yticks(np.arange(0, yLimit + 1, 100))
plot.grid(True, axis = 'y')
plot.subplots_adjust(bottom = 0.5)
plot.subplots_adjust(right = 0.82)
legend = plot.legend((line1[0], line2[0], line3[0]),
('DAU',
'WAU',
'MAU'),
'upper left',
bbox_to_anchor = [1, 1],
shadow = True)
frame = legend.get_frame()
frame.set_facecolor('0.80')
for t in legend.get_texts():
t.set_fontsize('small')
plot.show()
It would be fine as well with this date formatter:
ax.fmt_xdata = dates.DateFormatter('%Y-%m-%d')
but that gives me the timestamps as well, e.g. 2012-01-15 00:00:00+00:00 .
If someone could tell me how to cut the time off, it would be really great!!

It seems to me that the easiest way is to use real Datetime objects. This way you can use the datetime.timedelta(days=i) to make your date range. And matplotlib automatically takes spacing into account in case your dates are not regular. It also allows you to use the default date formatting options from matplotlib.
I left some code out to keep it simpler but you should be able to mix this with your script:
![import numpy as np
import matplotlib.pyplot as plot
import matplotlib.ticker as mticker
from matplotlib import dates
import datetime
fig = plot.figure(1)
DAU = ( 2, 20, 25, 60, 190, 210, 18, 196, 212, 200, 160, 150, 185, 175, 316, 320, 294, 260, 180, 145, 135, 97, 84, 80, 60, 45, 37, 20, 20, 24, 39, 73, 99)
WAU = ( 50, 160, 412, 403, 308, 379, 345, 299, 258, 367, 319, 381, 461, 412, 470, 470, 468, 380, 290, 268, 300, 312, 360, 350, 316, 307, 289, 321, 360, 378, 344, 340, 346)
MAU = (760, 620, 487, 751, 612, 601, 546, 409, 457, 518, 534, 576, 599, 637, 670, 686, 574, 568, 816, 578, 615, 520, 499, 503, 529, 571, 461, 614, 685, 702, 687, 649, 489)
firstDay = datetime.datetime(2012,1,15) #15. Januar 2012
dayArray = [firstDay + datetime.timedelta(days=i) for i in xrange(len(DAU))]
ax = plot.subplot(111)
line1 = ax.plot(dayArray, DAU, 'o-', color = '#336699')
line2 = ax.plot(dayArray, WAU, 'o-', color = '#993333')
line3 = ax.plot(dayArray, MAU, 'o-', color = '#89a54e')
ax.xaxis.set_major_formatter(dates.DateFormatter('%d.%m.%Y'))]
The main difference with your script is the way dayArray is created (and used as the x-value in the plotting command) and the last line which sets the format of the x-axis.

Related

How do I model serial correlation in a binomial model?

I'd like to test for trends in proportions of animals sampled over 12 years at six different beaches so that there is a separate trend test per beach. In the data below 'thisbeach' is the number of animals sampled at that particular beach and 'notthisbeach' is the number of animals sampled at all other beaches.
dat <- data.frame(fBeach = as.factor(rep(c("B6", "B5", "B2", "B1", "B4", "B3"), each=12)),
year = rep(seq(1:12),6),
notthisbeach = c(4990, 1294, 4346, 4082, 4628, 5576, 5939, 5664, 6108, 5195, 5564, 4079, 4694, 1224, 4052,
4019, 4457, 5242, 5259, 5198, 5971, 5208, 5168, 3722, 5499, 1288, 4202, 3988, 4773, 6018,
5952, 6100, 7308, 5821, 6030, 4546, 4698, 1300, 3884, 3943, 4717, 5911, 6110, 6076, 7606,
6138, 6514, 4767, 4830, 1307, 4886, 4327, 5285, 6344, 6627, 5824, 7305, 5991, 6073, 4647,
4584, 1162, 4200, 3956, 4710, 5664, 5533, 4828, 6082, 4697, 4721, 3529),
thisbeach = c(869, 221, 768, 781, 1086, 1375, 1145, 1074, 1968, 1415, 1250, 979, 1165, 291, 1062,
844, 1257, 1709, 1825, 1540, 2105, 1402, 1646, 1336, 360, 227, 912, 875, 941, 933,
1132, 638, 768, 789, 784, 512, 1161, 215, 1230, 920, 997, 1040, 974, 662, 470,
472, 300, 291, 1029, 208, 228, 536, 429, 607, 457, 914, 771, 619, 741, 411,
1275, 353, 914, 907, 1004, 1287, 1551, 1910, 1994, 1913, 2093, 1529))
glmmTMB indicates serial correlation is present;
require(glmmTMB)
require(DHARMa)
require(multcomp)
dat.TMB <- glmmTMB(cbind(notthisbeach,thisbeach) ~ year*fBeach, family = "betabinomial", data=dat)
simres <- simulateResiduals(dat.TMB,plot=T)
res = recalculateResiduals(simres, group = dat$year)
testTemporalAutocorrelation(res, time=unique(dat$year))
Durbin-Watson test
data: simulationOutput$scaledResiduals ~ 1
DW = 0.40903, p-value = 0.0002994
alternative hypothesis: true autocorrelation is not 0
However, I can't seem to find any examples including an autocorrelation structure in a model of this type.
Does anyone have any advice please?
I'm not sure I get the setup of the number of animals at this beach vs that beach, and depending on your research question you may need to do something different. However, basic patterns are easy enough to implement in glmmtmb. The example below shows an ar1.
dat <- data.frame(fBeach = as.factor(rep(c("B6", "B5", "B2", "B1", "B4", "B3"), each=12)),
year = rep(seq(1:12),6),
notthisbeach = c(4990, 1294, 4346, 4082, 4628, 5576, 5939, 5664, 6108, 5195, 5564, 4079, 4694, 1224, 4052,
4019, 4457, 5242, 5259, 5198, 5971, 5208, 5168, 3722, 5499, 1288, 4202, 3988, 4773, 6018,
5952, 6100, 7308, 5821, 6030, 4546, 4698, 1300, 3884, 3943, 4717, 5911, 6110, 6076, 7606,
6138, 6514, 4767, 4830, 1307, 4886, 4327, 5285, 6344, 6627, 5824, 7305, 5991, 6073, 4647,
4584, 1162, 4200, 3956, 4710, 5664, 5533, 4828, 6082, 4697, 4721, 3529),
thisbeach = c(869, 221, 768, 781, 1086, 1375, 1145, 1074, 1968, 1415, 1250, 979, 1165, 291, 1062,
844, 1257, 1709, 1825, 1540, 2105, 1402, 1646, 1336, 360, 227, 912, 875, 941, 933,
1132, 638, 768, 789, 784, 512, 1161, 215, 1230, 920, 997, 1040, 974, 662, 470,
472, 300, 291, 1029, 208, 228, 536, 429, 607, 457, 914, 771, 619, 741, 411,
1275, 353, 914, 907, 1004, 1287, 1551, 1910, 1994, 1913, 2093, 1529))
head(dat)
dim(dat)
require(glmmTMB)
require(DHARMa)
require(multcomp)
# function to test ar
testar <- function(mod, dat) {
simres <- simulateResiduals(mod,plot=T)
res <- recalculateResiduals(simres, group = dat$year)
print(testTemporalAutocorrelation(res, time=unique(dat$year)))
}
mod.TMB <- glmmTMB(cbind(notthisbeach,thisbeach) ~ year*fBeach, family = "betabinomial", data=dat)
testar(mod.TMB, dat)
# results
# Durbin-Watson test
#
# data: simulationOutput$scaledResiduals ~ 1
# DW = 0.40903, p-value = 0.0002994
# alternative hypothesis: true autocorrelation is not 0
mod.TMB.ar <- glmmTMB(cbind(notthisbeach,thisbeach) ~ as.factor(year) + fBeach + ar1(as.factor(year) + 0 | fBeach), family = "betabinomial", data=dat)
testar(mod.TMB.ar, dat)
#
# Durbin-Watson test
#
# data: simulationOutput$scaledResiduals ~ 1
# DW = 1.179, p-value = 0.1242
# alternative hypothesis: true autocorrelation is not 0
VarCorr(mod.TMB.ar)
# Conditional model:
# Groups Name Std.Dev. Corr
# fBeach as.factor(year)1 0.21692 0.464 (ar1)

Extracting Temperatures from .ravi file in Matlab

My Problem
Much like the post here: How can I get data from 'ravi' file?, I have a .ravi file (a radiometric video file, which is rather similar to an .avi) and I am trying to extract the Temperatures in it, to use them together with additional sensor data.
A Sample File can be found in the documentation (http://infrarougekelvin.com/en/optris-logiciel-eng/) when you download the "PIX Connect Software". Unfortunately, according to the documentation, the temperature information is stored in a 16 Bit Format, that Matlab seems to be rather unhappy with.
How I tried to solve my problem
I tried to follow the instructions from the before mentioned post, but I somehow struggle to reach results, which are even close to the correct temperatures.Original Picture with temperatures in the Optris Software
I tried to read the video with different methods:
At first I hoped to use the videorecorder Feature in Matlab:
video = VideoReader(videoPath);
frame1 = video.read(1);
imagesc(frame1)
But it only resulted in this poor picture, which is exactly what I can see, when I try to play the .ravi file in a media player like vlc.
First try with videorecorder function
Then I tried to look at the binary representation of my file and noticed, that I could separate the frames at a certain marker Beginning of a new frame in binary representation
So I tried to read the file with the matlab fread function:
fileID = fopen(videoPath);
[headerInfo,~] = fread(fileID,[1,123392],'uint8');
[imageMatrix,count] = fread(fileID,[video.width, video.height],'uint16', 'b');
imagesc(imageMatrix')
Now the image looks better, and you can at least see the brake disc, but it seems, as if the higher temperatures have some kind of offset, that is stil missing, for the picture to be right.
Also, the values that I read from the file are nowhere near actual temperatures, as the other post and the documentation suggests.
Getting somewhere!
My Question
Am I somehow missing something important? Could someone point me in the right direction, where to look or how to get the actual temperatures from my video? As it worked with the cpp code in the other post, I am guessing this might be a matlab problem.
A relatively simple solution for getting the raw frame data is converting the RAVI video file to raw video file format.
You can use FFmpeg (command line tool) for converting the RAVI to RAW format.
Example:
ffmpeg -y -f avi -i "Sequence_LED_Holder.ravi" -vcodec rawvideo "Sequence_LED_Holder.yuv"
The YUV (raw binary data) file, can be simply read by MATLAB using fread function.
Note: the .yuv is just a convention (used by FFmpeg) for raw video files - the actual pixel format is not YUV, but int16 format.
You can try parsing the RAVI file manually, but using FFmpeg is much simpler.
The raw file format is composed of raw video frames one after the other with no headers.
I our case, each frame is width*height*2 bytes.
The pixel type is int16 (may include negative values).
The IR video frames has no color information.
The colors are just "false colors" created using palette and used for visualization.
The code sample uses a palette from different IR camera manufacture.
Getting the temperature:
I could not find the way to convert the pixel value to the equivalent temperature.
I didn't read the documentation - there is a chance that the conversion is documented somewhere.
The MATLAB code sample applies the following stages:
Convert RAVI file format to RAW video file format using FFmpeg.
Read video frames as [cols, rows] size int16 matrix.
Remove the first line that probably contains data (not pixels).
Use linear contrast stretch - for visualization.
Apply false colors - for visualization.
Display the processed video frame.
Here is the code sample:
%ravi_file_name = 'Brake disc.ravi';
%ravi_file_name = 'Combustion process.ravi';
%ravi_file_name = 'Electronic board.ravi';
%ravi_file_name = 'Sequence_carwheels.ravi';
%ravi_file_name = 'Sequence_drop.ravi';
ravi_file_name = 'Sequence_LED_Holder.ravi';
%ravi_file_name = 'Steel workpiece with hole.ravi';
yuv_file_name = strrep(ravi_file_name, '.ravi', '.yuv'); % Same file name with .yuv extension.
% Get video resolution.
vidinfo = mmfileinfo(ravi_file_name);
cols = vidinfo.Video.Width;
rows = vidinfo.Video.Height;
% Execute ffmpeg (in the system shell) for converting RAVI to raw data file.
% Remark: download FFmpeg if needed, and make sure ffmpeg executable is in the execution path.
if ~exist(yuv_file_name, 'file')
% Remark: For some of the video files, cmdout returns a string with lots of meta-data
[status, cmdout] = system(sprintf('ffmpeg -y -f avi -i "%s" -vcodec rawvideo "%s"', ravi_file_name, yuv_file_name));
if (status ~= 0)
fprintf(cmdout);
error(['Error: ffmpeg status = ', num2str(status)]);
end
end
% Get the number of frames according to file size.
filesize = getfield(dir(yuv_file_name), 'bytes');
n_frames = filesize / (cols*rows*2);
f = fopen(yuv_file_name, 'r');
% Iterate the frames (skip the last frame).
for i = 1:n_frames-1
% Read frame as cols x rows and int16 type.
% The data is signed (int16) and not uint16.
I = fread(f, [cols, rows], '*int16')';
% It looks like the first line contains some data (not pixels).
data_line = I(1, :);
I = I(2:end, :);
% Apply linear stretch - in order to "see something"...
J = imadjust(I, stretchlim(I, [0.02, 0.98]));
% Apply false colors - just for visualization.
K = ColorizeIr(J);
if (i == 1)
figure;
h = imshow(K, []); %h = imshow(J, []);
impixelinfo
else
if ~isvalid(h)
break;
end
h.CData = K; %h.CData = J;
end
pause(0.05);
end
fclose(f);
imwrite(uint16(J+2^15), 'J.tif'); % Write J as uint16 image.
imwrite(K, 'K.png'); % Write K image (last frame).
% Colorize the IR video frame with "false colors".
function J = ColorizeIr(I)
% The palette apply different IR manufacture - don't expect the result to resemble OPTRIS output.
% https://groups.google.com/g/flir-lepton/c/Cm8lGQyspmk
colormapIronBlack = uint8([...
255, 255, 255, 253, 253, 253, 251, 251, 251, 249, 249, 249, 247, 247,...
247, 245, 245, 245, 243, 243, 243, 241, 241, 241, 239, 239, 239, 237,...
237, 237, 235, 235, 235, 233, 233, 233, 231, 231, 231, 229, 229, 229,...
227, 227, 227, 225, 225, 225, 223, 223, 223, 221, 221, 221, 219, 219,...
219, 217, 217, 217, 215, 215, 215, 213, 213, 213, 211, 211, 211, 209,...
209, 209, 207, 207, 207, 205, 205, 205, 203, 203, 203, 201, 201, 201,...
199, 199, 199, 197, 197, 197, 195, 195, 195, 193, 193, 193, 191, 191,...
191, 189, 189, 189, 187, 187, 187, 185, 185, 185, 183, 183, 183, 181,...
181, 181, 179, 179, 179, 177, 177, 177, 175, 175, 175, 173, 173, 173,...
171, 171, 171, 169, 169, 169, 167, 167, 167, 165, 165, 165, 163, 163,...
163, 161, 161, 161, 159, 159, 159, 157, 157, 157, 155, 155, 155, 153,...
153, 153, 151, 151, 151, 149, 149, 149, 147, 147, 147, 145, 145, 145,...
143, 143, 143, 141, 141, 141, 139, 139, 139, 137, 137, 137, 135, 135,...
135, 133, 133, 133, 131, 131, 131, 129, 129, 129, 126, 126, 126, 124,...
124, 124, 122, 122, 122, 120, 120, 120, 118, 118, 118, 116, 116, 116,...
114, 114, 114, 112, 112, 112, 110, 110, 110, 108, 108, 108, 106, 106,...
106, 104, 104, 104, 102, 102, 102, 100, 100, 100, 98, 98, 98, 96, 96,...
96, 94, 94, 94, 92, 92, 92, 90, 90, 90, 88, 88, 88, 86, 86, 86, 84, 84,...
84, 82, 82, 82, 80, 80, 80, 78, 78, 78, 76, 76, 76, 74, 74, 74, 72, 72,...
72, 70, 70, 70, 68, 68, 68, 66, 66, 66, 64, 64, 64, 62, 62, 62, 60, 60,...
60, 58, 58, 58, 56, 56, 56, 54, 54, 54, 52, 52, 52, 50, 50, 50, 48, 48,...
48, 46, 46, 46, 44, 44, 44, 42, 42, 42, 40, 40, 40, 38, 38, 38, 36, 36,...
36, 34, 34, 34, 32, 32, 32, 30, 30, 30, 28, 28, 28, 26, 26, 26, 24, 24,...
24, 22, 22, 22, 20, 20, 20, 18, 18, 18, 16, 16, 16, 14, 14, 14, 12, 12,...
12, 10, 10, 10, 8, 8, 8, 6, 6, 6, 4, 4, 4, 2, 2, 2, 0, 0, 0, 0, 0, 9,...
2, 0, 16, 4, 0, 24, 6, 0, 31, 8, 0, 38, 10, 0, 45, 12, 0, 53, 14, 0,...
60, 17, 0, 67, 19, 0, 74, 21, 0, 82, 23, 0, 89, 25, 0, 96, 27, 0, 103,...
29, 0, 111, 31, 0, 118, 36, 0, 120, 41, 0, 121, 46, 0, 122, 51, 0, 123,...
56, 0, 124, 61, 0, 125, 66, 0, 126, 71, 0, 127, 76, 1, 128, 81, 1, 129,...
86, 1, 130, 91, 1, 131, 96, 1, 132, 101, 1, 133, 106, 1, 134, 111, 1,...
135, 116, 1, 136, 121, 1, 136, 125, 2, 137, 130, 2, 137, 135, 3, 137,...
139, 3, 138, 144, 3, 138, 149, 4, 138, 153, 4, 139, 158, 5, 139, 163,...
5, 139, 167, 5, 140, 172, 6, 140, 177, 6, 140, 181, 7, 141, 186, 7,...
141, 189, 10, 137, 191, 13, 132, 194, 16, 127, 196, 19, 121, 198, 22,...
116, 200, 25, 111, 203, 28, 106, 205, 31, 101, 207, 34, 95, 209, 37,...
90, 212, 40, 85, 214, 43, 80, 216, 46, 75, 218, 49, 69, 221, 52, 64,...
223, 55, 59, 224, 57, 49, 225, 60, 47, 226, 64, 44, 227, 67, 42, 228,...
71, 39, 229, 74, 37, 230, 78, 34, 231, 81, 32, 231, 85, 29, 232, 88,...
27, 233, 92, 24, 234, 95, 22, 235, 99, 19, 236, 102, 17, 237, 106, 14,...
238, 109, 12, 239, 112, 12, 240, 116, 12, 240, 119, 12, 241, 123, 12,...
241, 127, 12, 242, 130, 12, 242, 134, 12, 243, 138, 12, 243, 141, 13,...
244, 145, 13, 244, 149, 13, 245, 152, 13, 245, 156, 13, 246, 160, 13,...
246, 163, 13, 247, 167, 13, 247, 171, 13, 248, 175, 14, 248, 178, 15,...
249, 182, 16, 249, 185, 18, 250, 189, 19, 250, 192, 20, 251, 196, 21,...
251, 199, 22, 252, 203, 23, 252, 206, 24, 253, 210, 25, 253, 213, 27,...
254, 217, 28, 254, 220, 29, 255, 224, 30, 255, 227, 39, 255, 229, 53,...
255, 231, 67, 255, 233, 81, 255, 234, 95, 255, 236, 109, 255, 238, 123,...
255, 240, 137, 255, 242, 151, 255, 244, 165, 255, 246, 179, 255, 248,...
193, 255, 249, 207, 255, 251, 221, 255, 253, 235, 255, 255, 24]);
lutR = colormapIronBlack(1:3:end);
lutG = colormapIronBlack(2:3:end);
lutB = colormapIronBlack(3:3:end);
% Convert I to uint8
I = im2uint8(I);
R = lutR(I+1);
G = lutG(I+1);
B = lutB(I+1);
J = cat(3, R, G, B);
end
Sample output:
Update:
Python code sample using OpenCV (without colorizing):
Using Python and OpenCV, we may skip the FFmpeg conversion part.
Instead of converting the RAVI file to YUV file, we may fetch undecoded RAW video from the RAVI file.
Open a video file and set CAP_PROP_FORMAT property for fetch undecoded RAW video:
cap = cv2.VideoCapture(ravi_file_name)
cap.set(cv2.CAP_PROP_FORMAT, -1) # Format of the Mat objects. Set value -1 to fetch undecoded RAW video streams (as Mat 8UC1).
When reading a video frame (using ret, frame = cap.read()), the undecoded frame is read as a "long" row vector of uint8 elements.
Converting the frame to int16 type, and reshaping to cols x rows:
First, we have to "view" the vector elements as int16 elements (opposed to uint8 elements): frame.view(np.int16)
Second, we have to reshape the vector into a matrix.
Conversion and reshaping code:
frame = frame.view(np.int16).reshape(rows, cols)
Complete Python code sample:
import numpy as np
import cv2
ravi_file_name = 'Sequence_LED_Holder.ravi'
cap = cv2.VideoCapture(ravi_file_name) # Opens a video file for capturing
# Fetch undecoded RAW video streams
cap.set(cv2.CAP_PROP_FORMAT, -1) # Format of the Mat objects. Set value -1 to fetch undecoded RAW video streams (as Mat 8UC1). [Using cap.set(cv2.CAP_PROP_CONVERT_RGB, 0) is not working]
cols = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) # Get video frames width
rows = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) # Get video frames height
while True:
ret, frame = cap.read() # Read next video frame (undecoded frame is read as long row vector).
if not ret:
break # Stop reading frames when ret = False (after the last frame is read).
# View frame as int16 elements, and reshape to cols x rows (each pixel is signed 16 bits)
frame = frame.view(np.int16).reshape(rows, cols)
# It looks like the first line contains some data (not pixels).
# data_line = frame[0, :]
frame_roi = frame[1:, :] # Ignore the first row.
# Normalizing frame to range [0, 255], and get the result as type uint8 (this part is used just for making the data visible).
normed = cv2.normalize(frame_roi, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
cv2.imshow('normed', normed) # Show the normalized video frame
cv2.waitKey(10)
cap.release()
cv2.destroyAllWindows()
Sample output:
Note:
In case colorization is required, you may use the following example: Thermal Image Processing
In most of ravi files processed with Ffmpeg, there are non-pixel values on the first line of the raw image.
This first line stores some redondant information such as image width and height.
We have to skip this line which corresponds to the image width. Since data values are 16-bit, we must multiply by 2, to get the exact offset of the binary data. We have also to calculate the exact size of the image: imageLength = Frame size - (image width * 2).
In the other case, data are from the start of the file and we can use the frame size (w * h * 2) to copy binary data and update the offset.
To know if it's necessary to calculate the data offset, we just look at image height. If this value is odd, that means there is a supplementary first line and thus we apply the correction. If the value is even, no correction for the data offset.
This is the same story when parsing the original ravi files. First we have to find the offset of the movi tag in the file. If the movi tag if followed by the ix00 tag, that means we have just after a series of values that give the offset and the size of each frames from the offset of the movi tag. Real data are elsewere in the file. If ix00 tag is not present, that means that data are just inside the movi chunck, after the 00db flag, and frame by frame. In this last case, we can also look for the idx1 tag (at the end of the file) which gives access to the exact offset and size of each frame.
Both approaches allow a rather correct image representation in grayscale or in pseudo-color, but the temperature formula provided by the libirimager tool-kit (float t = (float)data[i] / 10.f - 100.f) is incorrect and I do not undestand why, since the formula was correct when I was using raw data produced by the PI-160 camera.
Fmmpeg test
I found an alternative way. In recent ravi Optris file we can get the temperature range in the INFO chunk. Then, it's easy to find the minimal and maximal values in raw data and to interpolate in reference to the temperature scale.
with correct temperatures
Each frame holds 16-bit values by pixel with low byte first and high byte after. To find the temperature you have to apply this formula: temp = (hi * 256.0 + lo) / 10.0 - 100.0.
With low value, you can create a grayscale image. I used this approach with old Pi-160 Optris camera with success. However with new PI-450, it's more difficult since PI Connect does not support binary export now.
I tested the solution with Ffmpeg without success. You get a 16-bit data file, but the offset of real data is incorrect, and thus the temperature is aberrant.
Did you succeed ?
Sample of binary reading:

How to encode utf8 code units to String and read back Flutter

My user input is a string, which also has emojis, so I have converted it to a list of utf8 code units.
When I send this response to server, it stores it as a String
"[60, 115, 116, 114, 111, 110, 103, 62, 114, 105, 99, 104, 32, 116,
101, 120, 116, 240, 159, 152, 137, 240, 159, 152, 137, 240, 159, 152,
137, 240, 159, 152, 137, 240, 159, 152, 137, 60, 47, 115, 116, 114,
111, 110, 103, 62, 60, 98, 114, 62, 60, 98, 114, 62]"
but when I try to decode this, I cannot as utf8.decode(CodeUnits) require a List input but my input is now a String.
I am encoding this String to utf8 using:
utf8.encode("My String hereπŸ˜€");
Please let me know if there is some better approach to storing and displaying emojis.
We are using Django Rest Framework in our API.
Calling your string temp, you can convert temp to a List by:
List<int> intList = List<int>.from(json.decode(temp));
Then just convert it to UTF8 using:
String result = new Utf8Decoder().convert(intList);
print(result);
Result:
<strong>rich textπŸ˜‰πŸ˜‰πŸ˜‰πŸ˜‰πŸ˜‰</strong><br><br>
A more dense answer would be:
String result = new Utf8Decoder().convert(List<int>.from(json.decode(temp)))

Median, best fit of line from mesh skeleton

i got a skeleton of mesh, that is needed to determinate the orientation of the mesh, so i can position it correctly in 3d space automatically.
I don't really get how to get best fit of lines, so i can position it afterwards automatically from the set of points.
Below is the example of how it around how it should look and also set of sample data.
//skeleton vertices
vertices = [ 2.06104, 318.734, -149.29;
4.2207, 212.092, -145.141;
4.23213, 200.135, -144.811;
4.16573, 95.1567, -133.954;
4.7053, 126.626, -138.59;
4.16915, 171.645, -143.646;
4.18659, 183.173, -144.185;
4.17842, 179.964, -144.008;
2.76537, 288.063, -147.215;
-1.71817, 61.155, -124.25;
-0.492168, 66.2098, -127.702;
2.07608, 79.5012, -131.886;
4.03249, 238.699, -146.141;
4.23595, 206.822, -145.001;
4.23623, 203.704, -144.908;
4.17145, 220.543, -145.415;
4.12453, 228.514, -145.761;
2.41377, 301.021, -147.804;
4.03098, 236.064, -145.985;
2.48681, 298.432, -147.685;
4.68192, 129.093, -138.873;
2.65048, 292.424, -147.406;
4.54555, 104.737, -135.116;
-4.23707, 53.6538, -113;
1.93508, 317.106, -148.755;
-4.98045, 51.9036, -109.052;
-5.87157, 49.9703, -104.104;
-11.2433, 41.1865, -71.6569;
-15.1283, 30.7528, -33.5845;
-14.6647, 26.7291, -17.9213;
-13.1176, 23.9812, -5.55431;
-8.27057, 20.6161, 13.9826;
-4.49387, 19.0295, 25.4537;
4.5645, 139.344, -140.275;
-3.3737, 56.0265, -117.133;
4.21667, 192.056, -144.576;
3.99948, 93.2911, -133.447;
4.22894, 209.986, -145.081;
4.17824, 167.633, -143.406;
3.94993, 243.303, -146.26;
4.20391, 188.412, -144.431;
4.20673, 214.944, -145.239;
3.82056, 248.85, -146.382;
3.75634, 252.761, -146.523;
3.46093, 268.466, -147.033;
4.11554, 230.181, -145.836;
4.44799, 147.962, -141.303;
4.205, 165.551, -143.138;
4.7514, 117.603, -137.34;
4.25931, 161.613, -142.739;
4.15939, 222.571, -145.502;
4.38497, 152.519, -141.797;
3.12906, 279.268, -147.181;
3.05571, 282.333, -147.279;
-8.35374, 45.7208, -89.6802;
-6.43016, 19.7295, 20.0489;
3.51218, 265.85, -146.95;
4.22735, 196.569, -144.716;
3.60114, 261.275, -146.806;
3.87527, 245.264, -146.243;
3.70115, 255.948, -146.633;
3.33032, 273.433, -147.142;
4.14712, 224.649, -145.592;
4.11102, 231.178, -145.882;
4.17545, 177.258, -143.86;
4.07209, 234.491, -145.985;
4.13698, 226.37, -145.667;
4.50243, 144.003, -140.851;
4.74996, 120.215, -137.661;
3.00397, 283.765, -147.29;
4.22263, 164.279, -143.008;
4.19542, 216.684, -145.284;
4.74419, 122.387, -138.079;
4.24362, 162.754, -142.854;
4.1921, 217.204, -145.299;
3.1988, 276.954, -147.144;
2.24673, 307.266, -148.14;
4.70408, 113.726, -136.704;
1.55558, 75.9859, -131.562;
4.66136, 131.166, -139.157;
3.69856, 90.4963, -133.113;
4.42481, 149.643, -141.489;
2.02224, 313.586, -148.445;
3.39344, 270.742, -147.067;
-11.4507, 40.824, -70.2682;
-12.6325, 23.3651, -2.57932;
-7.10479, 20.0528, 17.8335;
-5.64725, 19.4808, 22.078;
-4.64392, 19.0822, 25.039;
2.02817, 318.148, -149.125;
2.33964, 303.688, -147.933;
3.66401, 257.998, -146.701;
4.63971, 109.538, -135.974;
1.92836, 315.331, -148.48;
-14.1808, 25.5312, -12.8091;
-9.41149, 21.1265, 10.4194;
-4.95978, 19.1884, 24.1908;
2.59159, 294.625, -147.508;
2.13575, 310.184, -148.256;
2.70921, 290.146, -147.303;
4.27225, 160.675, -142.644;
4.26101, 97.2627, -134.029;
4.48821, 103.188, -134.891;
4.49926, 103.485, -134.935;
-7.047, 47.7435, -97.5736;
-6.3594, 48.9598, -101.483;
-9.77383, 43.5659, -81.0321;
-8.9763, 44.7772, -85.911;
-10.8234, 41.8961, -74.4074;
-13.9937, 35.5167, -50.7962;
-14.3038, 34.7247, -47.9571;
-14.8257, 32.9699, -41.6747;
-14.6745, 33.5497, -43.7501;
-11.2159, 22.148, 3.98751;
-11.8022, 22.5945, 1.47664;
-8.85457, 20.8696, 12.1895;
-10.131, 21.5096, 7.94287;
4.63046, 134.052, -139.576;
-10.0719, 43.1032, -79.1781;
-10.7667, 21.8667, 5.68727;
-12.6587, 38.5085, -61.6436;
3.95535, 241.725, -146.174;
-15.1097, 31.1952, -35.2305;
-14.9267, 27.8269, -22.3608;
-15.0656, 31.6803, -37.016;
-9.86649, 21.3662, 8.86292;
4.58261, 106.635, -135.457;
-13.3958, 36.9225, -55.8701;
-14.8467, 27.4234, -20.75;
-9.26049, 44.3478, -84.1828;
-9.51023, 43.9693, -82.6548;
-13.9657, 25.1136, -10.9592;
-14.4925, 34.1899, -46.0425;
-10.5672, 42.3162, -76.0544;
-11.5984, 40.5562, -69.2552;
-0.993421, 64.013, -126.478;
0.765767, 72.1493, -130.141;
-2.1096, 59.7809, -122.798;
-12.1454, 39.5104, -65.3578;
-13.062, 37.65, -58.5144;
-6.68714, 48.3172, -99.7383;
4.33504, 156.127, -142.176;
4.39741, 100.816, -134.536;
2.55245, 82.6571, -132.286;
-14.4223, 26.1384, -15.3963;
-1.22434, 63.1015, -125.766;
-4.58353, 52.82, -111.184;
-5.20508, 51.4016, -107.822;
1.17634, 74.0686, -130.953;
-7.37625, 47.2239, -95.5894;
-14.9706, 32.2558, -39.1074;
-15.1378, 29.7063, -29.6614;
-13.3868, 24.326, -7.21636;
-2.6613, 58.1303, -120.338;
-5.48993, 50.7878, -106.232;
-15.0685, 28.7501, -25.9925;
-13.7052, 36.2122, -53.3032;
0.437576, 70.5848, -129.527;
];
//skeleton lines
lines = [
93, 24;
56, 44;
0, 89;
42, 59;
7, 64;
17, 90;
62, 50;
39, 59;
48, 77;
3, 36;
13, 14;
10, 157;
20, 79;
19, 17;
43, 60;
76, 90;
157, 136;
4, 72;
79, 117;
14, 2;
7, 6;
40, 6;
62, 66;
135, 10;
49, 73;
38, 5;
61, 75;
137, 153;
78, 148;
2, 57;
38, 47;
117, 33;
61, 83;
35, 40;
9, 137;
58, 56;
153, 34;
1, 41;
9, 145;
39, 121;
82, 98;
25, 147;
25, 146;
23, 34;
8, 99;
23, 146;
60, 91;
149, 54;
108, 27;
156, 127;
139, 120;
112, 132;
109, 110;
150, 111;
124, 122;
151, 28;
150, 124;
155, 123;
151, 155;
29, 144;
128, 123;
114, 85;
30, 85;
114, 113;
95, 115;
115, 31;
116, 125;
95, 125;
31, 86;
55, 86;
87, 96;
88, 96;
52, 53;
1, 37;
58, 91;
70, 47;
8, 69;
5, 64;
53, 69;
126, 22;
15, 50;
45, 16;
65, 63;
45, 63;
13, 37;
141, 51;
57, 35;
15, 74;
71, 74;
48, 68;
72, 68;
46, 67;
20, 4;
70, 73;
141, 100;
16, 66;
18, 12;
52, 75;
18, 65;
46, 81;
42, 43;
121, 12;
41, 71;
24, 89;
22, 103;
77, 92;
21, 97;
36, 80;
126, 92;
21, 99;
33, 67;
102, 103;
44, 83;
76, 98;
118, 133;
87, 55;
101, 3;
51, 81;
49, 100;
136, 148;
78, 11;
142, 101;
82, 93;
19, 97;
104, 140;
140, 105;
104, 149;
106, 118;
129, 107;
106, 130;
129, 130;
134, 84;
27, 84;
138, 134;
138, 120;
132, 110;
112, 111;
116, 119;
26, 105;
131, 94;
154, 147;
54, 107;
108, 133;
88, 32;
156, 109;
135, 145;
127, 139;
29, 128;
113, 119;
28, 122;
143, 80;
152, 30;
142, 102;
144, 94;
131, 152;
143, 11;
26, 154;
];
I would:
divide the mesh into separate polylines
and process each separately. So you need to reorder the points pnt[] so they are in the same topological order (just sequence of points no line[] structure).
compute |pnt[i]-sliding_average(pnt[i])|
the value will give you distance from sliding average so threshold it ... if too big divide your polyline.
Sliding average (averaging 10 points before and 10 after) for your data looks like this (red):
As threshold distance I used sqrt(20) so here start end endpoints of the individual polylines detected (yellow):
regress lines
after #2 your set should contain individual line segments. To improve precision ignore corner points (near edges of polyline) as it could contain points from next/previous line segment.
I choose to ignore 10 points from beginning and 10 from end. I computed avg position of the inside points. As direction I use difference between inside endpoints rescaled to size of outside endpoints. From that the regressed line is just avg position +/- half direction vector.
You can chose what ever fitting or regression of lines you want depends on the properties of result you want to achieve.
compute intersection between neighboring lines
reconstruct mesh
Here simple C++ code (sorry I do not use Matlab) for your data covering #1,#2,#3 without the recursive subdivision (branching of polyline) so you can see how the things are done in case the above description is not enough.
//---------------------------------------------------------------------------
//skeleton vertices
double pnt[]=
{
2.06104, 318.734, -149.29,
4.2207, 212.092, -145.141,
4.23213, 200.135, -144.811,
4.16573, 95.1567, -133.954,
4.7053, 126.626, -138.59,
4.16915, 171.645, -143.646,
4.18659, 183.173, -144.185,
4.17842, 179.964, -144.008,
2.76537, 288.063, -147.215,
-1.71817, 61.155, -124.25,
-0.492168, 66.2098, -127.702,
2.07608, 79.5012, -131.886,
4.03249, 238.699, -146.141,
4.23595, 206.822, -145.001,
4.23623, 203.704, -144.908,
4.17145, 220.543, -145.415,
4.12453, 228.514, -145.761,
2.41377, 301.021, -147.804,
4.03098, 236.064, -145.985,
2.48681, 298.432, -147.685,
4.68192, 129.093, -138.873,
2.65048, 292.424, -147.406,
4.54555, 104.737, -135.116,
-4.23707, 53.6538, -113,
1.93508, 317.106, -148.755,
-4.98045, 51.9036, -109.052,
-5.87157, 49.9703, -104.104,
-11.2433, 41.1865, -71.6569,
-15.1283, 30.7528, -33.5845,
-14.6647, 26.7291, -17.9213,
-13.1176, 23.9812, -5.55431,
-8.27057, 20.6161, 13.9826,
-4.49387, 19.0295, 25.4537,
4.5645, 139.344, -140.275,
-3.3737, 56.0265, -117.133,
4.21667, 192.056, -144.576,
3.99948, 93.2911, -133.447,
4.22894, 209.986, -145.081,
4.17824, 167.633, -143.406,
3.94993, 243.303, -146.26,
4.20391, 188.412, -144.431,
4.20673, 214.944, -145.239,
3.82056, 248.85, -146.382,
3.75634, 252.761, -146.523,
3.46093, 268.466, -147.033,
4.11554, 230.181, -145.836,
4.44799, 147.962, -141.303,
4.205, 165.551, -143.138,
4.7514, 117.603, -137.34,
4.25931, 161.613, -142.739,
4.15939, 222.571, -145.502,
4.38497, 152.519, -141.797,
3.12906, 279.268, -147.181,
3.05571, 282.333, -147.279,
-8.35374, 45.7208, -89.6802,
-6.43016, 19.7295, 20.0489,
3.51218, 265.85, -146.95,
4.22735, 196.569, -144.716,
3.60114, 261.275, -146.806,
3.87527, 245.264, -146.243,
3.70115, 255.948, -146.633,
3.33032, 273.433, -147.142,
4.14712, 224.649, -145.592,
4.11102, 231.178, -145.882,
4.17545, 177.258, -143.86,
4.07209, 234.491, -145.985,
4.13698, 226.37, -145.667,
4.50243, 144.003, -140.851,
4.74996, 120.215, -137.661,
3.00397, 283.765, -147.29,
4.22263, 164.279, -143.008,
4.19542, 216.684, -145.284,
4.74419, 122.387, -138.079,
4.24362, 162.754, -142.854,
4.1921, 217.204, -145.299,
3.1988, 276.954, -147.144,
2.24673, 307.266, -148.14,
4.70408, 113.726, -136.704,
1.55558, 75.9859, -131.562,
4.66136, 131.166, -139.157,
3.69856, 90.4963, -133.113,
4.42481, 149.643, -141.489,
2.02224, 313.586, -148.445,
3.39344, 270.742, -147.067,
-11.4507, 40.824, -70.2682,
-12.6325, 23.3651, -2.57932,
-7.10479, 20.0528, 17.8335,
-5.64725, 19.4808, 22.078,
-4.64392, 19.0822, 25.039,
2.02817, 318.148, -149.125,
2.33964, 303.688, -147.933,
3.66401, 257.998, -146.701,
4.63971, 109.538, -135.974,
1.92836, 315.331, -148.48,
-14.1808, 25.5312, -12.8091,
-9.41149, 21.1265, 10.4194,
-4.95978, 19.1884, 24.1908,
2.59159, 294.625, -147.508,
2.13575, 310.184, -148.256,
2.70921, 290.146, -147.303,
4.27225, 160.675, -142.644,
4.26101, 97.2627, -134.029,
4.48821, 103.188, -134.891,
4.49926, 103.485, -134.935,
-7.047, 47.7435, -97.5736,
-6.3594, 48.9598, -101.483,
-9.77383, 43.5659, -81.0321,
-8.9763, 44.7772, -85.911,
-10.8234, 41.8961, -74.4074,
-13.9937, 35.5167, -50.7962,
-14.3038, 34.7247, -47.9571,
-14.8257, 32.9699, -41.6747,
-14.6745, 33.5497, -43.7501,
-11.2159, 22.148, 3.98751,
-11.8022, 22.5945, 1.47664,
-8.85457, 20.8696, 12.1895,
-10.131, 21.5096, 7.94287,
4.63046, 134.052, -139.576,
-10.0719, 43.1032, -79.1781,
-10.7667, 21.8667, 5.68727,
-12.6587, 38.5085, -61.6436,
3.95535, 241.725, -146.174,
-15.1097, 31.1952, -35.2305,
-14.9267, 27.8269, -22.3608,
-15.0656, 31.6803, -37.016,
-9.86649, 21.3662, 8.86292,
4.58261, 106.635, -135.457,
-13.3958, 36.9225, -55.8701,
-14.8467, 27.4234, -20.75,
-9.26049, 44.3478, -84.1828,
-9.51023, 43.9693, -82.6548,
-13.9657, 25.1136, -10.9592,
-14.4925, 34.1899, -46.0425,
-10.5672, 42.3162, -76.0544,
-11.5984, 40.5562, -69.2552,
-0.993421, 64.013, -126.478,
0.765767, 72.1493, -130.141,
-2.1096, 59.7809, -122.798,
-12.1454, 39.5104, -65.3578,
-13.062, 37.65, -58.5144,
-6.68714, 48.3172, -99.7383,
4.33504, 156.127, -142.176,
4.39741, 100.816, -134.536,
2.55245, 82.6571, -132.286,
-14.4223, 26.1384, -15.3963,
-1.22434, 63.1015, -125.766,
-4.58353, 52.82, -111.184,
-5.20508, 51.4016, -107.822,
1.17634, 74.0686, -130.953,
-7.37625, 47.2239, -95.5894,
-14.9706, 32.2558, -39.1074,
-15.1378, 29.7063, -29.6614,
-13.3868, 24.326, -7.21636,
-2.6613, 58.1303, -120.338,
-5.48993, 50.7878, -106.232,
-15.0685, 28.7501, -25.9925,
-13.7052, 36.2122, -53.3032,
0.437576, 70.5848, -129.527,
};
const int pnts3=sizeof(pnt)/sizeof(pnt[1]); // number of points * 3
const int pnts=pnts3/3; // number of points
//skeleton lines
int lin[]=
{
93, 24,
56, 44,
0, 89,
42, 59,
7, 64,
17, 90,
62, 50,
39, 59,
48, 77,
3, 36,
13, 14,
10, 157,
20, 79,
19, 17,
43, 60,
76, 90,
157, 136,
4, 72,
79, 117,
14, 2,
7, 6,
40, 6,
62, 66,
135, 10,
49, 73,
38, 5,
61, 75,
137, 153,
78, 148,
2, 57,
38, 47,
117, 33,
61, 83,
35, 40,
9, 137,
58, 56,
153, 34,
1, 41,
9, 145,
39, 121,
82, 98,
25, 147,
25, 146,
23, 34,
8, 99,
23, 146,
60, 91,
149, 54,
108, 27,
156, 127,
139, 120,
112, 132,
109, 110,
150, 111,
124, 122,
151, 28,
150, 124,
155, 123,
151, 155,
29, 144,
128, 123,
114, 85,
30, 85,
114, 113,
95, 115,
115, 31,
116, 125,
95, 125,
31, 86,
55, 86,
87, 96,
88, 96,
52, 53,
1, 37,
58, 91,
70, 47,
8, 69,
5, 64,
53, 69,
126, 22,
15, 50,
45, 16,
65, 63,
45, 63,
13, 37,
141, 51,
57, 35,
15, 74,
71, 74,
48, 68,
72, 68,
46, 67,
20, 4,
70, 73,
141, 100,
16, 66,
18, 12,
52, 75,
18, 65,
46, 81,
42, 43,
121, 12,
41, 71,
24, 89,
22, 103,
77, 92,
21, 97,
36, 80,
126, 92,
21, 99,
33, 67,
102, 103,
44, 83,
76, 98,
118, 133,
87, 55,
101, 3,
51, 81,
49, 100,
136, 148,
78, 11,
142, 101,
82, 93,
19, 97,
104, 140,
140, 105,
104, 149,
106, 118,
129, 107,
106, 130,
129, 130,
134, 84,
27, 84,
138, 134,
138, 120,
132, 110,
112, 111,
116, 119,
26, 105,
131, 94,
154, 147,
54, 107,
108, 133,
88, 32,
156, 109,
135, 145,
127, 139,
29, 128,
113, 119,
28, 122,
143, 80,
152, 30,
142, 102,
144, 94,
131, 152,
143, 11,
26, 154,
};
const int lins2=sizeof(lin)/sizeof(lin[1]); // number of lines * 2
const int lins=lins2/2; // number of lines
//---------------------------------------------------------------------------
int pol[pnts],pols=0; // polyline
double avg[pnts3]; // sliding average
//---------------------------------------------------------------------------
const int _mesh=100;
double mesh[_mesh*3];
int meshs=0,meshs3=0;
//---------------------------------------------------------------------------
void compute()
{
int i,j,e,n,i0,i1;
int his[pnts];
int used[lins];
double x,y,z,w,rr,ll;
//--- compute polyline pol[pols] ----------------------------------------
// histogram of point usagge
for (i=0;i<pnts;i++) his[i]=0;
for (i=0;i<lins2;i++) his[lin[i]]++;
// clear tables
for (i=0;i<pnts;i++) pol[i]=-1; pols=0;
for (i=0;i<lins;i++) used[i]=0;
// find start point (his[i]!=2)
for (e=1,i=0;i<pnts;i++)
if ((his[i])&&(his[i]!=2))
{ e=0; break; }
if (e) return; // stop if none found
// add start point to polyline
pol[pols]=i; pols++;
// process polyline
for (e=1;e;)
for (e=0,j=0;j<lins;j++)
if (!used[j]) // ignore used lines
{
// is point i in line j ?
if (lin[j+j+0]==i) i=lin[j+j+1];
else if (lin[j+j+1]==i) i=lin[j+j+0];
else continue;
// add next point to polyline
pol[pols]=i; pols++;
used[j]=1;
if (his[i]==2) e=1; // loop if not end of polyline
break;
}
//--- compute sliding average -------------------------------------------
n=10; // sliding average half interval [poins]
w=1+n+n; w=1.0/w;
for (i=0;i<pols;i++)
{
e=3*pol[i];
x=pnt[e]; e++;
y=pnt[e]; e++;
z=pnt[e]; e++;
for (j=1;j<=n;j++)
{
e=i+j; if (e>=pols) e=pols-1;
e=3*pol[e];
x+=pnt[e]; e++;
y+=pnt[e]; e++;
z+=pnt[e]; e++;
e=i-j; if (e<0) e=0;
e=3*pol[e];
x+=pnt[e]; e++;
y+=pnt[e]; e++;
z+=pnt[e]; e++;
}
e=3*i;
avg[e]=x*w; e++;
avg[e]=y*w; e++;
avg[e]=z*w; e++;
}
//--- regress lines -----------------------------------------------------
meshs=0; meshs3=0;
ll=20.0;
for (e=0,i=0;i<pols;i++)
{
// distance to sliding average
j=3*pol[i];
x=pnt[j]; j++;
y=pnt[j]; j++;
z=pnt[j]; j++;
j=3*i;
x-=avg[j]; j++; x*=x;
y-=avg[j]; j++; y*=y;
z-=avg[j]; j++; z*=z;
rr=x+y+z;
if ((e==0)&&(rr<=ll)){ i0=i; e++; }// find start point
else if ((e==1)&&(rr> ll)) // find end point
{
i1=i-1; e=0; if (i0==i1) continue;
// regress line from pol[i0..i1]
int j0,j1;
double dx,dy,dz;
// ignore n edge points
n=10; while ((i1-i0)<(n<<2)) n--;
j0=i0+n;
j1=i1-n;
// direction
j=3*pol[j1];
dx=pnt[j]; j++;
dy=pnt[j]; j++;
dz=pnt[j]; j++;
j=3*pol[j0];
dx-=pnt[j]; j++;
dy-=pnt[j]; j++;
dz-=pnt[j]; j++;
// original line size
j=3*pol[i1];
x=pnt[j]; j++;
y=pnt[j]; j++;
z=pnt[j]; j++;
j=3*pol[i0];
x-=pnt[j]; j++;
y-=pnt[j]; j++;
z-=pnt[j]; j++;
// rescale direction to match original half size
rr=sqrt((x*x)+(y*y)+(z*z));
rr=divide(rr,sqrt((dx*dx)+(dy*dy)+(dz*dz)));
rr*=0.5;
dx*=rr;
dy*=rr;
dz*=rr;
// avg position
for (x=y=z=0.0,n=0,i=j0;i<=j1;i++)
{
j=3*pol[i];
x+=pnt[j]; j++;
y+=pnt[j]; j++;
z+=pnt[j]; j++;
n++;
}
x/=n;
y/=n;
z/=n;
// line is avg+/-half direction
mesh[meshs3]=x-dx; meshs3++; meshs++;
mesh[meshs3]=y-dy; meshs3++;
mesh[meshs3]=z-dz; meshs3++;
mesh[meshs3]=x+dx; meshs3++; meshs++;
mesh[meshs3]=y+dy; meshs3++;
mesh[meshs3]=z+dz; meshs3++;
// restore main loop
i=i1+1; e=0;
}
}
i=0;
}
//---------------------------------------------------------------------------
the mesh[meshs3] holds 3D points coordinates of the resulting polyline (2 points per line) meshs is the number of points so num_of_lines=meshs/2

How can I read an hex number with dlmread?

I'm trying to read a .csv file with Octave (I suppose it's equivalent on Matlab). One of the columns contains hexadecimal values identifying MAC addresses, but I'd like to have it parsed anyway, I don't mind if it's converted to decimal.
Is it possible to do this automatically with functions such as dlmread? Or do I have to create a custom function?
This is how the file looks like:
Timestamp, MAC, LastBsn, PRR, RSSI, ED, SQI, RxGain, PtxCoord, Channel: 26
759, 0x35c8cc, 127, 99, -307, 29, 237, 200, -32
834, 0x32d710, 183, 100, -300, 55, 248, 200, -32
901, 0x35c8cc, 227, 100, -300, 29, 238, 200, -32
979, 0x32d6a0, 22, 95, -336, 10, 171, 200, -32
987, 0x32d710, 27, 96, -328, 54, 249, 200, -32
1054, 0x35c8cc, 71, 92, -357, 30, 239, 200, -32
1133, 0x32d6a0, 122, 95, -336, 11, 188, 200, -32
I can accept any output value for the (truncated) MAC addresses, from sequence numbers (1-6) to decimal conversion of the value (e.g. 0x35c8cc -> 3524812).
My current workaround is to use a text editor to manually replace the MAC addresses with decimal numbers, but an automated solution would be handy.
The functions dlmread and csvread will handle numeric files. You can use textscan (which is also present in Matlab), but since you're using Octave, you're better off using csv2cell (part of Octave's io package). It basically reads a csv file and returns a cell array of strings and doubles:
octave-3.8.1> type test.csv
1,2,3,"some",1c:6f:65:90:6b:13
4,5,6,"text",0d:5a:89:46:5c:70
octave-3.8.1> plg load io; # csv2cell is part of the io package
octave-3.8.1> data = csv2cell ("test.csv")
data =
{
[1,1] = 1
[2,1] = 4
[1,2] = 2
[2,2] = 5
[1,3] = 3
[2,3] = 6
[1,4] = some
[2,4] = text
[1,5] = 1c:6f:65:90:6b:13
[2,5] = 0d:5a:89:46:5c:70
}
octave-3.8.1> class (data{1})
ans = double
octave-3.8.1> class (data{9})
ans = char
>> type mycsv.csv
Timestamp, MAC, LastBsn, PRR, RSSI, ED, SQI, RxGain, PtxCoord, Channel: 26
759, 0x35c8cc, 127, 99, -307, 29, 237, 200, -32
834, 0x32d710, 183, 100, -300, 55, 248, 200, -32
901, 0x35c8cc, 227, 100, -300, 29, 238, 200, -32
979, 0x32d6a0, 22, 95, -336, 10, 171, 200, -32
987, 0x32d710, 27, 96, -328, 54, 249, 200, -32
1054, 0x35c8cc, 71, 92, -357, 30, 239, 200, -32
1133, 0x32d6a0, 122, 95, -336, 11, 188, 200, -32
You can read the file with csv2cell. The values starting with "0x" will be automatically converted from hex to decimal values. See:
>> pkg load io % load io package for csv2cell
>> data = csv2cell ("mycsv.csv");
>> data(2,1)
ans =
{
[1,1] = 759
}
To access the cell values use:
>> data{2,1}
ans = 759
>> data{2,2}
ans = 3524812
>> data{2,5}
ans = -307