snmpv3 getone fails while trying via pysnmp (WrongValueError) - pysnmp

while trying to run get one with pysnmp for snmpv3, getting below error
pysnmp.smi.error.WrongValueError: WrongValueError({'msg': WrongValueError(), 'name': (1, 3, 6, 1, 6, 3, 15, 1, 2, 2, 1, 5, 24, 48, 48, 48, 48, 49, 100, 51, 98, 48, 48, 48, 48, 55, 53, 100, 49, 97, 99, 49, 48, 48, 49, 48, 49, 5, 107, 107, 48, 51, 48), 'idx': 3})
from pysnmp.hlapi import *
errorIndication, errorStatus, errorIndex, varBinds = next(
getCmd(SnmpEngine(),
UsmUserData('USERNAME',authKey='AUTHKEY', privKey='PRIVKEY', authProtocol='usmHMACSHAAuthProtocol', privProtocol='usmAESCfb256Protocol',
securityEngineId=OctetString(hexValue='0000303010')),
UdpTransportTarget(('<IP-ADDR>', <PORT>)),
ContextData(),
ObjectType(ObjectIdentity('<MIB-FILE-NAME>','<MIB-NAME>',<INDEX>)))
The same code works for SNMP-V2 with community string instead of UsmUserData. However, not working for SNMP-V3.
The traceback is too long and no clue
File "supy.py", line 15, in <module>
ObjectType(ObjectIdentity('<MIB-FILE-NAME>','<MIB-NAME>',<INDEX>)))
File "/usr/lib/python2.7/site-packages/pysnmp/hlapi/asyncore/sync/cmdgen.py", line 111, in getCmd
lookupMib=options.get('lookupMib', True)))
File "/usr/lib/python2.7/site-packages/pysnmp/hlapi/asyncore/cmdgen.py", line 124, in getCmd
addrName, paramsName = lcd.configure(snmpEngine, authData, transportTarget)
File "/usr/lib/python2.7/site-packages/pysnmp/hlapi/lcd.py", line 60, in configure
securityName=authData.securityName
File "/usr/lib/python2.7/site-packages/pysnmp/entity/config.py", line 159, in addV3User
(usmUserEntry.name + (13,) + tblIdx1, 'createAndGo'))
File "/usr/lib/python2.7/site-packages/pysnmp/smi/instrum.py", line 256, in writeVars
return self.flipFlopFsm(self.fsmWriteVar, varBinds, acInfo)
File "/usr/lib/python2.7/site-packages/pysnmp/smi/instrum.py", line 239, in flipFlopFsm
raise origExc
pysnmp.smi.error.WrongValueError: WrongValueError({'msg': WrongValueError(), 'name': (1, 3, 6, 1, 6, 3, 15, 1, 2, 2, 1, 5, 24, 48, 48, 48, 48, 49, 100, 51, 98, 48, 48, 48, 48, 55, 53, 100, 49, 97, 99, 49, 48, 48, 49, 48, 49, 5, 107, 107, 48, 51, 48), 'idx': 3})
Please help us with some clue here.

Make sure your authentication and private keys comply with the minimum length required by the underlying crypto algorithms. Perhaps the keys should be at least 8+ characters for any algorithm.

Please find below link:
https://pysnmp.readthedocs.io/en/latest/docs/api-reference.html#high-level-v3arch-asyncore
Please check and ensure that authProtocol & privProtocol are not mentioned as strings but as set of numbers for example: instead of using 'des' as authprotocol use (1, 3, 6, 1, 6, 3, 10, 1, 2, 2).
enter image description here

Related

Extracting Temperatures from .ravi file in Matlab

My Problem
Much like the post here: How can I get data from 'ravi' file?, I have a .ravi file (a radiometric video file, which is rather similar to an .avi) and I am trying to extract the Temperatures in it, to use them together with additional sensor data.
A Sample File can be found in the documentation (http://infrarougekelvin.com/en/optris-logiciel-eng/) when you download the "PIX Connect Software". Unfortunately, according to the documentation, the temperature information is stored in a 16 Bit Format, that Matlab seems to be rather unhappy with.
How I tried to solve my problem
I tried to follow the instructions from the before mentioned post, but I somehow struggle to reach results, which are even close to the correct temperatures.Original Picture with temperatures in the Optris Software
I tried to read the video with different methods:
At first I hoped to use the videorecorder Feature in Matlab:
video = VideoReader(videoPath);
frame1 = video.read(1);
imagesc(frame1)
But it only resulted in this poor picture, which is exactly what I can see, when I try to play the .ravi file in a media player like vlc.
First try with videorecorder function
Then I tried to look at the binary representation of my file and noticed, that I could separate the frames at a certain marker Beginning of a new frame in binary representation
So I tried to read the file with the matlab fread function:
fileID = fopen(videoPath);
[headerInfo,~] = fread(fileID,[1,123392],'uint8');
[imageMatrix,count] = fread(fileID,[video.width, video.height],'uint16', 'b');
imagesc(imageMatrix')
Now the image looks better, and you can at least see the brake disc, but it seems, as if the higher temperatures have some kind of offset, that is stil missing, for the picture to be right.
Also, the values that I read from the file are nowhere near actual temperatures, as the other post and the documentation suggests.
Getting somewhere!
My Question
Am I somehow missing something important? Could someone point me in the right direction, where to look or how to get the actual temperatures from my video? As it worked with the cpp code in the other post, I am guessing this might be a matlab problem.
A relatively simple solution for getting the raw frame data is converting the RAVI video file to raw video file format.
You can use FFmpeg (command line tool) for converting the RAVI to RAW format.
Example:
ffmpeg -y -f avi -i "Sequence_LED_Holder.ravi" -vcodec rawvideo "Sequence_LED_Holder.yuv"
The YUV (raw binary data) file, can be simply read by MATLAB using fread function.
Note: the .yuv is just a convention (used by FFmpeg) for raw video files - the actual pixel format is not YUV, but int16 format.
You can try parsing the RAVI file manually, but using FFmpeg is much simpler.
The raw file format is composed of raw video frames one after the other with no headers.
I our case, each frame is width*height*2 bytes.
The pixel type is int16 (may include negative values).
The IR video frames has no color information.
The colors are just "false colors" created using palette and used for visualization.
The code sample uses a palette from different IR camera manufacture.
Getting the temperature:
I could not find the way to convert the pixel value to the equivalent temperature.
I didn't read the documentation - there is a chance that the conversion is documented somewhere.
The MATLAB code sample applies the following stages:
Convert RAVI file format to RAW video file format using FFmpeg.
Read video frames as [cols, rows] size int16 matrix.
Remove the first line that probably contains data (not pixels).
Use linear contrast stretch - for visualization.
Apply false colors - for visualization.
Display the processed video frame.
Here is the code sample:
%ravi_file_name = 'Brake disc.ravi';
%ravi_file_name = 'Combustion process.ravi';
%ravi_file_name = 'Electronic board.ravi';
%ravi_file_name = 'Sequence_carwheels.ravi';
%ravi_file_name = 'Sequence_drop.ravi';
ravi_file_name = 'Sequence_LED_Holder.ravi';
%ravi_file_name = 'Steel workpiece with hole.ravi';
yuv_file_name = strrep(ravi_file_name, '.ravi', '.yuv'); % Same file name with .yuv extension.
% Get video resolution.
vidinfo = mmfileinfo(ravi_file_name);
cols = vidinfo.Video.Width;
rows = vidinfo.Video.Height;
% Execute ffmpeg (in the system shell) for converting RAVI to raw data file.
% Remark: download FFmpeg if needed, and make sure ffmpeg executable is in the execution path.
if ~exist(yuv_file_name, 'file')
% Remark: For some of the video files, cmdout returns a string with lots of meta-data
[status, cmdout] = system(sprintf('ffmpeg -y -f avi -i "%s" -vcodec rawvideo "%s"', ravi_file_name, yuv_file_name));
if (status ~= 0)
fprintf(cmdout);
error(['Error: ffmpeg status = ', num2str(status)]);
end
end
% Get the number of frames according to file size.
filesize = getfield(dir(yuv_file_name), 'bytes');
n_frames = filesize / (cols*rows*2);
f = fopen(yuv_file_name, 'r');
% Iterate the frames (skip the last frame).
for i = 1:n_frames-1
% Read frame as cols x rows and int16 type.
% The data is signed (int16) and not uint16.
I = fread(f, [cols, rows], '*int16')';
% It looks like the first line contains some data (not pixels).
data_line = I(1, :);
I = I(2:end, :);
% Apply linear stretch - in order to "see something"...
J = imadjust(I, stretchlim(I, [0.02, 0.98]));
% Apply false colors - just for visualization.
K = ColorizeIr(J);
if (i == 1)
figure;
h = imshow(K, []); %h = imshow(J, []);
impixelinfo
else
if ~isvalid(h)
break;
end
h.CData = K; %h.CData = J;
end
pause(0.05);
end
fclose(f);
imwrite(uint16(J+2^15), 'J.tif'); % Write J as uint16 image.
imwrite(K, 'K.png'); % Write K image (last frame).
% Colorize the IR video frame with "false colors".
function J = ColorizeIr(I)
% The palette apply different IR manufacture - don't expect the result to resemble OPTRIS output.
% https://groups.google.com/g/flir-lepton/c/Cm8lGQyspmk
colormapIronBlack = uint8([...
255, 255, 255, 253, 253, 253, 251, 251, 251, 249, 249, 249, 247, 247,...
247, 245, 245, 245, 243, 243, 243, 241, 241, 241, 239, 239, 239, 237,...
237, 237, 235, 235, 235, 233, 233, 233, 231, 231, 231, 229, 229, 229,...
227, 227, 227, 225, 225, 225, 223, 223, 223, 221, 221, 221, 219, 219,...
219, 217, 217, 217, 215, 215, 215, 213, 213, 213, 211, 211, 211, 209,...
209, 209, 207, 207, 207, 205, 205, 205, 203, 203, 203, 201, 201, 201,...
199, 199, 199, 197, 197, 197, 195, 195, 195, 193, 193, 193, 191, 191,...
191, 189, 189, 189, 187, 187, 187, 185, 185, 185, 183, 183, 183, 181,...
181, 181, 179, 179, 179, 177, 177, 177, 175, 175, 175, 173, 173, 173,...
171, 171, 171, 169, 169, 169, 167, 167, 167, 165, 165, 165, 163, 163,...
163, 161, 161, 161, 159, 159, 159, 157, 157, 157, 155, 155, 155, 153,...
153, 153, 151, 151, 151, 149, 149, 149, 147, 147, 147, 145, 145, 145,...
143, 143, 143, 141, 141, 141, 139, 139, 139, 137, 137, 137, 135, 135,...
135, 133, 133, 133, 131, 131, 131, 129, 129, 129, 126, 126, 126, 124,...
124, 124, 122, 122, 122, 120, 120, 120, 118, 118, 118, 116, 116, 116,...
114, 114, 114, 112, 112, 112, 110, 110, 110, 108, 108, 108, 106, 106,...
106, 104, 104, 104, 102, 102, 102, 100, 100, 100, 98, 98, 98, 96, 96,...
96, 94, 94, 94, 92, 92, 92, 90, 90, 90, 88, 88, 88, 86, 86, 86, 84, 84,...
84, 82, 82, 82, 80, 80, 80, 78, 78, 78, 76, 76, 76, 74, 74, 74, 72, 72,...
72, 70, 70, 70, 68, 68, 68, 66, 66, 66, 64, 64, 64, 62, 62, 62, 60, 60,...
60, 58, 58, 58, 56, 56, 56, 54, 54, 54, 52, 52, 52, 50, 50, 50, 48, 48,...
48, 46, 46, 46, 44, 44, 44, 42, 42, 42, 40, 40, 40, 38, 38, 38, 36, 36,...
36, 34, 34, 34, 32, 32, 32, 30, 30, 30, 28, 28, 28, 26, 26, 26, 24, 24,...
24, 22, 22, 22, 20, 20, 20, 18, 18, 18, 16, 16, 16, 14, 14, 14, 12, 12,...
12, 10, 10, 10, 8, 8, 8, 6, 6, 6, 4, 4, 4, 2, 2, 2, 0, 0, 0, 0, 0, 9,...
2, 0, 16, 4, 0, 24, 6, 0, 31, 8, 0, 38, 10, 0, 45, 12, 0, 53, 14, 0,...
60, 17, 0, 67, 19, 0, 74, 21, 0, 82, 23, 0, 89, 25, 0, 96, 27, 0, 103,...
29, 0, 111, 31, 0, 118, 36, 0, 120, 41, 0, 121, 46, 0, 122, 51, 0, 123,...
56, 0, 124, 61, 0, 125, 66, 0, 126, 71, 0, 127, 76, 1, 128, 81, 1, 129,...
86, 1, 130, 91, 1, 131, 96, 1, 132, 101, 1, 133, 106, 1, 134, 111, 1,...
135, 116, 1, 136, 121, 1, 136, 125, 2, 137, 130, 2, 137, 135, 3, 137,...
139, 3, 138, 144, 3, 138, 149, 4, 138, 153, 4, 139, 158, 5, 139, 163,...
5, 139, 167, 5, 140, 172, 6, 140, 177, 6, 140, 181, 7, 141, 186, 7,...
141, 189, 10, 137, 191, 13, 132, 194, 16, 127, 196, 19, 121, 198, 22,...
116, 200, 25, 111, 203, 28, 106, 205, 31, 101, 207, 34, 95, 209, 37,...
90, 212, 40, 85, 214, 43, 80, 216, 46, 75, 218, 49, 69, 221, 52, 64,...
223, 55, 59, 224, 57, 49, 225, 60, 47, 226, 64, 44, 227, 67, 42, 228,...
71, 39, 229, 74, 37, 230, 78, 34, 231, 81, 32, 231, 85, 29, 232, 88,...
27, 233, 92, 24, 234, 95, 22, 235, 99, 19, 236, 102, 17, 237, 106, 14,...
238, 109, 12, 239, 112, 12, 240, 116, 12, 240, 119, 12, 241, 123, 12,...
241, 127, 12, 242, 130, 12, 242, 134, 12, 243, 138, 12, 243, 141, 13,...
244, 145, 13, 244, 149, 13, 245, 152, 13, 245, 156, 13, 246, 160, 13,...
246, 163, 13, 247, 167, 13, 247, 171, 13, 248, 175, 14, 248, 178, 15,...
249, 182, 16, 249, 185, 18, 250, 189, 19, 250, 192, 20, 251, 196, 21,...
251, 199, 22, 252, 203, 23, 252, 206, 24, 253, 210, 25, 253, 213, 27,...
254, 217, 28, 254, 220, 29, 255, 224, 30, 255, 227, 39, 255, 229, 53,...
255, 231, 67, 255, 233, 81, 255, 234, 95, 255, 236, 109, 255, 238, 123,...
255, 240, 137, 255, 242, 151, 255, 244, 165, 255, 246, 179, 255, 248,...
193, 255, 249, 207, 255, 251, 221, 255, 253, 235, 255, 255, 24]);
lutR = colormapIronBlack(1:3:end);
lutG = colormapIronBlack(2:3:end);
lutB = colormapIronBlack(3:3:end);
% Convert I to uint8
I = im2uint8(I);
R = lutR(I+1);
G = lutG(I+1);
B = lutB(I+1);
J = cat(3, R, G, B);
end
Sample output:
Update:
Python code sample using OpenCV (without colorizing):
Using Python and OpenCV, we may skip the FFmpeg conversion part.
Instead of converting the RAVI file to YUV file, we may fetch undecoded RAW video from the RAVI file.
Open a video file and set CAP_PROP_FORMAT property for fetch undecoded RAW video:
cap = cv2.VideoCapture(ravi_file_name)
cap.set(cv2.CAP_PROP_FORMAT, -1) # Format of the Mat objects. Set value -1 to fetch undecoded RAW video streams (as Mat 8UC1).
When reading a video frame (using ret, frame = cap.read()), the undecoded frame is read as a "long" row vector of uint8 elements.
Converting the frame to int16 type, and reshaping to cols x rows:
First, we have to "view" the vector elements as int16 elements (opposed to uint8 elements): frame.view(np.int16)
Second, we have to reshape the vector into a matrix.
Conversion and reshaping code:
frame = frame.view(np.int16).reshape(rows, cols)
Complete Python code sample:
import numpy as np
import cv2
ravi_file_name = 'Sequence_LED_Holder.ravi'
cap = cv2.VideoCapture(ravi_file_name) # Opens a video file for capturing
# Fetch undecoded RAW video streams
cap.set(cv2.CAP_PROP_FORMAT, -1) # Format of the Mat objects. Set value -1 to fetch undecoded RAW video streams (as Mat 8UC1). [Using cap.set(cv2.CAP_PROP_CONVERT_RGB, 0) is not working]
cols = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) # Get video frames width
rows = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) # Get video frames height
while True:
ret, frame = cap.read() # Read next video frame (undecoded frame is read as long row vector).
if not ret:
break # Stop reading frames when ret = False (after the last frame is read).
# View frame as int16 elements, and reshape to cols x rows (each pixel is signed 16 bits)
frame = frame.view(np.int16).reshape(rows, cols)
# It looks like the first line contains some data (not pixels).
# data_line = frame[0, :]
frame_roi = frame[1:, :] # Ignore the first row.
# Normalizing frame to range [0, 255], and get the result as type uint8 (this part is used just for making the data visible).
normed = cv2.normalize(frame_roi, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
cv2.imshow('normed', normed) # Show the normalized video frame
cv2.waitKey(10)
cap.release()
cv2.destroyAllWindows()
Sample output:
Note:
In case colorization is required, you may use the following example: Thermal Image Processing
In most of ravi files processed with Ffmpeg, there are non-pixel values on the first line of the raw image.
This first line stores some redondant information such as image width and height.
We have to skip this line which corresponds to the image width. Since data values are 16-bit, we must multiply by 2, to get the exact offset of the binary data. We have also to calculate the exact size of the image: imageLength = Frame size - (image width * 2).
In the other case, data are from the start of the file and we can use the frame size (w * h * 2) to copy binary data and update the offset.
To know if it's necessary to calculate the data offset, we just look at image height. If this value is odd, that means there is a supplementary first line and thus we apply the correction. If the value is even, no correction for the data offset.
This is the same story when parsing the original ravi files. First we have to find the offset of the movi tag in the file. If the movi tag if followed by the ix00 tag, that means we have just after a series of values that give the offset and the size of each frames from the offset of the movi tag. Real data are elsewere in the file. If ix00 tag is not present, that means that data are just inside the movi chunck, after the 00db flag, and frame by frame. In this last case, we can also look for the idx1 tag (at the end of the file) which gives access to the exact offset and size of each frame.
Both approaches allow a rather correct image representation in grayscale or in pseudo-color, but the temperature formula provided by the libirimager tool-kit (float t = (float)data[i] / 10.f - 100.f) is incorrect and I do not undestand why, since the formula was correct when I was using raw data produced by the PI-160 camera.
Fmmpeg test
I found an alternative way. In recent ravi Optris file we can get the temperature range in the INFO chunk. Then, it's easy to find the minimal and maximal values in raw data and to interpolate in reference to the temperature scale.
with correct temperatures
Each frame holds 16-bit values by pixel with low byte first and high byte after. To find the temperature you have to apply this formula: temp = (hi * 256.0 + lo) / 10.0 - 100.0.
With low value, you can create a grayscale image. I used this approach with old Pi-160 Optris camera with success. However with new PI-450, it's more difficult since PI Connect does not support binary export now.
I tested the solution with Ffmpeg without success. You get a 16-bit data file, but the offset of real data is incorrect, and thus the temperature is aberrant.
Did you succeed ?
Sample of binary reading:

How to encode utf8 code units to String and read back Flutter

My user input is a string, which also has emojis, so I have converted it to a list of utf8 code units.
When I send this response to server, it stores it as a String
"[60, 115, 116, 114, 111, 110, 103, 62, 114, 105, 99, 104, 32, 116,
101, 120, 116, 240, 159, 152, 137, 240, 159, 152, 137, 240, 159, 152,
137, 240, 159, 152, 137, 240, 159, 152, 137, 60, 47, 115, 116, 114,
111, 110, 103, 62, 60, 98, 114, 62, 60, 98, 114, 62]"
but when I try to decode this, I cannot as utf8.decode(CodeUnits) require a List input but my input is now a String.
I am encoding this String to utf8 using:
utf8.encode("My String hereπŸ˜€");
Please let me know if there is some better approach to storing and displaying emojis.
We are using Django Rest Framework in our API.
Calling your string temp, you can convert temp to a List by:
List<int> intList = List<int>.from(json.decode(temp));
Then just convert it to UTF8 using:
String result = new Utf8Decoder().convert(intList);
print(result);
Result:
<strong>rich textπŸ˜‰πŸ˜‰πŸ˜‰πŸ˜‰πŸ˜‰</strong><br><br>
A more dense answer would be:
String result = new Utf8Decoder().convert(List<int>.from(json.decode(temp)))

Addition of elements in multiple list corresponding to a key column of RDD

I have PythonRDDs. I have to perform addition of elements for multiple lists. Add element 1 of list 1 to element 1 of list 2 then add to element 1 of list 3.
For Canada, add 47,59,77 as element 1, 97,98,63 as second element and so on.
I tried to flatten the list to add them and tried to convert to dataframe but I failed to do so. And I want to do it through all 3 ways
countryCounts = [
('CANADA','47;97;33;94;6'),
('CANADA','59;98;24;83;3'),
('CANADA','77;63;93;86;62'),
('CHINA','86;71;72;23;27'),
('CHINA','74;69;72;93;7'),
('CHINA','58;99;90;93;41'),
('ENGLAND','40;13;85;75;90'),
('ENGLAND','39;13;33;29;14'),
('ENGLAND','99;88;57;69;49'),
('GERMANY','67;93;90;57;3'),
('GERMANY','9;15;20;19'),
('GERMANY','77;64;46;95;48'),
('INDIA','90;49;91;14;70'),
('INDIA','70;83;38;27;16'),
('INDIA','86;21;19;59;4')
]
countryCountsRdd = sc.parallelize(countryCounts)
countryCountsSplit.collect()
countryCountsGroup=countryCountsSplit.groupByKey().mapValues(list)
countryCountsGroup.collect()
CountsSplit=countryCountsRdd.map(lambda x : (x[0], ",".join(x[1].split(';'))))
countryCountsSplit.collect()
Inputs :
Way 1
[('CANADA', [47, 97, 33, 94, 6]), ('CANADA', [59, 98, 24, 83, 3]), ('CANADA', [77, 63, 93, 86, 62]), ('CHINA', [86, 71, 72, 23, 27]), ('CHINA', [74, 69, 72, 93, 7]), ('CHINA', [58, 99, 90, 93, 41]), ('ENGLAND', [40, 13, 85, 75, 90]), ('ENGLAND', [39, 13, 33, 29, 14]), ('ENGLAND', [99, 88, 57, 69, 49]), ('GERMANY', [67, 93, 90, 57, 3]), ('GERMANY', [9, 15, 20, 19]), ('GERMANY', [77, 64, 46, 95, 48]), ('INDIA', [90, 49, 91, 14, 70]), ('INDIA', [70, 83, 38, 27, 16]), ('INDIA', [86, 21, 19, 59, 4])]
Way 2:
[('CANADA', [[47, 97, 33, 94, 6], [59, 98, 24, 83, 3], [77, 63, 93, 86, 62]]), ('CHINA', [[86, 71, 72, 23, 27], [74, 69, 72, 93, 7], [58, 99, 90, 93, 41]]), ('INDIA', [[90, 49, 91, 14, 70], [70, 83, 38, 27, 16], [86, 21, 19, 59, 4]]), ('ENGLAND', [[40, 13, 85, 75, 90], [39, 13, 33, 29, 14], [99, 88, 57, 69, 49]]), ('GERMANY', [[67, 93, 90, 57, 3], [9, 15, 20, 19], [77, 64, 46, 95, 48]])]
Way 3:
[('CANADA', '47 ,97 ,33 ,94 ,6'), ('CANADA', '59 ,98 ,24 ,83 ,3'), ('CANADA', '77 ,63 ,93 ,86 ,62'), ('CHINA', '86 ,71 ,72 ,23 ,27'), ('CHINA', '74 ,69 ,72 ,93 ,7'), ('CHINA', '58 ,99 ,90 ,93 ,41'), ('ENGLAND', '40 ,13 ,85 ,75 ,90'), ('ENGLAND', '39 ,13 ,33 ,29 ,14'), ('ENGLAND', '99 ,88 ,57 ,69 ,49'), ('GERMANY', '67 ,93 ,90 ,57 ,3'), ('GERMANY', '9 ,15 ,20 ,19'), ('GERMANY', '77 ,64 ,46 ,95 ,48'), ('INDIA', '90 ,49 ,91 ,14 ,70'), ('INDIA', '70 ,83 ,38 ,27 ,16'), ('INDIA', '86 ,21 ,19 ,59 ,4')]
Require same output for all 3 :
[('CANADA','183;258;150;263;71)]
[('CHINA','218,239,234,209,75')]
[('ENGLAND','178,114,175,173,153')]
[('GERMANY','144,166,151,172,70')]
[('INDIA','246,153,148,100,90')]
You want to combine the values for a given key by taking the sum. This is precisely what reduceByKey does. You just need to define an associative and commutative reduce function to combine the values as desired.
def myReducer(a, b):
a, b = map(int, a.split(";")), map(int, b.split(";"))
maxLength = max(len(a), len(b))
if len(a) < len(b):
a = a + [0]*(len(b)-len(a))
elif len(b) < len(a):
b = b + [0]*(len(a)-len(b))
return ";".join([str(a[i] + b[i]) for i in range(maxLength)])
The only real tricky part here is that your sample input lists are not all the same size. In this case, I defined the function to zero pad the shorter list.
Now call reduceByKey:
countryCountsRdd.reduceByKey(myReducer).collect()
#[('CANADA', '183;258;150;263;71'),
# ('CHINA', '218;239;234;209;75'),
# ('INDIA', '246;153;148;100;90'),
# ('ENGLAND', '178;114;175;173;153'),
# ('GERMANY', '153;172;156;171;51')]
So you can do it using a simple reduceByKey operation on RDD.
INPUT RDD - RDD[STRING, LIST]
Output RDD - input.reduceByKey(x,y -> addFunction(x,y))
addFunction (x,y) iterates over the input lists and add the elements index wise and returns the added list

Median, best fit of line from mesh skeleton

i got a skeleton of mesh, that is needed to determinate the orientation of the mesh, so i can position it correctly in 3d space automatically.
I don't really get how to get best fit of lines, so i can position it afterwards automatically from the set of points.
Below is the example of how it around how it should look and also set of sample data.
//skeleton vertices
vertices = [ 2.06104, 318.734, -149.29;
4.2207, 212.092, -145.141;
4.23213, 200.135, -144.811;
4.16573, 95.1567, -133.954;
4.7053, 126.626, -138.59;
4.16915, 171.645, -143.646;
4.18659, 183.173, -144.185;
4.17842, 179.964, -144.008;
2.76537, 288.063, -147.215;
-1.71817, 61.155, -124.25;
-0.492168, 66.2098, -127.702;
2.07608, 79.5012, -131.886;
4.03249, 238.699, -146.141;
4.23595, 206.822, -145.001;
4.23623, 203.704, -144.908;
4.17145, 220.543, -145.415;
4.12453, 228.514, -145.761;
2.41377, 301.021, -147.804;
4.03098, 236.064, -145.985;
2.48681, 298.432, -147.685;
4.68192, 129.093, -138.873;
2.65048, 292.424, -147.406;
4.54555, 104.737, -135.116;
-4.23707, 53.6538, -113;
1.93508, 317.106, -148.755;
-4.98045, 51.9036, -109.052;
-5.87157, 49.9703, -104.104;
-11.2433, 41.1865, -71.6569;
-15.1283, 30.7528, -33.5845;
-14.6647, 26.7291, -17.9213;
-13.1176, 23.9812, -5.55431;
-8.27057, 20.6161, 13.9826;
-4.49387, 19.0295, 25.4537;
4.5645, 139.344, -140.275;
-3.3737, 56.0265, -117.133;
4.21667, 192.056, -144.576;
3.99948, 93.2911, -133.447;
4.22894, 209.986, -145.081;
4.17824, 167.633, -143.406;
3.94993, 243.303, -146.26;
4.20391, 188.412, -144.431;
4.20673, 214.944, -145.239;
3.82056, 248.85, -146.382;
3.75634, 252.761, -146.523;
3.46093, 268.466, -147.033;
4.11554, 230.181, -145.836;
4.44799, 147.962, -141.303;
4.205, 165.551, -143.138;
4.7514, 117.603, -137.34;
4.25931, 161.613, -142.739;
4.15939, 222.571, -145.502;
4.38497, 152.519, -141.797;
3.12906, 279.268, -147.181;
3.05571, 282.333, -147.279;
-8.35374, 45.7208, -89.6802;
-6.43016, 19.7295, 20.0489;
3.51218, 265.85, -146.95;
4.22735, 196.569, -144.716;
3.60114, 261.275, -146.806;
3.87527, 245.264, -146.243;
3.70115, 255.948, -146.633;
3.33032, 273.433, -147.142;
4.14712, 224.649, -145.592;
4.11102, 231.178, -145.882;
4.17545, 177.258, -143.86;
4.07209, 234.491, -145.985;
4.13698, 226.37, -145.667;
4.50243, 144.003, -140.851;
4.74996, 120.215, -137.661;
3.00397, 283.765, -147.29;
4.22263, 164.279, -143.008;
4.19542, 216.684, -145.284;
4.74419, 122.387, -138.079;
4.24362, 162.754, -142.854;
4.1921, 217.204, -145.299;
3.1988, 276.954, -147.144;
2.24673, 307.266, -148.14;
4.70408, 113.726, -136.704;
1.55558, 75.9859, -131.562;
4.66136, 131.166, -139.157;
3.69856, 90.4963, -133.113;
4.42481, 149.643, -141.489;
2.02224, 313.586, -148.445;
3.39344, 270.742, -147.067;
-11.4507, 40.824, -70.2682;
-12.6325, 23.3651, -2.57932;
-7.10479, 20.0528, 17.8335;
-5.64725, 19.4808, 22.078;
-4.64392, 19.0822, 25.039;
2.02817, 318.148, -149.125;
2.33964, 303.688, -147.933;
3.66401, 257.998, -146.701;
4.63971, 109.538, -135.974;
1.92836, 315.331, -148.48;
-14.1808, 25.5312, -12.8091;
-9.41149, 21.1265, 10.4194;
-4.95978, 19.1884, 24.1908;
2.59159, 294.625, -147.508;
2.13575, 310.184, -148.256;
2.70921, 290.146, -147.303;
4.27225, 160.675, -142.644;
4.26101, 97.2627, -134.029;
4.48821, 103.188, -134.891;
4.49926, 103.485, -134.935;
-7.047, 47.7435, -97.5736;
-6.3594, 48.9598, -101.483;
-9.77383, 43.5659, -81.0321;
-8.9763, 44.7772, -85.911;
-10.8234, 41.8961, -74.4074;
-13.9937, 35.5167, -50.7962;
-14.3038, 34.7247, -47.9571;
-14.8257, 32.9699, -41.6747;
-14.6745, 33.5497, -43.7501;
-11.2159, 22.148, 3.98751;
-11.8022, 22.5945, 1.47664;
-8.85457, 20.8696, 12.1895;
-10.131, 21.5096, 7.94287;
4.63046, 134.052, -139.576;
-10.0719, 43.1032, -79.1781;
-10.7667, 21.8667, 5.68727;
-12.6587, 38.5085, -61.6436;
3.95535, 241.725, -146.174;
-15.1097, 31.1952, -35.2305;
-14.9267, 27.8269, -22.3608;
-15.0656, 31.6803, -37.016;
-9.86649, 21.3662, 8.86292;
4.58261, 106.635, -135.457;
-13.3958, 36.9225, -55.8701;
-14.8467, 27.4234, -20.75;
-9.26049, 44.3478, -84.1828;
-9.51023, 43.9693, -82.6548;
-13.9657, 25.1136, -10.9592;
-14.4925, 34.1899, -46.0425;
-10.5672, 42.3162, -76.0544;
-11.5984, 40.5562, -69.2552;
-0.993421, 64.013, -126.478;
0.765767, 72.1493, -130.141;
-2.1096, 59.7809, -122.798;
-12.1454, 39.5104, -65.3578;
-13.062, 37.65, -58.5144;
-6.68714, 48.3172, -99.7383;
4.33504, 156.127, -142.176;
4.39741, 100.816, -134.536;
2.55245, 82.6571, -132.286;
-14.4223, 26.1384, -15.3963;
-1.22434, 63.1015, -125.766;
-4.58353, 52.82, -111.184;
-5.20508, 51.4016, -107.822;
1.17634, 74.0686, -130.953;
-7.37625, 47.2239, -95.5894;
-14.9706, 32.2558, -39.1074;
-15.1378, 29.7063, -29.6614;
-13.3868, 24.326, -7.21636;
-2.6613, 58.1303, -120.338;
-5.48993, 50.7878, -106.232;
-15.0685, 28.7501, -25.9925;
-13.7052, 36.2122, -53.3032;
0.437576, 70.5848, -129.527;
];
//skeleton lines
lines = [
93, 24;
56, 44;
0, 89;
42, 59;
7, 64;
17, 90;
62, 50;
39, 59;
48, 77;
3, 36;
13, 14;
10, 157;
20, 79;
19, 17;
43, 60;
76, 90;
157, 136;
4, 72;
79, 117;
14, 2;
7, 6;
40, 6;
62, 66;
135, 10;
49, 73;
38, 5;
61, 75;
137, 153;
78, 148;
2, 57;
38, 47;
117, 33;
61, 83;
35, 40;
9, 137;
58, 56;
153, 34;
1, 41;
9, 145;
39, 121;
82, 98;
25, 147;
25, 146;
23, 34;
8, 99;
23, 146;
60, 91;
149, 54;
108, 27;
156, 127;
139, 120;
112, 132;
109, 110;
150, 111;
124, 122;
151, 28;
150, 124;
155, 123;
151, 155;
29, 144;
128, 123;
114, 85;
30, 85;
114, 113;
95, 115;
115, 31;
116, 125;
95, 125;
31, 86;
55, 86;
87, 96;
88, 96;
52, 53;
1, 37;
58, 91;
70, 47;
8, 69;
5, 64;
53, 69;
126, 22;
15, 50;
45, 16;
65, 63;
45, 63;
13, 37;
141, 51;
57, 35;
15, 74;
71, 74;
48, 68;
72, 68;
46, 67;
20, 4;
70, 73;
141, 100;
16, 66;
18, 12;
52, 75;
18, 65;
46, 81;
42, 43;
121, 12;
41, 71;
24, 89;
22, 103;
77, 92;
21, 97;
36, 80;
126, 92;
21, 99;
33, 67;
102, 103;
44, 83;
76, 98;
118, 133;
87, 55;
101, 3;
51, 81;
49, 100;
136, 148;
78, 11;
142, 101;
82, 93;
19, 97;
104, 140;
140, 105;
104, 149;
106, 118;
129, 107;
106, 130;
129, 130;
134, 84;
27, 84;
138, 134;
138, 120;
132, 110;
112, 111;
116, 119;
26, 105;
131, 94;
154, 147;
54, 107;
108, 133;
88, 32;
156, 109;
135, 145;
127, 139;
29, 128;
113, 119;
28, 122;
143, 80;
152, 30;
142, 102;
144, 94;
131, 152;
143, 11;
26, 154;
];
I would:
divide the mesh into separate polylines
and process each separately. So you need to reorder the points pnt[] so they are in the same topological order (just sequence of points no line[] structure).
compute |pnt[i]-sliding_average(pnt[i])|
the value will give you distance from sliding average so threshold it ... if too big divide your polyline.
Sliding average (averaging 10 points before and 10 after) for your data looks like this (red):
As threshold distance I used sqrt(20) so here start end endpoints of the individual polylines detected (yellow):
regress lines
after #2 your set should contain individual line segments. To improve precision ignore corner points (near edges of polyline) as it could contain points from next/previous line segment.
I choose to ignore 10 points from beginning and 10 from end. I computed avg position of the inside points. As direction I use difference between inside endpoints rescaled to size of outside endpoints. From that the regressed line is just avg position +/- half direction vector.
You can chose what ever fitting or regression of lines you want depends on the properties of result you want to achieve.
compute intersection between neighboring lines
reconstruct mesh
Here simple C++ code (sorry I do not use Matlab) for your data covering #1,#2,#3 without the recursive subdivision (branching of polyline) so you can see how the things are done in case the above description is not enough.
//---------------------------------------------------------------------------
//skeleton vertices
double pnt[]=
{
2.06104, 318.734, -149.29,
4.2207, 212.092, -145.141,
4.23213, 200.135, -144.811,
4.16573, 95.1567, -133.954,
4.7053, 126.626, -138.59,
4.16915, 171.645, -143.646,
4.18659, 183.173, -144.185,
4.17842, 179.964, -144.008,
2.76537, 288.063, -147.215,
-1.71817, 61.155, -124.25,
-0.492168, 66.2098, -127.702,
2.07608, 79.5012, -131.886,
4.03249, 238.699, -146.141,
4.23595, 206.822, -145.001,
4.23623, 203.704, -144.908,
4.17145, 220.543, -145.415,
4.12453, 228.514, -145.761,
2.41377, 301.021, -147.804,
4.03098, 236.064, -145.985,
2.48681, 298.432, -147.685,
4.68192, 129.093, -138.873,
2.65048, 292.424, -147.406,
4.54555, 104.737, -135.116,
-4.23707, 53.6538, -113,
1.93508, 317.106, -148.755,
-4.98045, 51.9036, -109.052,
-5.87157, 49.9703, -104.104,
-11.2433, 41.1865, -71.6569,
-15.1283, 30.7528, -33.5845,
-14.6647, 26.7291, -17.9213,
-13.1176, 23.9812, -5.55431,
-8.27057, 20.6161, 13.9826,
-4.49387, 19.0295, 25.4537,
4.5645, 139.344, -140.275,
-3.3737, 56.0265, -117.133,
4.21667, 192.056, -144.576,
3.99948, 93.2911, -133.447,
4.22894, 209.986, -145.081,
4.17824, 167.633, -143.406,
3.94993, 243.303, -146.26,
4.20391, 188.412, -144.431,
4.20673, 214.944, -145.239,
3.82056, 248.85, -146.382,
3.75634, 252.761, -146.523,
3.46093, 268.466, -147.033,
4.11554, 230.181, -145.836,
4.44799, 147.962, -141.303,
4.205, 165.551, -143.138,
4.7514, 117.603, -137.34,
4.25931, 161.613, -142.739,
4.15939, 222.571, -145.502,
4.38497, 152.519, -141.797,
3.12906, 279.268, -147.181,
3.05571, 282.333, -147.279,
-8.35374, 45.7208, -89.6802,
-6.43016, 19.7295, 20.0489,
3.51218, 265.85, -146.95,
4.22735, 196.569, -144.716,
3.60114, 261.275, -146.806,
3.87527, 245.264, -146.243,
3.70115, 255.948, -146.633,
3.33032, 273.433, -147.142,
4.14712, 224.649, -145.592,
4.11102, 231.178, -145.882,
4.17545, 177.258, -143.86,
4.07209, 234.491, -145.985,
4.13698, 226.37, -145.667,
4.50243, 144.003, -140.851,
4.74996, 120.215, -137.661,
3.00397, 283.765, -147.29,
4.22263, 164.279, -143.008,
4.19542, 216.684, -145.284,
4.74419, 122.387, -138.079,
4.24362, 162.754, -142.854,
4.1921, 217.204, -145.299,
3.1988, 276.954, -147.144,
2.24673, 307.266, -148.14,
4.70408, 113.726, -136.704,
1.55558, 75.9859, -131.562,
4.66136, 131.166, -139.157,
3.69856, 90.4963, -133.113,
4.42481, 149.643, -141.489,
2.02224, 313.586, -148.445,
3.39344, 270.742, -147.067,
-11.4507, 40.824, -70.2682,
-12.6325, 23.3651, -2.57932,
-7.10479, 20.0528, 17.8335,
-5.64725, 19.4808, 22.078,
-4.64392, 19.0822, 25.039,
2.02817, 318.148, -149.125,
2.33964, 303.688, -147.933,
3.66401, 257.998, -146.701,
4.63971, 109.538, -135.974,
1.92836, 315.331, -148.48,
-14.1808, 25.5312, -12.8091,
-9.41149, 21.1265, 10.4194,
-4.95978, 19.1884, 24.1908,
2.59159, 294.625, -147.508,
2.13575, 310.184, -148.256,
2.70921, 290.146, -147.303,
4.27225, 160.675, -142.644,
4.26101, 97.2627, -134.029,
4.48821, 103.188, -134.891,
4.49926, 103.485, -134.935,
-7.047, 47.7435, -97.5736,
-6.3594, 48.9598, -101.483,
-9.77383, 43.5659, -81.0321,
-8.9763, 44.7772, -85.911,
-10.8234, 41.8961, -74.4074,
-13.9937, 35.5167, -50.7962,
-14.3038, 34.7247, -47.9571,
-14.8257, 32.9699, -41.6747,
-14.6745, 33.5497, -43.7501,
-11.2159, 22.148, 3.98751,
-11.8022, 22.5945, 1.47664,
-8.85457, 20.8696, 12.1895,
-10.131, 21.5096, 7.94287,
4.63046, 134.052, -139.576,
-10.0719, 43.1032, -79.1781,
-10.7667, 21.8667, 5.68727,
-12.6587, 38.5085, -61.6436,
3.95535, 241.725, -146.174,
-15.1097, 31.1952, -35.2305,
-14.9267, 27.8269, -22.3608,
-15.0656, 31.6803, -37.016,
-9.86649, 21.3662, 8.86292,
4.58261, 106.635, -135.457,
-13.3958, 36.9225, -55.8701,
-14.8467, 27.4234, -20.75,
-9.26049, 44.3478, -84.1828,
-9.51023, 43.9693, -82.6548,
-13.9657, 25.1136, -10.9592,
-14.4925, 34.1899, -46.0425,
-10.5672, 42.3162, -76.0544,
-11.5984, 40.5562, -69.2552,
-0.993421, 64.013, -126.478,
0.765767, 72.1493, -130.141,
-2.1096, 59.7809, -122.798,
-12.1454, 39.5104, -65.3578,
-13.062, 37.65, -58.5144,
-6.68714, 48.3172, -99.7383,
4.33504, 156.127, -142.176,
4.39741, 100.816, -134.536,
2.55245, 82.6571, -132.286,
-14.4223, 26.1384, -15.3963,
-1.22434, 63.1015, -125.766,
-4.58353, 52.82, -111.184,
-5.20508, 51.4016, -107.822,
1.17634, 74.0686, -130.953,
-7.37625, 47.2239, -95.5894,
-14.9706, 32.2558, -39.1074,
-15.1378, 29.7063, -29.6614,
-13.3868, 24.326, -7.21636,
-2.6613, 58.1303, -120.338,
-5.48993, 50.7878, -106.232,
-15.0685, 28.7501, -25.9925,
-13.7052, 36.2122, -53.3032,
0.437576, 70.5848, -129.527,
};
const int pnts3=sizeof(pnt)/sizeof(pnt[1]); // number of points * 3
const int pnts=pnts3/3; // number of points
//skeleton lines
int lin[]=
{
93, 24,
56, 44,
0, 89,
42, 59,
7, 64,
17, 90,
62, 50,
39, 59,
48, 77,
3, 36,
13, 14,
10, 157,
20, 79,
19, 17,
43, 60,
76, 90,
157, 136,
4, 72,
79, 117,
14, 2,
7, 6,
40, 6,
62, 66,
135, 10,
49, 73,
38, 5,
61, 75,
137, 153,
78, 148,
2, 57,
38, 47,
117, 33,
61, 83,
35, 40,
9, 137,
58, 56,
153, 34,
1, 41,
9, 145,
39, 121,
82, 98,
25, 147,
25, 146,
23, 34,
8, 99,
23, 146,
60, 91,
149, 54,
108, 27,
156, 127,
139, 120,
112, 132,
109, 110,
150, 111,
124, 122,
151, 28,
150, 124,
155, 123,
151, 155,
29, 144,
128, 123,
114, 85,
30, 85,
114, 113,
95, 115,
115, 31,
116, 125,
95, 125,
31, 86,
55, 86,
87, 96,
88, 96,
52, 53,
1, 37,
58, 91,
70, 47,
8, 69,
5, 64,
53, 69,
126, 22,
15, 50,
45, 16,
65, 63,
45, 63,
13, 37,
141, 51,
57, 35,
15, 74,
71, 74,
48, 68,
72, 68,
46, 67,
20, 4,
70, 73,
141, 100,
16, 66,
18, 12,
52, 75,
18, 65,
46, 81,
42, 43,
121, 12,
41, 71,
24, 89,
22, 103,
77, 92,
21, 97,
36, 80,
126, 92,
21, 99,
33, 67,
102, 103,
44, 83,
76, 98,
118, 133,
87, 55,
101, 3,
51, 81,
49, 100,
136, 148,
78, 11,
142, 101,
82, 93,
19, 97,
104, 140,
140, 105,
104, 149,
106, 118,
129, 107,
106, 130,
129, 130,
134, 84,
27, 84,
138, 134,
138, 120,
132, 110,
112, 111,
116, 119,
26, 105,
131, 94,
154, 147,
54, 107,
108, 133,
88, 32,
156, 109,
135, 145,
127, 139,
29, 128,
113, 119,
28, 122,
143, 80,
152, 30,
142, 102,
144, 94,
131, 152,
143, 11,
26, 154,
};
const int lins2=sizeof(lin)/sizeof(lin[1]); // number of lines * 2
const int lins=lins2/2; // number of lines
//---------------------------------------------------------------------------
int pol[pnts],pols=0; // polyline
double avg[pnts3]; // sliding average
//---------------------------------------------------------------------------
const int _mesh=100;
double mesh[_mesh*3];
int meshs=0,meshs3=0;
//---------------------------------------------------------------------------
void compute()
{
int i,j,e,n,i0,i1;
int his[pnts];
int used[lins];
double x,y,z,w,rr,ll;
//--- compute polyline pol[pols] ----------------------------------------
// histogram of point usagge
for (i=0;i<pnts;i++) his[i]=0;
for (i=0;i<lins2;i++) his[lin[i]]++;
// clear tables
for (i=0;i<pnts;i++) pol[i]=-1; pols=0;
for (i=0;i<lins;i++) used[i]=0;
// find start point (his[i]!=2)
for (e=1,i=0;i<pnts;i++)
if ((his[i])&&(his[i]!=2))
{ e=0; break; }
if (e) return; // stop if none found
// add start point to polyline
pol[pols]=i; pols++;
// process polyline
for (e=1;e;)
for (e=0,j=0;j<lins;j++)
if (!used[j]) // ignore used lines
{
// is point i in line j ?
if (lin[j+j+0]==i) i=lin[j+j+1];
else if (lin[j+j+1]==i) i=lin[j+j+0];
else continue;
// add next point to polyline
pol[pols]=i; pols++;
used[j]=1;
if (his[i]==2) e=1; // loop if not end of polyline
break;
}
//--- compute sliding average -------------------------------------------
n=10; // sliding average half interval [poins]
w=1+n+n; w=1.0/w;
for (i=0;i<pols;i++)
{
e=3*pol[i];
x=pnt[e]; e++;
y=pnt[e]; e++;
z=pnt[e]; e++;
for (j=1;j<=n;j++)
{
e=i+j; if (e>=pols) e=pols-1;
e=3*pol[e];
x+=pnt[e]; e++;
y+=pnt[e]; e++;
z+=pnt[e]; e++;
e=i-j; if (e<0) e=0;
e=3*pol[e];
x+=pnt[e]; e++;
y+=pnt[e]; e++;
z+=pnt[e]; e++;
}
e=3*i;
avg[e]=x*w; e++;
avg[e]=y*w; e++;
avg[e]=z*w; e++;
}
//--- regress lines -----------------------------------------------------
meshs=0; meshs3=0;
ll=20.0;
for (e=0,i=0;i<pols;i++)
{
// distance to sliding average
j=3*pol[i];
x=pnt[j]; j++;
y=pnt[j]; j++;
z=pnt[j]; j++;
j=3*i;
x-=avg[j]; j++; x*=x;
y-=avg[j]; j++; y*=y;
z-=avg[j]; j++; z*=z;
rr=x+y+z;
if ((e==0)&&(rr<=ll)){ i0=i; e++; }// find start point
else if ((e==1)&&(rr> ll)) // find end point
{
i1=i-1; e=0; if (i0==i1) continue;
// regress line from pol[i0..i1]
int j0,j1;
double dx,dy,dz;
// ignore n edge points
n=10; while ((i1-i0)<(n<<2)) n--;
j0=i0+n;
j1=i1-n;
// direction
j=3*pol[j1];
dx=pnt[j]; j++;
dy=pnt[j]; j++;
dz=pnt[j]; j++;
j=3*pol[j0];
dx-=pnt[j]; j++;
dy-=pnt[j]; j++;
dz-=pnt[j]; j++;
// original line size
j=3*pol[i1];
x=pnt[j]; j++;
y=pnt[j]; j++;
z=pnt[j]; j++;
j=3*pol[i0];
x-=pnt[j]; j++;
y-=pnt[j]; j++;
z-=pnt[j]; j++;
// rescale direction to match original half size
rr=sqrt((x*x)+(y*y)+(z*z));
rr=divide(rr,sqrt((dx*dx)+(dy*dy)+(dz*dz)));
rr*=0.5;
dx*=rr;
dy*=rr;
dz*=rr;
// avg position
for (x=y=z=0.0,n=0,i=j0;i<=j1;i++)
{
j=3*pol[i];
x+=pnt[j]; j++;
y+=pnt[j]; j++;
z+=pnt[j]; j++;
n++;
}
x/=n;
y/=n;
z/=n;
// line is avg+/-half direction
mesh[meshs3]=x-dx; meshs3++; meshs++;
mesh[meshs3]=y-dy; meshs3++;
mesh[meshs3]=z-dz; meshs3++;
mesh[meshs3]=x+dx; meshs3++; meshs++;
mesh[meshs3]=y+dy; meshs3++;
mesh[meshs3]=z+dz; meshs3++;
// restore main loop
i=i1+1; e=0;
}
}
i=0;
}
//---------------------------------------------------------------------------
the mesh[meshs3] holds 3D points coordinates of the resulting polyline (2 points per line) meshs is the number of points so num_of_lines=meshs/2

Loading RSA private key

I'm trying to load a private key (generated with RSA in an external application) in a javacard. I've written some normal java code to generate a keypair and to print the exponent and modulus of the private key:
public class Main {
public static void main(String[] args) throws NoSuchAlgorithmException {
KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(512);
KeyPair kp = keyGen.generateKeyPair();
RSAPrivateKey privateKey = (RSAPrivateKey) kp.getPrivate();
BigInteger modulus = privateKey.getModulus();
BigInteger exponent = privateKey.getPrivateExponent();
System.out.println(Arrays.toString(modulus.toByteArray()));
System.out.println(Arrays.toString(exponent.toByteArray()));
}
}
I then copied the byte arrays to the javacard code
try {
RSAPrivateKey rsaPrivate = (RSAPrivateKey) KeyBuilder.buildKey(KeyBuilder.TYPE_RSA_PRIVATE, KeyBuilder.LENGTH_RSA_512, false);
byte[] exponent = new byte[]{113, 63, 80, -115, 103, 13, -90, 75, 85, -31, 83, 84, -15, -8, -73, -68, -67, -27, -114, 48, -103, -10, 27, -77, -27, 70, 61, 102, 17, 36, 0, -112, -10, 111, 40, -117, 116, -120, 76, 35, 54, -109, 115, 70, -11, 118, 92, -43, -15, -38, -67, 112, -13, -115, 7, 65, -41, 89, 127, 62, -48, -66, 8, 17};
byte[] modulus = new byte[]{0, -92, -30, 28, -59, 41, -57, 95, -61, 2, -50, -67, 0, 6, 67, -13, 22, 61, -96, -15, -95, 20, -86, 113, -31, -91, -92, 77, 124, 26, -67, -24, 40, -42, -41, 115, -66, 109, -115, -111, -6, 33, -51, 63, -72, 113, -36, 22, 99, 116, 18, 108, 106, 97, 95, -69, -118, 49, 9, 83, 67, -43, 50, -36, -55};
rsaPrivate.setExponent(exponent, (short) 0, (short) exponent.length);
rsaPrivate.setModulus(modulus, (short) 0, (short) modulus.length);
}
catch (Exception e) {
short reason = 0x88;
if (e instanceof CryptoException)
reason = ((CryptoException)e).getReason();
ISOException.throwIt(reason);
}
Now for some reason, a CryptoException is thrown when setting the modulus with reason 1. According to the API, this means CryptoException.ILLEGAL_VALUE if the input modulus data length is inconsistent with the implementation or if input data decryption is required and fails.
I really got no clue why this is failing. Generating the keys on card is not an option in this project.
And I know 512 bits is not safe anymore, it's just for testing purpose. It will be replaced by 2048 bits in the end.
I figured out that the RSAPrivateKey api expects unsigned values and the toByteArray of a BigInteger returns the signed version. This post ( BigInteger to byte[] ) helped to figure me out I could simply remove the leading zero byte in the modulus byte array. It's working ok now.