I would like to fill an array from a stream for around ten seconds.{I wish to do some processing on the data)So far I can:
(a) obtain the microphone stream using mediaRecorder
(b) use analyser and analyser.getFloatTimeDomainData(dataArray) to obtain an array but it is size limited to only a little over half a second of data.I can also successfully output the data after processing back onto a stream and to outDestination.
(c) I have also experimented with obtaining a 'chunks' array from mediaRecorder directly but the problem then is that I can't find any mime type that would give me a simple array of values - ie an uncompressed sample by sample single channel set of value - ie a longer version of 'dataArray' in (b).
I am wondering if I am missing a simple way round this problem?
Solutions I have seen tend to use step (b) and do regular polls then reassemble a longer array - however it seems the timing is a bit tricky ..
I'v also seen suggestions to use audio workouts - I might have to do this but would prefer a simpler solution!
Or again, if someone knows how to drive mediaRecorder to output the chunks array in a simple array format FLOAT32.of one channel.That would do the trick.
Or maybe I'm missing something simpler?
I have code showing those steps that have been successful and will upload if anyone requests.
I am working on a MIDI project using mido library in Python. I see in the manual a meta message for time signature with value: notated_32nd_notes_per_beat which has a default value of 8.
<meta message time_signature numerator=4 denominator=4 clocks_per_click=24 notated_32nd_notes_per_beat=8 time=0>
Which makes sense. However, can I define it like:
<meta message time_signature numerator=4 denominator=4 clocks_per_click=24 notated_32nd_notes_per_beat=32 time=0>
Does this increase the display resolution when shown in a score/typesetting software? What is the usage of this please?
time_signature (0x58) meta message in midi files
The file header specifies the number of ticks per beat, and the tempo messages specify the length of a beat, in microseconds. These value are needed to correctly play back the file.
The last field of the time signature message specifies how the tick values in the MIDI file relates to notes in a score. It does not affect at what time events are sent (so a pure playback program will ignore this message), but how notes are displayed.
For example, if the header says there are 100 ticks per beat, and the time signature has the default of 8 32th notes per beat, then a note-on/note-off pair with a distance of 100 ticks is displayed as a quarter note. If you change the time signature to 32 32th notes per beat, then a length of 100 ticks corresponds to a whole note.
I have a ".mat" file supposedly containing a [30720000x4 double] matrix (values from accelerometers). When I try to open this file with "Import data" in Matlab I get the following error:
Error using load
Can't read file F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat.
Error using load
Unknown text on line number 1 of ASCII file
F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat
"MATLAB".
Error in uiimport/runImportdata (line 456)
datastruct = load('-ascii', fileAbsolutePath);
Error in uiimport/gatherFilePreviewData (line 424)
[datastruct, textDelimiter, headerLines]= runImportdata(fileAbsolutePath,
type);
Error in uiimport (line 240)
[ctorPreviewText, ctorHeaderLines, ctorDelim] = ...
The filesize is 921MB which is the same as my other files that do open. I also tried opening the file using python, but no success. Any suggestions? I use MATLAB R2013b .
More info:
How the file was create:
%% acquisition of vibration data
% input:
% sample rate in Hz (max. 51200 Hz, should be used as bearing
% faults are high-frequent)
% time in seconds, stating the duration of the measurement
% (e.g. 600 seconds = 10 minutes)
% filename for the file to be saved
%
% examples:
% data = DAQ(51200, 600, 'NF1_1.mat');
% data = DAQ(51200, 600, 'NF1_2.mat');
function data = DAQ(samplerate,time,filename)
s = daq.createSession('ni'); % Creates the DAQ session
%%% Add the channels as accelerometer channels (meaning IEPE is turned on)
s.addAnalogInputChannel('cDAQ1Mod1','ai0','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai1','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai2','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai3','Accelerometer');
%s.addAnalogInputChannel('cDAQ1Mod2','ai0','Accelerometer');
s.Rate = samplerate;
s.NumberOfScans = samplerate*time;
%%% Defining the Sensitivities in V/g
s.Channels(1).Sensitivity = 0.09478; %31965, top outer
s.Channels(2).Sensitivity = 0.09531; %31966, back outer
s.Channels(3).Sensitivity = 0.09275; %31964, top inner
s.Channels(4).Sensitivity = 0.09363; %31963, back inner
data = s.startForeground(); %Acquiring the data
save(filename, 'data');
More info:
When I open the file using a simple text editor I can see a lot of characters that do not make sense but also the first line:
MATLAB 5.0 MAT-FILE, Platform: PCWIN64, Created on: Thu Apr 30
16:29:07 2015
More info:
The file itself: https://www.dropbox.com/s/r7mavil79j47xa2/GR_UB50n_1_2.mat?dl=0
It is 921MB.
EDIT:
How can I recover my data?
I've tried this, but got memory errors.
I've also tried this, but it did not work.
I fear I can't add many good news to what you know already, but it hasn't been mentioned yet.
The reason the .mat-file can't be load is due to the data beeing corrupted. What makes it 'unrecoverable' is the way it is stored internally. The exact format is specified in the MAT-File Format Documentation. So I decided to manually construct a simple reader to specifically read your .mat file.
It makes sense, that the splitmat.m can't recover anything, as it will basicly split the data into chunks, one stored variable per chunk, however in this case there is only 1 variable stored and thus only one chunk, which happens to be the corrupted one.
In this case, the data is stored as a miCOMPRESSED, which is a normal matlab array compressed using gzip. (Which, as a side note, doesn't seem like a good fit for 'random' vibration data.) This might explain previous comments about the smaller file size then the full data, as the filesize matches exatly with the internally stored value.
I extracted the compressed archive and tried to uncompress it in a variety of ways. Basicly it is a '.gz' without the header, that can be appended manually. Unfortunatly there seems to be a corrupted block near the start of the dataset. I am by no means an expert on gzip, but as far as I know the dictionary (or decryption key) is stored dynamicly which makes all data useless from the point the block is corrupted. If you are really eager, there seems to be a way to recover data even behind the point where data is corrupted, but that method is massively timeconsuming. Also the only way to validate data of those sections is manual inspection, which in your case might proof very difficult.
Below is the code, that I used to extract the .gz-file, so if you want to give it a try, this might get you started. If you manage to decrypt the data, you can read it as described in the MAT-File Format, 13f.
corrupted_file_id = fopen('corrupt.mat','r');
%% some header data
% can be skipped replacing this block with
% fread(id,132);
%header of .mat file
header_text = char(fread(corrupted_file_id,116,'char')');
subsystem_data_offset = fread(corrupted_file_id,8,'uint8');
version = fread(corrupted_file_id,1,'int16');
endian_indicator = char(fread(corrupted_file_id,2,'int8')');
data_type = fread(corrupted_file_id,4,'uint8');
%data_type is 15, so it is a compressed matlab array
%% save te content
data_size = fread(corrupted_file_id,1,'uint32');
gz_file_id = fopen('compressed_array.gz','w');
% first write a valid gzip head
fwrite(gz_file_id,hex2dec('1f8b080000000000'),'uint64',0,'b');
% then write the data sequentialy
step = 1:1e3:data_size;% 1MB steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1e3,'uint8'));
end
step = step(end):data_size;% 1B steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1,'uint8'));
end
fclose(gz_file_id);
fclose(corrupted_file_id);
To answer literally to the question, my suggestion would be to make sure first that the file is okay. This tool on File Exchange apparently knows how to diagnose corrupted .MAT files starting with version V5 (R8):
http://www.mathworks.com/matlabcentral/fileexchange/6893-matcat-mat-file-corruption-analysis-tool
The file's size (indices going out of range) seems to be a problem. Octave, which should read .mat files, gives the error
memory exhausted or requested size too large for range of Octave's index type
To find out what is wrong you may need to write a test program outside MatLab, where you have more control over memory management. Examples are here, including instructions on how to build them on your own platform. These stand-alone programs may not have the same memory issues. The program matdgns.c is specifically made to check .mat files for errors.
I'm using an ElNet energy&power meter that communicates with my processor via Modbus RTU protocol.
There are two 16-bit ElNet registers that contain information about the Date and Time (separately) in a Win Format (registers 85-86, page 6 of this document). I'm able to read these two registers. However, I'm unable to extract information about the Date and Time.
For example, Date register contains decimal value of 17841 for today's date (31/07/2015). Is there any person willing to explain me how to convert 17841 into 31/07/2015?
I have the same problem with the time. My time register contains a decimal value of 55296. Can you help me extract the time from this number?
This thread addresses the same problem:
HEX/Decimal to date and time from modbus
However, I'm not sure I understand the extraction algorithm applied there. My point of operation is processor with the code written in C or C++.
Thank you very much for your time and effort to help me.
Sincerely,
Bojan.
The MS-DOS date/time format is described here: https://archive.is/2bVlz (was http://proger.i-forge.net/MS-DOS_date_and_time_format/OFz but is gone)
It makes sense for the 17256 value mentioned in the other question, as it translates to 2013-11-08. See here how to do it:
Register bit description: 0bYYYYYYYMMMMDDDDD
Registervalue: 17256 0b0100001101101000
Yearmask: 0b1111111000000000
Yearpart: 0b0100001000000000
Yearpart rightshifted 9 steps: 0b0000000000100001 = 33 years after 1980
Monthmask: 0b0000000111100000
Monthpart: 0b0000000101100000
Monthpart rightshifted 5 steps: 0b0000000000001011 = 11
Daymask: 0b0000000000011111
Daypart: 0b0000000000001000 = 8
Unfortunately your register value 17841 does not make sense, as it translates to 2014-13-17 (That is month 13).
Are you sure that:
you read the correct register? (change the time setting in the instrument, and see what happens to the register value)
you do not mix up the two bytes in the register?
the time setting is correct?
due to Internet communication i could have two (or more) ASCII files in RINEX format (GPS ASCII format) of the same data period, which i would like to merge to one file.
Each data set (epoch) contain more then one line (in this example 19 lines). I would like to merge those files, where it could be that they in some parts overlap each other.
here is an example of RINEX epoch data set:
09 2 21 12 59 59.9000000 0 9G31G23G11G13G32G17G14G20G19
23152606.238 121667768.06047 94806069.43545 23152606.540 23152606.521
1262.605 43.750 31.500
22765313.352 119632547.53447 93220179.18745 22765312.252 22765311.072
3252.769 46.250 32.250
20798168.896 109295128.07748 85165036.96747 20798168.642 20798168.578
-2252.493 52.750 43.250
25363206.177 133284559.23845
3776.403 32.750
20350616.203 106943239.96448 83332404.31147 20350615.386 20350616.499
-929.443 51.000 44.500
21994260.713 115580595.93348 90062809.84446 21994260.826 21994260.114
416.327 49.500 38.250
23964108.994 125932271.15846 98129049.02843 23964107.689 23964107.603
-3561.500 39.250 20.250
20225257.452 106284459.64448 82819085.85247 20225256.341 20225256.964
956.944 52.750 45.250
25623383.323 134651746.21445 104923415.17742 25623386.202 25623384.504
-3991.096 34.250 12.250
The first line contains the time info and below are the raw data for each GPS satellite.
My idea was to open each file separate and stored the raw data in some kind of array relative to time. Each time i read new epoch, i ask my array if i already have something with time so and so, and if not i place the raw data there.
My question is how to store the raw data with respect to time, since it is not one line but something dynamic that could always change.
If you have better idea, please share it with me.
Regards
To store the raw data with respect to time, I would:
Encode the time as a number (# of seconds since Unix "epoch time" or since some arbitrary start time - use microseconds instead of seconds depending on what RINEX time precision is).
Store the raw data as an array (data for each line is 1 array element - stored either as a string, an arrayref of words or a hash of values).
Store the reference to that array as a value in a hash, with the key being the time-encoded-as-a-number.