I'm trying to read a lot of data coming from my Arduino, I've set my input buffer to 500000 to make sure that it can handle all these data. My data are 4 sensors readings each samples at 250 Hz. With the default buffer size (712), I used to get snags when I plot the readings in real time and the samples get disordered which makes the plot go crazy. I solved this by increasing the buffer size to 50000. But now, this will work for a while but if I want to run it for 15 minutes, I get the same misbehavior after 5 minutes, with the addition that the plotting gets slower. I do have some processing code along with the live plotting but it shouldn't be like this with such a bi buffer. I want to know whether the buffer will contain all the data from the beginning until it's full or will it keep erasing older data when it gets full (knowing that I already saved it in another vector and plotted it). I truly don't understand why this keeps happening.
kind regards
I.H
When the buffer gets full, once you get new data it erases the old data. The behavior you are seeing is because your processing and your plotting is slower than the flow of the data.
Try to make sure that you optimize you processing
Make sure that for plotting is done by "drawnow". Like this you are sure that if there is anything in the queue it is not executed
Try to avoid saving and keeping all the data
If the problem is still there, you can try to implement a timer to make sure that you are consistent with reading your data
I Have a Sensor (Gyro) that connected to my python program (with socket UDP) and send data to python console in real-time but with 200 Hz frequency.
I want to change this frequency of coming data to my console but could not find a good way to do it.
I was thinking about doing it with filters like Mean an waiting for idea?
If you want to have regular updates, use a windowing mechanism. Take the last n values and store the average. Then, discard the next two values and take the last n values again. This example would yield values with a frequency of 200 Hz/2.
If you only want to see events when changes have occured, store the last value, compare the current value with the last one and emit an event if it has changed, updating the stored value. As you're dealing with sensors (and thus, a little fuzziness), you probably want to implement a hysteresis.
You can even raise the frequency by creating extra values in between the received ones through interpolation. For a steady frequency, you would have to take care about your timing though.
I'm having an issue which is partially Matlab- and partially general programming-related, I'm hoping that somebody can help me brainstorm for solutions.
I have an external microcontroller that generates a large stream of binary data (~40kb) every 400ms and sends it via UART to a PC running Matlab scripts. The data is not encoded in hexa or dec characters, but true binary (hence, there's no terminator defined as all 256 values are possible, valid combinations of data). Baudrate is set at 1024000. In short, it takes roughly 375ms for a whole stream of data to be sent, with 25ms of dead time in between streams
In Matlab, the serial port is configured correctly (also 1024000, 8x bits, 1x stop bit, no parity, no hardware flow control, etc.). I am able to readout the data I'm sending via the microcontroller correctly (i.e. there's no corruption of data), but I'm not being able to synchronize the serial readout on Matlab. My script is as follows:
function data_show = GetDATA
if ~isempty(instrfind)
fclose(instrfind);
end
DATA_TOTAL_SIZE = 38400;
DATA_buffer = uint8(zeros(DATA_TOTAL_SIZE,1));
DATA_show = reshape(DATA_buffer(1:2:end)',[160,120])';
f_data_in = false;
f_data_out = true;
serialport = serial('COM11','BaudRate',1024000,'DataBits',8,'FlowControl','none','Parity','none','StopBits',1,...
'BytesAvailableFcnCount',DATA_TOTAL_SIZE,'BytesAvailableFcnMode','byte','InputBufferSize',DATA_TOTAL_SIZE * 2,...
'BytesAvailableFcn',#GetPortData);
fopen(serialport);
while (get(serialport,'BytesAvailable') ~= 0) % Skip first packet which might be incomplete
fread(serialport,DATA_TOTAL_SIZE,'uint8');
end
f_data_out = true;
while (1)
if (f_data_in)
DATA_buffer = fread(serialport,DATA_TOTAL_SIZE,'uint8');
DATA_show = reshape(DATA_buffer(1:2:end)',[160,120])'; %Reshape array as matrix
DATAsc(DATA_show);
disp('DATA');
end
pause(0.01);
end
fclose(serialport);
delete(serialport);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function GetPortData (obj,~)
if f_data_out
f_data_in = true;
end
end
end
The problem I see is that what I end up reading is always the correct size, but belongs to multiple streams, because I haven't found a way to tell Matlab that these 25ms of no data should be used to synchronize (i.e. data from before and after that blank period should belong to different streams).
Does anyone have any suggestions for this?
Thanks a lot!
For completeness, I would like to post the current implementation I have fixing this issue, which is probably not a suitable solution in all cases but might be useful in some.
The approach I took consists in moving into a bi-directional communication protocol, in which Matlab initiates the streaming by sending a very short command as a trigger (e.g. single, non-printable character). Given the high baudrate it does not add significant delay due to processing in the microcontroller's side.
The microcontroller, upon reception of this trigger, proceeds to transmit only one full package (as opposed to continuously streaming package at a 5Hz rate). By forcing Matlab to pickup a serial package of the known length right after issuing the trigger, it ensures that only one package and without synchronization issues is received.
Then it becomes just a matter of encapsulating the Matlab script in a routine with a 5Hz tick given by a timer, in which the sequence is repeated (send trigger, retrieve package, do whatever processing, and repeat).
Advantages of this:
It solves the synchronization problems
Disadvantages of this:
Having Matlab running on a timer tick does not ensure perfect periodicity, and hence the triggers might not always be sent at exactly 5Hz. If triggers are sent at "inconvenient" times for the microcontroller, packages might need to be skipped in order to avoid that a package is updated in memory while it is still being transmitted (since transmission takes a significant part of the 200ms time slot)
From experience, performance can vary a lot depending on what the PC running Matlab is doing. For example, it works fine when the PC is left on its own to do the acquisition, but if another program is used (e.g. Chrome), Matlab begins to lag and that results in delays in transmission of triggers.
As mentioned above, it's not a complete answer, but it is an approach that might be sufficient in some situations. If someone has a more efficient option, please fell free to share!
I'll briefly explain my purpose in case another solution would be much easier.
I have a server and a system. The system sends a "live-stat" packet twice a second and logs the time it was sent (epoch time). The server receives the packet and logs the receiving time.
I would like to know which of the packets arrived and the latency of their arrival.
Currently I'm inserting this data to an unorganized matrix and i would like to compare the two matrices (server and system) to calculate the latency.
Is there any matlab command that could help me?
Another point is that I don't know how to get a numerical value for the latency, is there any formula or tool that i could use? (I''m currently going to show the deviation of the latency on a graph).
I'm talking about 500,000 packets per system and i'm testing this on ~50 systems so i'm looking for the most efficient calculation.
I have the following problem in StreamInsight. I have a query where new tasks from an order came in and trigger an output adapter to make an prediction. The outputadapter writes the predicted task cycle time to a table (in Windows Azure). The prediction is based on neural networks and is plugged in in the outputadapter. After the prediction is written in the table I want to do something else with all the predicted times. So in a second query I want to count the number of written tasks in a time window of 5 minutes. When the number of predicted values saved in the table is equal to the number of tasks in an order, I want to get all the predicted values from the table and make a prediction of the order cycle time.
For this idea I need to make a new event in my outputadapter to know the predicted time is writen in the table. But I don't thinks its possible to enqueue new events in the streaminsight server from an outputadapter.
Maybe this figure makes the problem clear:
http://i40.tinypic.com/4h4850.jpg
Hope someone can help me.
Thanks Carlo
First off, I'm assuming you are using pre-2.1 StreamInsight based on your use of the term "output adapter".
From what you've posted, I would strongly recommend that your adapters do either input or output, but not both. This cuts down on the complexity, makes the implementation easier, and depending on how you wrote the adapter, you now have a reusable piece of code in your solution.
If you are wanting to send data from StreamInsight to your neural network prediction engine, you will need to write an output adapter to do that. Then I would create an input adapter that will get the results from the neural network prediction engine and enqueue the data into StreamInsight. After creating your stream from the neural network prediction engine input adapter, you can use dynamic query composition to share the stream to a Windows Azure storage output adapter and your next query.
If your neural network prediction engine can "push" data to your input adapter, that would be the way to do. If not, you'll have to poll for results.
There is a lot more to this, but it's difficult to drill in to more specifics without more details.
Hope this helps.