Syncing of buffer-transmission with ESP32, I2S MEMS-mic and SD-card (FreeRTOS, PlatformIO, ESP-PROG) - sd-card

i know this forum dislikes "open" questions like this, nevertheless i'd like somebody to help untie the knot in my head, much appreciated.
The goal is simple:
read a stereo 32bit 44100 S/s I2S signal from 2 adafruit sph0645 mics
create a wav-header and store the data onto an SD-card
I've been at this for a few days now and i know that this will be much more complicated than i originally thought. Main reason: signal quality. Like most tutorials on this subject the simplest "hello world" for these mics is a looped polling for I2S-samples. Poll, fill buffer, output via serial or write to SD-card. This returns a choppy, noisy, sped up version of RL-audio. The filling of the internal DMA-buffers can be seen as constant, but the rest is mostly chaos, so
how to i sync these DMA-buffers with the rest of my code?
From experience with the STM32 HAL i'd imagine some register which can be set to throw an interrupt whenever a buffer is full, or an event which can be sent between tasks via queues. Examples on this subject either poll in a main loop with mono an abysmal sample-rate and bit depth or use pages of overkill code and never adress what it does, "just copy and it works", not good. Does the ESP32-Arduino framework provide some way to to this properly? The espressif-documentation isn't something to look forward to, since some of their I2S interface functions don't even work (if you are researching this topic as well, you too might have noticed that i2s_read only returns zeros). Just a hint into the right direction would help, i'm writing my own code anyway. Interrupts? Events? Timers? Polling for full buffers? Only you might know.
have a good one, thx

Thanks to https://github.com/atomic14/ i now have an answer for a syncing-method which works very well. This method has been tried by https://esp32.com/viewtopic.php?t=12546 who also didn't fully understand what was going on: the espressif i2s-interface offers a flag stored in an event which is triggererd every time one of the specified dma-buffers has received a full set of data, ergo, is full. It looks like this:
while(<your condition>){
i2s_event_t evt;
if (xQueueReceive(<your queue>, &evt, portMAX_DELAY) == pdPASS){
if (evt.type == I2S_EVENT_RX_DONE){
size_t bytesRead = 0;
do{
//read data via i2s_read or i2s_read_bytes
} while (bytesRead > 0);
No data is stored in this queue, but rather a flag which can then be used to synchronize dma-filling and further buffering/calculating/sending the read data.
HOWEVER this only works if you install the i2s driver in a specific setup. Instead of using
i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
in your setup, you can activate the "affinity" for events by passing a queue-handle and a lenght:
i2s_driver_install(I2S_NUM_0, &i2s_config, 4, &<your queue>);
hope this helps getting started, it sure did help me.

Related

in web-audio api how to obtain an array(eg. FLOAT32 array) from a stream (eg a microphone stream) for several seconds

I would like to fill an array from a stream for around ten seconds.{I wish to do some processing on the data)So far I can:
(a) obtain the microphone stream using mediaRecorder
(b) use analyser and analyser.getFloatTimeDomainData(dataArray) to obtain an array but it is size limited to only a little over half a second of data.I can also successfully output the data after processing back onto a stream and to outDestination.
(c) I have also experimented with obtaining a 'chunks' array from mediaRecorder directly but the problem then is that I can't find any mime type that would give me a simple array of values - ie an uncompressed sample by sample single channel set of value - ie a longer version of 'dataArray' in (b).
I am wondering if I am missing a simple way round this problem?
Solutions I have seen tend to use step (b) and do regular polls then reassemble a longer array - however it seems the timing is a bit tricky ..
I'v also seen suggestions to use audio workouts - I might have to do this but would prefer a simpler solution!
Or again, if someone knows how to drive mediaRecorder to output the chunks array in a simple array format FLOAT32.of one channel.That would do the trick.
Or maybe I'm missing something simpler?
I have code showing those steps that have been successful and will upload if anyone requests.

Filtering an audio signal and then reading the meter without sending it to master

I'm trying to filter a signal and then analyse the values of the filtered signal using Tone.js / Web-Audio API.
I'm expecting to get values of the filtered signal, but I only get -Infinity, meaning that my connections between the nodes are wrong. I've made a small fiddle demonstrating this, however in my use-case I do not want to send this node to the destination of the context - I only want to analyse it, not hear it.
osc.connect(filter)
filter.connect(gainNode)
gainNode.connect(meter)
console.log(meter.getLevel())
I guess you tested the code in Chrome because there is a problem with Chrome which causes it to not process anything until it is connected to the destination. When using Tone.js that means you need to call .toMaster() at the end of your chain. I updated you fiddle to make it work: https://jsfiddle.net/8f7abzoL/.
In Firefox calling .toMaster() is not necessary therefore the following works in Firefox as well: https://jsfiddle.net/yrjgfdtz/.
After some digging I've found out that I need to have a scriptProcessorNode - which is apparently no longer recommended - so looking into Audio Worklet Nodes

Any way to block while reading an XBox 360 controller other than HID API?

I'm trying to read from my XBox 360 controller without polling it. (To be precise, I'm actually using a Logitech F310, but my Windows 10 PC sees it as an XBox 360 controller.) I've written some rather nasty HID code that uses overlapping I/O to block in a thread on two events, one that indicates there is a report ready to read from the HID device, the other indicating the UI thread has requested the HID thread to exit. That works fine, but the HID driver behaves somewhat differently than XInput does. In particular, it consolidates the two triggers into a single value, only passing their difference (on the curious claim that games expect HID values to be 0x80 when the player's finger is off the control). XInput treats them as two distinct values, which is a big improvement. Also, XInput reports the hat switches as four bits, which means you can actually get ten states out of it: unpressed, N, NE, E, SE, S, SW, W, NW, and all-down (that last might be hard to use successfully, but at least it's there if you want it; I've been using it to exit my polling loop).
The downside, to me, of XInput is that there appears to be no way to block on a read request until the controller changes one of its values or buttons. As an HID device, the ReadFile call will block (more exactly, WaitForMultipleEvents blocks until there is data available). XInput seems to anticipate polling. For a game that would naturally be written to poll the controller as often as it updated the game state (maybe once for each new video frame displayed, for example), that makes sense. But if you want to use the controller for some other purpose (I'm working on a theatrical application), you might want a purely asynchronous system like the HID API supplies. But, again, the HID API combines the two value triggers.
Now, when you read the device with XInput, not only do you get the state of all the controls, you also get a packet number. MSDN says the packet number only changes when the state of a control changes. That way, if consecutive packet numbers are the same, you don't have to bother with any processing after the first one, because you know the controller state hasn't changed. But you are still polling which, to me, is somewhat vulgar.
What intrigues me, however, is that when I put a big delay in between my polls (100ms) I can see that the packet numbers go up by more than one when the value controls (the triggers or sticks) are being moved. This, I think, suggests that the device is sending packets without waiting to be polled, and that I am only getting the most recent packet each time I poll. If that is the case, it seems that I ought to be able to block until a packet is sent, and react only when that happens, rather than having to poll at all. But I can't find any indication that this is an option. Because I can block with the HID API, I don't want to give up without trying (including asking for advice here).
Short of writing my own driver for the controller (which I'm not sure is even an option without proprietary documentation), does anyone know how I can use overlapping I/O (or any other blocking method) to read the XBox 360 controller the way XInput does, with the triggers as separate values, and the hat as four buttons?
Below is some code I wrote that reads the controller and shows that the packet numbers can jump by more than one between reads:
#include <Windows.h>
#include <Xinput.h>
#include <stdio.h>
#define MAX_CONTROLLERS 4
int main()
{
DWORD userIndex;
XINPUT_STATE xs;
XINPUT_VIBRATION v;
XInputEnable(TRUE);
// Which one are we?
for (userIndex = 0; userIndex < XUSER_MAX_COUNT; ++userIndex)
if (XInputGetState(userIndex, &xs) == ERROR_SUCCESS)
break;
if (userIndex == XUSER_MAX_COUNT)
{
printf("Couldn't find an Xbox 360 controller.\n");
getchar();
return -1;
}
printf("Using controller #%1d.\n", userIndex);
while (TRUE)
{
DWORD res = XInputGetState(userIndex, &xs);
printf("%5d %6d: %3d %3d %3d %3d %3d %3d 0x%04X\n",
res,
xs.dwPacketNumber,
xs.Gamepad.bLeftTrigger & 0xFF,
xs.Gamepad.bRightTrigger & 0xFF,
xs.Gamepad.sThumbLX & 0xFF,
xs.Gamepad.sThumbLY & 0xFF,
xs.Gamepad.sThumbRX & 0xFF,
xs.Gamepad.sThumbRY & 0xFF,
xs.Gamepad.wButtons);
if (xs.Gamepad.wButtons == 0x000F) // mash down the hat
break;
Sleep(100);
}
getchar();
return 0;
}
Please note that DirectInput isn't much help, as it also combines the triggers into one value.
Thanks!
Not sure there is any advantage to this, but could you write a thread that polls on a regular interval and then sets a semaphore (or some other signal) when the state has changed. Then your main thread could block waiting for the signal from the polling thread. But potentially there might not be any advantage to this system because on some controllers the values of the thumbsticks change slightly ever frame whether you move them or not. (Noise) You could of course ignore small changes and only signal your semaphore when a large change occurred.

Improve Arduino WiFly latency using protol

I have an Arduino with a WiFly shield, everything works perfectly!
The thing is, when I want to turn on an LED, I open in my
webbrowser:
192.168.1.120/ledon/
(I made a program which handles this URL).
But the thing is; when I make a request, I must wait 1-2 seconds before I can do another one.
So, it is very long, and if I want to control motors, it is just too long.
So, instead of using an HTTP request, I want to use something else which can be faster.
Something "super fast".
I just need to tell the Arduino:
- go direction 1
- go direction 2...
- turn on LED
- turn off LED
- tell me the light level (which return a int)
So it is just about a small amount of data.
Can you show me a way? (Telnet, UDP, OSC?)
For your arduino, have a look at just using sockets or even encoding the data in the URL requested.
You shouldnt get less than about 0.8 Seconds Lag maximum.
How big is your program for handling the Url /ledon/ ?
Using pure packets (usually TCP) from your computer to the arduino is faster sometimes..
But you may need to code a application to handle the packets on the pc.
There is the option of Javascript to parse data back and forth e.g. reading the light level and such.

iRobot Create not returning sensor data

I am trying to stream sensor data from the iRobot Create. I get tuple out of range errors when I try
bot.stream_sensors(somenumber) and bot.poll_sensors(somenumbers). Whenever I input bot.sensors, I just get an empty array {}. I have even tried sending bot.sensors while pushing in on the bump sensor, still getting an empty array. I am connected to the bot through the Serial port with a serial-to-usb converter on my side. The only code before trying to get the sensor data is
import openinterface
bot = openinterface.CreateBot(com_port="/dev/ttyUSB0", mode="full")
Does anyone have an idea of how to solve this issue? Everywhere else just uses stream_sensors(6) and it seems to work fine.
P.S. I posted a question similar to this topic not too long ago, but no one responded. Not trying to spam, but now I have a more clear question and what the apparent-problem is so I thought I would try again.
I downloaded openinterface.py from this site: which included some sample programs. I'd suggest you take a step back, try the sample code, try to find other, more sophisticated, sample code and play with that first before moving on to your real code. You may be missing a step somewhere.
I may be a bit late to answer this, but for reference purposes. Directly controlling the iRobot is simplified greatly by using
Pyrobot.