Web browser support for the opus audio codec is normally utilized by delivering an entire opus-encoded file to the browser, and this is known to work for firefox and chrome, for example. My scenario is different in that I stream opus packets from server to browser.
On a linux server I encode audio with opus_encode_float. It is delivered to a web browser client via WebSocket. In the browser I use the Web Audio API's decodeAudioData to try to decode that same data. It fails in firefox and chrome with a null exception.
Seems to me this should work, if not last year then Real Soon Now. Can anyone either tell me the status of the opus implementations in the browsers, or tell me what I'm doing wrong? Thanks in advance.
// .h
#define OPUS_FRAME_SIZE 1920
#define MAX_FRAME_SIZE 6*OPUS_FRAME_SIZE
#define MAX_PACKET_SIZE (4*OPUS_FRAME_SIZE)
class OpusEncoder; // forward declaration as we don't want to include opus.h here
class xxx {
OpusEncoder *encoder; // Holds the state of the opus encoder
unsigned char opusOutBuf[MAX_PACKET_SIZE];
}
// .cpp
#include <opus.h>
#define CHANNELS 2
#define SAMPLE_RATE 48000
#define APPLICATION OPUS_APPLICATION_AUDIO
#define BITRATE 64000
// one time code
// Create a new encoder state
int err;
encoder = opus_encoder_create(SAMPLE_RATE, CHANNELS, APPLICATION, &err);
if (err<0)
{
LogIt(prError) << "Failed to create an Opus encoder: " << opus_strerror(err);
return;
}
err = opus_encoder_ctl(encoder, OPUS_SET_BITRATE(BITRATE));
if (err<0)
{
LogIt(prError) << "Opus failed to set bitrate: " << opus_strerror(err);
return ;
}
// per packet code
int nbBytes = opus_encode_float(encoder, IncomingAudio, OPUS_FRAME_SIZE, opusOutBuf, MAX_PACKET_SIZE);
if (nbBytes < 0)
{
LogIt(prError) << "Opus encode failed: " << opus_strerror(nbBytes);
return;
}
// Client side javascript
// OpusEncodedArrayBuffer is an unmodified binary packet that
// arrived via websocket onmessage(evt); it is evt.data
window.context.decodeAudioData(OpusEncodedArrayBuffer, function(buffer) { // use "classic" callback
if (buffer) { // I would LIKE to arrive here, but so far no joy.
// ...
}
},
function(e){
if (e) {
tell("error in decodeAudioData: " + e) // I haven't seen this case yet
} else { // This ALWAYS happens, using latest firefox or chrome
tell("error in decodeAudioData"); // webaudio api spec says decodeAudioData does not return an object error
}
});
Chrome's decodeAudioData function does not support opus. There's an open bug on this issue.
Related
I am following Pixabay API documentation to retrieve/download images.
I don't have a lot of understanding of URI/REST/HTTP workings but I was able to follow some documentation and get boilerplate code:
int main()
{
auto fileStream = std::make_shared<ostream>();
//Open stream for output file
pplx::task<void> requestTask = fstream::open_ostream("results.html")
.then([=](ostream outFile) {
http_client client("https://pixabay.com/");
uri_builder builder("/api/");
builder.append_query("key", "xxxxxxx-xxxxxx-xxxxxxx");
builder.append_query("q", "yellow%20flowers");
builder.append_query("image_type", "photo");
std::cout << builder.to_string() << std::endl;
return client.request(methods::GET, builder.to_string()); })
// Handle the response headers arriving
.then([=](http_response response) {
printf("Received response status code: %u\n", response.status_code());
return response.body().read_to_end(fileStream->streambuf()); })
// Close the file stream.
.then([=](size_t) {
return fileStream->close(); });
// Wait for all the outstanding I/O to complete and handle any exceptions
try {
requestTask.wait();
}
catch (const std::exception &e) {
printf("Exception: %s\n", e.what());
}
return 0;
}
Problem : This code always gives me status code 301.
If I directly run https://pixabay.com/api/?key=xxxxxxx-xxxxxx-xxxxxxx&q=yellow+flowers&image_type=photo&pretty=true this link in the browser, I am getting the JSON data back. I am not sure if I am able to build this URI correctly through URI builder using the above code.
Some variation of the code that I tried involves commenting out query parameter q , removing/adding / from http_client/uri_builder but none of that worked.
Please help me understand what is the correct way of getting this done.
Thanks!
on adding an http_client_config the same code worked for me.
web::http::client::http_client_config httpConfig;
httpConfig.set_timeout(std::chrono::seconds(25));
httpConfig.set_validate_certificates(false);
and use this config while creating http_client
http_client client("https://pixabay.com/", httpConfig);
The C++ REST SDK has built-in support for automatically following 301 redirects when making HTTP requests. By default, the SDK will automatically follow up to 5 redirects.
I'm trying to read a MODBUS sensor via an ESP32.
I'm using the following library: https://github.com/emelianov/modbus-esp8266
I have the following code:
#include <ModbusRTU.h>
#include <SoftwareSerial.h>
SoftwareSerial modBusSerial;
ModbusRTU modbus;
#define startReg 100
#define endReg 123
uint16_t res[endReg - startReg + 1];
// Callback to monitor errors in the modbus
bool cb(Modbus::ResultCode event, uint16_t transactionId, void* data) {
if (event != Modbus::EX_SUCCESS) {
Serial.print("Request result: 0x");
Serial.print(event, HEX);
}
return true;
}
void setup() {
Serial.begin(115200); // Default serial port (Hardware serial)
modBusSerial.begin(9600, SWSERIAL_8E1, MB_RX, MB_TX); // modbus configuration SWSERIAL_8E1 = 8 bits data, even parity and 1 stop-bit
modbus.begin(&modBusSerial);
modbus.master();
Serial.println("starting modbus...");
while (true) {
Serial.println(modBusSerial.read());
res[endReg - startReg] = 0; // string terminator to allow to use res as char*
if (!modbus.slave()) {
modbus.readIreg(16, startReg, res, endReg - startReg, cb);
}
modbus.task();
Serial.print("result: ");
Serial.println((char*) res);
delay(1000); // every second
}
}
Response I get;
When I do the exact same in QModMaster, I do get the correct output. Anyone any idea what I'm doing wrong here?
These are the settings I use;
I am aware of the "wrong" address in my code. I have 2 identical sensors but one is connected to my computer while the other one is connected to my ESP32.
Thanks in advance!
I need to play sounds upon certain events, and want to minimize
processor load, because some image processing is being done too, and
processor performance is limited.
For the present, I play only one sound at a time, and I do it as
follows:
At program startup, sounds are read from .wav files
and the raw pcm data are loaded into memory
a sound device is opened (snd_pcm_open() in mode SND_PCM_NONBLOCK)
a worker thread is started which continously calls snd_pcm_writei()
as long as it is fed with data (data->remaining > 0).
Somewhat resumed, the worker thread function is
static void *Thread_Func (void *arg)
{
thrdata_t *data = (thrdata_t *)arg;
snd_pcm_sframes_t res;
while (1)
{ pthread_mutex_lock (&lock);
if (data->shall_stop)
{ data->shall_stop = false;
snd_pcm_drop (data->pcm_device);
snd_pcm_prepare (data->pcm_device);
data->remaining = 0;
}
if (data->remaining > 0)
{ res = snd_pcm_writei (data->pcm_device, data->bufptr, data->remaining);
if (res == -EAGAIN) continue;
if (res < 0) // error
{ fprintf (stderr, "snd_pcm_writeX() error: %s\n", snd_strerror(result));
snd_pcm_recover (data->sub_device, res);
}
else // another chunk has been handed over to sound hw
{ data->bufptr += res * bytes_per_frame;
data->remaining -= res;
}
if (data->remaining == 0) snd_pcm_prepare (data->pcm_device);
}
pthread_mutex_unlock (&lock);
usleep (sleep_us); // processor relief
}
} // Thread_Func
Ok, so this works well for one sound at a time. How do I play various?
I found dmix, but it seems a tool on user level, to mix streams coming
from separate programs.
Furthermore, I found the Simple Mixer Interface in the ALSA Project C
Library Interface, without any hint or example or tutorial about how
to use all these function described by one line of text each.
As a last resort I could calculate the mean value of all the buffers
to be played synchronously. So long I've been avoiding that, hoping
that an ALSA solution might use sound hardware resources, thus
relieving the main processor.
I'd be thankful for any hint about how to continue.
I am writing a cross-platform socket handling library (which also handles serial and a whole bunch of other protocols in a protocol agnostic way. - I am not re-inventing the wheel).
I need to emulate the Linux poll function. The code I started with used select and worked fine, but there was no way to interrupt it from another thread, and so I was forced to start using event objects. My initial attempt called:
WSACreateEvent()
WSAEventSelect() to associate the socket with the event object.
WaitForMultipleObjectsEx() to wait on all sockets plus my interrupt event object.
select() to work out what events actually occurred on the socket.
accept()/send()/recv() to process the sockets (later and elsewhere).
This failed. accept() was claiming that the file descriptor was not a socket. If I commented out the call to WSAEventSelect(), essentially reverting to my earlier code, it all works fine (except that I cannot interrupt).
I then realised that I did something wrong (according to the Microsoft dictatorship). Instead of using select() to work out what events have happened on each socket, I should be using WSAEnumNetworkEvents(). So I rewrote my code to do it the proper way, remembering to call WSAEventSelect() afterwards to disassociate the event object from the file descriptor so that (fingers crossed) accept() would now work.
Now WSAEnumNetworkEvents() is returning an error and WSAGetLastError() tells me that the error is WSAENOTSOCK.
This IS a socket. I am doing things the way MSDN tells me I should (allowing for the general poor quality of the documentation). It appears however that WSAEventSelect() is causing the file descriptor to be marked as a file rather than a socket.
I hate Microsoft so much right now.
Here is a cut down version of my code:
bool do_poll(std::vector<struct pollfd> &poll_data, int timeout)
{
...
for (const auto &fd_data : poll_data) {
event_mask = 0;
if (0 != (fd_data.events & POLLIN)) {
// select() will mark a socket as readable when it closes (read size = 0) or (for
// a listen socket) when there is an incoming connection. This is the *nix paradigm.
// WSAEventSelect() hasseparate events.
event_mask |= FD_READ;
event_mask |= FD_ACCEPT;
event_mask |= FD_CLOSE;
}
if (0 != (fd_data.events & POLLOUT)) {
event_mask |= FD_WRITE;
}
event_obj = WSACreateEvent();
events.push_back(event_obj);
if (WSA_INVALID_EVENT != event_obj) {
(void)WSAEventSelect((SOCKET)fd_data.fd, event_obj, event_mask);
}
}
lock.lock();
if (WSA_INVALID_EVENT == interrupt_obj) {
interrupt_obj = WSACreateEvent();
}
if (WSA_INVALID_EVENT != interrupt_obj) {
events.push_back(interrupt_obj);
}
lock.unlock();
...
(void)WaitForMultipleObjectsEx(events.size(), &(events[0]), FALSE, dw_timeout, TRUE);
for (i = 0u; i < poll_data.size(); i++) {
if (WSA_INVALID_EVENT == events[i]) {
poll_data[i].revents |= POLLERR;
} else {
if (0 != WSAEnumNetworkEvents((SOCKET)(poll_data[i].fd), events[i], &revents)) {
poll_data[i].revents |= POLLERR;
} else {
if (0u != (revents.lNetworkEvents & (FD_READ | FD_ACCEPT | FD_CLOSE))) {
poll_data[i].revents |= POLLIN;
}
if (0u != (revents.lNetworkEvents & FD_WRITE)) {
poll_data[i].revents |= POLLOUT;
}
}
(void)WSAEventSelect((SOCKET)(poll_data[i].fd), NULL, 0);
(void)WSACloseEvent(event_obj);
}
}
...
}
Is it possible to connect iMac to iPhone through the dock connector ? Also I am using the EAAccessory framework but I am not get any notifications when I connect the serial cable to iPhone. If any one knows about this please give me a suggestion.
There seems to be a lot of confusion regarding EAAccessoryManager and the accompanying classes. These are only for use with MFI sanctioned hardware i.e. those manufacturers that have taken the time to follow the MFI program and integrate the requisite hardware into their device.
The use of a chip and the establishment of a protocol allows for the facilitation of a communication stream between the app and the device.
Sam is correct in his statement but there is no need to jailbreak your app in order to bypass the MFI program however, you will not be able to submit to the App Store if you follow the procedure I am about to briefly outine:
Obtain all of the header files for the IOKit iOS OpenSource Browser
Add them to your solution
Include the IOKit.h framework
Head to http://www.arduino.cc/playground/Interfacing/Cocoa#IOKit for an example of how to use IOKit and icotl commands to achieve what you require
I have successfully implemented IOKit operations on an iOS device so this is definitely possible as long as you are willing to forego submission to the App Store.
Raj..
for get notification for connection accessory first of you have to register your accessory to MFI. otherwise you have to jailbreak your iPhone. for jail break check this code
#include <stdio.h> /* Standard input/output definitions */
#include <string.h> /* String function definitions */
#include <unistd.h> /* UNIX standard function definitions */
#include <fcntl.h> /* File control definitions */
#include <errno.h> /* Error number definitions */
#include <termios.h> /* POSIX terminal control definitions */
static struct termios gOriginalTTYAttrs;
static int OpenSerialPort()
{
int fileDescriptor = -1;
int handshake;
struct termios options;
// Open the serial port read/write, with no controlling terminal, and don't wait for a connection.
// The O_NONBLOCK flag also causes subsequent I/O on the device to be non-blocking.
// See open(2) ("man 2 open") for details.
fileDescriptor = open("/dev/tty.iap", O_RDWR | O_NOCTTY | O_NONBLOCK);
if (fileDescriptor == -1)
{
printf("Error opening serial port %s - %s(%d).\n",
"/dev/tty.iap", strerror(errno), errno);
goto error;
}
// Note that open() follows POSIX semantics: multiple open() calls to the same file will succeed
// unless the TIOCEXCL ioctl is issued. This will prevent additional opens except by root-owned
// processes.
// See tty(4) ("man 4 tty") and ioctl(2) ("man 2 ioctl") for details.
if (ioctl(fileDescriptor, TIOCEXCL) == -1)
{
printf("Error setting TIOCEXCL on %s - %s(%d).\n",
"/dev/tty.iap", strerror(errno), errno);
goto error;
}
// Now that the device is open, clear the O_NONBLOCK flag so subsequent I/O will block.
// See fcntl(2) ("man 2 fcntl") for details.
if (fcntl(fileDescriptor, F_SETFL, 0) == -1)
{
printf("Error clearing O_NONBLOCK %s - %s(%d).\n",
"/dev/tty.iap", strerror(errno), errno);
goto error;
}
// Get the current options and save them so we can restore the default settings later.
if (tcgetattr(fileDescriptor, &gOriginalTTYAttrs) == -1)
{
printf("Error getting tty attributes %s - %s(%d).\n",
"/dev/tty.iap", strerror(errno), errno);
goto error;
}
// The serial port attributes such as timeouts and baud rate are set by modifying the termios
// structure and then calling tcsetattr() to cause the changes to take effect. Note that the
// changes will not become effective without the tcsetattr() call.
// See tcsetattr(4) ("man 4 tcsetattr") for details.
options = gOriginalTTYAttrs;
// Print the current input and output baud rates.
// See tcsetattr(4) ("man 4 tcsetattr") for details.
printf("Current input baud rate is %d\n", (int) cfgetispeed(&options));
printf("Current output baud rate is %d\n", (int) cfgetospeed(&options));
// Set raw input (non-canonical) mode, with reads blocking until either a single character
// has been received or a one second timeout expires.
// See tcsetattr(4) ("man 4 tcsetattr") and termios(4) ("man 4 termios") for details.
cfmakeraw(&options);
options.c_cc[VMIN] = 1;
options.c_cc[VTIME] = 10;
// The baud rate, word length, and handshake options can be set as follows:
cfsetspeed(&options, B19200); // Set 19200 baud
options.c_cflag |= (CS8); // RTS flow control of input
printf("Input baud rate changed to %d\n", (int) cfgetispeed(&options));
printf("Output baud rate changed to %d\n", (int) cfgetospeed(&options));
// Cause the new options to take effect immediately.
if (tcsetattr(fileDescriptor, TCSANOW, &options) == -1)
{
printf("Error setting tty attributes %s - %s(%d).\n",
"/dev/tty.iap", strerror(errno), errno);
goto error;
}
// Success
return fileDescriptor;
// Failure "/dev/tty.iap"
error:
if (fileDescriptor != -1)
{
close(fileDescriptor);
}
return -1;
}
int main(int args, char *argv[])
{
int fd;
char somechar[8];
fd=OpenSerialPort(); // Open tty.iap with no hardware control, 8 bit, BLOCKING and at 19200 baud
if(fd>-1)
{
write(fd,"*",1); // Write handshaking message over serial
///////////////////////////////////////////////////////////////////////////////////////////////////
// After this, our device or our PC program should be strobing serial ground to gain access to the Iphone Serial Line
//////////////////////////////////////////////////////////////////////////////////////////////////
read(fd,&somechar[0],1); // Read 1 byte over serial. This will block (wait) untill the byte has been received
if(somechar[0]=='*') // Check if this byte is a "handshaking" message
{
printf("Serial connection established!\n"); // If it is, we have established a connection to the device and can freely read/write over serial!
while(1) // Do this forever or untill someone presses CTRL+C
{
read(fd,&somechar[0],1); // Read a character over serial!
putchar(somechar[0]); // Write the character to the Terminal!!
}
}
}
return 0;
}
If you connect your iPhone to an iMac and you have either iTunes or Xcode running, you would be able to see that the iPhone is connected.
In Xcode, go to the Organizer window. In iTunes, look under the Devices in the left-hand column.
The EAAccessory framework, which I have not used, should have nothing to do with this.
Here is the answer: Launch specific app when external accessory attached - not all accessories can signal to launch app when they are connected. I think that ordinal USB cable also can't.
About IOKit: You can add nesessary files from Xcode itself - open Xcode.app package. IOKit framework is located in Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks
simply copy absent files (or make link) to
Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.0.sdk/System/Library/Frameworks/IOKit.framework
and to
Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator7.0.sdk/System/Library/Frameworks/IOKit.framework
and also for all simulator platforms
You should know that not all functions of IOKit can work on real device (but they work under simulator) due to iOS sandbox security.
But You can work with IORegistry well. My first project on iOS was implemetation of the IORegistryExplorer.
Good luck!