real time audio processing tool in C++ - real-time

I am trying to record and calculate its spectrum in real time. I wonder what library I should use for this purpose. I need to integrate it into my original C++ code and combine with my image feature. I looked into OpenSmile and it seems not suit for this purpose.
Any suggestions are appreciated!!

Here are the pretty good comparison and evaluation of various audio feature extraction toolboxes.
Main conclusion based on the paper mention in the link below:
1) Essentia: Full function workflow environment for high and low-level features, facilitating audio input, preprocessing and statistical
analysis of output. Written in C++, with Python binding and
export data in YAML or JSON format.
2) Marsyas: Full real-time audio processing standalone framework
for data flow audio processing with GUI and CLI. This program
includes a low-level feature extraction tool built in C++, with an ability
to perform machine learning and synthesis within the framework.The
feature extraction aspects have also been translated to
Vamp plugin format
3)YAAFE Low level feature extraction library designed for computational
efficiency and batch processing by utilizing data flow
graphs, written in C++ with a CLI and bindings for Python and
Matlab
You can find the more details about Graph of Percentage Coverage of Multiple Feature Sets and Graph of Computational Time of Feature Extraction
Tools in the
link.

Related

How do I implement matlab code on hardware device to make it run?

My question is quite basic to most appropriate levels on consideration. I lack the perception of how can I dump or implement my matlab code on a hardware component like processors or fpga? For eg:
Suppose I create an image processing matlab (object classification/detection) code which needs to detect real time images from drone and identify whether object is human or animal while its one air through the vision of camera, how shall I proceed to implement this matlab code onto a processor or controller and make it run while the drone is on air?
For consideration, take the matlab code is in its raw form for processing any input data and put out an output classified data. What should I do next? Do I need to convert the matlab code to any hdl or .exe format to run it across the hardware platform or is it possible to implement matlab code(.m format) directly into a device for processing and classification. Basically I am not getting how to practically use matlab code and put it into a system. Do I need to use some sort of toolbox or extension code ?
Could you please list down the steps for this process or share some link of website or youtube videos where it has been shown in detail how to harbour this mechanism.
You could use MATLAB Coder to convert your MATLAB code to an executable that can run on your hardware.
Here are a couple of articles from the official MathWorks documentation regarding Code Generation for Image Processing to get you started:
https://www.mathworks.com/help/images/code-generation-for-image-processing.html
https://www.mathworks.com/help/images/code-generation-with-cell-detection.html

Matlab: interfacing with Windows C++ executable

In my team, we are using a high-performance C++ program to read data from the network. We want to place such data in a shared memory buffer in our C++ process for reading in a separate Matlab process that will further asynchronously process the data and provide a display interface.
All this is running on Windows.
What of the many cross-language mechanisms in Matlab is best suited to this purpose?
Thanks!
The best strategy is to use a memory-mapped file to provide data from one component and parse it from another. It appears Matlab's locking primitives are somewhat primitive, but fully worked out examples are on the Matworks website, including a simple chat application passing data between two Matlab instances.

learn training file new letters in windows (for c# app)

I would like to learn my training file for tesseract new letters. I want use win 10 (I won't use linux) - for use tesseract Nuget-package in c#.net app.
I tried jTessBoxEditor but it's not working (first time error in registry, than cannot found fonts, than problem with java, than text2image doesn't work properly...). Editor SunnyPage could not even load the image without fail.
which program use for separating letters and creating training file as windows user
should I use tesseract or other OCR engine? It looks like tesseract isn't windows-user friendly
please post example training file for this three images - if there is any need of preprocessing (scale etc.) it should be done programaticaly (c#.net)
Which program use for separating letters and creating a training file?
Try this one: https://github.com/skotz/captcha-breaking-library
or:
OpenCV
OpenCV is a popular framework for computer vision and image processing. It is easy to use OpenCV to process the CAPTCHA images. It has a Python API so you can use it directly from Python.
Keras
Keras is a deep learning framework written in Python. It makes it easy to define, train and use deep neural networks with minimal coding.
TensorFlow
TensorFlow is Google’s library for machine learning. If you will be coding in Keras, but Keras doesn’t actually implement the neural network logic itself. Instead, it uses Google’s TensorFlow library behind the scenes to do the heavy lifting.
This involves either brute-forcing the captcha or running OCR algorithms on it to try and detect what is written in the captcha.
If you want to implement your own CAPTHA an algorithm please look into that abstract: http://cmp.felk.cvut.cz/~cernyad2/TextCaptchaPdf/DESIGNING%20CAPTCHA%20ALGORITHM%20SPLITTING%20AND%20ROTATING.pdf
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.800.3065&rep=rep1&type=pdf

Extensive comparison between SIMULINK and LabVIEW

I am trying to determine which of these two to buy for my work. I have used SIMULINK but not LabVIEW. Is there anyone who has used both and would like to provide some details? My investigation criteria are the user friendliness, availability of libraries and template functions, real-time probing facility, COTS hardware interfacing opportunity, quality of code generation, design for testability (i.e. ease of generating unit/acceptance tests), etc. However, if anyone would like to educate me with more criteria, please do so by all means!
For anyone who does not know about SIMULINK and LabVIEW - These are both Domain-Specific Languages (DSLs) intended for graphical dataflow modelling (and also code generation). These are multi-industrial tools and quite heavily used for engineering design and modelling.
IMPORTANT - I am quite interested to know if SIMULINK and LabVIEW offer real-time probing. For example, I have a model that I want to simulate. If there are variables associated to certain building blocks in that model, could I view them changing as the simulation continues? I know that it is certainly not possible with SIMULINK as it has a step-by-step debugger. I am not aware of anything similar in LabVIEW.
I really have not used LabVIEW and cannot obtain it temporarily as my work internet has got download restrictions and administrative privilege issues. This is the reason why I simply cannot use only NI website to draw conclusions. If there is any white paper available that addresses this issue, I would also love to know :)
UPDATE SINCE LAST POST
I have used MATLAB code generator and will not say that it is the best. However, I hear now that SIMULINK Embedded Coder is the best code generator and almost one of its own kind. Can anyone confirm whether or not this is good for safety critical system design i.e. generating code from safety-critical subsystem models. I know that the Mathworks is constantly trying to close the gap to achieve fully-flexible production-level C/C++ code generation.
I know that an ideal answer would be,"Depending on what you are trying to do, use a bit of both". And interestingly, I think I am heading to that direction. ATEOTD, it is a lot of money and need to be spent "nicely".
Thanks in advance.
I used labVIEW from 1995, and Simulink from 2000. Now I am involved in control system design, and simulation of robotic systems using labVIEW Real Time and automotive ECUs using MATALAB/Simulink/DSPACE .
LabVIEW is focus on measurement systems, and MATLAB/SIMULINK in dynamic simulation, so,
If you run complex simulations, and your work is create/debug complex simulation models of controllers or plants, use Simulink+RealTimeWorkShop+StateFlowChart. LabVIEW has no eficient code generators for dynamic simulation. RTW generates smaller and fastest code.
If your main work is developing systems with controllers and GUI for machines, or you want to deploy the controllers on field, use labVIEW.
If your main work is developing flexible HIL or SIL systems, with a good GUI, you can use VeriStand. Veristand can mix Simulink and LabVIEW code.
And if you have a big budget ( VERY BIG ) and you are working in automotive control prototypes, DSPACE hardware is a very good choice for fast development of automotive ECUS, or OPAL to develope electric power circuits. But only for prototype or HIL testing of controllers.
From the point of view of COTS hardware:
Mathworks don´t manufacture hardware -> Matlab/Simulink support hardware from several vendors.
National Instruments produce/sell hardware->LabVIEW Real Time is focused in support NationalInstruments hardware. There are no COTS full replacement.
I have absolutely no experience with Simulink, so I'll comment only on LV, although a quick read about Simulink on Wikipedia seems to indicate that it's focused mainly on simulation and modelling, which is certainly not the case with LabVIEW.
OK, so first of all, LV is NOT a DSL. While you wouldn't want to use it for any project, it's a general purpose programming language and you should take that into account. I know that NI has a simulation toolkit for LV, which might help you if that's what you're after, but I have absolutely no experience with it. The images I saw of it seemed to indicate that it adds a special kind of diagram to LV for simulation.
Second, LV is not restricted to any kind of hardware. It's a general purpose language, so you can write code which won't use any hardware at all, code which will use or run on NI's hardware or code which will use any hardware (be it through DLL calls, .NET assemblies, RS232, TCP, GPIB or any other option you can think of). There is quite a large collection of LV drivers for various devices and the quality of the driver usually depends on who wrote it.
Third, you can certainly probe in real time in LV. You write your code, just as you would in C or Java, and when you run it, you have several debugging options:
Single stepping. This isn't actually all that common, partially because LV is parallel.
Execution highlighting. This runs the code in slow motion, while showing all the values in the various wires.
Probes, which show you the last value that each wire had, where wires fill the same function that variables do in text based languages. This updates in real time and I assume is what you want.
Retain wire values, which allows you to probe a wire even after data passed through it. This is similar to what you get in text based IDEs with variables. In LV you don't usually have it because wire values are transient, so the value is not kept around unless you explicitly ask for it.
Of course, since you're talking about code, you could also simply write the code to display the values to the screen on a graph or a numeric indicator or to log them to a file, so there should be no need for actual probing. You could also add analysis code, etc.
Fourth, you could try downloading and running LV in a fully functional evaluation mode. If I remember correctly, NI currently gives you 7 days and then 45 days if you register on their site. If you can't do that on a work computer, you could try at home. If your problem is only with downloading, you could try contacting your local NI office and asking them to send you a DVD.
Note that I don't really know anything about modelling and simulation, so I have no idea what kind of code you would actually have to write in order to do what you want. I assume that if NI has a special module for it, then it's not something that you can completely cover in regular code (at least not if you want the original notation), but I would say that if you could write the code that does what you want in C, there's no reason you shouldn't be able to write it in LV (assuming, of course, that you know how to write code in LV).
A lot of the best answer would have to depend on your ultimate design requirements. Are you developing a product? If so, in what stage of development are you? Or are you doing research?
I recently did a comparison just as you are doing. I know LV, but was wanting to move towards a more hardware-scalable option, since NI HW is very expensive in volume. That is, my company was wanting to move towards a product. What LV and NI HW give you is flexibility. You can change code very quickly compared to C. On the other hand, LV does not run on nearly as many different HW platforms as C. So I wanted to find an inexpensive platform that would work well for real-time control and data acquisition, such that if we wanted to sell a product for, say, $30k, our controller wouldn't be costing $15k of that. We ended up with Diamond Systems Linux SBC's. Interestingly, Simulink ended up using the most expensive hardware! It did have a lot of flexibility, and could generate code, as well as model plants and controllers. But then, LV can do that as well.
As Yair wrote, LV has plenty of good debugging tools. One of the more interesting tools that is not so well known is the Suspend when Called option for a SubVI. This allows you to play with the inputs and outputs of a SubVI as much as you want while execution is paused.
MATLAB and Simulink are the defacto standard for control system design and simulation. Simulink controller models can be used for offline simulation in conjunction with plant models, all the way to realtime implementation on embedded targets. It is a general simulation framework with extensive built-in libraries as well as a la carte special purpose libraries, and can be extended through creation of custom blocks (S-function blocks) in C and other languages. It includes the ability to display values in graphs, numeric displays, gages, etc. while a nonrealtime simulation is taking place. Realtime target support from The Mathworks includes x86 (xPC Target) and several embedded targets (MPC555, etc.), and there is 3rd party support for other targets. The aforementioned dSPACE provides complete prototyping controllers including support for their quite powerful hardware. xPC Target includes support for a plethora of COTS PC data acquisition cards. Realtime target support includes GUI elements such as graphs, numeric displays gages, etc.
As I understand it (I have never really used it in anger), LabView only supports NI hardware, and is more hardware-oriented. Simulink supports hardware from multiple vendors, be it for data acquisition, or real-time implementation, but it may require a bit more work for the user to interface to his or her own hardware (less plug & play than LabView). On the other hand, Simulink provides tools to support the whole model-based design process, from modelling & simulation, control design, verification & validation, code generation, hardware-in-the-loop, etc...
Disclaimer: I used to work for MathWorks.
You guys may really be interested in Control Design adn Simulation Module for LabVIEW. It does a lot of simulations and in the future may be competitive to Simulink. I'm not a control engineer but I use it sometimes for simple testing and I'm glad that I don't have to learn Simulink from the beginning to do some work since I'm familiar with LabVIEW philosophy.

easiest tool to use for a extreme beginner for classification/clustering

I saw that the tool weka is having a gui interface. This gui interface is very easy for non coding users to classify data sets into classes. Matlab is very difficult since say for example making a neural network you need to write code and to do that you need to have a solid understanding of whats going on. Are there other tools like weka or else is there a plugin to matlab that gives more power to it?
RapidMiner has a functional GUI, and will work for both classification and clustering. It is the most popular open-source (free) data mining application available as of 2012.
RapidMiner: http://rapid-i.com/
It also has numerous training videos and tutorials that you can follow along with - I learned basic clustering methods using a K-means cluster method in about 3 hours. See the Vancouver Data blog for some great RapidMiner analytics videos. Top-notch stuff, really.
Vancouver Data (Neil McGuigan): http://vancouverdata.blogspot.com/
As a bonus, you can install the Weka plug-in, which then gives you GUI Weka. All of the add-ons are free and well-integrated. Other add-ons include a GUI 'R' (the stats program), Reporting Services, Text and Web Analytics, etc. It is fairly simple to use straight 'out of the box' (IMO).
Weka is very (very) powerful and you can write your own classifier if that's what you need to do.
Between Matlab and Weka there's pretty much nothing you can't do in terms of Machine Learning.
You might want to check out Netlab toolkit for Matlab, which is a neural network toolkit developed by a Professor at Aston University - it is available from http://www1.aston.ac.uk/eas/research/groups/ncrg/resources/netlab/