Audio widget within Jupyter notebook is **not** playing. How can I get the widget to play the audio? - visual-studio-code

I writing my code within a Jupyter notebook in VS Code. I am hoping to play some of the audio within my data set. However, when I execute the cell, the console reports no errors, produces the widget, but the widget displays 0:00 / 0:00 (see below), indicating there is no sound to play.
Below, I have listed two ways to reproduce the error.
I have acquired data from the hub data store. Looking specifically at the spoken MNIST data set, I cannot get the data from the audio tensor to play
import hub
from IPython.display import display, Audio
from ipywidgets import interactive
# Obtain the data using the hub module
ds = hub.load("hub://activeloop/spoken_mnist")
# Create widget
sample = ds.audio[0].numpy()
display(Audio(data=sample, rate = 8000, autoplay=True))
The second example is a test (copied from another post) that I ran to see if it was something wrong with the data or something wrong with my console, environment, etc.
# Same imports as shown above
# Toy Function to play beats in notebook
def beat_freq(f1=220.0, f2=224.0):
max_time = 5
rate = 8000
times = np.linspace(0,max_time,rate*max_time)
signal = np.sin(2*np.pi*f1*times) + np.sin(2*np.pi*f2*times)
display(Audio(data=signal, rate=rate))
return signal
v = interactive(beat_freq, f1=(200.0,300.0), f2=(200.0,300.0))
display(v)
I believe that if it is something wrong with the data (this is a well-known data set so, I doubt it), then only the second one will play. If it is something to do with the IDE or something else, then neither will work, as is the case now.

Apologies for the late reply! In the future, please tag the questions with activeloop so it's easier to sort through (or hit us up directly in community slack -> slack.activeloop.ai).
Regarding the Free Spoken Digit Dataset, I managed to track the error with your usage of activeloop hub and audio display.
adding [:,0] to 9th line will help fixing display on Colab as Audio expects one-dimensional data
%matplotlib inline
import hub
from IPython.display import display, Audio
from ipywidgets import interactive
# Obtain the data using the hub module
ds = hub.load("hub://activeloop/spoken_mnist")
# Create widget
sample = ds.audio[0].numpy()[:,0]
display(Audio(data=sample, rate = 8000, autoplay=True))
(When we uploaded the dataset, we decided to upload the audio as (N,C) where C is the number of channels, which happens to be 1 for the particular dataset. The added dimension wasn't added automatically)
Regarding the VScode... the audio, unfortunately, would still not work (not because of us, but VScode), but you can still try visualizing Free Spoken Digit Dataset (you can play the music there, too). Hopefully this addresses your needs!
Let us know if you have further questions.
Mikayel from Activeloop

Related

in web-audio api how to obtain an array(eg. FLOAT32 array) from a stream (eg a microphone stream) for several seconds

I would like to fill an array from a stream for around ten seconds.{I wish to do some processing on the data)So far I can:
(a) obtain the microphone stream using mediaRecorder
(b) use analyser and analyser.getFloatTimeDomainData(dataArray) to obtain an array but it is size limited to only a little over half a second of data.I can also successfully output the data after processing back onto a stream and to outDestination.
(c) I have also experimented with obtaining a 'chunks' array from mediaRecorder directly but the problem then is that I can't find any mime type that would give me a simple array of values - ie an uncompressed sample by sample single channel set of value - ie a longer version of 'dataArray' in (b).
I am wondering if I am missing a simple way round this problem?
Solutions I have seen tend to use step (b) and do regular polls then reassemble a longer array - however it seems the timing is a bit tricky ..
I'v also seen suggestions to use audio workouts - I might have to do this but would prefer a simpler solution!
Or again, if someone knows how to drive mediaRecorder to output the chunks array in a simple array format FLOAT32.of one channel.That would do the trick.
Or maybe I'm missing something simpler?
I have code showing those steps that have been successful and will upload if anyone requests.

music21: get the voice/program/instrument of midi voice from a flat score?

I have a simple script that uses music21 to process the notes in a midi file:
import music21
score = music21.converter.parse('www.vgmusic.com/music/console/nintendo/nes/zanac1a.mid')
for i in score.flat.notes:
print(i.offset, i.quarterLength, i.pitch.midi)
Is there a way to also obtain a note's voicing / midi program using a flat score? Any pointers would be appreciated!
MIDI channels and programs are stored on Instrument instances, so use getContextByClass(instrument.Instrument) to find the closest Instrument in the stream, and then access its .midiProgram.
Be careful:
.midiChannel and .midiProgram are 0-indexed, so MIDI channel 10 will be 9 in music21, etc., (we're discussing changing this behavior in the next release)
Some information might be missing if you're not running the bleeding edge version (we merged a patch yesterday on this topic), so I advise pulling from git: pip install git+https://github.com/cuthbertLab/music21
.flat is going to kill you, though, if the file is multitrack. If you follow my advice you'll just get the last instrument on every track. 90% of the time people doing .flat actually want .recurse().

Create Jupyter Notebook object that changes after the cell has run

I want to make a progress bar that updates asynchronously within the Jupyter notebook (with an ipython kernel)
Example
In [1]: ProgressBar(...)
Out[1]: [|||||||||------------------] 35% # this keeps moving
In [2]: # even while I do other stuff
I plan to spin up a background thread to check and update progress. I'm not sure how to update the rendered output though (or even if this is possible.)
This might help put you on the right track, code taken from lightning-viz, which borrowed a lot of it from matplotlib. warning this is all pretty underdocumented.
In python you have to instantiate a comm object
from IPython.kernel.comm import Comm
comm = Comm('comm-target-name', {'id': self.id})
full code here https://github.com/lightning-viz/lightning-python/blob/master/lightning/visualization.py#L15-L19. The id is just there in case you want to manage multiple different progress bars for example.
then do the same in javascript:
var IPython = window.IPython;
IPython.notebook.kernel.comm_manager.register_target('comm-target-name', function(comm, data) {
// the data here contains the id you set above, useful for managing
// state w/ multiple comm objects
// register the event handler here
comm.on_msg(function(msg) {
})
});
full example here. Note the javascript on_msg code is untested as I've only used comm to go from js -> python. To see what that handler looks like see https://github.com/lightning-viz/lightning-python/blob/master/lightning/visualization.py#L90
finally to send a message in python:
comm.send(data=data)
https://ipython.org/ipython-doc/3/api/generated/IPython.kernel.comm.comm.html#IPython.kernel.comm.comm.Comm.send

PIL to Qimage conversion: QImage constructor does not free memory

I am developping a Qt application loading pictures with PIL, modifying colors and alpha channels, then converting them as QImage.
Here is the problematic piece of code: normal repeated usage of the ImageQt function: # memory is filled around 7 mB/s
if name == 'main':
while True:
im = Image.open('einstein.png') #small picture
imQt = QtGui.QImage(ImageQt.ImageQt(im)) # convert to PySide.QtGui.QImage
imQt.save('outtest.png')# -> rendered picture is correct
#del(imQt) and del(im) does not change anything
time.sleep(0.02)
The problem here is the crazy memory filling, when the picture is supposed to be erased by the garbage collector. I checked with gc.collect(), but it did not change anything.
This example shows what happends with the imageQt function, but in fact, I noticed this is a problem caused by QImage: if you repeatedly use the QImage constructor with data, the memory used by python process increases: im= Image.load('mypic.png').convert('RGBA')
data = im.toString('raw','RGBA')
qIm = QtGui.QImage(data,im.size[0],im.size[1],QtGui.QImage.Format_ARGB32)
qIm.save('myConvertedPic.png')# -> picture is perfect
If you put this code in a loop, memory will increase, as 1st example. From there i am a bit lost because this is a PySide problem...
I tried to use a workaround, but it does not work either:
#Workaround, but not working ....
if name == 'main':
while True:
im = Image.open('einstein.png') #small picture
imRGBA = im.convert('RGBA') # convert to RGBA
imRGBA.save('convtest.png') # ->picture is looks perfect
imBytes = imRGBA.tostring('raw','RGBA')
#print("size %d %d" % (imRGBA.size[0],imRGBA.size[1]))
qImage = QtGui.QImage(imRGBA.size[0],imRGBA.size[1],QtGui.QImage.Format_ARGB32) # create new empty picture
qImage.fill(QtCore.Qt.blue) # fill with blue, otherwise it catches pieces of the picture still in memory
loaded = qImage.loadFromData(imBytes,'RGBA') # load from raw data
print("success %d" % loaded)# -> returns 0
qImage.save('outtest.png')# -> rendered picture is blue
time.sleep(0.02)
I am really stuck here, if you could help find a solution with this workaround ? Because I'm really stuck here!
Also I would like to discuss the QImage problem. Is there any reliable way to free this memory ? Could the fact I am using python3.2(32bits) be a problem in this case ? Am I the only one in this case ?
The imports I am using in case of:
import time
import sys
import PySide
sys.modules['PyQt4'] = PySide # this little hack allows to solve naming problem when using PIL with Pyside (instead of PyQt4)
from PIL import Image, ImageQt
from PySide import QtCore,QtGui
After further unsuccessful searching, I noticed, that the PIL function image.tostring() associated with a QImage constructor caused this problem
im = Image.open('einstein.png').convert('RGBA')
data = im.tostring('raw','RGBA') # the tostring() function is out of the loop
while True:
imQt = QtGui.QImage(data,im.size[0],im.size[1],QtGui.QImage.Format_ARGB32)
#imQt.save("testpic.png") #image is valid
time.sleep(0.01)
#no memory problem !I think I am really close to find what is wrong, but I cannot point it out.
It definitely has something to do with the data variable being held in memory.

need for tool for video processing

I have a 2giga mpeg file of people runnig,jogging,walking etc. in it. I will use it in a image classification project but I need to segmentate the video depending on per person an per action.
for example;
there are 25 people in video which repeat these actions in order
1st person
-runs
-walks
2nd person
-runs
-walks
and goes on....
and what I want is to have 2 different mpeg file for each person
such as;
firstperson_runs.mpeg
firstperson_waves.mpeg
so I need a tool to split big file into these files. Splitting shall be due to time.
such as;
pick t1:start of action
pick t2:end of action
create a new video from big file for the interval t1 and t2
of course I will select time intervals for each video.
OS:Winxp pro
if it can be done by matlab ,can you describe it?
any help???
I imagine there are a number of tools available to do this without MATLAB, but if you really want to use MATLAB I would check out these submissions on The MathWorks File Exchange:
Gerald Dalley's videoIO Toolbox for Matlab
Micah Richert's mmread
David Foti's mpgread and mpgwrite
EDIT:
As mentioned by M456, you can also use the built-in function MMREADER for creating a multimedia reader object for your movie file (and subsequently reading selected movie frames from it with the READ method). However, I don't know which version of MATLAB this function was introduced in. It is in versions 7.7 and 7.8 (R2008b and R2009a, respectively), but it is not in version 7.1.
Matlab can do such video split operations. There are two built in functions (aviread and mmreader) for reading video files. Both will create objects which contain the individual frames of the video. You can save these as separate frames or make a new video out of by using avifile.