I am trying to use Libvlcsharp to play back HLS video. That part works well, but I also want to extract timed ID3 metadata from the live stream, is this supported?
I have tried to use the event:
_mediaPlayer.Media.MetaChanged
But this does not appear to fire when timed ID3 metadata arrives. I have seen docs on a function called "AddMetadataListener", but I am unsure how to use this:
https://videolan.videolan.me/vlc/group__vlc__player____metadata.html#gace6a4fc0f59e7ff3824d19446f19b423
Related
I am making a python application that takes text converts it into audio using ibm cloud Watson TTS, then return an audio using
content = watson_tts.synthesize(text, voice), accept=format).get_result().content
then I want to take this content and stream using Gstreamer, without saving it to a file.
I know how to play files from uri using this:
player = Gst.ElementFactory.make("playbin", "player")
player.set_property("uri", uri)
player.set_state(Gst.State.PLAYING)
but that's not what I want,
what I want is being able to stream the audio directly without downloading
After executing
content = watson_tts.synthesize(text, voice), accept=format).get_result()
synthesized audio is already "downloaded" from IBM's service so instead of "what I want is being able to stream the audio directly without downloading" I suppose it's better to say "... without saving to a file".
Anyways... to "programmatically" feed gstreamer's pipeline with (audio) bytes from Python's content object, you can utilize appsrc element.
For example, the pipeline can be implemented something like this
and it will produce MPEG Transport Stream with aac encoded audio streamed via UDP.
Question
Is it possible to resume the content of a media object once a user has tried to interact with a Google Home device during the playing of that media?
The problem
Say you have started playing an mp3 file using conv.ask. Your call to conv.ask will look something along the lines of this:
conv.ask(`<speak><audio src="${someUrl1}"><desc>${someDescription}</desc></audio></speak>`)
.add(new MediaObject({
url: someUrl2
}))
.add(new Suggestions(['suggestion1', 'suggestion2']));
This plays all fine and well. But then say a user says something along the lines of 'Ok Google, Gobbledygoop', you then might want to tell the user that their request was nonsensical, and then continue the playing of the media in the media object.
What I have tried already
app.fallback(): This does not seem compatible with the actions SDK. It does not seem possible under any circumstance to get the callback (the one provided to app.fallback) to be called.
Providing conv.ask with null/empty string responses: This was a desperate attempt to see what would happen if you provided nothing to conv.ask. There was a hope that it would see the empty response and just keep playing the media.
There is nothing that is part of Actions on Google itself that will do this for you.
The best you can probably do requires a lot of effort on your part:
You can include as part of the session state (or in a context) when you replied to the user with the Media result.
If you get another request before the Media Status event, you can determine the difference between the two, and this will roughly be how long the audio had been playing.
You can then return a URL for audio that includes starting at this point in the audio. However, the audio offset isn't something that the Assistant does, you'll have to support this on the server that contains the audio as well.
As for the two things you attempted - app.fallback() should have worked to handle any intent that you had set to go to your webhook that didn't have any other handler defined, but that still wouldn't just be able to "resume" the audio. conv.ask() requires you to ask something - null replies aren't allowed.
In this case, you at least want to tell the user that what they said made no sense... before resuming the audio.
Does anybody know of a way I can extract timing data from any kind of live video stream?
I have tried with JWPlayer, but it is not capable of doing that.
My encoder is Streamcoders Mediasuite and I am happy to stream using whatever streamtype is necessary, in order for me to get cuepoint info (or any kind of timing info) from the stream.
Caveat - Flash and Silverlight are not an option, as the viewer-base is restricted by policy.
Thanks in advance.
Neil.
HLS has timed metadata, which can be used from within iOS / OS X (and some Flash-based players) to launch JavaScript events at a certain point in a live video stream by running a JavaScript event handler when the metadata arrives: HTTP Live Streaming: how to listen for timed metadata embedded as ID3 tags using Javascript in iOS8?
RTMP (Flash) has cue points, which can be used for the same effect.
Is there any way to do something like this with a live (not VOD) MPEG DASH stream?
With MPEG Dash you can make use of Inline and Inband events. Those events have a presentation time and a unique combination of schemeIdURI and value. In your DASH player you can usually register for that events and will get a callback if they occur.
Inline events are signalled directly in the manifest file, while inband events are multiplexed into specific segments. You can find a working demo and inband events here. In that example an event is used to trigger a reload of the manifest file. Nevertheless you also use that mechanism for your own custom events.
I'm folowing Apple's proposal in https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-01.
Trying a dummy HTTP Live Streaming to my iphone, I wrote a webservice with Django corresponding to a .m3u8 file. I'm begining the response with
#EXTM3U
#EXT-X-TARGETDURATION:#10
#EXT-X-MEDIA-SEQUENCE:#0
I then write the URLs of the segments (6 segments of 10 seconds each )inside the response:
#EXTINF:10,
http://...../sample_low-1.ts
...
and that's all. I change the part containing URLs of segments every minute, so in theory I'm expecting a continuous live stream.
However, when I check the stream with my iphone I observe the following:
The phone connects to ...m3u8 , gets its contents, starts downloading .ts files and starts showing the video. Then, after downloading 6th segment(last segment in the .m3u8) it reaches end of file, sees no
EXT-X-ENDLIST
and searches for the new .m3u8. The new .m3u8 is ready at the server at this point, as I renew the contents of .m3u8 every 60 seconds.
However, the phone pauses, and I cannot achieve a continuous stream on the phone.
So, obviously I make a huge mistake somewhere. Any helps and suggestions are very welcome.
Edit : Turns out that incrementing media sequence works.
How do you send the response back?
If you return the Django response object, then the server is simply sending a response with the six segments, and then will sit quietly, waiting for a new request from the client.
If you want to continuously send data from the server, you should instead yield the result, and use some kind of synchronization, so that you are sure you are not sending the same data over and over again.