I was wondering if it was possible to not include text in my SSML, since my audio file says 'Are you ready to play?', I dont need any speech from the google assistant itself.
app.intent('Default Welcome Intent',(conv) =>{
const reply = `<speak>
<audio src="intro.mp3"></audio>
</speak>`;
conv.ask(reply);
});
The code above produces an error since I do not have any text input.
The error you probably got was something like
expected_inputs[0].input_prompt.rich_initial_prompt.items[0].simple_response: 'display_text' must be set or 'ssml' must have a valid display rendering.
As it notes, there are conditions where the Assistant runs on a device with a display (such as your phone), and it should show a message that is substantively the same as what the audio plays.
You have a couple of options that are appropriate for these cases.
First, you can provide optional text inside the <audio> tag that will be shown, but not read out (unless the audio file couldn't be loaded for some reason).
<speak>
<audio src="intro.mp3">Are you ready to play?</audio>
</speak>
Alternately, you can provide separate strings that represent the SSML version and the plain text version of what you're saying.
const ssml = `<speak><audio src="intro.mp3"></audio></speak>`;
const text = "Are you ready to play?";
conv.ask( new SimpleResponse({
speech: ssml,
text: text
}) );
Found a hacky work around for this, By adding a very short string and then putting it in a prosody tag with a silent volume:
app.intent('Default Welcome Intent',(conv) =>{
const reply = `<speak>
<audio src="intro.mp3"></audio>
<prosody volume ="silent">a</prosody> </speak>`;
conv.ask(reply);
});
This plays the audio and does not speak the 'a' text.
The other way to trick, try to use blank space to don't get No Response error (... is not responding now)
conv.ask(new SimpleResponse(" "))
const reply = `<speak>
<audio src="intro.mp3"></audio>
</speak>`;
conv.ask(reply);
Related
I have fragmented mp4 which I want to send users through HLS. It's ok if I just send it as is. But I need opportunity to reorder fragments in this video.
For example initial video, which looks like this:
original video format
I want reorganize fragments and get this:
expected video format
I try make it locally, and it's work in VLS player (HLS). For this I modified sequence number for fragments in moof (mfhd). But when I try play it remotely (HLS) it does not work. I think, that some players (js) expect some additional information from each fragment, probably for example time offset. But I can not find which atom (box) contain this information. I spent a lot of time searching and I'm still at the very beginning of the problem.
I tried to modify the fragment sequence number, but it doesn't work.
The "Track Fragment Media Decode Time Box" (tfdt) stores the baseMediaDecodeTime which is the accumulative decode time.
Consider the following...
baseMediaDecodeTime must increase monotonically for each chunk.
This means you must update (replace) the tfdt entry of chunk with expected next tftd entry.
When you naively reorder the chunks, the baseMediaDecodeTime will be invalid.
The "Track Fragment Media Decode Time Box" (tfdt) is located inside each moof header at:
moof --> traf --> tfdt
Project Requirement: Music player app which will download audio files, encrypt and save them. The audio files should be playable in the app only. No other app should be able to play the files. Nor the user should be able to copy the files.
Approach: I don't want the entire decryted audio file to exist at any moment. So I want to encrypt the audio file as soon as it is downloaded. Then when the file is to be played, I want it to be decrypted chunk-by-chunk and played. I believe this can be achieved by using stream. As far as I searched, a package named "just_audio" can play the audio from stream source.
Problem: I cannot find any encryption package for Flutter/Dart which will output the decrypted data in the form of a stream. This is the first time I am trying to implement encryption/decryption, so my knowledge is very poor in this regard.
Notes:
The encryption does not need to be heavy. A typical user not being able to copy the files and play them elsewhere would suffice.
Audio files are going to be large, some of them even hours in length.
I need all the usual functions of a music player (e.g. albums, playlists, progress bars with seeking function, etc.)
Options:
It will be best if there is a package which can do what I need, off the shelf.
To find a basic package and then modifying it into doing what is needed.
Some radically different solution, which takes entirely different path but provides all the solutions.
Firstly, to encrypt or decrypt data, have a look at https://pub.dev/packages/cryptography or https://pub.dev/packages/encrypt or something like that.
Secondly, since you want seeking, it may not be an optimal solution to use streams - the "Stream" abstraction is more like a sequence of data, instead of arbitrarily jumping (seeking) here and there. Instead, divide whole audio (say 1hour) into chunks (say 1minute), and upload/encrypt/decrypt/download each chunk separately and as a whole without using streams. If your chunk is small enough, download/decrypt a chunk will be fast and you do not stuck the UI. If the decrypting is still too slow, have a look at isolates, which are "threads" in Flutter. Run decrypt in a separate isolate then your UI will be smooth.
I need all the usual functions of a music player (e.g. albums, playlists, progress bars with seeking function, etc.)
Seeking is implemented above. For albums/playlists, you may modify any normal audio players in flutter to implement that, or directly implement by your own. it is more like just some UI things, no special things, and anyone familiar with Flutter can write it, no worry.
if you're open to a 3rd package, don't reinvent the wheel..
try this here https://morioh.com/p/34a06006b299 with various CipherStream Options
If you can forego stream encryption and do it after you have the file then try this package, Credit: I used the sample from this answer by Hoaea Varghese
With AES all you need is the path to the file, and you can encrypt files or albums with something as simple as
encrypted_file_path = EncryptData.encrypt_file('your/file/path');
With the code below
import 'dart:io';
import 'package:aes_crypt/aes_crypt.dart';
class EncryptData {
static String encrypt_file(String path) {
AesCrypt crypt = AesCrypt();
crypt.setOverwriteMode(AesCryptOwMode.on);
crypt.setPassword('my cool password');
String encFilepath;
try {
encFilepath = crypt.encryptFileSync(path);
print('The encryption has been completed successfully.');
print('Encrypted file: $encFilepath');
} catch (e) {
if (e.type == AesCryptExceptionType.destFileExists) {
print('The encryption has been completed unsuccessfully.');
print(e.message);
}
else{
return 'ERROR';
}
}
return encFilepath;
}
static String decrypt_file(String path) {
AesCrypt crypt = AesCrypt();
crypt.setOverwriteMode(AesCryptOwMode.on);
crypt.setPassword('my cool password');
String decFilepath;
try {
decFilepath = crypt.decryptFileSync(path);
print('The decryption has been completed successfully.');
print('Decrypted file 1: $decFilepath');
print('File content: ' + File(decFilepath).path);
} catch (e) {
if (e.type == AesCryptExceptionType.destFileExists) {
print('The decryption has been completed unsuccessfully.');
print(e.message);
}
else{
return 'ERROR';
}
}
return decFilepath;
}
}
getUserMedia(constrains).then(stream => {
var recorder = new MediaRecorder(stream)
})
recorder.start()
recorder.pause()
// get new stream getUserMedia(constrains_new)
// how to update recorder stream here?
recorder.resume()
Is it possible? I've try to create MediaStream and use addTrack and removeTrack methods to change stream tracks but no success (recorder stops when I try to resume it with updated stream)
Any ideas?
The short answer is no, it's not possible. The MediaStream recording spec explicitly describes this behavior: https://w3c.github.io/mediacapture-record/#dom-mediarecorder-start. It's bullet point 15.3 of that algorithm which says "If at any point, a track is added to or removed from stream’s track set, the UA MUST immediately stop gathering data ...".
But in case you only want to record audio you can probably use an AudioContext to proxy your streams. Create a MediaStreamAudioDestinationNode and use the stream that it provides for recording. Then you can feed your streams with MediaStreamAudioSourceNodes and/or MediaStreamTrackAudioSourceNodes into the audio graph and mix them in any way you desire.
Last but not least there are currently plans to add the functionality you are looking for to the spec. Maybe you just have to wait a bit. Or maybe a bit longer depending on the browser you are using. :-)
https://github.com/w3c/mediacapture-record/issues/167
https://github.com/w3c/mediacapture-record/pull/186
I've been looking around Swift documentation to save an audio output from AVAudioEngine but I couldn't find any useful tip.
Any suggestion?
Solution
I found a way around thanks to matt's answer.
Here a sample code of how to save an audio after passing it through an AVAudioEngine (i think that technically it's before)
newAudio = AVAudioFile(forWriting: newAudio.url, settings: nil, error: NSErrorPointer())
//Your new file on which you want to save some changed audio, and prepared to be bufferd in some new data...
var audioPlayerNode = AVAudioPlayerNode() //or your Time pitch unit if pitch changed
//Now install a Tap on the output bus to "record" the transformed file on a our newAudio file.
audioPlayerNode.installTapOnBus(0, bufferSize: (AVAudioFrameCount(audioPlayer.duration)), format: opffb){
(buffer: AVAudioPCMBuffer!, time: AVAudioTime!) in
if (self.newAudio.length) < (self.audioFile.length){//Let us know when to stop saving the file, otherwise saving infinitely
self.newAudio.writeFromBuffer(buffer, error: NSErrorPointer())//let's write the buffer result into our file
}else{
audioPlayerNode.removeTapOnBus(0)//if we dont remove it, will keep on tapping infinitely
println("Did you like it? Please, vote up for my question")
}
}
Hope this helps !
One issue to solve:
Sometimes, your outputNode is shorter than the input: if you accelerate the time rate by 2, your audio will be 2 times shorter. This is the issue im facing for now since my condition for saving the file is (line 10)
if(newAudio.length) < (self.audioFile.length)//audiofile being the original(long) audio and newAudio being the new changed (shorter) audio.
Any help here?
Yes, it's quite easy. You simply put a tap on a node and save the buffer into a file.
Unfortunately this means you have to play through the node. I was hoping that AVAudioEngine would let me process one sound file into another directly, but apparently that's impossible - you have to play and process in real time.
Offline rendering Worked for me using GenericOutput AudioUnit. Please check this link, I have done mixing two,three audios offline and combine it to a single file. Not the same scenario but it may help you for getting some idea. core audio offline rendering GenericOutput
I have a simple mail system developed with G.W.T and i am trying add a feature to play audio and video files if there is a video or audio file comes as an attachment.
I have been trying bst player and HTML video tag to get work , but i can not play some video formats such as .avi, .mpeg ,.mpg and so on.
What else can be done to play those kind of video formats?
On the other hand, i am thinking of converting the video file in a java servlet then giving that url to player, but i do not know if that makes sense.Is that should be the way?
Last thing is; is there a general format (maybe .flv?) that a video file first have to be converted into so it can be playable by VlcPlayerPlugin or another video player? Any other tip will be helpful.
Thank you for your helps.
the html5 video tag can only play certain formats. You can find a list of the supported browser formats here.
I also had some problems with the BST player but at least it worked with the following code:
public YoutubeVideoPopup( String youtubeUrl )
{
// PopupPanel's constructor takes 'auto-hide' as its boolean parameter.
// If this is set, the panel closes itself automatically when the user
// clicks outside of it.
super( true );
this.setAnimationEnabled( true );
Widget player = null;
try
{
player = new YouTubePlayer( youtubeUrl, "500", "375" );
player.setStyleName( "player" );
}
catch ( PluginVersionException e )
{
// catch plugin version exception and alert user to download plugin first.
// An option is to use the utility method in PlayerUtil class.
player = PlayerUtil.getMissingPluginNotice( Plugin.Auto, "Missing Plugin",
"You have to install a flash plaxer first!",
false );
}
catch ( PluginNotFoundException e )
{
// catch PluginNotFoundException and tell user to download plugin, possibly providing
// a link to the plugin download page.
player = new HTML( "You have to install a flash plaxer first!" );
}
setWidget( player );
}
As you can see, we used the youtube player here, which has the positive effect that the vide can be placed at youtube and must not be pished to server every time the GWT app is redeployed.
You also can play flash other formats, simply use the correct Player class in the try block; example for flash:
player = new com.bramosystems.oss.player.core.client.ui.FlashMediaPlayer( GWT.getHostPageBaseURL( ) +
f4vFileName, true, "375", "500" );
player.setWidth( 500 + "px" );
player.setHeight( "100%" );
Sorry for the delay, did not have the chance to reply.Because of VlcPlayer was behaving strange and showing different control buttons on Ubuntu and Windows, i decided to use FlashPlayerPlugin of BstPlayer.I first convert file to flv by using jave is described here in documentation, then it serves the converted video to FlashPlayer, it works without problem now , thank you all for your helps.