Replace audio on video with Azure Media Encoder - azure-media-services

Question

I'm using Azure Media services to encode uploaded videos for streaming. I would like to replace the audio track with a different audio source. I'm already creating a custom preset configuration to the encoding but I haven't found a way to replace or overlay different audio. Is this possible?

Answer 1

First of all, I presume you are attempting to use Media Encoder Standard, and not the deprecated Azure Media Encoder.
Is the replacement audio source in sync/aligned in time with respect to the video - same starting timestamp, same duration etc?
While the overlay feature will not work in this case (it'll end up mixing the audio from the original with the replacement), you can attempt a simple workaround which requires the content to be time-aligned. I'll be able to share sample code later today.

Answer 2

The workaround below has three sections:
Describes the preset that does video-only encoding of the video file
Describes the preset that does audio-only encoding of the overlay/replacement file
The code block that shows how to submit a Job with two Tasks that write to the same output Asset
Video Only Encoding Preset
Save the JSON below to a suitable file, say "C:\TEMP\VideoOnly.json". I'll use a single bitrate setting as an example to keep the JSON brief
{
"Version": 1.0,
"Codecs": [
{
"KeyFrameInterval": "00:00:02",
"H264Layers": [
{
"Profile": "Auto",
"Level": "auto",
"Bitrate": 2500,
"MaxBitrate": 2500,
"BufferWindow": "00:00:05",
"Width": 1280,
"Height": 720,
"BFrames": 3,
"Type": "H264Layer",
"FrameRate": "0/1"
}
],
"Type": "H264Video"
}
],
"Outputs": [
{
"FileName": "{Basename}_{Resolution}_{Bitrate}.mp4",
"Format": {
"Type": "MP4Format"
}
}
]
}
Audio Only Encoding Preset
Save the JSON below to a suitable file, say "C:\TEMP\AudioOnly.json".
{
"Version": 1.0,
"Codecs": [
{
"Profile": "AACLC",
"Channels": 2,
"SamplingRate": 48000,
"Bitrate": 128,
"Type": "AACAudio"
}
],
"Outputs": [
{
"FileName": "{Basename}_AAC_{AudioBitrate}.mp4",
"Format": {
"Type": "MP4Format"
}
}
]
}
Encoding
The code below assumes that you have the video file uploaded as an Asset myVideoAsset and the audio file uploaded as an Asset myAudioAsset.
string videoConfig = File.ReadAllText(_presetFiles + #"C:\TEMP\VideoOnly.json");
string audioConfig = File.ReadAllText(_presetFiles + #"C:\TEMP\AudioOnly.json");
// Prepare a job with two Tasks that write to the same Asset
IJob job = _context.Jobs.Create(#"Encoding " + myVideoAsset.Name + #" and " + myAudioAsset.Name);
IMediaProcessor mediaProcessor = GetLatestMediaProcessorByName("Media Encoder Standard");
ITask videoTask = job.Tasks.AddNew("Video Task", mediaProcessor, videoConfig, TaskOptions.DoNotCancelOnJobFailure | TaskOptions.DoNotDeleteOutputAssetOnFailure);
videoTask.InputAssets.Add(myVideoAsset);
IAsset outputAsset = videoTask.OutputAssets.AddNew(myVideoAsset.Name + #" plus " + myAudioAsset.Name + # - Encoded", options: AssetCreationOptions.None, formatOption: AssetFormatOption.None);
ITask audioTask = job.Tasks.AddNew("Audio Task", mediaProcessor, audioConfig, TaskOptions.DoNotCancelOnJobFailure | TaskOptions.DoNotDeleteOutputAssetOnFailure);
audioTask.InputAssets.Add(myAudioAsset);
audioTask.OutputAssets.Add(outputAsset); // Note the re-use of outputAsset here
Console.WriteLine("Submitting transcoding job...");
job.Submit();
// Wait for job to succeed, etc.