Flutter Web Record Audio to Stream - flutter

Question

I am working on a Flutter Web project that involves a Speech to Text component. I intend to use Google's Speech to Text API (https://cloud.google.com/speech-to-text/docs). One of the critical requirements that I have is to use the API's Single Utterance capability to recognize when a speaker is done speaking automatically. This requires that I stream audio directly to Google's API from the Flutter client so that I can receive that event and do something with it. I am using the google_speech dart plugin (https://pub.dev/packages/google_speech) and am fairly certain it will meet my needs.
What I am struggling with is finding an implementation that can successfully record audio to a stream that I can then send to Google that works in Flutter Web.
So far the only one I cant find that seems to meet my needs is the flutter_sound (https://pub.flutter-io.cn/packages/flutter_sound) plugin as it claims to support Flutter Web and seems to have the ability to record to a dart Stream without the use of a File. I have an initial implementation but I seem to be missing something somewhere as the library seems to hang.
Here is my implementation so far:
class _LandingPageState extends State<LandingPage> {
FlutterSoundRecorder _mRecorder = FlutterSoundRecorder();
String _streamText = 'not yet recognized';
_LandingPageState(this.interviewID);
#override
Widget build(BuildContext context) {
//Build method works fine
}
// Called elsewhere to start the recognition
_submit() async {
// Do some stuff
await recognize();
// Do some other stuff
}
Future recognize() async {
// Set up the Google Speech (google_speech plugin) recongition apparatus
String serviceAccountPath = await rootBundle
.loadString('PATH TO MY SERVICE ACCOUNT CREDENTIALS');
final serviceAccount = ServiceAccount.fromString(serviceAccountPath);
final speechToText = SpeechToText.viaServiceAccount(serviceAccount);
final config = _getConfig();
// Create the stream controller (flutter_sound plugin)
StreamController recordingDataController = StreamController<Food>();
// Start the recording and specify what stream sink it is using
// which is the above stream controller's sink
await _mRecorder.startRecorder(
toStream: recordingDataController.sink,
codec: Codec.pcm16,
numChannels: 1,
sampleRate: 44000,
);
// Set up the recognition stream and pass it the stream
final responseStream = speechToText.streamingRecognize(
StreamingRecognitionConfig(
config: config,
interimResults: true,
singleUtterance: true,
),
recordingDataController.stream,
);
responseStream.listen((data) {
setState(() {
_streamText =
data.results.map((e) => e.alternatives.first.transcript).join('\n');
});
}, onDone: () {
setState(() {
print("STOP LISTENING");
print("STREAM TEXT = ");
print("--------------------------------------");
print(_streamText);
print("--------------------------------------");
// Stop listening to the mic
recordingDataController.close();
});
});
}
init() async {
await Future.delayed(Duration(seconds: 1));
await _sumbit();
}
#override
void initState() {
super.initState();
_openRecorder();
}
#override
void dispose() {
super.dispose();
_stopRecorder();
}
RecognitionConfig _getConfig() => RecognitionConfig(
encoding: AudioEncoding.LINEAR16,
model: RecognitionModel.basic,
enableAutomaticPunctuation: true,
sampleRateHertz: 16000,
languageCode: 'en-US');
Future<void> _openRecorder() async {
// These Permission calls dont seem to work on Flutter web
// var status = await Permission.microphone.request();
// if (status != PermissionStatus.granted) {
// throw RecordingPermissionException('Microphone permission not granted');
// }
await _mRecorder.openAudioSession();
}
Future<void> _stopRecorder() async {
await _mRecorder.stopRecorder();
}
}
When this is debugged the library hangs on starting the recorder and states "Waiting for the recorder being opened" but it just waits there forever. I've tried to debug the library but it is very unclear what is going on. I worry that this library does not support Flutter Web after all. Could it be that because Microphone permissions have not been granted that the library would hang?
I've been using this example for flutter_sound to implement this: https://github.com/Canardoux/tau/blob/master/flutter_sound/example/lib/recordToStream/record_to_stream_example.dart
Is there another library or approach that supports recording audio to a dart stream in Flutter web?

Answer 1

The problem is that to record to stream you must use the PCM16 codec, but this codec is not compatible to record audio in the browser. You can see the associated documentation https://tau.canardoux.xyz/guides_codec.html