Generating image files from CameraImage - flutter

Question

We would like to use CameraController.startImageStream() instead of CameraController.takePicture() to get a return value equivalent to CameraController.takePicture(). We know that various answers have been documented for these but we have not yet succeeded with any of the methods.
We have implemented the following. We tried setting the imageFormatGroup during CameraController generation, but this did not improve things. We have to make these work on both Android and iOS. We want a way to run it faster. It can be written in Flutter or using C++ and ffi.
import 'package:image/image.dart' as imglib;
final cameras = await availableCameras();
final firstCamera = cameras.first;
cameraController = CameraController(
firstCamera,
ResolutionPreset.medium,
// imageFormatGroup: ImageFormatGroup.yuv420,
);
await cameraController!.initialize();
await cameraController!.startImageStream((CameraImage cameraImage) async {
cameraController!.stopImageStream();
Uint8List list = cameraImage.planes.first.bytes;
final width = cameraImage.width;
final height = cameraImage.height;
imglib.Image test = imglib.Image.fromBytes(width, height, list);
List<int> _imageBytes = imglib.encodeJpg(test);
Uint8List convert = Uint8List.fromList(_imageBytes);
File file = File.fromRawPath(convert);
XFile? result = XFile(file.path);
});
Index out of range occurred when we called encodeJpg(). How should we fix this?
fvm flutter doctor -v
[✓] Flutter (Channel stable, 3.3.6, on macOS 12.6 21G115 darwin-x64, locale en-JP)
• Flutter version 3.3.6 on channel stable at /Users/xxxxx/fvm/versions/stable
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 6928314d50 (3 weeks ago), 2022-10-25 16:34:41 -0400
• Engine revision 3ad69d7be3
• Dart version 2.18.2
• DevTools version 2.15.0
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
• Android SDK at /Users/xxxxx/src/android-sdks
• Platform android-33, build-tools 33.0.0
• Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 13F100
• CocoaPods version 1.11.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
[✓] VS Code (version 1.73.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.52.0
[✓] Connected device (3 available)
• SHV43 (mobile) • XXXXXXXXXXXXXXX • android-arm64 • Android 10 (API 29)
• macOS (desktop) • macos • darwin-x64 • macOS 12.6 21G115 darwin-x64
• Chrome (web) • chrome • web-javascript • Google Chrome 107.0.5304.110
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!

Answer 1

I assume your problem comes from mixing different image formats.
The camera stream in this line
Uint8List list = cameraImage.planes.first.bytes
returns an image that is encoded with YUV instead of the common RGB. So before further using this image, I would recommend converting it.
There are multiple sources already out there on how to do that conversion. Let me point you to this blog post and the following stackoverflow questions: 1 2.
You certainly can implement this conversion via ffi, but it is not strictly necessary.