I have a book with pictures. The task is that a each picture is attached to video, and when the camera hovers over the picture, the application should open another screen and play the video associated with the photo. i tried to use teachablemachine, but it cant detect if there's too many photos. Any ideas is highly appreciated. Thanks
You could use firebase's Object Detection and Tracking and Camera Plugin's image stream feature.
Basically, you would process each frame you get from camera plugin with Firebase's ML feature, and once you detect an object you can perform any action with it.
You can use Tensoflow Lite: https://www.tensorflow.org/lite
You have some dependencies for flutter, for example: https://pub.dev/packages/tflite
Related
I am looking into creating a Flutter mobile app that live streams to YouTube using the YouTube Live Streaming API. I have checked the API and found that it does not offer a way to overlay text and images onto the livestream. How would I achieve this using Flutter?
I imagine this involves using the Stack widget to overlay content on top of the user's video feed. However this would somehow need to be encoded into the video stream to be sent to YouTube.
this type of work is usually done with FFmpeg
See this discussion for more info: https://video.stackexchange.com/questions/12105/add-an-image-overlay-in-front-of-video-using-ffmpeg
FFmpeg for mobile devices is made available by this project:
https://github.com/tanersener/mobile-ffmpeg
And then, as always, we have a flutter package called flutter_ffmpeg to allow us these features on flutter
https://pub.dev/packages/flutter_ffmpeg
TLDR: You can use CameraController (Camera package) and Canvas in Flutter for drawing the text. Unfortunately CameraController.startImageStream is not documented in the API docs, and is a 1 year+ GitHub issue.
Everytime the camera plugin gives you a video frame controller.startImageStream((CameraImage img) { /* your code */}, you can draw the image onto the canvas, draw the text, capture the video and call the YouTube API. You can see an example of using the video buffer in Tensorflow Lite package here or read more info at this issue.
On this same canvas, you can draw whatever you want, like drawArc, drawParagraph, drawPoints. It gives you ultimate flexibility.
A simple example of capturing the canvas contents is here, where I have previously saved the strokes in state. (You should use details about the text instead, and just pull the latest frame from the camera.):
Future<img.Image> getDrawnImage() async {
ui.PictureRecorder recorder = ui.PictureRecorder();
Canvas canvas = Canvas(recorder);
canvas.drawColor(Colors.white, BlendMode.src);
StrokesPainter painter = StrokesPainter(
strokes: InheritedStrokesHistory.of(context).strokes);
painter.paint(canvas, deviceData.size);
ui.Image screenImage = await (recorder.endRecording().toImage(
deviceData.size.width.floor(), deviceData.size.height.floor()));
ByteData imgBytes =
await screenImage.toByteData(format: ui.ImageByteFormat.rawRgba);
return img.Image.fromBytes(deviceData.size.width.floor(),
deviceData.size.height.floor(), imgBytes.buffer.asUint8List());
}
I was going to add a link to an app I made which allows you to draw and screenshot the drawing into your phone gallery (but also uses Tensorflow Lite), but the code is a little complicated. Its probably best to clone it and see what it does if you are struggling with capturing the canvas.
I initially could not find the documentation on startImageStream and forgotten I have used it for Tensorflow Lite, and suggested using MethodChannel.invokeMethod and writing iOS/ Android specific code. Keep that in mind if you find any limitations in Flutter, although I don't think Flutter will limit you in this problem.
Im building an app that allows the user to record a video (in app) by pressing a button on the main screen. I don't want the user to be taken to the photo app because the video will only be able to be viewed on the app (Max of 15 seconds) and I can't quite get it. Anyone have the code to do this? A good example of what i want the camera to do is the camera in the app Cinemagram. Thanks for any help.
If you plan on saving the movie to the user's photo library, then you can use UIImagePickerController. In particular, you should read the guide that accompanies the class.
However, if you only want the video to be temporary, then you will probably want to use AVFoundation. You would then need to configure an AVCaptureSession with an AVCaptureMovieFileOutput to write the video to disk. Then, when you are ready to play the video, create an AVURLAsset with the file url that you just wrote, use that to create an AVPlayer to play the video, and add an AVPlayerLayer to your view, with said player, to display the video.
Either way, I would recommend studying the examples that Apple provides.
AVCam and
AVPlayerDemo should be more than enough to get you started (especially the AVCam example project).
So I have an application that can currently capture video with the front facing iphone camera and then do some processing on the video feed real-time. What I'm trying to do, however, is make this process run in the background and put other controls onscreen. So for example, say I'd like to run the camera and process the image feed, but I want the user to see a black screen with some buttons on it. Any ideas on how to do this?
Just so we get terminology right, by "in the background", you mean running the camera capture while your application is in the foreground, but not displaying the actual video feed. This is possible, but I wanted to make clear that if you move your whole application into the background you will not have access to the camera then.
There are a few ways to do this, but the one that I've spent the most time with is grabbing frames of video (or photos) via AV Foundation. Using an AVCaptureDevice and AVCaptureSession, you can grab the frames of video and route them to an encoder for saving to disk or for processing using your own custom code. None of this requires the camera feed to be displayed onscreen, so you can put up whatever interface you like and do this video recording or photo capture without any onscreen indication.
I would caution that you should make it explicit to your users what you are doing, so that you do not run the risk of violating someone's privacy. Apple does not react kindly to those who do this (for good reason).
I encapsulate a lot of this within my open source GPUImage video and photo processing framework, so you could look at the code for the GPUImageVideoCamera class there to see how I configure the capture inputs. I hand the video frames off to OpenGL ES for the application of filters and other processing operations, but you could ignore that portion of it if you just wanted to do your own encoding or processing.
Heres an exemple code from Apple's doc:
http://developer.apple.com/library/ios/#samplecode/PhotoPicker/Introduction/Intro.html
there is also the way to customize the camera interface.
Please help me with my question.
Is there any way to get image from camera without UIImagePickerController?
I need to render current image(from camera) into image on my view and update it by timer.
May be AVCaptureStillImageOutput? I didn't find any examples.
Any ideas?
Yes, you can do it easily using AVCamCaptureManager and AVCamRecorder classes. Apple has a demo program build on its developer site here. It is named AVCam. In simple words what it does is when you click to open the camera, it calls the classes and methods which are responsible for opening the iPhone's camera and record video or capture audio. It calls the same classes which are called by UIImagePickerController.
I hope it helps.
I am working an iphone app that needs to record a vedio automatically.
I used mobile coreservices framework and using that. I made it to came into video mode and clicking on record option its start capturing a vedio. But I want it automatically that is.. I should able to record a video without clicking on record option. That is when video mode comes up its automatically start record video.
Could any one help?
You can look at UIImagePickerControllers startVideoCapture method which is used to start taking video from the camera, this is to be used when you arent using the camera standard controlors and you provide and overlay view. Here is a reference UIImagePickerCOntroller ref. If this is not enough for you, you might want to look into AVFoundation framework which gives you a lot more control over video capturing process...hope that helps