How can I implement my Hand Pose Classification (WWDC21) ML Model in a Swift app - swift

Last month I created a Hand Pose Classification .mlmodel using CreateML for a school project, my model works great in the live preview and when I upload an image to it. I tried following the video that apple provided in their Developer website and follow the steps and I didn't understand anything.. I searched the web for demos and I found nothing, All the demos and tutorials I found takes an image as an input but my model takes a MultiArray (Float32 1 × 3 × 21) and I don't know what that even is. So I need your help guys. Do you know where can I find a tutorial or at least a demo to complete my project. Thanks in advance ❤️

Related

How to create a dialog with Open AI GPT using CoreML?

With the introduction of CoreML 3 in the slides they show Open AI GPT. So my assumptions is they integrated GPT into CoreML. Would that make it possible to have a conversation with a bot?
The problem is I have spent hours on the internet trying to find how to implement this and was not able to find the answer. I did find:
"NLTagger(tagSchemes: [.lemma])" for Lemmatization.
"NLTagger(tagSchemes: [.nameType])" for identifying names.
"NLTagger(tagSchemes: [.sentimentScore])" for sentiment.
"NLLanguageRecognizer" to determine dominant language.
"UITextChecker" for spell checking and correction.
This is all used for analyzing text. But from my understanding GPT can be used for dialog and text generation as well.
https://github.com/huggingface/swift-coreml-transformers
Came close with GPT-2 text generation. However if I download it won't compile because of model issues. Is there anyone that knows how this can be accomplished? Preferabele using CoreML?

How to display input values to ionic grid?

I am new to Ionic and I am trying to do this, I want to pass the value from my input to my ionic grid. Can anyone give me an idea on how to do this? Thank you for answering.
First you need to get first name value and last name value as form data. Then add it to an array. In Html side you can loop that array and value can add to the rows of grid.
Check this angular tutorial to get clear idea
https://www.syntaxsuccess.com/viewarticle/angular-2.0-input-values-and-binding
I think you should learn angular instead of ionic/react, here is why...
[the following is a personal opinion from personal experience]
I started learning ionic react but by the time I had finished the documentation and my project, I left having learned nothing. This half full feeling. It was odd that I couldn't fully grasp a simple object class container language as I program almost every day of the year for fun. I couldn't figure it out, it bugged me for a while but I took the L.
a few months pass and then I begin learning Angular as a second attempt at hybrid PWAs.
Having used Angular and Ionic/React, Angular blows Ionic/React out of the water when it comes to learning and understanding the material. Angular s docs are well orchestrated and the framework is as enjoyable as it is popular.
Comparing Official docs between Angular and Ionic.
Ionic/react documentation demos were outdated making it impossible to follow the example code vs the interactive demo
Ionic/react documentation is unreliable. at the time it was a gamble between 'updated', 'outdated','missing-information' (makes it impossible to build a proper project)
Angular's documentation is updated, accurate, thorough and overall superb
Angular has a greater online presence with an active online community, Q and A.
Easier to work with, less headaches, constant updates
Ionic documentation is a travel brochure.
Angular documentation is a map.
If you are going hiking in the woods, do you want a map or a travel brochure?
Posted some links below.
https://angular.io/docs
Covers all topics, demos are accurate
https://material.angular.io/components/categories
Ready Components

is there any way to deploy custom ML models in flutter?

I am trying to create a flutter app using the ML model from the below link.
https://www.tensorflow.org/lite/models/style_transfer/overview
I haven't played with Flutter but there are some promising articles article 1 article 2 article 3 that gives code snippets to insert tensorflow lite dependencies, to make assets folder and load models. You can check them and I hope you get a first idea how to proceed. Style transfer is a medium to hard coding project though. I suggest to start from simpler tasks as classification problems.
Happy coding
I have used this style_transfer model in flutter app before. unfortunately, I've lost that code (I did not use git). But I can give suggestions on that.
For using this custom model you will have to use https://pub.dev/packages/tflite_flutter.
The trick here is you will have to see model's input shape (image_shape=(384,384,3), style_shape=(256,256,3) and output size (shape=(384,384,3) and also shape of bottleneck). You can resize image using https://pub.dev/packages/image.
with these two, you are good to go. Sorry for bad english and I know I am too much late but maybe someone else will find it useful.
Happy coding!
Edited: So I have found the code. I stored a copy of it in my google drive. I have uploaded it on github https://github.com/Rizwan2613/style-transfer. Please do read Readme file. Thanks
There is a recently published flutter plugin for integrating an arbitrary TFLite model.
https://pub.dev/packages/tflite_flutter
Please see the README of the pub package and see how you can bundle the .tflite models and how to load / run them in flutter.
There is a blog post on how to use this flutter plugin, but it uses a different model (text classification) as an example.
https://medium.com/#am15hg/text-classification-using-tensorflow-lite-plugin-for-flutter-3b92f6655982
You can upload custom tensorwflow model to firebase ML KIT (custom tab), and integrate with firebase API in your flutter project.

Render openCTM files on android (especially flutter)

I'm currently developing an App in Flutter which i need to display 3D models. I know. Flutter is, at least at the moment, not the best choice (or shouldn't be even a choice) to build 3d apps but unfortunately i didn't know this when i started developing. I managed to render a simple '.obj' file with a simple engine I found on the internet:
https://levelup.gitconnected.com/a-simple-3d-engine-in-flutter-cc4a7edc943e
Now my problem:
My app needs to contain multiple hundred 3D models so I need to reduce the app's size. I experimented with openCTM to compress my '.obj' models and got a good result.
Now i want to display these '.ctm' files in my app. And thats the point where I got stuck.
I've researched for days if there's a sample or a tutorial to solve my problem but I haven't found one.
I've read the official openCTM documents and found openCtm reader ported to javascript (https://github.com/jcmellado/js-openctm) and dart (https://github.com/nelsonsilva/dart-openctm). And now I dont know where to begin.
I guess I need to, if my problem is solvable in flutter, modify the engine im currently using and add the openCTM reading part to it.
I hope that anyone of you can help me with my problem. Maybe even solved it already. Furthermore I hope I explained my problem clear enough.
If you can provide a native solution it would helpful, too.
Thank you in advance!

Create a location-based rotating globe similar to Google Earth using C# and Unity?

I am currently in the process of developing an iOS app involving a touch-enabled rotating globe. I have already found past links for this such as: http://answers.unity3d.com/questions/255118/rotating-an-object-with-touch.html
The question is:
Is possible to make this object location-oriented? I plan to create a cel-shaded globe with different points on the object related to specific locations.
Edit: These locations are not GPS oriented, but just meaning points on an object (globe) represented by objects (tree).
Here is an example I created for a class using Dreamweaver many years ago that would look similar, but touch-oriented instead of using 'West' and 'East' buttons located in the top left of the video: https://youtu.be/UZACQtnQfkA
Sorry the example is in video form, I don't have the code available for comparison at the moment.
This question is being naive and you seem to lack knowledge in Unity3D. I suggest you to go through their official tutorials and get and idea. Then ask if you have any specific questions on issues you're encountering.
You can follow some tutorials to create such scene by PlannetaryTerrain. You can see some efforts on YouTube as well.
Apart from that creating terrains would be a plus. Follow the tutorial by Kostiantyn. Also Follow the Faux gravity tutorial by Sebastian