Unable to upload an image by a celery task in fastapi - celery

I am using Fastapi, celery and cloudinary
I want to upload an image to cloudinary by a celery task.
I got an error telling the file object is not JSON serializable.
Here is code:
app.py
#app.post("/upload3")
async def upload(file: UploadFile = File(...),):
result=tasks.create_image.delay(file)
url=result.get()
return {"message":"ok"}
tasks.py
#shared_task
async def create_image(file:_fastapi.UploadFile):
urlFile = cloudinary.uploader.upload(file.file)
url = urlFile.get("url")
return url
Error is as below:
kombu.exceptions.EncodeError: Object of type UploadFile is not JSON serializable
I want to get the image URL when uploading succesfully completed.

Related

FastAPI with MongoDB

Well im kind of new with all the Back-end stuff so pardon if i ask silly questions or my code makes no sense ;).
so what im trying to do is to transfer data between my api and the database. right now focusing on get-all and post methods. the data that im transferring is Als files (abelton live set).
i have a main.py file which is where the routes sit, database.py containing the engine and the functions and file_model.py which is the pydantic model.
im able to post a file and indeed i can see it in my DB, it has the default mongo _id : objectid
and the file.
when i try the get_all_files_db() function the errors start to show, well there was one problem with the _id which python cant read properly so i tried to fix it with some web searching.
its hard to define my problem because I probably have many, so specifying the error will not be relevant.
so this is where you guys come in, your answers could be code recommendations or a link to some docs that may help me understand better or even using some different libraries/dependencies because i see that fastapi and mongoDB is annoying when dealing with bigger files and not regular json.
those are my 2 routes in main.py (used Uploadfile because of the file type)
`#app.post("/api/files" , response_model=Als)
async def post_file(title : str = Form(...), file : UploadFile = File(...)):
Als(title=title)
response = await upload_file(title,file.file)
if response :
return response
raise HTTPException(400, "Something went wrong")
#app.get("/api/files" , response_model= Als )
async def get_all_files():
response =await get_all_files_db()
return response`
here are my functions in the database.py and the connection to database.
client = motor.motor_asyncio.AsyncIOMotorClient('localhost', 27017)
db = client['Files']
collection = db['ALS_Files']
async def upload_file(title,file):
await collection.insert_one({title:file.read()})
return file
async def get_all_files_db():
files = []
cursor = collection.find({})
async for document in cursor:
#doc_copy = document.copy()
files.append(document)
return files()
and my file_model.py
class PyObjectId(ObjectId):
""" Custom Type for reading MongoDB IDs """
#classmethod
def __get_validators__(cls):
yield cls.validate
#classmethod
def validate(cls, v):
if not ObjectId.is_valid(v):
raise ValueError("Invalid object_id")
return ObjectId(v)
#classmethod
def __modify_schema__(cls, field_schema):
field_schema.update(type="string")
class Als(BaseModel) :
id: PyObjectId = Field(default_factory=PyObjectId, alias="_id")
title: str
class Config:
allow_population_by_field_name = True
arbitrary_types_allowed = True
json_encoders = {ObjectId: str}

Gathering performance metrics for calls to Cloud Functions from Flutter

I am attempting to use Firebase Performance monitoring with Firebase Cloud Functions. When I run the following code I get the error URL host is null or invalid. Clearly that is because I am passing in a function name rather than a URL with a host.
import 'package:flutter/widgets.dart';
import 'package:cloud_functions/cloud_functions.dart';
import 'package:firebase_performance/firebase_performance.dart';
/// Handles tracking metrics while calling cloud functions.
class MetricCloudFunction {
Future<HttpsCallableResult> call(String functionName, [dynamic parameters]) async {
final HttpMetric metric = FirebasePerformance.instance
.newHttpMetric(functionName, HttpMethod.Get);
final HttpsCallable callable = CloudFunctions.instance.getHttpsCallable(
functionName: functionName,
);
await metric.start();
HttpsCallableResult response;
try {
response = await callable.call(parameters);
} catch(e) {
debugPrint('failed: ${e.toString()}');
} finally {
await metric.stop();
}
return response;
}
}
Is there some way to get the URL of the callable function so that I can create the metric for it? If not, is there another way that I can create an HTTP metric for it?
There is currently no way to programmatically get the URL for an HTTP callable type function. You can compose it by what you know about the function (the three variables are deployment region, project ID, name), or you can simply copy it from the Firebase console Functions dashboard and hard code it into your app.

Scala: Http client read http response

I am using httpClient lib to do some REST API calls using scala. I am able to post the data.
I am using following code to read the content. However, When I run on Spark Databricks Cluster it give me error.
val entity = response.getEntity
var content = ""
if (entity != null) {
val inputStream = entity.getContent
content = io.Source.fromInputStream(inputStream).getLines.mkString
inputStream.close()
}
Error
error: object Source is not a member of package io
content = io.Source.fromInputStream(inputStream).getLines.mkString
is there a way I can fix this error, or a different way to read HTTP response content.
Please try to import scala.io.Source
OR use the completed package name like this:
content = scala.io.Source.fromInputStream(inputStream).getLines.mkString

S3 File upload using Play-S3 fails for images but works for text files

I am trying to upload a file to S3 using Scala Playframe work 2.4.1 Specifically using the module play-s3 7.0.2
I can get it to work for text files but when I try and upload an image I get this message, The provided 'x-amz-content-sha256' header does not match what was computed.
If I swap "avatar.jpeg" for "text.txt" and "image/jpeg" for "plain/text" in the following code it works. The text file gets uploaded to S3. But if I try and upload an image (I've tried jpeg and png) it fails saying the header doesn't match. I have no idea what I am doing wrong at this point and it is driving me nuts.
import fly.play.s3.{BucketFile, S3, S3Exception}
import java.nio.file.{Files, Paths}
def test() = Action
{ implicit request =>
val file_path = "/path/to/file/avatar.jpeg"
val bucket = S3("path_to_bucket")
val byte_array = Files.readAllBytes(Paths.get(file_path))
val result = bucket + BucketFile("avatar.jpeg", "image/jpeg", byte_array)
result.map { unit =>
Logger.info("Saved the file")
}
.recover {
case S3Exception(status, code, message, originalXml) =>
{
Logger.info("Error: " + message)
Logger.info("originalXml: " + originalXml)
}
}
Ok("Yay")
}
The solution to this is to not use play 2.4.1. There is something wrong with the hashing of binary data with 2.4.1. 2.4.2 works just fine though.
https://github.com/Kaliber/play-s3/issues/70

Amazon AWS get object URI

I am using following code to get URI but it gives exception:
s3.putObject(bucket, fileToPut, file.ref.file)
val s3Object = s3.getObject(bucket, fileToPut).orNull
if(s3Object != null){
val assetUrl = s3Object.getObjectContent.getHttpRequest.getURI
}
I am unable to get uploaded object URI
Please help
If you just want the url of the object you could use
s3Client.getResourceUrl("your-bucket", "path/file.txt");