I am trying to upload a file to S3 using Scala Playframe work 2.4.1 Specifically using the module play-s3 7.0.2
I can get it to work for text files but when I try and upload an image I get this message, The provided 'x-amz-content-sha256' header does not match what was computed.
If I swap "avatar.jpeg" for "text.txt" and "image/jpeg" for "plain/text" in the following code it works. The text file gets uploaded to S3. But if I try and upload an image (I've tried jpeg and png) it fails saying the header doesn't match. I have no idea what I am doing wrong at this point and it is driving me nuts.
import fly.play.s3.{BucketFile, S3, S3Exception}
import java.nio.file.{Files, Paths}
def test() = Action
{ implicit request =>
val file_path = "/path/to/file/avatar.jpeg"
val bucket = S3("path_to_bucket")
val byte_array = Files.readAllBytes(Paths.get(file_path))
val result = bucket + BucketFile("avatar.jpeg", "image/jpeg", byte_array)
result.map { unit =>
Logger.info("Saved the file")
}
.recover {
case S3Exception(status, code, message, originalXml) =>
{
Logger.info("Error: " + message)
Logger.info("originalXml: " + originalXml)
}
}
Ok("Yay")
}
The solution to this is to not use play 2.4.1. There is something wrong with the hashing of binary data with 2.4.1. 2.4.2 works just fine though.
https://github.com/Kaliber/play-s3/issues/70
Related
In Firefox(http/https) or Chrome/Safari(http) I can download it without a problem.
Code for saving:
saveBase64BinaryFile(SaveBinaryFile.builder()
.namespace("namespace")
.fileName(s"Schët-faktura_№143_ot_21_fevralya_2020.pdf")
.contentType("application/pdf; charset=utf-8")
.data(getFile)
.build())
def saveBase64BinaryFile(cmd: SaveBinaryFile)(implicit context: Context): Future[FileInfo] =
upload(
namespace = cmd.getNamespace,
fileName = cmd.getFileName,
src = Source.fromFuture(Future.successful(ByteString(cmd.getData))),
contentType = cmd.getContentType
)
def upload(namespace: String, fileName: String, src: Source[ByteString, Any], contentType: String = null) = Future({
val inputStream = src.runWith(StreamConverters.asInputStream())
val fileId = UUID.randomUUID().toString
val meta = new ObjectMetadata
val bytes = IOUtils.toByteArray(inputStream)
meta.setContentLength(bytes.length)
meta.setUserMetadata(Map(
FileNameHeader → Base64.getEncoder.encodeToString(fileName.getBytes(Charsets.UTF_8)),
ContentTypeHeader → contentType,
).asJava)
s3Client.putObject(bucketName, s3Path(namespace, fileId), new ByteArrayInputStream(bytes), meta)
FileInfo.builder()
.fileId(fileId)
.size(meta.getContentLength)
.contentType(contentType)
.name(fileName)
.build()
})(blockEc)
Code for downloading:
def download(namespace: String, fileId: String): Future[(FileInfo, InputStream)] = Future({
val obj = try s3Client.getObject(bucketName, s3Path(namespace, fileId)) catch {
case e: AmazonS3Exception if e.getStatusCode == 404 ⇒ throw new NotFoundError("File not found!")
}
val fileName = obj.getObjectMetadata.getUserMetadata.get(FileNameHeader) match {
case null ⇒ "file"
case encoded ⇒ new String(Base64.getDecoder.decode(encoded), Charsets.UTF_8)
}
(FileInfo.builder()
.fileId(fileId)
.name(fileName)
.size(obj.getObjectMetadata.getContentLength)
.contentType(obj.getObjectMetadata.getUserMetadata.getOrDefault(ContentTypeHeader, null))
.build(),
obj.getObjectContent)
})(blockEc)
Result when I trying to open link to file from Chrome/Safari:
This site can’t be reached
The webpage at [link] might be temporarily down or it may have moved
permanently to a new web address. ERR_HTTP2_PROTOCOL_ERROR
where could the problem be?
I tried to change the encoding, add metadata to S3. Nothing helps.
In the past, the archive with pdf docs was able to download and everything worked. But for a single document happens something strange.
The problem was a special character '№'. For downloadable archives, the names were simple, such as "ARCHIVE.zip".
So check to save the file with a simple name and if it can be downloaded you have a problem with naming. But I can't figure out why chrome and safari with https connection do not want to save the file. Maybe someone will explain why some browsers behave so strangely?
I want to test my method, which requires uploading a file. It is initialized like this:
val tempFile = TemporaryFile(new java.io.File("/home/ophelia/Desktop/students"))
val part = FilePart[TemporaryFile](
key = "students",
filename = "students",
contentType = Some("text/plain"),
ref = tempFile)
val files = Seq[FilePart[TemporaryFile]](part)
val formData = MultipartFormData(
dataParts = Map(),
files = Seq(part),
badParts = Seq(),
missingFileParts = Seq())
I pass it into the FakeRequest:
val result = route(
FakeRequest(POST, "/api/courses/"+"4f3c4ec9-46bf-4a05-a0b2-886c2040f2f6"+"/import" )
.withHeaders("Authorization" -> ("Session " + testSessionA.id.string))
.withMultipartFormDataBody(formData)
)
But when I run the test I get the following error:
Cannot write an instance of play.api.mvc.AnyContentAsMultipartFormData to HTTP response. Try to define a Writeable[play.api.mvc.AnyContentAsMultipartFormData]
What am I doing wrong and how to fix it? I looked on the internet, I didnt find any useful way to understand and resolve this problem.
It's important to remember that http requests are entirely text. route() takes an implicit Writeable to convert the body type of the provided request into text. Without the right Writeable, there is no way to know how to turn MultipartFormData into a request body.
There doesn't seem to be a Writeable for MultipartFormData, but you can provide your own. jroper has a great Writeable you could use for reference. (EDIT: That code is buggy, here's a working Writeable for AnyContentAsMultipartFormData)
Once you have your Writeable, you will need to make it accessible to your call to route(). Bear in mind, you currently have a FakeRequest[AnyContentAsMultipartFormData], not a FakeRequest[MultipartFormData]. You can either convert your request first:
val request = FakeRequest(POST,
"/api/courses/"+"4f3c4ec9-46bf-4a05-a0b2-886c2040f2f6"+"/import" )
.withHeaders("Authorization" -> ("Session "))
.withMultipartFormDataBody(formData)
route(request.map(_.mdf).asInstanceOf[FakeRequest[MultipartFormData[TemporaryFile]]])
or make your Writeable a Writeable[AnyContentAsMultipartFormData].
route for a given Request[T] requires an implicit parameter of type Writeable[T] that knows how to serialize the request body, because it will actually call the controller action just like an actual web request would, by pushing bytes onto it.
The problem is that there is no Writeable[MultipartFormData] predefined (you can see which are in play.api.test.Writeables).
This means you basically have two options:
write your own Writeable that serializes a MultipartFormData into bytes
Skip the routing part and call the action directly instead, like in the accepted answer in Play Framework Testing using MultipartFormData in a FakeRequest. This way of testing actions takes a shortcut and does not actually serialize and deserialize the request.
IMHO the first option is way too much pain for the gain, but if you go down that road, maybe contribute it to play when you succeed.
One of the possible solutions is to use wsUrl. For example
"File uploading action" should {
"upload sent file and result in ID" in {
val file = Paths.get(getClass.getResource("/1.txt").toURI)
val action = wsUrl("/upload").post(Source.single(FilePart("file", "hello.txt", Option("text/plain"), FileIO.fromPath(file))))
val res = Await.result(action, timeout)
res.status mustBe OK
res.body contains "123"
}
}
I am uploading a videos and images using web-service and save the images in our application. When i save the files, the files are save on root of application folder. I want to access those images and videos with localhost url, like: I upload the file and save under app-root/upload/image.jpg. In my route mapping file, i declare routing as below:
GET /uploads/ staticDir:/upload
As define in Play Documentation. But still getting an compile time error: Controller method call expected. I want to access image like this http://localhost:9999/uploads/image.jpg
Well... One way of doing this is by adding following routes,
GET /uploads/*file controllers.Assets.at(path="/uploads", file)
But, it will interfere with the reverse-routing of already existing route which is,
GET /assets/*file controllers.Assets.at(path="/public", file)
And then you will have to use your these two assets routes as - #route.Assets.at("public", filename) and #route.Assets.at("uploads", filename) which means all your templates which use you public assets route as - #route.Assets.at(filename) will have to be changed. Which can be a hassle in an existing big project.
You can avoid this by using following method,
Create another controller as,
package controllers
object FileServer extends Controller {
def serveUploadedFiles1 = controllers.Assets.at( dicrectoryPath, file, false )
// Or... following is same as above
def serveUploadedFiles2( file: String ) = Action.async {
implicit request => {
val dicrectoryPath = "/uploads"
controllers.Assets.at( dicrectoryPath, file, false ).apply( request )
}
}
}
The above should have worked... but seems like play does a lot of meta-data checking on the requested "Assets" which somehow results in empty results for all /uploads/filename requests. I tried to look into the play-source code to check, but it seems like it may take sometime to figure it out.
So I think we can make do with following simpler method ( It can be refined further in so many ways.).
object FileServer extends Controller {
import play.api.http.ContentTypes
import play.api.libs.MimeTypes
import play.api.libs.iteratee.Enumerator
import play.api.libs.concurrent.Execution.Implicits.defaultContext
def serveUploadedFiles(file: String) = Action { implicit request =>
val fileResUri = "uploads/"+file
val mimeType: String = MimeTypes.forFileName( fileResUri ).fold(ContentTypes.BINARY)(addCharsetIfNeeded)
val serveFile = new java.io.File(fileResUri)
if( serveFile.exists() ){
val fileContent: Enumerator[Array[Byte]] = Enumerator.fromFile( serveFile )
//Ok.sendFile(serveFile).as( mimeType )
val response = Result(
ResponseHeader(
OK,
Map(
CONTENT_LENGTH -> serveFile.length.toString,
CONTENT_TYPE -> mimeType
)
),
fileContent
)
response
}
else {
NotFound
}
}
def addCharsetIfNeeded(mimeType: String): String =
if (MimeTypes.isText(mimeType)) s"$mimeType; charset=$defaultCharSet" else mimeType
lazy val defaultCharSet = config(_.getString("default.charset")).getOrElse("utf-8")
def config[T](lookup: Configuration => Option[T]): Option[T] = for {
app <- Play.maybeApplication
value <- lookup(app.configuration)
} yield value
}
But this method will cause some troubles in case of packaged-build deployments.
Which means, using the Play's Asset thing would be wiser choice. So looking again, the controllers.Assets.at which is actually controllers.Assets.assetAt uses this method at one place,
def resource(name: String): Option[URL] = for {
app <- Play.maybeApplication
resource <- app.resource(name)
} yield resource
Which means, it tries to locate the resource in the directories which are part of application's classpath and our uploads folder sure is not one of them. So... we can make play's Assets.at thingy work by adding uploads to classpath.
But... thinking again... If I recall all folders in the classpath are supposed to be packaged in the package to be deployed in-case of packaged-build deployments. And uploaded things will be created by the users, which means they should not be a part of package. Which again means... we should not be trying to access our uploaded things using Play's Assets.at thingy.
So... I think we are better off using our own simpler rudimentary implementation of serveUploadedFiles.
Now add a route in route file as,
GET /uploads/*file controllers.FileServer.serveUploadedFiles( file:String )
Also... Keep in mind that you should not be thinking of using play to serve your uploaded assets. Please use nginx or something similar.
What is the preferred method to serve a static file from a Play Framework 2 Scala controller?
The file is bundled with my application, so it's not possible to hardcode a filesystem absolute /path/to/the/file, because its location depends on where the Play app happens to be installeld.
The file is placed in the public/ dir, but not in app/assets/, because I don't want Play to compile it.
(The reason I don't simply add a route to that file, is that one needs to login before accessing that file, otherwise it's of no use.)
Here is what I've done so far, but this breaks on my production server.
object Application ...
def viewAdminPage = Action ... {
... authorization ...
val adminPageFile = Play.getFile("/public/admin/index.html")
Ok.sendFile(adminPageFile, inline = true)
}
And in my routes file, I have this line:
GET /-/admin/ controllers.Application.viewAdminPage
The problem is that on my production server, this error happens:
FileNotFoundException: app1/public/admin/index.html
Is there some other method, rather than Play.getFile and OK.sendFile, to specify which file to serve? That never breaks in production?
(My app is installed in /some-dir/app1/ and I start it from /some-dir/ (without app1/) — perhaps everything would work if I instead started the app from /some-dir/app1/. But I'd like to know how one "should" do, to serve a static file from inside a controller? So that everything always works also on the production servers, regardless of from where I happen to start the application)
Check Streaming HTTP responses doc
def index = Action {
Ok.sendFile(
content = new java.io.File("/tmp/fileToServe.pdf"),
fileName = _ => "termsOfService.pdf"
)
}
You can add some random string to the fileName (individual for each logged user) to avoid sharing download link between authenticated and non-authinticated users and also make advanced download stats.
I did this: (but see the Update below!)
val fileUrl: java.net.URL = this.getClass().getResource("/public/admin/file.html")
val file = new java.io.File(adminPageUrl.toURI())
Ok.sendFile(file, inline = true)
(this is the controller, which is (and must be) located in the same package as the file that's being served.)
Here is a related question: open resource with relative path in java
Update
Accessing the file via an URI causes an error: IllegalArgumentException: URI is not hierarchical, if the file is then located inside a JAR, which is the case if you run Play like so: play stage and then target/start.
So instead I read the file as a stream, converted it to a String, and sent that string as HTML:
val adminPageFileString: String = {
// In prod builds, the file is embedded in a JAR, and accessing it via
// an URI causes an IllegalArgumentException: "URI is not hierarchical".
// So use a stream instead.
val adminPageStream: java.io.InputStream =
this.getClass().getResourceAsStream("/public/admin/index.html")
io.Source.fromInputStream(adminPageStream).mkString("")
}
...
return Ok(adminPageFileString) as HTML
Play has a built-in method for this:
Ok.sendResource("public/admin/file.html", classLoader)
You can obtain a classloader from an injected Environment with environment.classLoader or from this.getClass.getClassLoader.
The manual approach for this is the following:
val url = Play.resource(file)
url.map { url =>
val stream = url.openStream()
val length = stream.available
val resourceData = Enumerator.fromStream(stream)
val headers = Map(
CONTENT_LENGTH -> length.toString,
CONTENT_TYPE -> MimeTypes.forFileName(file).getOrElse(BINARY),
CONTENT_DISPOSITION -> s"""attachment; filename="$name"""")
SimpleResult(
header = ResponseHeader(OK, headers),
body = resourceData)
The equivalent using the assets controller is this:
val name = "someName.ext"
val response = Assets.at("/public", name)(request)
response
.withHeaders(CONTENT_DISPOSITION -> s"""attachment; filename="$name"""")
Another variant, without using a String, but by streaming the file content:
def myStaticRessource() = Action { implicit request =>
val contentStream = this.getClass.getResourceAsStream("/public/content.html")
Ok.chunked(Enumerator.fromStream(contentStream)).as(HTML)
}
I'm trying to serve an image from a play-mini application.
object App extends Application {
def route = {
case GET(Path("/image")) => Action { request =>
Ok( Source.fromInputStream(getClass.getResourceAsStream("image.gif")).toArray ).as("image/gif")
}
}
}
Unfortunately, this does noe work :) I get the following error
Cannot write an instance of Array[Char] to HTTP response. Try to define a Writeable[Array[Char]]
Don't know about play-mini, but in play20 there is predefined Writeable[Array[Byte]], so you need to provide Array[Byte] for file handling. Also, there is a bit of documentation about serving files in play20.
I had the same problem and kept scratching my head for almost a week. Turned out the solution that worked for me was the following piece of code in my controller class:
def getPhoto(name: String) = Action {
val strPath = Paths.get(".").toAbsolutePath.toString() + "/public/photos/" + name
val file1: File = strPath
.toFile
val fileContent: Enumerator[Array[Byte]] = Enumerator.fromFile(new java.io.File(file1.path.toString))
Ok.stream(fileContent).as("image/jpeg")
}
And the route was defined as below:
GET /photos/:name controllers.myController.getPhoto(name)
Hence typing the URL with the photos extension displayed the photo on the browser like so: http://localhost:9000/photos/2018_11_26_131035.jpg
The image is saved in a folder "public/photos" in the root folder of the application and not necessarily the assets folder. Hope this helps someone :-)