In Gatling, I'd like to perform a check on some JSON included in an HTML response like below:
<!doctype html>
<html lang="fr">
<head>
<script>
var documentLoaded = performance.now();
</script>
<link rel="stylesheet" href="/styles/main.f14d8fab5a7e.css">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/manifest.json">
<link rel="preconnect" href="https://www.gstatic.com">
<title data-react-helmet="true">Asus Discount</title>
<meta data-react-helmet="true" name="description" content="Asus discount”/><meta data-react-helmet="true" name="keywords" content="Asus"/>
</head>
<body>
<div>Some content</div>
<script>
var parseStart = performance.now();
</script>
<script>
window.__INITIAL_STATE__ = {some JSON}; <!-- This is what I need -->
window.__ENV_VARIABLES__ = {some other JSON};
window.renderTime = '76';
window.fetchTime = '349';
</script>
<script type="text/javascript" charset="utf-8" src="/vendor.e33d9940372.js"></script>
<script type="application/ld+json" src="/schema.fr.json"></script>
</body>
</html>
My actual solution (which is working) looks as follow:
def loadPageJsonInHTML(requestName: String, link: String): ChainBuilder ={
exec(
http(requestName)
.get(link)
.check(regex("""window[.]__INITIAL_STATE__ = ([^;]+)""").find.transform(s => parseSToProdList(s)).saveAs("prod_list")
)
)
doIf("${prod_list.size()}" == 0){
exec{session => session.markAsFailed}
}
}
def parseSToProdList(jsonString: String): Seq[String] ={
val jsonMap = jsonStrToMap(jsonString)
val buffer = mutable.Buffer.empty[String]
jsonMap("products").asInstanceOf[Map[String, Any]].foreach{f =>
if(f._2.asInstanceOf[Map[String, Any]].keySet.exists(_ == "code"))
buffer.append(f._2.asInstanceOf[Map[String, Any]]("code").asInstanceOf[String])
}
buffer.toSeq
}
def jsonStrToMap(jsonStr: String): Map[String, Any] = {
implicit val formats = org.json4s.DefaultFormats
parse(jsonStr).extract[Map[String, Any]]
}
However, this solution has several drawbacks:
The check will always succeed as long as the regex is found and doesn't care if there's any product at all in the JSON -> I have to manually check for it later;
Having a function extracting the required data is more difficult to maintain than if I could use a Json Path expression like "$.products.*.code" which could be stored on a centralised paths file for ease of maintenance;
This is the only place where I have to use a transform to check the JSON of a request, making it more difficult to read and understand.
What I'd like to achieve is something that would look a bit like this:
def loadPageJsonInHTML(requestName: String, link: String): ChainBuilder ={
exec(
http(requestName)
.get(link)
.check(jsonPath("""$.products.*.code""").findAll.saveAs("prod_list")
)
Or
def loadPageJsonInHTML(requestName: String, link: String): ChainBuilder ={
exec(
http(requestName)
.get(link)
.check(jsonpJsonPath("""$.products.*.code""").findAll.saveAs("prod_list")
)
Of course, jsonPath doesn't work since most of the answer is HTML. jsonpJsonPath doesn't work either as there's several Json strings in the response.
Any good input as to how could I do this more effectively (and nicely) while avoiding a regex on some HTML? Thanks in advance
So, after some digging I found a workaround using ".transformResponse", in order to extract the string before it's actually checked, giving it a default value that is parsable in Json. Then, to make sure we actually did find the regex, we make sure that it's not our default value :
def loadPageJsonInHTML(requestName: String, link: String): ChainBuilder = {
exec(
http(requestName)
.get(link)
.transformResponse{(session, response) =>
response.copy(body = new StringResponseBody(
(for(m <- """window[.]__INITIAL_STATE__ = ([^;]+)""".r
.findFirstMatchIn(response.body.string)
) yield m.group(1)
).getOrElse("""{"error":"chain not found"}"""),
UTF_8
)
)
}
.check(bodyString.not("""{"error":"chain not found"}"""))
.check(jsonPath("""$.products.*.code""").findAll.saveAs("prod_list")
)
)
}
Related
I have some basic Server and Client Side code with FastAPI, but I am trying to find the best way to read in that data for a live application that does some audio processing. I am not sure how to approach this, as FastAPI can only read the data from the client as bytes. Should I even bother converting it to a Blob after recording? Maybe use HTTP for simplicity (just building a prototype so it does not need to be optimal)? I also read about RTSP but I can't seem to find any good resources for it, so I would appreciate some way to point me in the right direction. Thanks for any advice!
Client
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<h1>helloworld</h1>
<script>
var ws = new WebSocket("ws://localhost:8000/ws", 'echo-protocol');
navigator.mediaDevices.getUserMedia({ audio: true })
.then(stream => {
const mediaRecorder = new MediaRecorder(stream);
mediaRecorder.start(5000);
mediaRecorder.addEventListener("dataavailable", event => {
console.log("sending audio");
const audioBlob = new Blob([event.data], {type: 'audio/mp3'});
ws.send(audioBlob);
console.log("sent audio");
});
});
</script>
</body>
</html>
Server
from fastapi import FastAPI, WebSocket, Request
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.get("/")
def read_root(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
#app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
while True:
data = await websocket.receive_bytes()
I am trying to use the bootstrap input spinner in my scalajs app. But when I try to use it I am getting the error "jQuery is not defined", even though I have included it in my project. Can someone tell me what I am missing?
So if I define a build.sbt:
name := "repro"
version := "1.0"
scalaVersion := "2.12.8"
scalaJSUseMainModuleInitializer := true
mainClass in Compile := Some("App")
enablePlugins(ScalaJSPlugin)
enablePlugins(ScalaJSBundlerPlugin)
webpackBundlingMode := BundlingMode.LibraryOnly()
libraryDependencies ++= Seq(
"org.querki" %%% "jquery-facade" % "1.2"
)
npmDependencies in Compile ++= Seq(
"bootstrap" -> "4.3.1",
"jquery" -> "3.2.1",
"bootstrap-input-spinner" -> "1.11.8",
)
And then try to use it in my app as follows:
#js.native
trait BootstrapInputSpinner extends JQuery {
def inputSpinner(options: js.Object = js.Dynamic.literal()): BootstrapInputSpinner = js.native
}
object BootstrapInputSpinner {
#JSImport("bootstrap-input-spinner", JSImport.Default)
#js.native
object Import extends BootstrapInputSpinner
val _import = Import // explicitly import it
implicit def bisExtensions(query: JQuery): BootstrapInputSpinner =
query.asInstanceOf[BootstrapInputSpinner]
}
object App {
import BootstrapInputSpinner._
def main(args: Array[String]): Unit = {
$(dom.document.getElementById("spinner")).inputSpinner()
}
}
My html file is defined as follows:
<!DOCTYPE html>
<html>
<head>
<title>Test User interface</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<div class="app-container" id="root">
<input id="spinner" type="number"/>
</div>
<script src="repro-fastopt-library.js"></script>
<script src="repro-fastopt-loader.js"></script>
<script src="repro-fastopt.js"></script>
</body>
</html>
Note that if I don't refer to the bootstrap-input-spinner library and try to use jQuery on its own it works fine. For example there are no errors if I change my App object to:
object App {
// import BootstrapInputSpinner._
def main(args: Array[String]): Unit = {
println($(dom.document.getElementById("spinner")))
}
}
Also, I checked the -library.js file and it has the following code:
module.exports = {
"require": (function(x1) {
return {
"jquery": __webpack_require__(2),
"bootstrap-input-spinner": __webpack_require__(3)
}[x1]
})
}
Which tells me that jquery should be imported first?
I could guess from looking at the code, that you might need to add jQuery script too, before the library script that depend on.
Side note: You must check the compatibility of the bootstrap version and the jquery it depends
<script
src="https://code.jquery.com/jquery-3.4.1.min.js"
integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo="
crossorigin="anonymous"></script>
<script src="repro-fastopt-library.js"></script>
<script src="repro-fastopt-loader.js"></script>
<script src="repro-fastopt.js"></script>
Which of the scripts depend on jquery?
repro-fastopt-library or
repro-fastopt-loader or
repro-fastopt
It could be the import order when you bundle them.
I am trying to fetch a page from the internet then save it into a HTML file. The page has this in the header:
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="ja" >
<head>
<meta http-equiv="Content-Type" content="text/html;charset=Shift_JIS" />
</head>
No matter what I try, the saved HTML page looks horrible and I just can't get it to save the Japanese characters properly.
I am using node-fetch, fs.writeFile and a module named jconv. I have tried all combinations but nothing works. Right now, the code is supposed to convert from SJIS to UTF-8, then fs should write the file with UTF-8 encoding.
fetch(link).
then((res) => {
if (res.ok) {
return res.text();
}
console.log("Invalid data");
}).
then((body) => {
// this is supposed to convert from SJIS to UTF-8
var buf = jconv.convert(body, 'SJIS', 'UTF-8');
// save file
fs.writeFile(path, buf, 'UTF-8', (err) => {
if (!err) {
console.log('Saved');
}
});
});
I have tried other encodings but the final HTML document still does not show the proper special characters, just like on the online page from which is taken. A page that I am testing right now is this
The line:
<meta http-equiv="Content-Type" content="text/html;charset=Shift_JIS" />
must also be modified to:
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8" />
to have the charset information in the header match the new encoding.
I can succesfully make POST requests to Mailgun and receive the emails as expected. I'm trying to inline an image into an email and can't work out how to do it.
Looking at https://documentation.mailgun.com/user_manual.html#sending-via-api and selecting Java, I can see that the example given constructs a FileDataBodyPart with "inline", the File reference and the MediaType. Looking at the curl example, this seems rather unnecessary as that just references a file.
Here is my method for sending an email:
def send(message:EmailMessage) = {
val postMessage = Map("from" -> Seq(message.from), "to" -> Seq(message.to), "subject" -> Seq(message.subject), "text" -> Seq(message.text), "html" -> Seq(message.html.toString()))
val logo = FileBody(Play.getExistingFile("/public/images/logo.png").get)
WS.url(apiUrl).withAuth("api", myKey, WSAuthScheme.BASIC).withBody(logo).post(postMessage)
}
The message.html.toString looks like the following:
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
</head>
<body style="background-color:#9B59B6; padding:10px">
<img src="cid:logo.png">
<h1 style="color:#FFF">Activate!</h1>
</body>
</html>
The logo.png file is found when sending the email and the email comes through fine, but with no image. This is what the email source looks like once it arrives at gmail:
Mime-Version: 1.0
Content-Type: text/html; charset="ascii"
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
</head>
<body style="background-color:#9B59B6; padding:10px">
<img src="cid:logo.png">
<h1 style="color:#FFF">Activate!</h1>
</body>
</html>
I can't see any base64 encoding of the image in the email. As the curl example appeared to just be passing a file as part of the POST, I though I'd try that. Here is what I did:
def send(message:EmailMessage) = {
val logoFile = Play.getExistingFile("/public/images/logo.png").get
val source = Files.readAllBytes(Paths.get(logoFile.getAbsolutePath))
val logoBase64 = Base64.encodeBase64String(source)
val postMessage = Map("from" -> Seq(message.from), "to" -> Seq(message.to), "subject" -> Seq(message.subject), "text" -> Seq(message.text), "html" -> Seq(message.html.toString()), "inline" -> Seq(logoBase64))
WS.url("https://api.mailgun.net/v2/sandboxaa9afcea1f2e4d5db5e2c080f7784b74.mailgun.org/messages").withAuth("api", "key-f165695d4c72e929ff8215115e648c95", WSAuthScheme.BASIC).post(postMessage)
}
I converted the logo into base64 and POSTed that like the other parameters. Still no joy.
What am I missing here? Do I need to pass this in the body, but somehow specify that this is an "inline" file?
I solved this by using Jersey, as suggested in the section on libraries: https://documentation.mailgun.com/libraries.html#java
I imported Jersey in sbt using the following:
libraryDependencies += "com.sun.jersey" % "jersey-core" % "1.18.3"
libraryDependencies += "com.sun.jersey" % "jersey-client" % "1.18.3"
libraryDependencies += "com.sun.jersey.contribs" % "jersey-multipart" % "1.18.3"
and then created my Email sending object like so:
object Email {
val client = Client.create()
client.addFilter(new HTTPBasicAuthFilter("api", current.configuration.getString("mailgun.api.key").get))
val webResource = client.resource(current.configuration.getString("mailgun.api.url").get)
def send(message:EmailMessage) = {
val form = new FormDataMultiPart
form.field("from", message.from)
form.field("to", message.to)
form.field("subject", message.subject)
form.field("text", message.text)
form.field("html", message.html.toString())
val logo = Play.getExistingFile("/public/images/logo.png").get
form.bodyPart(new FileDataBodyPart("inline", logo, MediaType.APPLICATION_OCTET_STREAM_TYPE))
webResource.`type`(MediaType.MULTIPART_FORM_DATA_TYPE).post(form)
}
}
I hope this helps someone.
Hello there Grammarians,
The documentation only shows:
flashing("success")
Do failures never happen if I use play? I've tried "failure" & "error" they don't do anything
You can either pass in a Flash instance or tuples of Strings. It doesn't have to be a specific String. Important is that you handle whatever you stick into the flash scope.
consider this example (Play 2.3.4):
Application.scala
package controllers
import play.api.mvc._
object Application extends Controller {
def index = Action { implicit req =>
Redirect(routes.Application.flash()).flashing("something" -> "show this text")
}
def flash = Action { implicit req =>
Ok(views.html.index("Flash!"))
}
}
index.scala.html
#(title: String)(implicit flash: Flash)
<!DOCTYPE html>
<html>
<head>
<title>#title</title>
</head>
<body>
<h1>#flash.get("something")</h1>
</body>
</html>
routes
# Home page
GET / controllers.Application.index
GET /flash controllers.Application.flash