I want to call value of 'myActionID' variable. How do I do that?
If i pass static value like "actionId":1368201 to myActionID then it works, but If I use "actionId" : ${actionIdd} it gives error.
Here's the relevant code:
class LaunchWorkflow_Act extends Simulation {
val scenarioRepeatCount = 1
val userCount = 1
val myActionID = "13682002351"
val scn = scenario("LaunchMyFile")
.repeat (scenarioRepeatCount) {
exec(session => session.set("counter", (globalVar.getAndIncrement+" "+timeStamp.toString())))
.exec(http("LaunchRequest")
.post("""/api/test""")
.headers(headers_0)
.body(StringBody(
"""{ "actionId": ${myActionID} ,
"jConfig": "{\"wflow\":[{\"Wflow\":{\"id\": \"13500145349\"},\"inherit-variables\": true,\"workflow-context-variable\": [{\"variable-name\": \"externalFilePath\",\"variable-value\": \"/var/nem/nem/media/mount/assets/Test.mp4\"},{\"variable-name\": \"Name\",\"variable-value\": \"${counter}\"}]}]}"
}""")))
.pause(pause)
}
}
setUp(scn.inject(atOnceUsers(userCount))).protocols(httpProtocol)
Everything works fine If I put value 13682002351 instead of myActionID. While executing this script in Gatling I am Getting this error
ERROR i.g.http.action.HttpRequestAction - 'httpRequest-3' failed to
execute: No attribute named 'myActionID' is defined
Scala has various mechanisms for String Interpolation (see docs), which can be used to embed variables in strings. All of them can be used in conjunction with the triple quotes """ used to create multi-line strings.
In this case, you can use:
val counter = 12
val myActionID = "13682002351"
val str = s"""{
"actionId": $myActionID ,
"jConfig": "{\"wflow\":[{\"Wflow\":{\"id\": \"13500145349\"},\"inherit-variables\": true,\"workflow-context-variable\": [{\"variable-name\": \"externalFilePath\",\"variable-value\": \"/var/nem/nem/media/mount/assets/Test.mp4\"},{\"variable-name\": \"Name\",\"variable-value\": \"${counter}\"}]}]}"
}"""
Notice the s prepended to the string literal, and the dollar sign prepended to the variable names.
Using S interpolated String we can do this easily:
s"""Hello Word , Welcome Back!
How are you doing ${userName}"""
Related
I am currently importing the dataset from an excel sheet which has a column name with a dot character like this "abc.xyz".
I went through a couple of stackOverflow questions and it says that we can replace it with the column names with backtick like this: "'abc.xyz'". So, I renamed all the column names which have a dot in it with the same name but enclosed in backticks like this:
df.columns.foreach(item => {
if(item.contains("."))
{
df.withColumnRenamed(item, s"`$item`")
}
})
Now when I pass this dataframe inside the ConstraintSuggestionRunner class like this:
val suggestionResult = ConstraintSuggestionRunner()
.onData(df)
.addConstraintRules(Rules.DEFAULT)
.setKLLParameters(KLLParameters(sketchSize = 2048, shrinkingFactor = 0.64, numberOfBuckets = 10))
.run()
I am getting errors like :
ERROR Main: org.apache.spark.sql.AnalysisException: cannot resolve
'`abc.xyz`' given input columns:
How can I resolve this error?
The escaping must be handled in Deequ but the issue is always open. What you did here is adding the backticks as part of the column names, not escaping them.
You can try to replace the dots by another caracheter like underscore _ then pass the dataframe with the renamed columns to the ConstraintSuggestionRunner:
val df1 = df.toDF(df.columns.map(_.replaceAll("[.]+", "_")):_*)
val suggestionResult = ConstraintSuggestionRunner()
.onData(df1)
.addConstraintRules(Rules.DEFAULT)
.setKLLParameters(KLLParameters(sketchSize = 2048, shrinkingFactor = 0.64, numberOfBuckets = 10))
.run()
I've got a Scala Akka App where I execute python scripts inside Futures with ProcessBuilder.
Unfortunately are special character not displayed correct, so do I get instead of mädchen-> m�dchen
(äöü -> �)
If I execute the python script via command line do I get the right output of "mädchen", so I assume it has nothing to do with the python script instead somehow related to my Scala input read.
Python Spider:
print("mädchen")
Scala:
val proc = Process("scrapy runspider spider.py")
var output : String = ""
val exitValue = proc ! ProcessLogger (
(out) => if( out.trim.length > 0 )
output += out.trim,
(err) =>
System.err.printf("e:%s\n",err)
)
println(exitValue) // 0 -> succ.
println(output) // m�dchen -> should be mädchen
I already tried many thinks and also read that Strings are by default UTF-8 so I am not sure why I get those question marks.
Also did I tried with no success:
var byteBuffer : ByteBuffer = StandardCharsets.UTF_8.encode(output.toString())
val str = new String(output.toString().getBytes(), "UTF-8")
Update:
It seems to be a windows related issue, following instruction will solve this problem: Using UTF-8 Encoding (CHCP 65001) in Command Prompt / Windows Powershell (Windows 10)
I am new to scala and gatling . I am trying to fetch values from feeder and post zip file to service . but ${extensionId} is not been updated with fetched value instead it remain as ${extensionId} . Could some one please help me know If I miss some thing here .
def installExtension() =
exec(http("template - Install Extension")
.post(url + "/v1/extensions")
.basicAuth("jack", "password")
.headers(namespaceHeader)
// using testUtils to get InputStream conte
.body(InputStreamBody(TestUtils.toStream(hashMap.get("${extensionId}").getOrElse(null))))
.check(status.is( 201)))
class extmgrSimulations extends Simulation {
val extensionIds = csv(s"${Configuration.dataDirectory}/extensionId.csv").circular
val extMgrScenerio = scenario("extensionMgr - Scenario")
.during(Configuration.duration) {
exitBlockOnFail(
group("load-test") {
exec(
pace(Configuration.paceFrom, Configuration.paceTo),
feed(extensionIds),feed(extensionIds)
randomSwitch(
50.00 -> group("Install and delete") {
exec(
extmgrChain.installExtension(),
extmgrChain.deleteExtension(),
)
},
50.00 -> extmgrChain.listExtension()
)
)
}
)
}
That can't work. Gatling EL (the ${} syntax in strings) doesn't magically work anywhere. This is explained in the documentation.
Warning
This Expression Language only works on String values being passed to Gatling DSL methods. Such Strings are parsed only once, when the Gatling simulation is being instantiated.
For example queryParam("latitude", session => "${latitude}") wouldn’t work because the parameter is not a String, but a function that returns a String.
Also, queryParam("latitude", "${latitude}".toInt) wouldn’t because the toInt would happen before passing the parameter to the queryParam method.
The solution here would be to pass a function:
session => session("latitude").validate[Int].
I wrote this:
df.select(col("colname")).distinct().collect.map(_.toString()).toList
the result is
List("[2019-06-24]", "[2019-06-22]", "[2019-06-23]")
Whereas I want to get :
List("2019-06-24", "2019-06-22", "2019-06-23")
How to change this please
You need to change .map(_.toString()) to .map(_.getAs[String]("colname")).With .map(_.toString()), you are calling org.apache.spark.sql.Row.toString, that's why the output is like List("[2019-06-24]", "[2019-06-22]", "[2019-06-23]").Correct way is:
val list = df.select("colname").distinct().collect().map(_.getAs[String]("colname")).toList
Output will be:
List("2019-06-24", "2019-06-22", "2019-06-23")
Sample data:
val df=sc.parallelize(Seq(("2019-06-24"),( "2019-06-22"),("2019-06-23"))).toDF("cn")
Now select column then apply map to get first index value then add quotes and convert to string.
df.select("cn").collect().map(x => x(0)).map(x => s""""$x"""".toString)
//res36: Array[String] = Array("2019-06-24", "2019-06-22", "2019-06-23")
(or)
df.select("cn").collect().map(x => x(0)).map(x => s""""$x"""".toString).toList
//res37: List[String] = List("2019-06-24", "2019-06-22", "2019-06-23")
We are using Scala Play, and I am trying to ensure that all SQL queries are using Anorm's String Interpolation. It works with some queries, but many are not actually replacing the variables before the query is executing.
import anorm.SQL
import anorm.SqlStringInterpolation
object SecureFile
{
val table = "secure_file"
val pk = "secure_file_idx"
...
// This method works exactly as I would hope
def insert(secureFile: SecureFile): Option[Long] = {
DBExec { implicit connection =>
SQL"""
INSERT INTO secure_file (
subscriber_idx,
mime_type,
file_size_bytes,
portal_msg_idx
) VALUES (
${secureFile.subscriberIdx},
${secureFile.mimeType},
${secureFile.fileSizeBytes},
${secureFile.portalMsgIdx}
)
""" executeInsert()
}
}
def delete(secureFileIdx: Long): Int = {
DBExec { implicit connection =>
// Prints correct values
println(s"table: ${table} pk: ${pk} secureFileIdx: ${secureFileIdx} ")
// Does not work
SQL"""
DELETE FROM $table WHERE ${pk} = ${secureFileIdx}
""".executeUpdate()
// Works, but unsafe
val query = s"DELETE FROM ${table} WHERE ${pk} = ${secureFileIdx}"
SQL(query).executeUpdate()
}
}
....
}
Over in the PostgreSQL logs, it's clear that the delete statement has not acquired the correct values:
2015-01-09 17:23:03 MST ERROR: syntax error at or near "$1" at character 23
2015-01-09 17:23:03 MST STATEMENT: DELETE FROM $1 WHERE $2 = $3
2015-01-09 17:23:03 MST LOG: execute S_1: ROLLBACK
I've tried many variations of execute, executeUpdate, and executeQuery with similar results. For the moment, we are using basic string replacement but of course this is bad because it's not using PreparedStatements.
For anyone else sitting on this page scratching their head and wondering what they might be missing...
SQL("select * from mytable where id = $id")
is NOT the same as
SQL"select * from mytable where id = $id"
The former does not do String interpolation whereas the latter does.
This is easily overlooked in the aforementioned docs as all the samples provided just happen to have a (non-related) closing parenthesis on them (like this sentence does)
Anorm String interpolation was introduced to pass parameter (e.g. SQL"Select * From Test Where id = $x), with interpolation arguments (e.g. $x) set on underlying PreparedStament according proper type conversion (see use cases on https://www.playframework.com/documentation/2.3.x/ScalaAnorm ).
Next Anorm release will also have the #$foo syntax to mix interpolation for parameter with standard string interpolation. This will allow to write DELETE FROM #$table WHERE #${pk} = ${secureFileIdx} and having it executed as DELETE FROM foo WHERE bar = ? (if literal table is "foo" and pk is "bar"), with literal secureFileIdx passed as parameter. See related pull request.
Until next revision is release, you can build Anorm from its master sources ti benefit from this change.