Jenkins: Active Choices Parameter + Groovy to build a list based on REST responde - rest

I have a REST client that returns me a list of systems.
I need this list to be as a parameter for a jenkins job.
I think I need Actice Choices Parameter plugin with Groovy and HTTPBuilder in order to do that.
What do you guys think?
I did not find a way to install HTTPBuilder into Jenkins.
Is there any other way that you guys think it is possible?

I have run into the same problem trying to parse parameters via groovy script. Arun's answer did not work for me. However, I have managed to get it to work using the following:
import java.io.BufferedReader
import java.io.InputStreamReader
import java.io.OutputStreamWriter
import java.net.URL
import java.net.URLConnection
import groovy.json.JsonSlurper
def choices = []
def url = new URL("some.data.url")
def conn = url.openConnection()
conn.setDoOutput(true)
def reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))
def results = new JsonSlurper().parseText(reader.getText());
reader.close()
results.each { data -> choices.push(data.field) }
return choices.sort()

First paste the JSON body snapshot output -or whatever your REST client is going to return. That'll help.
For ex: if it'll return a JSON object then you can use Active Choice Parameter's Groovy script step - OR Scriptler script (within the Active Choice Parameter plugin). PS: Scriptler script runs in the same JVM of Jenkins process so it has access to Jenkins/etc object for free. You don't need HTTPBuilder or anything. See the code sample below.
Assuming if your REST client is returning a JSON object and from that object if you want to list hostname of the system or some field name then replace the following variable with that and you'll get it listed while doing "Build with parameters" from Jenkins job's dashboard.
import groovy.json.JsonSlurper
//this will be your URL which will return something, tweak it if you want to pass parameters or username/password acc.
def SOME_URL = "https://koba.baby.com/some_url"
// now connect to the URL and create a connection variable 'conn'
def conn = SOME_URL.toURL().openConnection()
// create a list variable 'servernames'
def servernames = []
// if connection response was successful i.e. http protocol return code was 200, then do the following
if( conn.responseCode == 200 ) {
// get the results / output of the URL connection in a variable 'results'
def results = new JsonSlurper().parseText(conn.content.text)
// to see results variable output uncomment the next line
//println results
// now read each element in the 'results' variable and pick servername/somefield variable into the list variable 'servernames'
results.each { id, data -> servernames.push(data.someField_or_HostName) }
}
return servernames.sort().unique()
// return servernames.sort()

Related

Build Jenkins pipeline using HttpRequest

I try to execute a jenkins pipeline from a Scala app, but the authentication is required. So, I think before this request, I need to call the auth endpoint and get a token, then assign it on the headers.
val userToken = "user_token_generated_in_jenkins"
val pipelineToken = "pipeline_token"
val pipelineName = "HttpRequestTest"
// test1
val baseUrl = s"http://jenkins_address/view/Development/job/$pipelineName/build?token=$pipelineToken"
val response = Http(baseUrl).headers("token" -> "tried_a_valid_token_auth").asString
// test2
val url = s"http://auto:$userToken#jenkins_address/job/$pipelineName/build?token=$pipelineToken"
val response2 = Http(url).asString
If I can be sure that first test is wrong, why for test2, where I use the user token (generated in Users manager), it asks me to use extra token (generated by authentication)? Is a double verification and useless in my opinion.
As an emergency solution, I can execute the pipeline with curl using sys.process. But I prefer to solve it using Http library because I want to use generated token and not a specific username in my project code.
// test3
import sys.process._
val data = List("curl", "-u", "username:password", baseUrl).!!
How can I execute an HttpRequest using Jenkins User Token?
Thanks
If your Http(..) is from Akk-HTTP, the solution could be derived from here:
val authorization = headers.Authorization(BasicHttpCredentials("username", "password"))
HttpRequest(
PUT,
uri = "/user",
entity = HttpEntity(`text/plain` withCharset `UTF-8`, userData),
headers = List(authorization),
protocol = `HTTP/1.0`)
In case of scalaj-http it could be derived from here
With applying to your existing code, it should be like this:
Http(baseUrl).auth("username", "password").asString
By the way
I want to use generated token and not a specific username in my
project code
It is not a problem. You can construct your process' auth string dynamically. You are able to get the credentials from somewhere in the "native" Http solution, right? So you can construct your sys.process with the same getted-from-somewhere credentials by concatinating strings or string interpolation:
// test3
import sys.process._
val (username, password) = gettingFromSomewhereCreadentials()
val data = List("curl", "-u", s"$username:$password", baseUrl).!!

PlayFramework Testing: Uploading File in Fake Request Errors

I want to test my method, which requires uploading a file. It is initialized like this:
val tempFile = TemporaryFile(new java.io.File("/home/ophelia/Desktop/students"))
val part = FilePart[TemporaryFile](
key = "students",
filename = "students",
contentType = Some("text/plain"),
ref = tempFile)
val files = Seq[FilePart[TemporaryFile]](part)
val formData = MultipartFormData(
dataParts = Map(),
files = Seq(part),
badParts = Seq(),
missingFileParts = Seq())
I pass it into the FakeRequest:
val result = route(
FakeRequest(POST, "/api/courses/"+"4f3c4ec9-46bf-4a05-a0b2-886c2040f2f6"+"/import" )
.withHeaders("Authorization" -> ("Session " + testSessionA.id.string))
.withMultipartFormDataBody(formData)
)
But when I run the test I get the following error:
Cannot write an instance of play.api.mvc.AnyContentAsMultipartFormData to HTTP response. Try to define a Writeable[play.api.mvc.AnyContentAsMultipartFormData]
What am I doing wrong and how to fix it? I looked on the internet, I didnt find any useful way to understand and resolve this problem.
It's important to remember that http requests are entirely text. route() takes an implicit Writeable to convert the body type of the provided request into text. Without the right Writeable, there is no way to know how to turn MultipartFormData into a request body.
There doesn't seem to be a Writeable for MultipartFormData, but you can provide your own. jroper has a great Writeable you could use for reference. (EDIT: That code is buggy, here's a working Writeable for AnyContentAsMultipartFormData)
Once you have your Writeable, you will need to make it accessible to your call to route(). Bear in mind, you currently have a FakeRequest[AnyContentAsMultipartFormData], not a FakeRequest[MultipartFormData]. You can either convert your request first:
val request = FakeRequest(POST,
"/api/courses/"+"4f3c4ec9-46bf-4a05-a0b2-886c2040f2f6"+"/import" )
.withHeaders("Authorization" -> ("Session "))
.withMultipartFormDataBody(formData)
route(request.map(_.mdf).asInstanceOf[FakeRequest[MultipartFormData[TemporaryFile]]])
or make your Writeable a Writeable[AnyContentAsMultipartFormData].
route for a given Request[T] requires an implicit parameter of type Writeable[T] that knows how to serialize the request body, because it will actually call the controller action just like an actual web request would, by pushing bytes onto it.
The problem is that there is no Writeable[MultipartFormData] predefined (you can see which are in play.api.test.Writeables).
This means you basically have two options:
write your own Writeable that serializes a MultipartFormData into bytes
Skip the routing part and call the action directly instead, like in the accepted answer in Play Framework Testing using MultipartFormData in a FakeRequest. This way of testing actions takes a shortcut and does not actually serialize and deserialize the request.
IMHO the first option is way too much pain for the gain, but if you go down that road, maybe contribute it to play when you succeed.
One of the possible solutions is to use wsUrl. For example
"File uploading action" should {
"upload sent file and result in ID" in {
val file = Paths.get(getClass.getResource("/1.txt").toURI)
val action = wsUrl("/upload").post(Source.single(FilePart("file", "hello.txt", Option("text/plain"), FileIO.fromPath(file))))
val res = Await.result(action, timeout)
res.status mustBe OK
res.body contains "123"
}
}

Gatling 2 dynamic queryParam on each request

I am trying to run a load test using Gatling 2. I need to generate one of the query parameters dynamically on each request.
My scenario is defined like this:
val scn = scenario("Load Test Scenario")
.exec(
http("Test API")
.post(url)
.body(StringBody("Some XML"))
.queryParam("x", DigestUtils.md5Hex(generateX().getBytes("UTF-8")))
)
def generateX() : String = {
// generate random string and return
}
This only calls generateX once and uses the result in each request. Is there anyway to have the generateX call on every request?
You have to pass a function, not a value. See Gatling documentation about Expression.
Here, you can just discard the session input parameter as you don't use it, so you can simply write:
.queryParam("x", _ => DigestUtils.md5Hex(generateX().getBytes("UTF-8")))

How to compare two SOAP responses in SoapUI for discrepancies, using Groovy script or something else

I would like to compare for discrepancies(namespaces, content) two SOAP responses in SoapUI that i get at the same time, and display the differences in the SoapUI log or log.info, currently i got only a piece of code to get the response and put it in a variable:
import java.util.regex.Matcher
import java.util.regex.Pattern
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context ) ;
def holder = groovyUtils.getXmlHolder( "GetAttachementList#Response" ) ;
log.info(holder.getXml());
You can use soapui assertions.
http://www.soapui.org/Functional-Testing/getting-started-with-assertions.html

Serving static /public/ file from Play 2 Scala controller

What is the preferred method to serve a static file from a Play Framework 2 Scala controller?
The file is bundled with my application, so it's not possible to hardcode a filesystem absolute /path/to/the/file, because its location depends on where the Play app happens to be installeld.
The file is placed in the public/ dir, but not in app/assets/, because I don't want Play to compile it.
(The reason I don't simply add a route to that file, is that one needs to login before accessing that file, otherwise it's of no use.)
Here is what I've done so far, but this breaks on my production server.
object Application ...
def viewAdminPage = Action ... {
... authorization ...
val adminPageFile = Play.getFile("/public/admin/index.html")
Ok.sendFile(adminPageFile, inline = true)
}
And in my routes file, I have this line:
GET /-/admin/ controllers.Application.viewAdminPage
The problem is that on my production server, this error happens:
FileNotFoundException: app1/public/admin/index.html
Is there some other method, rather than Play.getFile and OK.sendFile, to specify which file to serve? That never breaks in production?
(My app is installed in /some-dir/app1/ and I start it from /some-dir/ (without app1/) — perhaps everything would work if I instead started the app from /some-dir/app1/. But I'd like to know how one "should" do, to serve a static file from inside a controller? So that everything always works also on the production servers, regardless of from where I happen to start the application)
Check Streaming HTTP responses doc
def index = Action {
Ok.sendFile(
content = new java.io.File("/tmp/fileToServe.pdf"),
fileName = _ => "termsOfService.pdf"
)
}
You can add some random string to the fileName (individual for each logged user) to avoid sharing download link between authenticated and non-authinticated users and also make advanced download stats.
I did this: (but see the Update below!)
val fileUrl: java.net.URL = this.getClass().getResource("/public/admin/file.html")
val file = new java.io.File(adminPageUrl.toURI())
Ok.sendFile(file, inline = true)
(this is the controller, which is (and must be) located in the same package as the file that's being served.)
Here is a related question: open resource with relative path in java
Update
Accessing the file via an URI causes an error: IllegalArgumentException: URI is not hierarchical, if the file is then located inside a JAR, which is the case if you run Play like so: play stage and then target/start.
So instead I read the file as a stream, converted it to a String, and sent that string as HTML:
val adminPageFileString: String = {
// In prod builds, the file is embedded in a JAR, and accessing it via
// an URI causes an IllegalArgumentException: "URI is not hierarchical".
// So use a stream instead.
val adminPageStream: java.io.InputStream =
this.getClass().getResourceAsStream("/public/admin/index.html")
io.Source.fromInputStream(adminPageStream).mkString("")
}
...
return Ok(adminPageFileString) as HTML
Play has a built-in method for this:
Ok.sendResource("public/admin/file.html", classLoader)
You can obtain a classloader from an injected Environment with environment.classLoader or from this.getClass.getClassLoader.
The manual approach for this is the following:
val url = Play.resource(file)
url.map { url =>
val stream = url.openStream()
val length = stream.available
val resourceData = Enumerator.fromStream(stream)
val headers = Map(
CONTENT_LENGTH -> length.toString,
CONTENT_TYPE -> MimeTypes.forFileName(file).getOrElse(BINARY),
CONTENT_DISPOSITION -> s"""attachment; filename="$name"""")
SimpleResult(
header = ResponseHeader(OK, headers),
body = resourceData)
The equivalent using the assets controller is this:
val name = "someName.ext"
val response = Assets.at("/public", name)(request)
response
.withHeaders(CONTENT_DISPOSITION -> s"""attachment; filename="$name"""")
Another variant, without using a String, but by streaming the file content:
def myStaticRessource() = Action { implicit request =>
val contentStream = this.getClass.getResourceAsStream("/public/content.html")
Ok.chunked(Enumerator.fromStream(contentStream)).as(HTML)
}