Gatling / Scala remove Vector from string value in POST request - scala

I'm trying to send a POST request within a Gatling test.
2 values have to be sent, the first one is extracted from my page content, the second one is hardcoded.
My issue is that when i extract a value from my page content, i end up with a string submitted in my POST request but polluted with the "Vector()" wrapper.
Here is my scenario and how my variable is extracted:
val dossier = exec(http("Content creation - Extract vars")
.get("/node/add/dossier")
.check(css("""input[name="form_token"]""", "value").findAll.saveAs("form_token_node"))
.headers(headers_0))
.pause(2)
.exec(http("Content creation")
.post("/node/add/dossier")
.headers(headers_1)
.formParam("form_token", "${form_build_id_node}")
.formParam("form_id", "node_dossier_form")
.check(status.is(303))
)
And here is how the data look like when they are sent in the POST request:
form_token: Vector(HciBSyvuZ14NIj9HHuebgHYc06gL62B0iKAQ-E-KhvA)
form_id: node_dossier_form
As you can the the form_token variable should not look like this at all, it's breaking the form submission for a unvalid reason.
So my question is, how do i get ride of the Vector() part of the string?
And the answer is use ${form_build_id_node(0)} instead of ${form_build_id_node} to access to the value. Thanks to sschaef.

Here the issue is at saving the attribute.
you have used .findAll.saveAs - Which will save as list taking all the occurrences
If you want to pass only the first occurrences, then it should be
.check(css("""input[name="form_token"]""","value").saveAs("form_token_node"))
instead of
.check(css("""input[name="form_token"]""","value").findAll.saveAs("form_token_node"))
if your going to use foreach or repeat to get more values then you can .findAll.saveAs list and create a logic to iterate the session attribute
${form_build_id_node(i)} in your scenario

Related

pyTelegramBotAPI pagination: how to pass a data between pages in a message

I have a Telegram bot for the forwarding of database requests. It returns long messages, so,
I use pagination from python-telegram-bot-pagination, for splitting this message to blocks. But I can't find a way to split the message to blocks effectively.
Now it looks like this:
by user request, the #bot.message_handler runs send_info_to_user function.
send_info_to_user calls the some_long_operation and gets the whole dataset as a result.
send_info_to_user gets a slice from some_long_operation result and sends it as a message for user
in case of using pagination inline buttons, the #bot.callback_query_handler will call send_info_to_user again
it re-calls the some_long_operation and gets the whole dataset again, just for another slice from the result and edit the content of the previous message.
# a time-consumable function
def some_long_operation(user_id):
return [f'some_text{i}\n{user_id}' for i in range(40)]
# send info to user by pages
def send_info_to_user(message, user_id, isCallback=False, page=1):
result = some_long_operation(user_id)
paginator = InlineKeyboardPaginator(
len(answer),
current_page=page,
data_pattern=f'{user_id}#'+'items#{page}' # wrap user_id for callback
)
if isCallback:
bot.edit_message_text(message.chat.id, result[page-1], message.message_id, paginator.markup)
else:
bot.send_message(message.chat.id, result[page-1],paginator.markup)
#bot.message_handler(content_types='text')
def message_reply(message):
send_info_to_user(message, message.from_user.id)
# parse callback from inline buttons
#bot.callback_query_handler(func=lambda call: call.data.split('#')[1]=='items')
def items_page_callback(call):
page = int(call.data.split('#')[2])
user_id = call.data.split('#')[0] # unwrap user_id
get_items(call.message, user_id, True, page)
Obviously, re-call of some_long_operation every time and using a slice of it isn't a good solution. But I can't find a way to pass the data between pages in a message.
The result of some_long_operation is user-specific and changes in time, so, store it anywhere isn't a good idea.
As I think, the some_long_operation should be called once, the result of it should be available as a slices for different pages of the message.

How to crawl data from encrypted url?

I'm trying to use scrapy to collect the university's professors' contact information from its directory. Since I can't post more than 2 links, I put all links in the following picture.
I set last name equals from the drop-down menu as shown in the picture. Then I search all professors by last name.
Usually, the url will have some pattern from other universities' website. However, for this one, the original url is (1). It becomes (2)when I search 'An' as last name. It seems like 'An' is replaced by something like 529385FD5FF90A198625819E002B8B41? I'm not sure. Is there any way I can get the url that I need to send as a request? I mean, this time I search 'An'. If I search another last name like Lee. It will be another request. They are irregular. I can't find a pattern.
The scraper is not as complex as you think it is. It just makes a POST call from the form and that returns a GET request. Below would work
import scrapy
from scrapy.utils.response import open_in_browser
class univSpider(scrapy.Spider):
name = "univ"
start_urls = ["http://appl103.lsu.edu/dir003.nsf/(NoteID)/5903C096337C2AA28625819E0038E3E4?OpenDocument"]
def parse(self, response):
yield FormRequest.from_response(response, formname="_DIRVNAM", formdata={"LastName": "Lalwani"},callback = self.search_result)
def search_result(self, response):
open_in_browser(response)
print(response.body)

Sharepoint URL to reference a lookup

I have two SharePoint lists used for Support case management. The first list contains Case Numbers and information about the case. The second list contains exhibits that support the case itself.
We have a convention that the Case Number is a String supplied by the worker, ex 20150205-001. When the exhibits are joined to the Case it is through a Lookup. I want the Exhibit ID, a String, to be of the form Case Number + _[A-Z] -- and be auto-assigned.
I want to use a Workflow (MS Sharepoint Designer 2013) to assign the Exhibit ID. The problem I face is that I cannot get the actual Case Number from the Lookup. The closest I have gotten so far is to get the ID (1, 2, etc) but not the actual String value represented.
Tried working with the following URL:
http://[mySiteURL]/_api/web/lists/getbytitle([listName])/items?$select=Title,Case/Id&$expand=Case/Id&$filter=Case/Id%20eq%2020150205%45001
substituted ascii: $filter=Case/ID eq 20150205-001
without the filter I get all list items (understandably) but the filter does not work properly because the ID is not the actual lookup value.
This is a SPD 2013 limitation. You have to use a web service call from within Designer to get the specifics of a lookup column from SharePoint. You make a REST call ad then parse the JSON response for the specific data from the lookup column. It gives you access to all of the columns from the list item that you looked up:
Build {...} Dictionary (Output to Variable: requestHeader )
Call [%Workflow Context:Current Site URL%]... HTTP web service with Variable: Request (ResponseContent to Variable: PoleIDData |ResponseHeaders to Variable: dictionary |ResponseStatusCode to Variable: responseCode )
Get d/Pole_x0020_ID from Variable: PoleIDData (Output to Variable: PoleID )
Set Name to Variable: PoleID
Your actual web service call will be formatted like this:
[%Workflow Context:Current Site URL%]/_api/web/lists/GetByTitle('List Name')/Items([%Current Item:ID%])/LookupColumnNameOnOtherList
Sorry for the formatting, I would post a screenshot but I cannot.
This article is good for showing you some of the other specifics about formatting your HTTP Request, especially the Request Headers which must be setup right.
http://www.fiechter.eu/Blog/Post/37/SharePoint-2013--HTTP-Web-Service-Action---Use-Managed-Metadata-as-Text-in-Workflow

Scala/Gatling: How do I append to an array within a foreach?

I'm uber new with Scala/Gatling, so I apologize in advance if this question is dumb.
Writing gatling tests, and I've got a foreach that iterates through a list of items, makes a request to a URL, and verifies the response.
Now, I'd like to take an element from each response and create an array of them in the session for future use.
Here's some code:
foreach("collection","currentItem"){
exec(
http(requestName)
.post([redacted])
.param([redacted])
.check(
regex(""""error":\s*"([^"]*)"""").find.notExists
)
.check(
regex(""""id":\s*"([^"]*)"""").findAll.saveAs([HERE'S WHERE I'M STUCK])
)
)
}
I don't want to just use saveAs("someString"), or it'll be overwritten with every iteration of the foreach. What I need is all of the ids from each response to be appended to an array that I can then throw into the session.
I've been googling all morning, but every other search takes me to a github page that says "the documentation has been moved", and then provides a link to nowhere. I also didn't see any relevant stackoverflow posts.
Anyone have any ideas?
You can't do that directly in the check. You have to add an extra exec step after your request, but inside the loop that would fetch from the Session the previously stored accumulator and what was saved by the check in the current iteration, and push the result back into the Session.

HTML form POST method with querystring in action URL

Lets say I have a form with method=POST on my page.
Now this form has some basic form elements like textbox, checkbox, etc
It has action URL as http://example.com/someAction.do?param=value
I do understand that this is actually a contradictory thing to do, but my question is will it work in practice.
So my questions are;
Since the form method is POST and I have a querystring as well in my URL (?param=value)
Will it work correctly? i.e. will I be able to retrieve param=value on my receiving page (someAction.do)
Lets say I use Java/JSP to access the values on server side. So what is the way to get the values on server side ? Is the syntax same to access value of param=value as well as for the form elements like textbox/radio button/checkbox, etc ?
1) YES, you will have access to POST and GET variables since your request will contain both. So you can use $_GET["param_name"] and $_POST["param_name"] accordingly.
2) Using JSP you can use the following code for both:
<%= request.getParameter("param_name") %>
If you're using EL (JSP Expression Language), you can also get them in the following way:
${param.param_name}
EDIT: if the param_name is present in both the request QueryString and POST data, both of them will be returned as an array of values, the first one being the QueryString.
In such scenarios, getParameter("param_name) would return the first one of them (as explained here), however both of them can be read using the getParameterValues("param_name") method in the following way:
String[] values = request.getParameterValues("param_name");
For further info, read here.
Yes. You can retrieve these parameters in your action class.
Just you have to make property of same name (param in your case) with there getters and setters.
Sample Code
private String param;
{... getters and setters ...}
when you will do this, the parameters value (passed via URL) will get saved into the getters of that particular property. and through this, you can do whatever you want with that value.
The POST method just hide the submitted form data from the user. He/she can't see what data has been sent to the server, unless a special tool is used.
The GET method allows anybody to see what data it has. You can easily see the data from the URL (ex. By seeing the key-value pairs in the query string).
In other words it is up to you to show the (maybe unimportant) data to the user by using query string in the form action. For example in a data table filter. To keep the current pagination state, you can use domain.com/path.do?page=3 as an action. And you can hide the other data within the form components, like input, textarea, etc.
Both methods can be catched in the server with the same way. For example in Java, by using request.getParameter("page").