I need to perform a load test against a pgbouncer. All the clients perform an SSH Tunnel before enstablishing a database connection to the database (through the pgbouncer). It's something like this:
sshpass -p 'MY_PSW' ssh -o StrictHostKeyChecking=no -N -L LOCAL_PORT:127.0.0.1:63666 PGBOUNCER_USER#PGBOUNCER_ADDRESS -p PORT >/dev/null 2>&1 &
My Jmeter project has three thread Groups at the moment:
SetUp Thread Group: In which I make a connection to a different database to select a random username and schema
Query Thread Group: In which I perform the JDBC connection using the previous user (which became a property using props.put("schema", vars.get("schema_1")); into the BeanShell Assertion) and the queries
TearDown Thread Group: In which I close the ssh Tunnel.
Now every first time I run the test from the GUI, the data select (JDBC request) into the Query Thread Group gives me an error:
Cannot create PoolableConnectionFactory (FATAL: "trust" authentication failed)
After that, if I run the test again, everything works. I checked the content of the variables and properties with a Debug sampler and everything is correct.
The main problem starts when I run the test without the GUI. It always fails because of that error.
I actually don't like the fact that I have to enstablish the SSH tunnel running the command with an OS Process Sampler, but I can't find any better solution. The SSH tunnel is a part of the test, I don't need that for the master/slave configuration of JMeter.
I would appreciate a lot for a solution or a suggestion to make this work. Thanks.
If you're using the command in the OS Process Sampler it's being run in the background therefore my expectation is that the OS Process Sampler returns the SampleResult immediately and the tunnel is not up yet.
Then when the "Query Thread Group" starts as per JMeter Test Elements execution order JDBC Connection Configuration tries to establish the connection using local port which is not fully established. The fact that the issue is reproducible in non-GUI mode might be the confirmation for my guess as JMeter works much faster in non-GUI as it doesn't need to waste time and resources for GUI refreshing and propagating sample results to listeners.
My expectation is that if you add i.e. Flow Control Action sampler to the setUp Thread Group and configure it to "sleep" for a couple of seconds it should resolve your issue. If it doesn't - try increasing JMeter logging verbosity for the JDBC Test Elements by adding the next line to log4j2.xml file
<Logger name="org.apache.jmeter.protocol.jdbc" level="debug" />
and compare the entries for "successful" and "failed" executions in the jmeter.log file.
Ok, I managed to find the solution. Thank to Dmitri T Answer, I could track the problem, which was about the property not being properly set.
I was using the BeanShell Assertion to set the property after the result of the JDBC request in the setUp Thread Group. Apparently, the BeanShell Assertion is executed at the end of the entire run, so the property where being set at the end. During the first execution, the property is empty.
I used the Beanshell Sampler instead and now it works.
Related
I've been using Mongo for a while now, and I never had any kind of errors. But today, I tried running the mongo command in my terminal and I got the following error:
Error connecting to 127.0.0.1:27017 :: caused by :: No connection could be made because the target machine actively refused it. :
I have my PATH variable for Mongo properly configured in my environment variables as follows:
C:\Program Files\MongoDB\Server\4.4\bin
so I doubt that is the issue. I remember going through my task manager yesterday and I accidentally terminated a program running in the background related to Mongo, but I can't seem to remember exactly what it was called, and I really think that that's the root of my problem, because before having terminated that Mongo program in my task manager I had never ran across this connection problem before.
By terminating a program in the background, I'm going to assume you didn't just end process, otherwise a simple computer restart would fix your issue. And in some cases, that same program would've relaunched when you launched MongoDB. But if you disabled a service and need to find which service needs to be running to be able to connect to your MongDB then I would suggest going through your Windows Services list and seeing which ones you disabled and looking one relating to TCP or SNMP.
This is because MongoDB Wire Protocol is a simple socket-based, request-response style protocol. You communicate with the database server through a regular TCP/IP socket and since you can't remember which one you "terminated" and any number of services related to networking can cause a dependency to be absent, I can't be more specific in helping you determine which one you need to turn back on and you'll have to do it through trial and error but I can at least offer you some guidance, hopefully.
Specifically you can either
Run system configuration using
msconfig
In a run box, navigating to the Services tab, order the list by Date Disabled to find the service that was disabled which correlates with when you when snooping through task manager, or
Run Task manager and navigate to the Services Tab, then Open Services, and order them by Status or by Name, and look for any service that includes TCP/IP, COM+, Port direction, etc. to see which one is disabled and change the configuration from anything but Disabled and then stat it manually and run MongDB again.
It's about as specific as I can get without knowing anything more than you terminated some program running in the background but I hope it helps.
The background process (daemon) for MongoDB is called 'mongod'. It's an executable in your bin directory inside your mongodb installation. You can just execute it in the terminal.
Run:
C:\Program Files\MongoDB\Server\4.4\bin\mongod.exe
I am using ORACLE database in a windows environment and running a JSP/servlet web application in tomcat. After I do some operations with the application it gives me the following error.
ORA-12518, TNS: listener could not hand off client connection
can any one help me to identify the reason for this problem and propose me a solution?
The solution to this question is to increase the number of processes :
1. Open command prompt
2. sqlplus / as sysdba; //login sysdba user
3. startup force;
4. show parameter processes; // This shows 150(some default) processes allocated, then increase the count to 800
5. alter system set processes=800 scope=spfile;
As Tried and tested.
In my case I found that it is because I haven't closed the database connections properly in my application. Too many connections are open and Oracle can not make more connections. This is a resource limitation. Later when I check with oracle forum I could see some reasons that have mentioned there about this problem. Some of them are.
In most cases this happens due to a network problem.
Your server is probably running out of memory and need to swap memory to disk.One cause can be an Oracle process consuming too much memory.
if it is the second one, please verify large_pool_size or check dispatcher were enough for all connection.
You can refer bellow link for further details.
https://community.oracle.com/message/1874842#1874842
I ran across the same problem, in my case it was a new install of the Oracle client on a new desktop that was giving the error, other clients were working so I knew it wouldn't be a fix to the database configuration. tnsping worked properly but sqlplus failed with the ora-12518 listener error.
I had the tnsnames.ora entry with a SID instead of a service_name, then once I fixed that, still the same error and found I had the wrong service_name as well. Once I fixed that, the error went away.
If from one day to another the issue shows for no apparent reasons, add these following lines at the bottom of the listner.ora file. If your oracle_home environment variable is set like this:
(ORACLE_HOME = C:\oracle11\app\oracle\product\11.2.0\server)
The lines to add are:
ADR_BASE_LISTENER = C:\oracle11\app\oracle\
DIRECT_HANDOFF_TTC_LISTENER=OFF
I had the same problem when executing queries in my application. I'm using Oracle client with Ruby on Rails.
The problem started when I accidentally started several connections with the DB and didn't close them.
When I fixed this, everything started to work fine again.
Hope this helps another one with the same problem.
I experienced the same error after upgrading to Windows 10. I solved it by starting services for Oracle which are stopped.
Start all the services as shown in the following image:
I had the same issue. After restarting all Oracle services it worked again.
same problem encountered for me.
And from oracle server listener log, can see more information.
and I found that the SERVICE_NAME is not match the tnsnames.ora configured Service name. so I changed the application's data source configuration from SID value to Service_NAME value and it fixed.
23-MAY-2019 02:44:21 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=XXXXXX$))(SERVICE_NAME=orclaic)) * (ADDRESS=(PROTOCOL=tcp)(HOST=::1)(PORT=50818)) * establish * orclaic * 12518
TNS-12518: TNS:listener could not hand off client connection
TNS-12560: TNS:protocol adapter error
TNS-00530: Protocol adapter error
64-bit Windows Error: 203: Unknown error
I had the same issue in real time application and the issue gone by itself next day. upon checking, it was found that server ran out of memory due to additional processes running.
So in my case, the reason was server run out of memory
first of all
check the listener log
check the show parameter processes vs select count(*) from v$processes;
increase the process, it may require SGA increase
;
Once a Login script is executed with few user, I don't see connection reset problem, whereas, when the same is run 100 users, "java.net.SocketException: Connection reset" starts throwing for very first link.
What I don't understand is if there is connection problem, then it should even show the same error for single or few users as well.
This means that your server is rejecting connections because it is either overloaded or misconfigured.
It is regular that you don't face it with 1 user and face it with 100, this is typically what load testing brings, ie simulate traffic on your server
It might be the case described in Connection Reset since JMeter 2.10 ? wiki page.
If you are absolutely sure that your server is not overloaded and is configured to accept 100+ connections (defaults are good for development, not for production, they need to be tweaked) you can try work it around as follows:
In user.properties file add the next 2 lines:
httpclient4.retrycount=1
hc.parameters.file=hc.parameters
In hc.parameters file add the following line:
http.connection.stalecheck$Boolean=true
Both files live in JMeter's bin folder.
You need to restart JMeter to pick the properties up.
Above instructions are applicable for HttpClient4 implementation, make sure you use it, the fastest and the easiest way to set HttpClient4 implementation for all the HTTP Request samplers is using HTTP Request Defaults
Jmeter Environment Details
I am performing Jmeter testing on Microsoft Azure Cloud. I have created on VM(Virtual Machine) on the same cloud and from there I am hitting the application server on the same cloud environment. So in this case there is no network latency.
Problem Statement:
I am trying to run the load test for 300 users for 30 mins , but after 5 mins my script started failing, because of Socket connection refused error.
My Analysis based on information available on net:
I have read somewhere that this problem is because of limited socket connection limit on server, but when i run the same test from VM then my scripts run's just fine. so its definitely not server's issue. Can somebody please help me resolve this issue? Are there any settings needs to be done in jmeter, increase the socket connections?
Actual Screenshot of Error
enter image description here
Most likely:
Looks like situation described at Connection Reset since JMeter 2.10 ? wiki page. If you're absolutely sure that nothing is wrong with your server, you can follow the next recommendations:
Switch all your HTTP Request Samplers "Implementation" to be "HTTPClient4". The fastest and the easiest way of doing it is using HTTP Request Defaults.
Add the next lines to user.properties file (in JMeter's /bin folder)
httpclient4.retrycount=1
hc.parameters.file=hc.parameters
Add (or uncomment and edit) the following line in hc.parameters file
http.connection.stalecheck$Boolean=true
Alternative assumption:
"Good" browsers send "Connection: close" with the last request to the web server. "Bad" browsers don't and keep connection open. You can control this behaviour via "Use KeepAlive" checkbox in the HTTP Request Sampler/Defaults. If it's unchecked - you can try ticking it.
We're currently using chef to provision our servers and we want our recipe/cookbook to automatically add some data to the mongo database once its installed and running.
This is where we start to run into problems. We were using an execute resource to run the mongo script like this:
execute "install-mongodb-config" do
command "mongo #{node[:mongodb][:mongo_db_host]}/#{node[:mongodb][:mongo_db]} \"#{node[:mongodb][:mongo_add_config_script]}\""
action :run
end
This part of the recipe always failed no matter what we tried! I won't get into the details of everything we tried here (unless i need to) but lets just say that i've exhausted all possibilities of subscribes and notifies (i think).
The problem originates from the fact that we are using the mongodb::10gen_repo to install mongodb. The recipe exits when apt-get installs the package and then chef continues on to execute more resourses.
We have tried executing the above resource directly after mongodb::10gen_repo but it doesn't seem like mongodb is available and the mongo shell cannot connect and run the script. The error we see is somewhat like this:
MongoDB shell version: 2.0.2
Thu Sep 6 18:40:45 ReferenceError: setTimeout is not defined mongotest.js:2
failed to load: mongoAddConfig.js
Nothing we have tried has been able to get around this in a nice chef way. The thing that we resorted to was to replace the execute resource with the following:
execute "install-mongodb-config" do
command "sleep 60; mongo #{node[:mongodb][:mongo_db_host]}/#{node[:mongodb][:mongo_db]} \"#{node[:mongodb][:mongo_add_config_script]}\""
action :run
end
Which just makes the command sleep for 60 seconds before the mongo script is run. I know this isn't the Right way to do this but it works for now.
Can anyone suggest the Right way to do this? I have a feeling that I will need to talk to the guys that created the mongodb chef script and request a feature!
First of all. Remove this "sleep 60". This can be done by chef: All resources have common attributes and "retries" and "retry_delay" are part of them. So the easiest way would be:
execute "install-mongodb-config" do
command "mongo some_command"
action :run
retries 6
retry_delay 10
end
If you have more than 2-3 places, where you have to run some command on mongo database, consider creating LWRP, similar to one created in this mongodb cookbook. (Particularly check the libraries/mongodb.rb file). You can hide the logic that waits for the server to respond there.
Is it important that the same Chef run that installs the software also injects the initial configuration? The 'chefly' method to constructing cookbooks and recipes is to guard against idempotency in order to ensure that they can be run over and over again without producing unintended results.
In this particular case, I would limit the first recipe to only just installing and starting up mongodb. This recipe would do nothing if it saw that mongodb was already running on the host. Then, I'd have another recipe that would run only if it saw that mongo had been setup and was running. It would query the mongodb to see if the initial configuration had been done. If so, it would simply return. If not, it would run your configuration routine.
In this way, these recipes could run all the time, anytime, on your machine. Even if someone uninstalled mongodb, chef would get around to ensuring that it was set back up again and pristine.
So, I don't know much at all about chef. But your problem seems to be that you try to immediately connect after bringing the server up.
Server's are not immediately available when you bring them up since there is a bit of overhead that goes into electing a primary, getting all the server status's etc.
You can recreate this without chef by trying to bring up a replica set and immediately trying to connect to it in a simple script. So it's not chef specific.
Not sure if there is a way around the server startup lag since bringing up a primary is expected to be a relatively infrequent occurrence compared to just adding nodes to a set.
The only potential solution I see that is cleaner is adding a longer Timeout for the connection to be formed in the configuration. You can find how to do this in the mongodb documentation here: http://www.mongodb.org/display/DOCS/Connections
The flag of interest for you is likely connectTimeoutMS