I made a wrapper for a Poly API function. The wrapper is supposed to fire some Locust events. An API function "authorize" is called successfully (if wrong parameters are supplied to "authorize" then it throws an exception in the console). It looks though that Locust event function is not. The statistics is not reflected in the Locust interface, like nothing is happening.
import time
from business_logic_refactored import BusinessLogic
from locust import User, task, between, events
class PolyClinet(BusinessLogic):
_locust_environment = None
def __getattr__(self, name):
#func = BusinessLogic.__getattr__(self, name)
func = BusinessLogic.authorize
def wrapper(*args, **kwargs):
start_time = time.time()
try:
result = func(*args, **kwargs)
events.request_success.fire(request_type="TEST", name=name, response_time=total_time, response_length=0)
self._locust_environment.events.request_success.fire(request_type="TEST", name=name, response_time=total_time, response_length=0)
except Fault as e:
total_time = int((time.time() - start_time) * 1000)
self._locust_environment.events.request_failure.fire(request_type="TEST", name=name, response_time=total_time, exception=e)
else:
total_time = int((time.time() - start_time) * 1000)
events.request_success.fire(request_type="grpc", name=name, response_time=total_time, response_length=0)
###self._locust_environment.events.request_success.fire(request_type="TEST", name=name, response_time=total_time, response_length=0)
# In this example, I've hardcoded response_length=0. If we would want the response length to be
# reported correctly in the statistics, we would probably need to hook in at a lower level
return wrapper
class PolyUser(User):
abstract = True
def __init__(self, *args, **kwargs):
super(PolyUser, self).__init__(*args, **kwargs)
self.client = PolyClient()
self.client._locust_environment = self.environment
class ApiUser(PolyUser):
#task(10)
def get_time(self):
self.client.authorize("user", "password","URL")
UPD 06/11/20:
Basically, the problem is that .getattr is only called on a non-existing function. Here is the script to try and see the difference/problem:
class BusinessLogic():
def __getattr__(self, name):
return name
def authorize(self):
print("authorize")
class PolyClinet(BusinessLogic):
_locust_environment = None
def __getattr__(self, name):
print("getattr")
func = BusinessLogic.__getattr__(self, name)
def wrapper(*args, **kwargs):
print("wrapper")
func()
return wrapper
class PolyUser():
abstract = True
def __init__(self, *args, **kwargs):
super(PolyUser, self).__init__(*args, **kwargs)
self.client = PolyClinet()
#self.client._locust_environment = self.environment
class ApiUser(PolyUser):
def get_time(self):
print("get_time")
self.client.authorize()
def get_time2(self):
print("get_time2")
self.client.authorize2()
c= ApiUser()
c.get_time()
print("*******")
c.get_time2()
To log an event, you’re supposed to do self.environment.events.request_success.fire(...) in your User class
See https://github.com/SvenskaSpel/locust-plugins/blob/2958d265857a8730af3b6cd9fd05f8839ec8f383/locust_plugins/users.py#L70 for example.
I’m not sure what happens when you call events.request_success directly. Maybe it should throw an exception.. It could probably be a lot better documented as well...
Edit: The root problem was that BusinessLogic has its own regular (non-getattr) methods, in which case, subclassing it will make a client unable to wrap the BusinessLogic method (because the client's getattr will not get called) The example in the documentation should be improved. https://github.com/locustio/locust/issues/1423
In my case the problem was the Locust version. In version 1.4.4 events.request_success.fire works (and named EventHook) but in version 1.5.1 it should be just request object and events.request_success.fire has already DeprecatedEventHook name.
https://docs.locust.io/en/stable/api.html#locust.event.Events.request
Related
I would like to use this command to set my custom variable, how to do this:
self.ent1.configure(my_custom_var='teste')
I want my custom variable to be part of the .configure dictionary
sample code:
from tkinter import *
class My_Entry(Entry):
def __init__(self, parent, my_custom_var='', *args, **kwargs):
super().__init__(parent, *args, **kwargs)
#print('my_custom value: ', my_custom_var)
print(self['my_custom_var'])
return
def configure(self, **kwargs):
super().configure(**kwargs)
print(kwargs) #<--- my custom var here in kwargs
#--------------
class Mainframe(Tk):
def __init__(self):
Tk.__init__(self)
#self.ent1 = My_Entry(self, my_custom_var='teste')
self.ent1 = My_Entry(self)
self.ent1.configure(show='*')
#self.ent1.configure(show='*', my_custom_var='teste')
self.ent1.pack()
return
if __name__== '__main__':
app = Mainframe()
app.mainloop()
Tkinter doesn't have a way to add an option that works exactly like the built-in options. However, you can override configure and cget to handle both your custom options and the default options.
Here's an example of one way to do it, though it's not the only way.
class My_Entry(tk.Entry):
# tuple of supported custom option names
custom_options = ("my_custom_var",)
def __init__(self, parent, *args, my_custom_var='', **kwargs):
super().__init__(parent)
self.configure(my_custom_var=my_custom_var, **kwargs)
def configure(self, **kwargs):
for key in self.custom_options:
if key in kwargs:
setattr(self, key, kwargs.pop(key))
if kwargs:
super().configure(**kwargs)
def cget(self, key):
if key in self.custom_options:
return getattr(self, key)
else:
return super().cget(key)
This lets you use either cget or directly access the class attribute:
entry = My_Entry(root, width=40, my_custom_var="Hello, world")
print(f"custom var via cget: {entry.cget('my_custom_var')}")
print(f"custom var via attribute: {entry.my_custom_var}")
And within the class, you can do likewise:
print(f"custom var via cget: {self.cget('my_custom_var')}")
print(f"custom var via attribute: {self.my_custom_var}")
I'm trying to run unit test for a ray remote function. I am using a #patch decorator to patch the remote function. The
foo.py
class Foo(object):
def __init__(self):
self.value = 0
def bar(self):
self.value = 100
print("In original method")
assert False
test_foo.py
from unittest.mock import patch
import pytest
import unittest
import ray
from tests.foo import Foo
#pytest.fixture
def ray_fixture():
print("Initializing ray")
if not ray.is_initialized():
ray.init()
yield None
print("Terminating ray")
ray.shutdown()
def fake_bar(self):
print("In fake method")
assert True
#pytest.mark.usefixtures("ray_fixture")
class FooTestCase(unittest.TestCase):
"""Test cases for Foo module"""
#patch("foo.Foo.bar", new=fake_bar)
def test_bar(self):
Foo().bar()
#patch("foo.Foo.bar", new=fake_bar)
def test_bar_remote(self):
foo_actor = ray.remote(Foo).remote()
obj_ref = foo_actor.bar.remote()
ray.get(obj_ref)
The test test_bar passes and test_bar_remote fails.
If I use ray.init(local_mode=True) then both tests pass. I can not use local_mode=True due to other limitations.
How can we patch ray actor's remote method using #patch?
Here's an alternative. Subclass Foo with a stubbed/mocked implementation and use it in ray. That way, the Foo class would be intact, you would only update those that needs to be mocked e.g. the method bar().
test_foo.py
...
class FooStub(Foo):
def bar(self, *args, **kwargs):
print("In another fake method")
assert True
# Optionally, you can also call the real method if you want. You may update the arguments as needed.
# super().bar(*args, **kwargs)
#pytest.mark.usefixtures("ray_fixture")
class FooTestCase(unittest.TestCase):
...
def test_bar_remote(self):
foo_actor = ray.remote(FooStub).remote()
obj_ref = foo_actor.bar.remote()
ray.get(obj_ref)
...
Output
$ pytest -q -rP
..
================================================================================================= PASSES ==================================================================================================
__________________________________________________________________________________________ FooTestCase.test_bar ___________________________________________________________________________________________
------------------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------------------
Initializing ray
------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------
In fake method
---------------------------------------------------------------------------------------- Captured stdout teardown -----------------------------------------------------------------------------------------
Terminating ray
_______________________________________________________________________________________ FooTestCase.test_bar_remote _______________________________________________________________________________________
------------------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------------------
Initializing ray
---------------------------------------------------------------------------------------- Captured stdout teardown -----------------------------------------------------------------------------------------
Terminating ray
2 passed, 1 warning in 5.03s
I have found a below hacky way which involves change in original function (and check env variable to provide test implementation)
import os
class Foo(object):
def __init__(self):
self.value = 0
def bar(self):
self.value = 100
if os.environ.get['TEST'] == 'True':
print("In fake method")
assert True
else:
print("In original method")
assert False
runtime_env = {"env_vars": {"TEST": "True"}}
ray.remote(Foo) .options(runtime_env=runtime_env) .remote()
My goal is to do some database queries from the async controller, then return the answer.
I'm playing with the example project, for now just simulating the DB queries with a sleep, but what I noticed is that whatever I do, the REST interface won't even start the sleep of the second query until the first one finishes.
E.g.: If I call the REST interface from one tab in the browser, then 1 second later again from an another tab, I'd expect that the second one gets the reply too in 10 seconds, but actually it's 19.
Also it doesn't seem to use the "database-io" pool either:
1: application-akka.actor.default-dispatcher-2
2: application-akka.actor.default-dispatcher-5
My code:
#Singleton
class AsyncController #Inject()(cc: ControllerComponents, actorSystem: ActorSystem) extends AbstractController(cc) {
implicit val executionContext = actorSystem.dispatchers.lookup("database-io")
def message = Action.async {
getFutureMessage().map { msg => Ok(msg) }
}
private def getFutureMessage(): Future[String] = {
val defaultThreadPool = Thread.currentThread().getName;
println(s"""1: $defaultThreadPool""")
val promise: Promise[String] = Promise[String]()
actorSystem.scheduler.scheduleOnce(0 second) {
val blockingPool = Thread.currentThread().getName;
println(s"""2: $blockingPool""")
Thread.sleep(10000)
promise.success("Hi!")
}(actorSystem.dispatcher)
promise.future
}
}
It could be two reasons for this behavior:
You use the development mode (1 thread), or your product configuration is configured only for one thread.
The browser blocks the second request until receiving the response from the first. This phrase: "If I call the REST interface from one tab in the browser." Try to do the same from different browsers.
You need to avoid blocking code. Basically:
You can a method that returns the Future.
You map into it.
You recover any failure the Future result might bring.
Lets say I have:
def userAge (userId: String): Future[Int] = ???
Then you map into it:
userAge.map{
age => ??? //everything is ok
}.recover{ case e: Throwable => ??? //Do something when it fails
Note that if you have more than one call the other map becomes flatMap because you want Future[...] something and not Future[Future[...]].
Sometimes I create a class and its only used to store some values or static methods and I never want to create an instance.
Is there some way to express this in Python3?
Eg:
class MyClass:
#staticmethod
def hello():
print("world")
# is there a better way to do this?
def __init__(self):
raise Exception("instantiation isnt supported for this class!")
# OK
MyClass.hello()
# NOT OK
c = MyClass()
You can use the objectless baseclass found in this answer (which is actually an answer to a different question).
class objectless:
def __new__(cls, *args, **kwargs):
raise RuntimeError('%s should not be instantiated' % cls)
class UninstantiateableClass(objectless):
#classmethod
def foo(cls):
return 'bar'
...
Note that in python2, objectless should explicitly subclass object, in order to be a new-style class.
Please excuse my poor understanding of Scala. I'm just a Java developer that wants to get something working in the Play Framework. I even attempted to implement a trait using Java code, but I got even more obscure errors. I have the following Scala code:
package models
import scala.concurrent.Future
class SettableFuture[T](name: String) extends Future[T] {
var assignedValue: T
def set(newValue: T) =
assignedValue = newValue
//override abstract methods in Future[T]
def ready(atMost: scala.concurrent.duration.Duration)(implicit permit: scala.concurrent.CanAwait): models.SettableFuture[T] =
this
def result(atMost: scala.concurrent.duration.Duration)(implicit permit: scala.concurrent.CanAwait): T =
assignedValue
def isCompleted: Boolean =
false
def onComplete[U](func: scala.util.Try[T] => U)(implicit executor: scala.concurrent.ExecutionContext): Unit =
null
def value: Option[scala.util.Try[T]] =
null
}
And here's my error:
overriding method ready in trait Awaitable of type (atMost: scala.concurrent.duration.Duration)(implicit permit: scala.concurrent.CanAwait)SettableFuture.this.type; method ready has incompatible type
Ignore the return values of the methods for now, they're nonsensical because I'm just trying to fix all the compile errors.
I simply copied the method stubs from the compile time exception when extending a trait without overriding its abstract methods and pasted them into my source file. I don't understand why I'm still getting errors. I took a look at the signature for ready() in Awaitable and it looks like the return type should in fact be the class.
EDIT: the reason why I want to implement this is because in the Promise/Future Scala API, I can only find things that let me asynchronously execute long-running, blocking tasks. What I'm after is something that lets execution of a request be suspended until something of interest sets a value in the SettableFuture instance, which completes a Promise to send a response. In this way, it's somewhat like a continuation. Anyway, here is the working code that I ended up with:
package models
import java.util.concurrent.CountDownLatch
import java.util.concurrent.TimeUnit
import java.util.concurrent.atomic.AtomicInteger
import scala.concurrent.CanAwait
import scala.concurrent.ExecutionContext
import scala.concurrent.Future
import scala.concurrent.duration.Duration
import scala.util.Try
class SettableFuture[T]() extends Future[T] {
private final val ValueNotSet = 0
private final val ValueBeingSet = 1
private final val ValueSet = 2
private val valueStatus: AtomicInteger = new AtomicInteger(ValueNotSet)
private val onCompleteWaitHandle: CountDownLatch = new CountDownLatch(1)
private var onComplete: Try[T] => _ = _
private var assignedValue: T = _
/** Set a value and complete this Future.
*
* Returns false if the value has already been set by a past call to this method.
* Otherwise, marks this Future as complete, executes the function passed to
* onComplete, and finally returns true.
*/
def set(newValue: T): Boolean = {
//set value and trigger onComplete only once
if (valueStatus.compareAndSet(ValueNotSet, ValueBeingSet)) {
assignedValue = newValue
valueStatus.set(ValueSet)
onCompleteWaitHandle.countDown()
if (onComplete != null)
onComplete(Try(assignedValue))
true
}
false
}
//override abstract methods in the Future[T] trait
def ready(atMost: Duration)(implicit permit: CanAwait): this.type = {
onCompleteWaitHandle.await(atMost.length, atMost.unit)
this
}
def result(atMost: Duration)(implicit permit: CanAwait): T = {
ready(atMost)
assignedValue
}
def isCompleted: Boolean = (valueStatus.get() == ValueSet)
def onComplete[U](func: Try[T] => U)(implicit executor: ExecutionContext): Unit =
onComplete = func
def value: Option[Try[T]] = {
if (!isCompleted)
None
Some(Try(assignedValue))
}
}
In terms of the specific error you are getting, the ready method you are overriding has a return type of Awaitable.this.type - ie. the specific type of this instance of Awaitable (the super type of Future, so that in Future, this method is seen to have return type Future.this.type). For your SettableFuture class, this means the return type for the ready method needs to be models.SettableFuture.this.type.
Other minor problems you can expect to hit: the onComplete method's implementation should be {}, not null, as the latter is a return of type Null.type, not Unit, and the var assignedValue needs to be initialised in a non-abstract class, which can be done by adding = _ to the line defining the variable (although you really want to make it at least protected, and provide an accessor which will check if it has been set - maybe by changing the variable to an Option[T] initialised to None, or else maintaining a Boolean flag that can be checked in the accessor, and which is set to true by the set method).
In terms of what it looks like you are trying to achieve, however, you might just want to check out scala.concurrent.Promise, which represents a "promise of a future result". It has a method future that returns a Future, and various complete, completeWith, and similar methods that can be used to set the Promise's value, which in turn will cause the associated Future to become ready/completed.
The SettableFuture class unnecessarily mixes the two concerns that the Future and Promise traits were designed to separate:
asynchronous provision of a value (Promise), and
waiting for and reacting to that provision (Future)
Rather than thinking of a Future as an asynchronous, long-running, blocking computation, it may help to think of it simply as a value that may be provided at some future time. You can react to provision of that value in a number of ways, including registering a callback or mapping it to some other value. For example, in Play one will often suspend processing of a request with a pattern like this (in Scala):
def handleRequest = Action {
Async {
gimmeAFuture().map(value => Ok(value))
}
}
def gimmeAFuture(): Future[JsValue] = // ...
The gimmeAFuture method returns a Future, but the request handling code doesn't care about how the value is computed. It could be
computed immediately with Future.successful,
computed asynchronously with Future.apply, or
supplied by the completion of a Promise
As an example of the latter, the gimmeAFuture method might be implemented as follows:
def gimmeAFuture: Future[JsValue] = {
val p = Promise.apply[JsValue]()
// asynchronously complete the promise 30 seconds from now
scheduler.scheduleOnce(30.seconds)(p.complete(Success(someJsObject)))
// return the future immediately
p.future
}
Of course, you could implement that method however you want. The key is that something will need to hold on to that Promise object and complete it with a value in order to resume handling of the request. The request handler itself does not get a reference to the Promise object, since it only cares about the value that will be computed (i.e., the Promise's Future).
I revisited this question after finding this link and I realized why there was so much confusion.
I was planning on using the SettableFuture class because I couldn't find anything like it that already existed in the Play Java API. I wanted something equivalent to a TaskCompletionSource in .NET, and Aaron's answer made it really clear that Scala had exactly what I needed. Unfortunately, I couldn't find the equivalent in Play's Java API.
That link cleared up exactly why I was having so much difficulty over something that should have been so simple. Thanks all for answering my question!