How do I create emit macros that don't add a stacktrace?
import macros
macro emitWithoutStacktrace(): stmt =
result = newStmtList()
result.add quote do:
{.push stacktrace: off.}
result.add(newNimNode(nnkPragma).add(newNimNode(nnkExprColonExpr).add(newIdentNode("emit"), newStrLitNode("foobar"))))
result.add quote do:
{.pop stacktrace: off.}
proc main =
emitWithoutStacktrace()
main()
Produces
N_NIMCALL(void, main_104042)(void) {
nimfr("main", "test.nim")
nimln(7, "test.nim");
foobar
popFrame();
}
Where I would prefer
N_NIMCALL(void, main_104042)(void) {
foobar
popFrame();
}
Without any additional compiler arguments like --stacktrace: off.
Note that the same happens with this code:
proc main =
{.push stacktrace: off.}
echo "hi"
{.pop.}
main()
Instead you need to surround the entire proc with the push/pop:
{.push stacktrace: off.}
proc main =
echo "hi"
{.pop.}
main()
I just noticed this yesterday when trying to do {.push rangechecks: on.}.
Related
Apologies for a length post. I have been trying to beat my head around reading about mock, MagicMock, and all the time getting confused. Hence, decided to write this post.
I know several questions, and pages have been written on this. But, still not able to wrap my head around this.
My Setup:
All the test code, and the 2 module files come under one "folder" mymodule
my_module_1.py file contains
class MyOuterClass(object):
MyInnerClass(object):
attribute1: str
attribute2: str
attribute3: str
def get(self) -> MyInnerClass:
'''
pseudocode
1. a call to AWS's service is made
2. the output from call in step 1 is used to set attributes of this InnerClass
3. return innerclass instance
'''
I use the OuterClass in another file(my_module_2.py), to set some values and return a string as follows:
class MyModule2():
def get_foo(self, some_boolean_predicate):
if some_boolean_predicate:
temp = my_module_1.OuterClass().get()
statement = f'''
WITH (
BAR (
FIELD_1 = '{temp.attribute1}',
FIELD_2 = '{temp.attribute2}',
FIELD_3 = '{temp.attribute3}'
)
)
'''
else:
statement = ''
return statement
I want to write the unit tests for the file my_module_2.py file, and test the function get_foo
How I am writing the tests(or planning on)
a test file by name test_my_module2.py
I started with creating a pytest.fixture for the MyOuterClass's get function as follows since I will be reusing this piece of info again in my other tests
#pytest.fixture
def mock_get(mocker: MockerFixture) -> MagicMock:
return mocker.patch.object(MyOuterClass, 'get')
Finally,
Then I proceeded to use this fixture in my test as follows:
from unittest import mock
from unittest.mock import MagicMock, Mock, patch, PropertyMock
import pytest
from pytest_mock import MockerFixture
from my_module.my_module_1 import myOuterClass
def test_should_get_from_inner_class(self, mock_get):
# mock call to get are made
output = mock_get.get
#update the values for the InnerClass's attributes here
output.attribute1.side_effect = 'attr1'
output.attribute2.side_effect = 'attr2'
output.attribute3.side_effect = 'attr3'
mock_output_str = '''
WITH (
BAR (
FIELD_1 = 'attr1',
FIELD_2 = 'attr2',
FIELD_3 = 'attr3'
)
)
'''
module2Obj = MyModule2()
response = module2Obj.get_foo(some_boolean_predicate=True)
# the following assertion passes
assert mock_get.get.called_once()
# I would like match `response to that with mock_output_str instance above
assert response == mock_output_str
But, the assertion as you might have guessed failed, and I know I am comparing completely different types, since I see
errors such as
FAILED [100%]
WITH (
BAR (
FIELD1 = '<MagicMock name='get().attr1' id='4937943120'>',
FIELD3 = '<MagicMock name='get().attr2' id='4937962976'>',
FIELD3 = '<MagicMock name='get().attr3' id='4937982928'>'
)
)
Thank you for being patient with me till here, i know its a really lengthy post, but stuck on this for a few days, ended up creating a post here.
How do i get to validate the mock's value with the mock_output_str?
yess! the hint was in the #gold_cy's answer. I was only calling my mock and never setting its values
this is what my test case ended up looking
mock_obj = OuterClass.InnerClass()
mock_obj.attribute1='some-1'
mock_obj.attribute2='some-2'
mock_obj.attribute3='some-3'
mock_get.return_value = mock_obj
once my mock was setup properly, then the validation became easy! Thank you!
Please refer to this trivial block of code shown below. My goal is to use defaultdict to come up with a relatively simple dictionary, and further print the results out as a yaml file.
When I manually define the dictionary, it seems to work just fine and the YAML is displayed exactly the way I want it, but when I use defaultdict to come up with a dictionary, I get an error message and unfortunately I am not able to decipher that.
When I print the dictionary as a JSON, it prints the exact same output. What I am missing?
import sys,ruamel.yaml
import json
from collections import defaultdict
def dict_maker():
return defaultdict(dict_maker)
S = ruamel.yaml.scalarstring.DoubleQuotedScalarString
app = "someapp"
d = {'beats':{'name':S(app), 'udp_address':S('239.1.1.1:10101')}}
foo = dict_maker()
foo["beats"]["name"] = S(app)
foo["beats"]["udp_address"] = S("239.1.1.1:10101")
print "Regular dictionary"
print json.dumps(d, indent=4)
print "defaultdict dictionary"
print json.dumps(foo, indent=4)
print "dictionary as a yaml\n"
ruamel.yaml.dump(d, sys.stdout, Dumper=ruamel.yaml.RoundTripDumper)
print "defaultdict dictionary as a yaml\n"
ruamel.yaml.dump(foo, sys.stdout, Dumper=ruamel.yaml.RoundTripDumper)
Error Message
raise RepresenterError("cannot represent an object: %s" % data)
ruamel.yaml.representer.RepresenterError: cannot represent an object: defaultdict(<function dict_maker at 0x7f1253725a28>, {'beats': defaultdict(<function dict_maker at 0x7f1253725a28>, {'name': u'someapp', 'udp_address': u'239.1.1.1:10101'})})
You seem to be using the word "dictionary" when refering to a Python dict. There is however no such thing as a "defaultdict based dictionary", that would imply that foo after
foo = dict_maker()
would be a dict, and of course it is not: foo is a defaultdict which is dict based (i.e. exactly the other way around from what you write).
That JSON dumps this, is not surprising, as it cannot do more than stupidly dump the key-value pairs as if it were a dict. But when you try to load that JSON back, you see how useless this is as, you cannot continue working with it (at least not in the way expected):
import sys
import json
from collections import defaultdict
import io
def dict_maker():
return defaultdict(dict_maker)
app = "someapp"
foo = dict_maker()
foo["beats"]["name"] = app
foo["beats"]["udp_address"] = "239.1.1.1:10101"
io = io.StringIO()
json.dump(foo, io, indent=4)
io.seek(0)
bar = json.load(io)
bar['otherapp']['name'] = 'some_alt_app'
print(bar['beats']['udp_address'])
The above throws: KeyError: 'otherapp'. And that is because JSON doesn't keep all the information needed.
However, if you use the unsafe YAML dumper, then ruamel.yaml can dump and load this fine:
import sys
from ruamel.yaml import YAML
from collections import defaultdict
import io
def dict_maker():
return defaultdict(dict_maker)
app = "someapp"
yaml = YAML(typ='unsafe')
foo = dict_maker()
foo["beats"]["name"] = app
foo["beats"]["udp_address"] = "239.1.1.1:10101"
io = io.StringIO()
yaml.dump(foo, io)
io.seek(0)
print(io.getvalue())
bar = yaml.load(io)
bar['otherapp']['name'] = 'some_alt_app'
print(bar['beats']['udp_address'])
this doesn't throw an error, as bar is again a defaultdict with dict_maker as the function it defaults to. The above prints
239.1.1.1:10101
as you would expect.
That the RoundTripDumper/Loader doesn't support this out-of-the-box, is because it is based on the SafeDumper/Loader, which cannot dump/load arbitrary Python instances like defaultdict and its dict_maker function reference. Enabling that would make the loading unsafe.
So if you need to use the RoundTripDumper you should add a representer for defaultdict or a subclass thereof (and possible one for dict_maker as well). To be able to load that, you need constructor(s) as well. How to do that is described in the documentation (Dumping Python classes)
I am trying to write a macro for debug print in the Nim language.
Currently this macro adds filename andline to the output by instantiationInfo().
import macros
macro debugPrint(msg: untyped): typed =
result = quote do:
let pos = instantiationInfo()
echo pos.filename, ":", pos.line, ": ", `msg`
proc hello() =
debugPrint "foo bar"
hello()
currently output:
debug_print.nim:9: foo bar
I would like to add the name of the procedure (or iterator) of the place where the macro was called.
desired output:
debug_print.nim:9(proc hello): foo bar
How can I get the name of procedure (or iterator) in Nim, like __func__ in C?
At runtime you can do getFrame().procname, but it only works with stacktrace enabled (not in release builds).
At compile-time surprisingly I can't find a way to do it. There is callsite() in macros module, but it doesn't go far enough. It sounds like something that might fit into the macros.LineInfo object.
A hacky solution would be to also use __func__ and parse that back into the Nim proc name:
template procName: string =
var name: cstring
{.emit: "`name` = __func__;".}
($name).rsplit('_', 1)[0]
building on answer from #def- but making it more robust to handle edge cases of functions containing underscores, and hashes containing trailing _N or not
also using more unique names as otherwise macro would fail if proc defines a variable name
import strutils
proc procNameAux*(name:cstring): string =
let temp=($name).rsplit('_', 2)
#CHECKME: IMPROVE; the magic '4' chosen to be enough for most cases
# EG: bar_baz_9c8JPzPvtM9azO6OB23bjc3Q_3
if temp.len>=3 and temp[2].len < 4:
($name).rsplit('_', 2)[0]
else:
# EG: foo_9c8JPzPvtM9azO6OB23bjc3Q
($name).rsplit('_', 1)[0]
template procName*: string =
var name2: cstring
{.emit: "`name2` = __func__;".}
procNameAux(name2)
proc foo_bar()=
echo procName # prints foo_bar
foo_bar()
NOTE: this still has some issues that trigger in complex edge cases, see https://github.com/nim-lang/Nim/issues/8212
I have a few .csv files I want to use for the same data in Gatling. Each of these files has a certain number of ID's that I want to be accessed fairly equally. I don't want to put them all in the same file because the .csv files are generated from SQL queries and, while I may have a lot of IDs in one file, I only have a few in another. What's important to me is that I have a random sample from each of my files and a way to specify the distribution.
I found an example of how to do this but I'm having trouble applying it in my case. Here is the code I have so far. I try to both 1) print out the value from the feeder in the session and 2) try to use the value from the feeder in a get request. Both attempts fail with various errors which I detail below:
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
import util.Random
class FeederTest extends Simulation {
//headers...
val userCreds = csv("user_creds.csv")
val sample1 = csv("sample1.csv")
val sample2 = csv("sample2.csv")
def randFeed(): String = {
val foo = Random.nextInt(2)
var retval = ""
if (foo == 0) retval = "file1"
if (foo == 1) retval = "file2"
return retval
}
val scn = scenario("feeder test")
.repeat(1) {
feed(userCreds)
.doSwitch(randFeed)(
"file1" -> feed(sample1),
"file2" -> feed(sample2)
)
.exec(http("request - login")
.post("<URL>")
.headers(headers_login)
.formParam("email", "${username}")
.formParam("password", "<not telling>"))
.exec(session => {
println(session)
println(session("first").as[String])
session})
.exec(http("goto_url")
.get("<my url>/${first}"))
}
setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}
This is the error I get when I attempt to printout the feeder value in the session (as in the above code using session(<value>).as[String]):
[ERROR] [03/13/2015 10:22:38.221] [GatlingSystem-akka.actor.default-dispatcher-8] [akka://GatlingSystem/user/sessionHook-2] key not found: first
java.util.NoSuchElementException: key not found: first
at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59)
at scala.collection.MapLike$class.apply(MapLike.scala:141)
at scala.collection.AbstractMap.apply(Map.scala:59)
at io.gatling.core.session.SessionAttribute.as(Session.scala:40)
at FeederTest$$anonfun$2.apply(feeder_test.scala:81)
at FeederTest$$anonfun$2.apply(feeder_test.scala:79)
at io.gatling.core.action.SessionHook.executeOrFail(SessionHook.scala:35)
at io.gatling.core.action.Failable$class.execute(Actions.scala:71)
at io.gatling.core.action.SessionHook.execute(SessionHook.scala:28)
at io.gatling.core.action.Action$$anonfun$receive$1.applyOrElse(Actions.scala:29)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at io.gatling.core.akka.BaseActor.aroundReceive(BaseActor.scala:22)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
at akka.dispatch.Mailbox.run(Mailbox.scala:221)
at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
I have also tried just using the EL expression ${first} in the session. All this does is print out the string ${first}. Similarly in the last .get line I get an error saying "No attribute named 'first' is defined.
The CSV files I'm currently using just have two columns of first and last in them like so:
sample1.csv:
first, last
george, bush
bill, clinton
barak, obama
sample2.csv:
first, last
super, man
aqua, man
bat, man
I'm using Gatling 2.1.4.
doSwitch takes an Expression[Any], which is a type alias for Session => Validation[Any]. Gatling has an implicit conversion that let you pass a static value instead, see documentation.
Which is exactly what you do. Even if randFeed is a def, it still doesn't return a function, but a String.
As you want randFeed to be called every time a virtual user pass through this step, you have to wrap the randFeed inside a function, even if you don't use the Session input parameter.
doSwitch(_ => randFeed)
Then, your randFeed is both ugly (no offense) and inefficient (Random is synchronized):
import scala.concurrent.forkjoin.ThreadLocalRandom
def randFeed(): String =
ThreadLocalRandom.current().nextInt(2) match {
case 0 => "file1"
case 1 => "file2"
}
I've never used feeders, but from the documentation, I see two possible problems:
randFeed returns either "foo" or "bar" but you're mapping with "file1" and "file2", so is anything being loaded? (The documentation says "If no switch is selected, the switch is bypassed.")
The examples for feeders I see don't show accessing fed data with session(varname) but rather "${varname}".
I am embedding python in my C++ application, using boost python.
I would like to be able to call a boost python function object, and associate a global name space with that function call. Specifically, the simplified relevant code is:
bp::object main = bp::import("__main__");
bp::object main_namespace = main.attr("__dict__");
//Put the function name runPyProg in the main_namespace
bp::object PyProg = exec(
"import cStringIO\n"
"import sys\n"
"sys.stderr = cStringIO.StringIO()\n"
"def runPyProg(exp):\n"
" print exp\n"
" exec(exp)\n"
" return\n"
"\n",main_namespace);
//Now call the python function runPyProg with an argument
bp::object py_fn = main.attr("runPyProg");
py_fn(expStr)
I know that when I use the boost python exec() function, I can send in the global namespace, as shown above. My question is how do I associate main_namespace with the python function when I call py_fn? My final goal is that local variables from runPyProg will be placed in the main_namespace.
Thank you.
If I understand the question correctly, then it should be as simple as specifying the context in which exec will execute. A function or method can access the namespace in which it is defined via globals(). Thus, calling globals() from within runPyProg() will return the Python equivalent of main_namespace. Additionally, exec takes two optional arguments:
The first argument specifies the dictionary that will be used for globals(). If the second argument is omitted, then it is also used for locals().
The second argument specifies the dictionary that will be used for locals(). Variable changes occurring within exec are applied to locals().
Therefore, change:
exec exp
to
exec exp in globals()
and it should provide the desired behavior, where exp can interact with global variables in main_namespace.
Here is a basic example:
#include <boost/python.hpp>
int main()
{
Py_Initialize();
namespace python = boost::python;
python::object main = python::import("__main__");
python::object main_namespace = main.attr("__dict__");
//Put the function name runPyProg in the main_namespace
python::exec(
"def runPyProg(exp):\n"
" print exp\n"
" exec exp in globals()\n"
" return\n"
"\n", main_namespace);
// Now call the python function runPyProg with an argument
python::object runPyProg = main.attr("runPyProg");
// Set x in python and access from C++.
runPyProg("x = 42");
std::cout << python::extract<int>(main.attr("x")) << std::endl;
// Set y from C++ and access within python.
main.attr("y") = 100;
runPyProg("print y");
// Access and modify x in python, then access from C++.
runPyProg("x += y");
std::cout << python::extract<int>(main.attr("x")) << std::endl;
}
Commented output:
x = 42 // set from python
42 // print from C++
// y set to 100 from C++
print y // print y from python
100 //
x += y // access and modify from python
142 // print x from C++