I defined a macro like this:
defmodule Test do
defmacro __using__ do
quote do
require Test
import Test
end
end
defmacro my_macro do
quote do
Gernserver.call() # generic server call from other module
end
end
end
I have this code inside a lib which I am building. When testing this lib in a different application I get a compile error on the file which has the use Test.
Error: exited in: GenServer.call ... exit no process (on the line of the my_macro call, more specifically on the genserver call). As if the genserver was not running, and it is not because it is compile time.
Adding a simple Application.ensure_all_started(:lib_name) to the using macro seems to fix the problem. But then again, at compile time the code shouldn't be running, am I right ? Or is here something I am not seeing ? Maybe I can not use genserver calls on a macro ?
Thanks.
You are calling every in your TestScheduler during compilation stage.
One should either put all the schedule initialization code inside a function, that would be called explicitly during run stage, or simply
introduce a new GenServer, add it to supervised list (to make Elixir to run it automatically on app start,) and perform this every initialization in it’s start_link (or any other function that would be called later explicitly):
defmodule Test.Scheduler do
use GenServer
use Cronex.Scheduler
def start_link ... do
every :minute do
IO.puts "every minute job"
end
every :minute do
IO.puts "every minute job 2"
end
end
end
Another way round (if you want to keep the file above as clean as possible,) would be to collect data in every macro, and to spawn all the calls during run stage.
To become more familiar with stages, put the following code into /tmp/test.ex:
defmodule A do
IO.puts "Here “every” macro was called: compilation"
def a do
IO.puts "Is not called during compilation stage"
end
end
IO.puts "=== compilation finished ==="
A.a
And issue elixir /tmp/test.ex:
Here “every” macro was called: compilation
=== compilation finished ===
Is not called during compilation stage
Related
I am currently experimenting with defining and using my own slightly adjusted receive macro, based on its default implementation. As a classic example, let’s say I want to log every time a Process starts receiving a message from the mailbox. Could I define my own version of the receive macro that does the logging, then calls/uses the default receive macro and import this custom receive into my existing code?
Below is a not-working example to better illustrate what I am trying to achieve:
defmodule MyWeirdReceive do
def receive(args) do
IO.puts "I just started receiving a message from the mailbox"
Kernel.SpecialForms.receive(args)
end
end
defmodule Stack do
import Kernel, except: [receive: 1]
import MyWeirdReceive
def loop(state, ctr) do
receive do
{_from, :push, value} ->
loop([value | state], ctr + 1)
{from, :pop} ->
[h | t] = state
send(from, {:reply, h})
loop(t, ctr)
end
loop(state, ctr)
end
end
It's hard to override Kernel.SpecialForms macros because you can't import them like you could macros from anywhere else (you get a CompileError). I think your best bet is to name your macro something other than "receive".
Other problems you may be running into:
You are defining your receive as a function instead of a macro. Elixir will try evaluating your do/after block as a literal keyword list, and that won't work well.
receive/1 comes from Kernel.SpecialForms, not Kernel, so your import/:except won't work.
I'm currently working on some elixir macro fun stuff. I have a module like that:
defmodule MapUtils do
#handlers "I don't want you"
defmacro __using__(_) do
quote do
import MapUtils
Module.register_attribute __MODULE__, :handlers, accumulate: true
#handlers "I want you"
end
end
defmacro test do
IO.inspect #handlers
quote do
IO.inspect(#handlers)
end
end
end
defmodule Test do
use MapUtils
def testowa do
MapUtils.test
end
end
Test.testowa
Which yields in such result:
"I don't want you"
["I want you"]
I want to access #handlers from callers module outside of the quote block to generate some code based on what it is. As far as my understanding goes first inspect is being executed and second is being transformed to AST and executed in differnt context.
Is there a way to get access to that #handlers from caller module in compile time?
If I understand your question correctly, you want this:
Module.register_attribute __MODULE__, :handlers,
accumulate: true,
persist: true
before the change:
iex(6)> Test.module_info(:attributes)
[vsn: [95213125195364189087674570096731471099]]
after the change:
iex(13)> Test.module_info(:attributes)
[vsn: [95213125195364189087674570096731471099], handlers: ["I want you"]]
I stumbled upon this.
I realized I can call it with __CALLER__ like that:
Module.get_attribute(__CALLER__.module, :handlers)
And in fact it returned value it suppose to return.
I have some py.test tests that have multiple dependent and parameterized fixtures and I want to measure the time taken by each fixture. However, in the logs with --durations it only shows time for setup for actual tests, but doesn't give me a breakdown of how long each individual fixture took.
Here is a concrete example of how to do this:
import logging
import time
import pytest
logger = logging.getLogger(__name__)
#pytest.hookimpl(hookwrapper=True)
def pytest_fixture_setup(fixturedef, request):
start = time.time()
yield
end = time.time()
logger.info(
'pytest_fixture_setup'
f', request={request}'
f', time={end - start}'
)
With output similar to:
2018-10-29 20:43:18,783 - INFO pytest_fixture_setup, request=<SubRequest 'some_data_source' for <Function 'test_ruleset_customer_to_campaign'>>, time=3.4723987579345703
The magic is hookwrapper:
pytest plugins can implement hook wrappers which wrap the execution of other hook implementations. A hook wrapper is a generator function which yields exactly once. When pytest invokes hooks it first executes hook wrappers and passes the same arguments as to the regular hooks.
One fairly important gotcha that I ran into is that the conftest.py has to be in your project's root folder in order to pick up the pytest_fixture_setup hook.
There isn't anything builtin for that, but you can easily implement yourself by using the new pytest_fixture_setup hook in a conftest.py file.
I'm trying to write the macro that generates a module:
defmodule GenModules do
defmacro module(name, do: body) do
quote do
defmodule unquote(String.to_atom(name)) do
unquote(body)
end
end
end
end
What I'm missing is to how to inject the 'import' statement that would refer the module the macro will be called from?
I.e., the following code:
defmodule Test do
import GenModules
module "NewModule" do
def test_call do
hello_world
end
end
def hello_world do
IO.puts "Hello"
end
end
won't compile because hello_world function is not visible from the generated NewModule.
So I need to generate
import Test
before the body of the module, somehow getting the name of the module the macro was called from. How do I do this?
Thank you,
Boris
To get the module your macro was called from you can use the special form __CALLER__. It contains a bunch of information, but you can extract the calling module like so:
__CALLER__.context_modules |> hd
But more generally I don't think what you want is possible - you cannot import a module before you finish defining it. For example:
defmodule A do
defmodule B do
import A
end
end
results in an error:
** (CompileError) test.exs:3: module A is not loaded but was defined.
This happens because you are trying to use a module in the same context it is defined.
Try defining the module outside the context that requires it.
I am building some tests for python3 code using py.test. The code accesses a Postgresql Database using aiopg (Asyncio based interface to postgres).
My main expectations:
Every test case should have access to a new asyncio event loop.
A test that runs too long will stop with a timeout exception.
Every test case should have access to a database connection.
I don't want to repeat myself when writing the test cases.
Using py.test fixtures I can get pretty close to what I want, but I still have to repeat myself a bit in every asynchronous test case.
This is how my code looks like:
#pytest.fixture(scope='function')
def tloop(request):
# This fixture is responsible for getting a new event loop
# for every test, and close it when the test ends.
...
def run_timeout(cor,loop,timeout=ASYNC_TEST_TIMEOUT):
"""
Run a given coroutine with timeout.
"""
task_with_timeout = asyncio.wait_for(cor,timeout)
try:
loop.run_until_complete(task_with_timeout)
except futures.TimeoutError:
# Timeout:
raise ExceptAsyncTestTimeout()
#pytest.fixture(scope='module')
def clean_test_db(request):
# Empty the test database.
...
#pytest.fixture(scope='function')
def udb(request,clean_test_db,tloop):
# Obtain a connection to the database using aiopg
# (That's why we need tloop here).
...
# An example for a test:
def test_insert_user(tloop,udb):
#asyncio.coroutine
def insert_user():
# Do user insertion here ...
yield from udb.insert_new_user(...
...
run_timeout(insert_user(),tloop)
I can live with the solution that I have so far, but it can get cumbersome to define an inner coroutine and add the run_timeout line for every asynchronous test that I write.
I want my tests to look somewhat like this:
#some_magic_decorator
def test_insert_user(udb):
# Do user insertion here ...
yield from udb.insert_new_user(...
...
I attempted to create such a decorator in some elegant way, but failed. More generally, if my test looks like:
#some_magic_decorator
def my_test(arg1,arg2,...,arg_n):
...
Then the produced function (After the decorator is applied) should be:
def my_test_wrapper(tloop,arg1,arg2,...,arg_n):
run_timeout(my_test(),tloop)
Note that some of my tests use other fixtures (besides udb for example), and those fixtures must show up as arguments to the produced function, or else py.test will not invoke them.
I tried using both wrapt and decorator python modules to create such a magic decorator, however it seems like both of those modules help me create a function with a signature identical to my_test, which is not a good solution in this case.
This can probably solved using eval or a similar hack, but I was wondering if there is something elegant that I'm missing here.
I’m currently trying to solve a similar problem. Here’s what I’ve come up with so far. It seems to work but needs some clean-up:
# tests/test_foo.py
import asyncio
#asyncio.coroutine
def test_coro(loop):
yield from asyncio.sleep(0.1)
assert 0
# tests/conftest.py
import asyncio
#pytest.yield_fixture
def loop():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
yield loop
loop.close()
def pytest_pycollect_makeitem(collector, name, obj):
"""Collect asyncio coroutines as normal functions, not as generators."""
if asyncio.iscoroutinefunction(obj):
return list(collector._genfunctions(name, obj))
def pytest_pyfunc_call(pyfuncitem):
"""If ``pyfuncitem.obj`` is an asyncio coroutinefunction, execute it via
the event loop instead of calling it directly."""
testfunction = pyfuncitem.obj
if not asyncio.iscoroutinefunction(testfunction):
return
# Copied from _pytest/python.py:pytest_pyfunc_call()
funcargs = pyfuncitem.funcargs
testargs = {}
for arg in pyfuncitem._fixtureinfo.argnames:
testargs[arg] = funcargs[arg]
coro = testfunction(**testargs) # Will no execute the test yet!
# Run the coro in the event loop
loop = testargs.get('loop', asyncio.get_event_loop())
loop.run_until_complete(coro)
return True # TODO: What to return here?
So I basically let pytest collect asyncio coroutines like normal functions. I also intercept text exectuion for functions. If the to-be-tested function is a coroutine, I execute it in the event loop. It works with or without a fixture creating a new event loop instance per test.
Edit: According to Ronny Pfannschmidt, something like this will be added to pytest after the 2.7 release. :-)
Every test case should have access to a new asyncio event loop.
The test suite of asyncio uses unittest.TestCase. It uses setUp() method to create a new event loop. addCleanup(loop.close) is close automatically the event loop, even on error.
Sorry, I don't know how to write this with py.test if you don't want to use TestCase. But if I remember correctly, py.test supports unittest.TestCase.
A test that runs too long will stop with a timeout exception.
You can use loop.call_later() with a function which raises a BaseException as a watch dog.