I have some code that requires an Environment Variable to run correctly. But when I run my unit tests, it bombs out once it reaches that point unless I specifically export the variable in the terminal. I am using Scala and sbt. My code does something like this:
class something() {
val envVar = sys.env("ENVIRONMENT_VARIABLE")
println(envVar)
}
How can I mock this in my unit tests so that whenever sys.env("ENVIRONMENT_VARIABLE") is called, it returns a string or something like that?
If you can't wrap existing code, you can change UnmodifiableMap System.getenv() for tests.
def setEnv(key: String, value: String) = {
val field = System.getenv().getClass.getDeclaredField("m")
field.setAccessible(true)
val map = field.get(System.getenv()).asInstanceOf[java.util.Map[java.lang.String, java.lang.String]]
map.put(key, value)
}
setEnv("ENVIRONMENT_VARIABLE", "TEST_VALUE1")
If you need to test console output, you may use separate PrintStream.
You can also implement your own PrintStream.
val baos = new java.io.ByteArrayOutputStream
val ps = new java.io.PrintStream(baos)
Console.withOut(ps)(
// your test code
println(sys.env("ENVIRONMENT_VARIABLE"))
)
// Get output and verify
val output: String = baos.toString(StandardCharsets.UTF_8.toString)
println("Test Output: [%s]".format(output))
assert(output.contains("TEST_VALUE1"))
Ideally, environment access should be rewritten to retrieve the data in a safe manner. Either with a default value ...
scala> scala.util.Properties.envOrElse("SESSION", "unknown")
res70: String = Lubuntu
scala> scala.util.Properties.envOrElse("SECTION", "unknown")
res71: String = unknown
... or as an option ...
scala> scala.util.Properties.envOrNone("SESSION")
res72: Option[String] = Some(Lubuntu)
scala> scala.util.Properties.envOrNone("SECTION")
res73: Option[String] = None
... or both [see envOrSome()].
I don't know of any way to make it look like any/all random env vars are set without actually setting them before running your tests.
You shouldn't test it in unit-test.
Just extract it out
class F(val param: String) {
...
}
In your prod code you do
new Foo(sys.env("ENVIRONMENT_VARIABLE"))
I would encapsulate the configuration in a contraption which does not expose the implementation, maybe a class ConfigValue
I would put the implementation in a class ConfigValueInEnvVar extends ConfigValue
This allows me to test the code that relies on the ConfigValue without having to set or clear environment variables.
It also allows me to test the base implementation of storing a value in an environment variable as a separate feature.
It also allows me to store the configuration in a database, a file or anything else, without changing my business logic.
I select implementation in the application layer.
I put the environment variable logic in a supporting domain.
I put the business logic and the traits/interfaces in the core domain.
Related
I'm using other teams api(let's name it otherTeamAPI) to call data, so in my function, my code looks like this:
def getData(host:String, port:Int, date: String): Map[String, String] = {
val data = new otherTeamAPI(host,port)
val latestData = data.getLatestData(date)
}
Could someone teach me how to use Mockito to do the same thing to get data in unit test? I'm not sure whether to use something like below to new an api:
val otherTeamAPI = Mock[otherTeamAPI]
otherTeamAPI.getLatestData(date)
How to get data everytime i trigger my function getData? Do i need to do somthing new a mock otherTeamAPI?
Your code, written as is, is not testable. You have to be able to pass your method an instance of the OtherTeamAPI so that your production code uses a real instance but test code can use a fake one (a "mock").
How you pass this instance depends on the structure of the rest of your code: either as a parameter of this method getData or as an attribute of the class that contains it.
The first one would look like this:
def getData(api: OtherTeamApi, date: String): Map[String, String] = {
val latestData = api.getLatestData(date)
// ...
}
And then in your test, you can do something like:
val fakeApi = mock[OtherTeamAPI]
when(fakeApi.getLatestData(anyString())).the return(...)
val result = getData(fakeApi, ...)
// Then assert on result
This is a high level answer. You'll need to learn more about Mockito to find out what you want to do.
I'm using pytest for testing with mixer library for generating model data. So, now I'm trying to setup my tests once before they run. I grouped them into TestClasses, set to my fixtures 'class' scope, but this doesn't work for me.
#pytest.mark.django_db
class TestCreateTagModel:
#classmethod
#pytest.fixture(autouse=True, scope='class')
def _set_up(cls, create_model_instance, tag_model, create_fake_instance):
cls.model = tag_model
cls.tag = create_model_instance(cls.model)
cls.fake_instance = create_fake_instance(cls.model)
print('setup')
def test_create_tag(self, tag_model, create_model_instance, check_instance_exist):
tag = create_model_instance(tag_model)
assert check_instance_exist(tag_model, tag.id)
conftest.py
pytest.fixture(scope='class')
#pytest.mark.django_db(transaction=True)
def create_model_instance():
instance = None
def wrapper(model, **fields):
nonlocal instance
if not fields:
instance = mixer.blend(model)
else:
instance = mixer.blend(model, **fields)
return instance
yield wrapper
if instance:
instance.delete()
#pytest.fixture(scope='class')
#pytest.mark.django_db(transaction=True)
def create_fake_instance(create_related_fields):
"""
Function for creating fake instance of model(fake means that this instance doesn't exists in DB)
Args:
related (bool, optional): Flag which indicates create related objects or not. Defaults to False.
"""
instance = None
def wrapper(model, related=False, **fields):
with mixer.ctx(commit=False):
instance = mixer.blend(model, **fields)
if related:
create_related_fields(instance, **fields)
return instance
yield wrapper
if instance:
instance.delete()
#pytest.fixture(scope='class')
#pytest.mark.django_db(transaction=True)
def create_related_fields():
django_rel_types = ['ForeignKey']
def wrapper(instance, **fields):
for f in instance._meta.get_fields():
if type(f).__name__ in django_rel_types:
rel_instance = mixer.blend(f.related_model)
setattr(instance, f.name, rel_instance)
return wrapper
But I'm catching exception in mixer gen_value method: Database access not allowed, use django_db mark(that I'm already use). Do you have any ideas how this can be implemented?
You can set things up once before a run by returning the results of the setup, rather than modifying the testing class directly. From my own attempts, it seems any changes to the class made within class-scope fixtures are lost when individual tests are run. So here's how you should be able to do this. Replace your _setup fixture with these:
#pytest.fixture(scope='class')
def model_instance(self, tag_model, create_model_instance):
return create_model_instance(tag_model)
#pytest.fixture(scope='class')
def fake_instance(self, tag_model, create_fake_instance):
return create_fake_instance(tag_model)
And then these can be accessed through:
def test_something(self, model_instance, fake_instance):
# Check that model_instance and fake_instance are as expected
I'm not familiar with Django myself though, so there might be something else with it going on. This should at least help you solve one half of the problem, if not the other.
I have service that returns a ZIO[Has[MyCustomHeader]], and I'm having trouble testing it.
Other services in our organisation are tested by converting ZIO to Twitterfuture using runtime.unsafeRunToFuture (where runtime is a Runtime[ZEnv] ) and then awaiting the future, thus running the tests in blocking mode.
However this service has a Has[] requirement and runtime.unsafeRunToFuture doesnt handle those. So far my approach has been to try to convert my ZIO[Has[MyCustomHeader]] to a ZIO[ZEnv], but I've yet to succeed at this.
from what I gather I need to provide a ZLayer via ZIO.provideSomeLayer() but I'm simply too stupid to understand how to construct a ZLayer properly?
Am I even on the right path here? and if so, How do I construct a ZLayer with a static value for MyCustomHeader to use in my tests?
This is how far along I am at trying to add a header for testing purposes: it doesn't work, but might illustrate what I'm trying to achieve..maybe... I'm pretty confused myself:
object effectAwait {
implicit class ZioEffect[A](private val value: ZIO[Has[EnvironmentHeader], RequestFailure, A]) extends AnyVal {
final def await(implicit runtime: Runtime[ZEnv] = Runtime.default): A = {
val zmanaged = ZManaged.fromEffect(value).provide(Has(EnvironmentHeader("test")))
val layered = value.provideSomeLayer(zmanaged.toLayer)
val sf = runtime.unsafeRunToFuture(layered)
Await.result(sf, 10.seconds)
}
}
}
this however gives me the error:
could not find implicit value for izumi.reflect.Tag[A]. Did you
forget to put on a Tag, TagK or TagKK context bound on one of the
parameters in A? e.g. def x[T: Tag, F[_]: TagK] = ...
<trace>:
deriving Tag for A, dealiased: A:
could not find implicit value for Tag[A]: A is a type parameter without an implicit Tag!
val layered = value.provideSomeLayer(zmanaged.toLayer)
I think you can just use ZIO.provideLayer (instead of provideSomeLayer) here :)
Also, there's a runtime.unsafeRun that will wait for the result as well, so you don't necessarily have to convert it to a Future. Also, also, instead of relying on an implicit runtime, there's always zio.Runtime.default that you can use anywhere (it's a Runtime[ZEnv] so it should work just as well, unless you've otherwise customized the runtime's behavior)
After Reading the official flink testing documentation (https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/testing.html)
I was able to develop tests for a ProcessFunction, using a Test Harness, something like this:
pendingPartitionBuilder = new PendingPartitionBuilder(":::some_name", "")
testHarness =
new OneInputStreamOperatorTestHarness[StaticAdequacyTilePublishedData, PendingPartition](
new ProcessOperator[StaticAdequacyTilePublishedData, PendingPartition](pendingPartitionBuilder)
)
testHarness.open()
now, I’m trying to do the same for a ProcessAllWindowFunction, that looks like this:
class MapVersionValidationDistributor(batchSize: Int) extends
ProcessAllWindowFunction[MapVersionValidation, Seq[StaticAdequacyTilePublishedData],TimeWindow] {
lazy val state: ValueState[Long] = getRuntimeContext .getState(new ValueStateDescriptor[Long]("latestMapVersion", classOf[Long]))
(...)
First I realized I can’t use TestHarness for ProcessAllWindowFunction, because it doesn’t have a processElement method. In this case, what unit test strategy should I follow?
EDIT: At the moment my test code looks like this:
val collector = mock[Collector[Seq[StaticAdequacyTilePublishedData]]]
val mvv = new MapVersionValidationDistributor(1)
val input3 = Iterable(new MapVersionValidation("123",Seq(TileValidation(1,true,Seq(1,3,4)))))
val ctx = mock[mvv.Context]
val streamContext = mock[RuntimeContext]
mvv.setRuntimeContext(streamContext)
mvv.open(mock[Configuration])
mvv.process(ctx,input3,collector)
and I'm getting this error:
Unexpected call: <mock-3> RuntimeContext.getState[T](ValueStateDescriptor{name=latestMapVersion, defaultValue=null, serializer=null}) Expected: inAnyOrder { }
You don't really need test harness to unit test the process method of the ProcessAllWindowFunction. The process function takes 3 arguments: Context, Iterable[IN], Collector[OUT]. You can use some library depending on the language used to mock the Context. You can also easily implement or mock the Collector depending on your prerefences here. And the Iterable[IN] is just an Iterable containing the elements of Your window, that would be passed to the function after the window is triggered.
I'm unsure if this is possible but I was wondering if there is a way to get a variable from an outer scope without passing it as an argument.
I've played around with global() and inspect but i'm having issues trying to get the attribute.
Here's what i'm trying to do:
class Example:
#staticmethod
def query(**kwargs):
print(f.read())
with open(__file__) as f:
Example.query(foo='bar')
So after a while of back and forth I have finally found a solution to my issue.
As Matthias suggested I use global to find the object, i decided to use inspect to add it myself like so:
Assigning
def __enter__(self):
inspect.stack()[1][0].f_globals["_ExampleName"] = self
Retrieving (Fixed)
#staticmethod
def _find_example():
stack = inspect.stack()
for stack_index in range(2, len(stack)):
stack_globals = inspect.stack()[stack_index][0].f_globals
if "_ExampleName" in stack_globals.keys():
return stack_globals["_ExampleName"]
This is a bit 'dodgy' as inspect is not ment to be used in a production environment, However works and solves my issue
Here is a MCVE of what you're trying to do.
class Example:
#staticmethod
def query(**kwargs):
print(f.read())
with open(__file__) as f:
Example.query(foo='bar')
Works as expected.
What you should do is have the Client class set a class variable to the current session.
class Client:
last_session = None
def Session():
# code that creates new session, in variable s
Client.last_session = s
return s
client = Client()
with client.Session as s:
Example.query(foo=bar)
Class Example:
#staticmethod
def query(**kwargs):
s = Client.last_session
s.magic_stuff()