As I understood from the "Erlang and OTP in action" book, the word behavior refers to:
the behaviour interface, which is a set of functions;
the behaviour implementation, which is the application-specific code (a callback module);
the behaviour container, which is a process.
Question:
What an Erlang/OTP beginner should know about behaviours? Is it possible to describe and understand the notion of OTP behaviour in a nutshell?
What 'callback function' does actually mean in the context of Elang/OTP?
Can we consider the callbacks in a behaviour implemenation as methods overriden in Java?
The book says that the associated callback function for the library function 'gen_server:start_link/4' in the following code is 'Module:init/1'.
Does that mean that with init/1 we call the gen_server:start_link/4 library function? Or does that mean anything else?
-module(tr_server).
-behaviour(gen_server).
-include_lib("eunit/include/eunit.hrl").
%% API
-export([
start_link/1,
start_link/0,
get_count/0,
stop/0
]).
%% gen_server callbacks
-export([init/1, handle_call/3, handle_cast/2, handle_info/2,
terminate/2, code_change/3]).
-define(SERVER, ?MODULE).
-define(DEFAULT_PORT, 1055).
-record(state, {port, lsock, request_count = 0}).
%%%===================================================================
%%% API
%%%===================================================================
%%--------------------------------------------------------------------
%% #doc Starts the server.
%%
%% #spec start_link(Port::integer()) -> {ok, Pid}
%% where
%% Pid = pid()
%% #end
%%--------------------------------------------------------------------
start_link(Port) ->
gen_server:start_link({local, ?SERVER}, ?MODULE, [Port], []).
%% #spec start_link() -> {ok, Pid}
%% #doc Calls `start_link(Port)' using the default port.
s tart_link() ->
start_link(?DEFAULT_PORT).
%%--------------------------------------------------------------------
%% #doc Fetches the number of requests made to this server.
%% #spec get_count() -> {ok, Count}
%% where
%% Count = integer()
%% #end
%%--------------------------------------------------------------------
get_count() ->
gen_server:call(?SERVER, get_count).
%%--------------------------------------------------------------------
%% #doc Stops the server.
%% #spec stop() -> ok
%% #end
%%--------------------------------------------------------------------
stop() ->
gen_server:cast(?SERVER, stop).
%%%===================================================================
%%% gen_server callbacks
%%%===================================================================
init([Port]) ->
{ok, LSock} = gen_tcp:listen(Port, [{active, true}]),
{ok, #state{port = Port, lsock = LSock}, 0}.
handle_call(get_count, _From, State) ->
{reply, {ok, State#state.request_count}, State}.
handle_cast(stop, State) ->
{stop, normal, State}.
handle_info({tcp, Socket, RawData}, State) ->
do_rpc(Socket, RawData),
RequestCount = State#state.request_count,
{noreply, State#state{request_count = RequestCount + 1}};
handle_info(timeout, #state{lsock = LSock} = State) ->
{ok, _Sock} = gen_tcp:accept(LSock),
{noreply, State}.
terminate(_Reason, _State) ->
ok.
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%%===================================================================
%%% Internal functions
%%%===================================================================
do_rpc(Socket, RawData) ->
try
{M, F, A} = split_out_mfa(RawData),
Result = apply(M, F, A),
gen_tcp:send(Socket, io_lib:fwrite("~p~n", [Result]))
catch
_Class:Err ->
gen_tcp:send(Socket, io_lib:fwrite("~p~n", [Err]))
end.
split_out_mfa(RawData) ->
MFA = re:replace(RawData, "\r\n$", "", [{return, list}]),
{match, [M, F, A]} =
re:run(MFA,
"(.*):(.*)\s*\\((.*)\s*\\)\s*.\s*$",
[{capture, [1,2,3], list}, ungreedy]),
{list_to_atom(M), list_to_atom(F), args_to_terms(A)}.
args_to_terms(RawArgs) ->
{ok, Toks, _Line} = erl_scan:string("[" ++ RawArgs ++ "]. ", 1),
{ok, Args} = erl_parse:parse_term(Toks),
Args.
%% test
start_test() ->
{ok, _} = tr_server:start_link(1055).
Rather than try to address your specific questions as other answers have already done, I'll try to explain in simple terms the basics behind behaviors, and let you answer your own questions based on understanding those basics.
A behavior is basically a message handling framework, where by "framework" I mean the classical definition of a partial solution to a problem that can be completed and customized by the end user. OTP behaviors essentially supply:
a message loop
integration with underlying OTP support for code upgrade, tracing, system messages, etc.
Behaviors delegate message handling to callback modules, or behavior implementations as "Erlang and OTP In Action" calls them. Upon the invocation of its init/1 function, the callback module generally creates state for the message loop to keep on its behalf. The behavior loop then passes this state to each subsequent invocation of a callback module message handling function, and each of these invocations can return a modified state. Callback functions also return instructions telling the behavior message loop what to do next.
Here's a greatly simplified version of the message loop at the heart of a behavior:
loop(Callbacks, State) ->
{Next, NState} =
receive
M1 ->
Callbacks:handle_m1(M1,State);
M2 ->
Callbacks:handle_m2(M2,State);
Other ->
Callbacks:handle_other(Other,State)
end,
case Next of
stop -> ok;
_ -> loop(Callbacks, NState)
end.
This tail-recursive loop has the Callbacks module and the State variable as arguments. Before this loop is first invoked, you've already told the behavior what your callback module is, and then base OTP behavior support code has already called your init/1 callback function to get the initial value of State.
Our example behavior loop receives messages of form M1, M2, and any other message, the details of which don't matter here, and for each message, invokes a different callback function in the Callbacks module. In this example, the handle_m1 and handle_m2 callback functions handle messages M1 and M2 respectively, while the callback handle_other handles all other kinds of messages. Note that State is passed to each callback function. Each function is expected to return a tuple with the first element telling the loop what to do next and the second element containing possible new state for the loop — either the same value as State or a new different value — which the loop stores in its variable NState. In this example, if Next is the atom stop, the loop stops, but if it's anything else, the loop invokes itself recursively, passing the new state NState to the next iteration. And since it's tail recursive, the loop won't ever blow out the stack.
If you dig through the sources of standard OTP behaviors such as gen_server and gen_fsm, you'll find a loop much like this, but they're much more complex due to handling system messages, timeouts, tracing, exceptions, etc. Standard behaviors also start their loops in a separate process, so they also contain code for starting the loop process and passing messages to it.
Q: What an Erlang/OTP beginner should know about behaviours? Is it
possible to describe and understand the notion of OTP behaviour in a
nutshell?
A behaviour is usually used in code so that the compiler can generate more intuitive error messages depending upon its behaviour i.e application/supervisor/gen_server/gen_event/gen_fsm.
It enables the compiler give error messages specific to the behaviour for ex: gen_server
Q: What 'callback function' does actually mean in the context of
Elang/OTP?
Callback function can be said to be taken from GUI programming (at least similar). Whenever an event occurs for ex. a mouse click there is a separate function that handles mouse click.
Thus whenever for eg. an exported function of a gen_server is called from another module, that function can have a callback function (handle_call/handle_cast) having different patterns.
Q: Can we consider the callbacks in a behaviour implementation as methods
overridden in Java?
Yeah...maybe...no :)
Q: The book says that the associated callback function for the library
function 'gen_server:start_link/4' in the following code is
'Module:init/1'.
gen_server:start_link calls init function by itself as answered by w55.... (sorry quite a big name).
Hope I have answered all your queries :)
What an Erlang/OTP beginner should know about behaviours?
Probably what's written here.
Is it possible to describe and understand the notion of OTP behaviour in a nutshell?
Reading from the doc: "Behaviours are formalizations of these common patterns. The idea is to divide the code for a process in a generic part (a behaviour module) and a specific part (a callback module)."
What 'callback function' does actually mean in the context of Elang/OTP?
Look at the link above where examples of callback functions are provided.
Can we consider the callbacks in a behaviour implemenation as methods overriden in Java?
In Java terms, a behaviour would probably be a Java Interface, while a callback would be the implementation of one of the methods defined in the interface.
The book says that the associated callback function for the library function 'gen_server:start_link/4' in the following code is 'Module:init/1'.
Does that mean that with init/1 we call the gen_server:start_link/4 library function? Or does that mean anything else?
It means that, every time you call the gen_server:start_link/4, the function Module:init/1 will be called, where Module is the second parameter you passed to the start_link function, with the arguments you provided as the forth argument. In other words, this is what happens behind the scenes of the start_link/4:
...
start_link(Name, Module, Args, Opts) ->
...
Module:init(Args)
...
...
look at the source code of gen_server module in your erlang lib directory. It is very well explained in the source code, the comments are very elaborate.
gen_server:start_link calls init.
Related
So, I was trying to learn about Continuation. I came across with the following saying (link):
Say you're in the kitchen in front of the refrigerator, thinking about a sandwich. You take a continuation right there and stick it in your pocket. Then you get some turkey and bread out of the refrigerator and make yourself a sandwich, which is now sitting on the counter. You invoke the continuation in your pocket, and you find yourself standing in front of the refrigerator again, thinking about a sandwich. But fortunately, there's a sandwich on the counter, and all the materials used to make it are gone. So you eat it. :-) — Luke Palmer
Also, I saw a program in Scala:
var k1 : (Unit => Sandwich) = null
reset {
shift { k : Unit => Sandwich) => k1 = k }
makeSandwich
}
val x = k1()
I don't really know the syntax of Scala (looks similar to Java and C mixed together) but I would like to understand the concept of Continuation.
Firstly, I tried to run this program (by adding it into main). But it fails, I think that it has a syntax error due to the ) near Sandwich but I'm not sure. I removed it but it still does not compile.
How to create a fully compiled example that shows the concept of the story above?
How this example shows the concept of Continuation.
In the link above there was the following saying: "Not a perfect analogy in Scala because makeSandwich is not executed the first time through (unlike in Scheme)". What does it mean?
Since you seem to be more interested in the concept of the "continuation" rather than specific code, let's forget about that code for a moment (especially because it is quite old and I don't really like those examples because IMHO you can't understand them correctly unless you already know what a continuation is).
Note: this is a very long answer with some attempts to describe what a continuations is and why it is useful. There are some examples in Scala-like pseudo-code none of which can actually be compiled and run (there is just one compilable example at the very end and it references another example from the middle of the answer). Expect to spend a significant amount of time just reading this answer.
Intro to continuations
Probably the first thing you should do to understand a continuation is to forget about how modern compilers for most of the imperative languages work and how most of the modern CPUs work and particularly the idea of the call stack. This is actually implementation details (although quite popular and quite useful in practice).
Assume you have a CPU that can execute some sequence of instructions. Now you want to have a high level languages that support the idea of methods that can call each other. The obvious problem you face is that the CPU needs some "forward only" sequence of commands but you want some way to "return" results from a sub-program to the caller. Conceptually it means that you need to have some way to store somewhere before the call all the state of the caller method that is required for it to continue to run after the result of the sub-program is computed, pass it to the sub-program and then ask the sub-program at the end to continue execution from that stored state. This stored state is exactly a continuation. In most of the modern environments those continuations are stored on the call stack and often there are some assembly instructions specifically designed to help handling it (like call and return). But again this is just implementation details. Potentially they might be stored in an arbitrary way and it will still work.
So now let's re-iterate this idea: a continuation is a state of the program at some point that is enough to continue its execution from that point, typically with no additional input or some small known input (like a return value of the called method). Running a continuation is different from a method call in that usually continuation never explicitly returns execution control back to the caller, it can only pass it to another continuation. Potentially you can create such a state yourself, but in practice for the feature to be useful you need some support from the compiler to build continuations automatically or emulate it in some other way (this is why the Scala code you see requires a compiler plugin).
Asynchronous calls
Now there is an obvious question: why continuations are useful at all? Actually there are a few more scenarios besides the simple "return" case. One such scenario is asynchronous programming. Actually if you do some asynchronous call and provide a callback to handle the result, this can be seen as passing a continuation. Unfortunately most of the modern languages do not support automatic continuations so you have to grab all the relevant state yourself. Another problem appears if you have some logic that needs a sequence of many async calls. And if some of the calls are conditional, you easily get to the callbacks hell. The way continuations help you avoid it is by allowing you build a method with effectively inverted control flow. With typical call it is the caller that knows the callee and expects to get a result back in a synchronous way. With continuations you can write a method with several "entry points" (or "return to points") for different stages of the processing logic that you can just pass to some other method and that method can still return to exactly that position.
Consider following example (in pseudo-code that is Scala-like but is actually far from the real Scala in many details):
def someBusinessLogic() = {
val userInput = getIntFromUser()
val firstServiceRes = requestService1(userInput)
val secondServiceRes = if (firstServiceRes % 2 == 0) requestService2v1(userInput) else requestService2v2(userInput)
showToUser(combineUserInputAndResults(userInput,secondServiceRes))
}
If all those calls a synchronous blocking calls, this code is easy. But assume all those get and request calls are asynchronous. How to re-write the code? The moment you put the logic in callbacks you loose the clarity of the sequential code. And here is where continuations might help you:
def someBusinessLogicCont() = {
// the method entry point
val userInput
getIntFromUserAsync(cont1, captureContinuationExpecting(entry1, userInput))
// entry/return point after user input
entry1:
val firstServiceRes
requestService1Async(userInput, captureContinuationExpecting(entry2, firstServiceRes))
// entry/return point after the first request to the service
entry2:
val secondServiceRes
if (firstServiceRes % 2 == 0) {
requestService2v1Async(userInput, captureContinuationExpecting(entry3, secondServiceRes))
// entry/return point after the second request to the service v1
entry3:
} else {
requestService2v2Async(userInput, captureContinuationExpecting(entry4, secondServiceRes))
// entry/return point after the second request to the service v2
entry4:
}
showToUser(combineUserInputAndResults(userInput, secondServiceRes))
}
It is hard to capture the idea in a pseudo-code. What I mean is that all those Async method never return. The only way to continue execution of the someBusinessLogicCont is to call the continuation passed into the "async" method. The captureContinuationExpecting(label, variable) call is supposed to create a continuation of the current method at the label with the input (return) value bound to the variable. With such a re-write you still has a sequential-looking business logic even with all those asynchronous calls. So now for a getIntFromUserAsync the second argument looks like just another asynchronous (i.e. never-returning) method that just requires one integer argument. Let's call this type Continuation[T]
trait Continuation[T] {
def continue(value: T):Nothing
}
Logically Continuation[T] looks like a function T => Unit or rather T => Nothing where Nothing as the return type signifies that the call actually never returns (note, in actual Scala implementation such calls do return, so no Nothing there, but I think conceptually it is easy to think about no-return continuations).
Internal vs external iteration
Another example is a problem of iteration. Iteration can be internal or external. Internal iteration API looks like this:
trait CollectionI[T] {
def forEachInternal(handler: T => Unit): Unit
}
External iteration looks like this:
trait Iterator[T] {
def nextValue(): Option[T]
}
trait CollectionE[T] {
def forEachExternal(): Iterator[T]
}
Note: often Iterator has two method like hasNext and nextValue returning T but it will just make the story a bit more complicated. Here I use a merged nextValue returning Option[T] where the value None means the end of the iteration and Some(value) means the next value.
Assuming the Collection is implemented by something more complicated than an array or a simple list, for example some kind of a tree, there is a conflict here between the implementer of the API and the API user if you use typical imperative language. And the conflict is over the simple question: who controls the stack (i.e. the easy to use state of the program)? The internal iteration is easier for the implementer because he controls the stack and can easily store whatever state is needed to move to the next item but for the API user the things become tricky if she wants to do some aggregation of the stored data because now she has to save the state between the calls to the handler somewhere. Also you need some additional tricks to let the user stop the iteration at some arbitrary place before the end of the data (consider you are trying to implement find via forEach). Conversely the external iteration is easy for the user: she can store all the state necessary to process data in any way in local variables but the API implementer now has to store his state between calls to the nextValue somewhere else. So fundamentally the problem arises because there is only one place to easily store the state of "your" part of the program (the call stack) and two conflicting users for that place. It would be nice if you could just have two different independent places for the state: one for the implementer and another for the user. And continuations provide exactly that. The idea is that we can pass execution control between two methods back and forth using two continuations (one for each part of the program). Let's change the signatures to:
// internal iteration
// continuation of the iterator
type ContIterI[T] = Continuation[(ContCallerI[T], ContCallerLastI)]
// continuation of the caller
type ContCallerI[T] = Continuation[(T, ContIterI[T])]
// last continuation of the caller
type ContCallerLastI = Continuation[Unit]
// external iteration
// continuation of the iterator
type ContIterE[T] = Continuation[ContCallerE[T]]
// continuation of the caller
type ContCallerE[T] = Continuation[(Option[T], ContIterE[T])]
trait Iterator[T] {
def nextValue(cont : ContCallerE[T]): Nothing
}
trait CollectionE[T] {
def forEachExternal(): Iterator[T]
}
trait CollectionI[T] {
def forEachInternal(cont : ContCallerI[T]): Nothing
}
Here ContCallerI[T] type, for example, means that this is a continuation (i.e. a state of the program) the expects two input parameters to continue running: one of type T (the next element) and another of type ContIterI[T] (the continuation to switch back). Now you can see that the new forEachInternal and the new forEachExternal+Iterator have almost the same signatures. The only difference in how the end of the iteration is signaled: in one case it is done by returning None and in other by passing and calling another continuation (ContCallerLastI).
Here is a naive pseudo-code implementation of a sum of elements in an array of Int using these signatures (an array is used instead of something more complicated to simplify the example):
class ArrayCollection[T](val data:T[]) : CollectionI[T] {
def forEachInternal(cont0 : ContCallerI[T], lastCont: ContCallerLastI): Nothing = {
var contCaller = cont0
for(i <- 0 to data.length) {
val contIter = captureContinuationExpecting(label, contCaller)
contCaller.continue(data(i), contIter)
label:
}
}
}
def sum(arr: ArrayCollection[Int]): Int = {
var sum = 0
val elem:Int
val iterCont:ContIterI[Int]
val contAdd0 = captureContinuationExpecting(labelAdd, elem, iterCont)
val contLast = captureContinuation(labelReturn)
arr.forEachInternal(contAdd0, contLast)
labelAdd:
sum += elem
val contAdd = captureContinuationExpecting(labelAdd, elem, iterCont)
iterCont.continue(contAdd)
// note that the code never execute this line, the only way to jump out of labelAdd is to call contLast
labelReturn:
return sum
}
Note how both implementations of the forEachInternal and of the sum methods look fairly sequential.
Multi-tasking
Cooperative multitasking also known as coroutines is actually very similar to the iterations example. Cooperative multitasking is an idea that the program can voluntarily give up ("yield") its execution control either to the global scheduler or to another known coroutine. Actually the last (re-written) example of sum can be seen as two coroutines working together: one doing iteration and another doing summation. But more generally your code might yield its execution to some scheduler that then will select which other coroutine to run next. And what the scheduler does is manages a bunch of continuations deciding which to continue next.
Preemptive multitasking can be seen as a similar thing but the scheduler is run by some hardware interruption and then the scheduler needs a way to create a continuation of the program being executed just before the interruption from the outside of that program rather than from the inside.
Scala examples
What you see is a really old article that is referring to Scala 2.8 (while current versions are 2.11, 2.12, and soon 2.13). As #igorpcholkin correctly pointed out, you need to use a Scala continuations compiler plugin and library. The sbt compiler plugin page has an example how to enable exactly that plugin (for Scala 2.12 and #igorpcholkin's answer has the magic strings for Scala 2.11):
val continuationsVersion = "1.0.3"
autoCompilerPlugins := true
addCompilerPlugin("org.scala-lang.plugins" % "scala-continuations-plugin_2.12.2" % continuationsVersion)
libraryDependencies += "org.scala-lang.plugins" %% "scala-continuations-library" % continuationsVersion
scalacOptions += "-P:continuations:enable"
The problem is that plugin is semi-abandoned and is not widely used in practice. Also the syntax has changed since the Scala 2.8 times so it is hard to get those examples running even if you fix the obvious syntax bugs like missing ( here and there. The reason of that state is stated on the GitHub as:
You may also be interested in https://github.com/scala/async, which covers the most common use case for the continuations plugin.
What that plugin does is emulates continuations using code-rewriting (I suppose it is really hard to implement true continuations over the JVM execution model). And under such re-writings a natural thing to represent a continuation is some function (typically called k and k1 in those examples).
So now if you managed to read the wall of text above, you can probably interpret the sandwich example correctly. AFAIU that example is an example of using continuation as means to emulate "return". If we re-sate it with more details, it could go like this:
You (your brain) are inside some function that at some points decides that it wants a sandwich. Luckily you have a sub-routine that knows how to make a sandwich. You store your current brain state as a continuation into the pocket and call the sub-routine saying to it that when the job is done, it should continue the continuation from the pocket. Then you make a sandwich according to that sub-routine messing up with your previous brain state. At the end of the sub-routine it runs the continuation from the pocket and you return to the state just before the call of the sub-routine, forget all your state during that sub-routine (i.e. how you made the sandwich) but you can see the changes in the outside world i.e. that the sandwich is made now.
To provide at least one compilable example with the current version of the scala-continuations, here is a simplified version of my asynchronous example:
case class RemoteService(private val readData: Array[Int]) {
private var readPos = -1
def asyncRead(callback: Int => Unit): Unit = {
readPos += 1
callback(readData(readPos))
}
}
def readAsyncUsage(rs1: RemoteService, rs2: RemoteService): Unit = {
import scala.util.continuations._
reset {
val read1 = shift(rs1.asyncRead)
val read2 = if (read1 % 2 == 0) shift(rs1.asyncRead) else shift(rs2.asyncRead)
println(s"read1 = $read1, read2 = $read2")
}
}
def readExample(): Unit = {
// this prints 1-42
readAsyncUsage(RemoteService(Array(1, 2)), RemoteService(Array(42)))
// this prints 2-1
readAsyncUsage(RemoteService(Array(2, 1)), RemoteService(Array(42)))
}
Here remote calls are emulated (mocked) with a fixed data provided in arrays. Note how readAsyncUsage looks like a totally sequential code despite the non-trivial logic of which remote service to call in the second read depending on the result of the first read.
For full example you need prepare Scala compiler to use continuations and also use a special compiler plugin and library.
The simplest way is a create a new sbt.project in IntellijIDEA with the following files: build.sbt - in the root of the project, CTest.scala - inside main/src.
Here is contents of both files:
build.sbt:
name := "ContinuationSandwich"
version := "0.1"
scalaVersion := "2.11.6"
autoCompilerPlugins := true
addCompilerPlugin(
"org.scala-lang.plugins" % "scala-continuations-plugin_2.11.6" % "1.0.2")
libraryDependencies +=
"org.scala-lang.plugins" %% "scala-continuations-library" % "1.0.2"
scalacOptions += "-P:continuations:enable"
CTest.scala:
import scala.util.continuations._
object CTest extends App {
case class Sandwich()
def makeSandwich = {
println("Making sandwich")
new Sandwich
}
var k1 : (Unit => Sandwich) = null
reset {
shift { k : (Unit => Sandwich) => k1 = k }
makeSandwich
}
val x = k1()
}
What the code above essentially does is calling makeSandwich function (in a convoluted manner). So execution result would be just printing "Making sandwich" into console. The same result would be achieved without continuations:
object CTest extends App {
case class Sandwich()
def makeSandwich = {
println("Making sandwich")
new Sandwich
}
val x = makeSandwich
}
So what's the point? My understanding is that we want to "prepare a sandwich", ignoring the fact that we may be not ready for that. We mark a point of time where we want to return to after all necessary conditions are met (i.e. we have all necessary ingredients ready). After we fetch all ingredients we can return to the mark and "prepare a sandwich", "forgetting that we were unable to do that in past". Continuations allow us to "mark point of time in past" and return to that point.
Now step by step. k1 is a variable to hold a pointer to a function which should allow to "create sandwich". We know it because k1 is declared so: (Unit => Sandwich).
However initially the variable is not initialized (k1 = null, "there are no ingredients to make a sandwich yet"). So we can't call the function preparing sandwich using that variable yet.
So we mark a point of execution where we want to return to (or point of time in past we want to return to) using "reset" statement.
makeSandwich is another pointer to a function which actually allows to make a sandwich. It's the last statement of "reset block" and hence it is passed to "shift" (function) as argument (shift { k : (Unit => Sandwich).... Inside shift we assign that argument to k1 variable k1 = k thus making k1 ready to be called as a function. After that we return to execution point marked by reset. The next statement is execution of function pointed to by k1 variable which is now properly initialized so finally we call makeSandwich which prints "Making sandwich" to a console. It also returns an instance of sandwich class which is assigned to x variable.
Not sure, probably it means that makeSandwich is not called inside reset block but just afterwards when we call it as k1().
I am trying to implement a "multithreaded" program in Matlab. It is not actually multithreaded, but it consists on a main script (A) interacting with other scripts (B,C...) in other Matlab instances via TCP/IP.
The way I tried to do this is to create a listener on the tcpip object (srv) :
func = #(src, evnt) disp('booh');
l = listener(srv, 'BytesAvailable', 'PostSet', func );
which returns me
Undefined function 'listener' for input arguments of type 'tcpip'.
I'm new to listeners in Matlab, so I might be missing something here. In case not, is there a clean way to know when data has been received ? (I prefer not having to do an infinite loop checking the value of srv.BytesAvailable).
A way to achieve this is to set the srv.BytesAvailableFcn property to a function callback.
I'm currently following the book Learn You Some Erlang for Great Good by Fred Herbert and one of the sections is regarding Macros.
I understand using macros for variables (constant values, mainly), however, I don't understand the use case for macros as functions. For example, Herbert writes:
Defining a "function" macro is similar. Here's a simple macro used to subtract one number from another:
-define(sub(X, Y), X-Y).
Why not just define this as a function elsewhere? Why use a macro? Is there some sort of performance advantage from the compiler or is this merely just a "this function is so simple, let's just define it in one line" type of thing?
I'm not trying to start a debate or preference argument, but after seeing some production Erlang code, I've started noticing lots of macros function usage.
In this case, the one obvious advantage of the macro not being a function (-define(sub(X, Y), X-Y), which would be safer as -define(sub(X, Y), (X-Y))) is that it can be used as a guard since custom function calls are forbidden.
In many cases it would otherwise be safer to define the function as an inlined one.
On the other hand, there are other interesting cases, such as assertions in tests or shortcuts where what you want is to keep some local context in the final place.
For example, let's say I want to make a generic call for a test where the objective is 'match a given pattern and return a given value, or fail after M milliseconds'.
I cannot make this generic with code since patterns are not data structures you are allowed to carry around. However, with macros:
-define(wait_for(PAT, Timeout),
receive
PAT -> VAL
after Timeout ->
error(timeout)
end).
This macro can then be used as:
my_test() ->
Pid = start_whatever(),
%% ...
?wait_for({'EXIT', Pid, Reason}, 5000),
?assertMatch(shutdown, Reason).
By doing this, I'm able to simplify the form of text in some tests without needing a bunch of nesting, and in a way that is not possible with functions.
Do note that the assertion itself as defined by eunit is using a function macro, and does something akin to
-define(assertMatch(PAT, TERM),
%% funs to avoid leaking bindings into parent scope
(fun() ->
try
PAT = TERM,
true
catch _:_ ->
error({assertion_failed, ?LINE, ...})
end
end)()).
This similarly lets you carry patterns and bindings and do fancy forms that couldn't be possible otherwise.
In this last case, you'll notice I used the ?LINE macro. That's another advantage of macros: you preserve information and locality about the call site, such as its module name, line number, and so on. This is useful when such metadata is required, such as when you're reporting test failures.
If you're looking at old code, there might be macros used as a way of inlining small functions under the assumption that function calls are very expensive. I'm not sure if that was ever true, but it's not something you need to worry about today.
Macros can be used to define constants, like
-define(MAX_TIMEOUT, 30 * 1000).
%% ...
gen_server:call(my_server, {do_stuff, Data}, ?MAX_TIMEOUT),
%% ...
I mostly prefer to pass in environment variables for this job, but it's more work to read them on startup and stash them somewhere and write accessors.
Finally, you can do some simple metaprogramming:
-define(MAKE_REQUEST_FUN(Method),
Method(Request, HTTPOptions, Options) ->
httpc:request(Method, Request, HTTPOptions, Options)).
?MAKE_REQUEST_FUN(get).
?MAKE_REQUEST_FUN(put).
%% Now we've defined a get/3 that can be called as
%% get(Request, [], []).
I have the source of the messages, which is an Observable. For every message I would like to make an HTTP call which will produce another Observable, so I combine them together with the flatMap and then sink them to some subscriber. Here the code of this scenario:
Rx.Observable.interval(1000)
.flatMap (tick) ->
// returns an `Observable`
loadMessages()
.flatMap (message) ->
// also returns and `Observable`
makeHttpRequest(message)
.subscribe (result) ->
console.info "Processed: ", result
this example is written in coffeescript, but I think the problem statement would be valid for any other Rx implementation.
The issue I have with this approach is that loadMessages produces a lot of messages very quickly. This means, that I make a lot of HTTP requests in a very short period of time. This is not acceptable in my situation, so I would like to limit amount of the parallel HTTP requests to 10 or so. In other words I would like to throttle the pipelene or apply some kind of backpresure, when I making HTTP requests.
Is there any standard approach or best practices for the Rx to deal with this kind of situations?
Currently I implemented very simple (and pretty suboptimal) backpresure mechanism, that ignores tick if system has too many massages in processing. It looks like this (simplified version):
Rx.Observable.interval(1000)
.filter (tick) ->
stats.applyBackpressureBasedOnTheMessagesInProcessing()
.do (tick) ->
stats.messageIn()
.flatMap (tick) ->
// returns an `Observable`
loadMessages()
.flatMap (message) ->
// also returns and `Observable`
makeHttpRequest(message)
.do (tick) ->
stats.messageOut()
.subscribe (result) ->
console.info "Processed: ", result
I'm not sure though, whether this can be done better, or maybe Rx already has some mechanisms in-place to deal with this kind of requirements.
This isn't strictly backpressure, this is just limiting concurrency. Here's an easy way to do it (ignore my possibly wrong syntax, coding via TextArea):
Rx.Observable.interval(1000)
.flatMap (tick) ->
// returns an `Observable`
loadMessages()
.map (message) ->
// also returns and `Observable`, but only when
// someone first subscribes to it
Rx.Observable.defer ->
makeHttpRequest(message)
.merge 10 // at a time
.subscribe (result) ->
console.info "Processed: ", result
In C#, the equivalent idea is, instead of SelectMany, it's Select(Defer(x)).Merge(n). Merge(int) subscribes to at most n in-flight Observables, and buffers the rest until later. The reason we have a Defer, is to make it such that we don't do any work until the Merge(n) subscribes to us.
In RXJS you can use the backpressure submodule
http://rxjs.codeplex.com/SourceControl/latest#src/core/backpressure/
disclaimer I have never used the RX version of JS but you did ask for a standard way of implementing backpressure and the core library seems to have support for it. RX c# does not yet have this support. Not sure why.
It sounds like you want to pull from a queue rather than push your http requests. Is Rx really the right choice of technology here?
EDIT:
In general, I would not design a solution using Rx where I had complete imperative control over the source events. It's just not a reactive scenario.
The backpressure module in Rxjs is clearly written to deal with situations where you don't own the source stream. Here you do.
TPL Dataflow sounds like a far better fit here.
If you must use RX, you could set up a loop like this: If you want to limit to X concurrent events, set up a Subject to act as your message source and imperatively push (OnNext) X messages into it. In your subscriber, you can push a new message to the subject in each iteration of the OnNext handler until the source is exhausted. This guarantees a maximum of X messages in flight.
Just getting started on Scala Actors. The Scala website says:
Thread-blocking operations can be avoided by using react to wait for
new messages (the event-based pendant of receive). However, there is a
(usually small) price to pay: react never returns.
...
Note that using react inside a while loop does not work! However,
since loops are common there is special library support for it in form
of a loop function. It can be used like this:
loop {
react {
case A => ...
case B => ...
}
}
I'm now confused - there seems to be a contradiction:
a) If react never returns, then what's the point of putting it in a loop?
b) Since loop repeatedly executes a block, how is it any different to while(true) - why doesn't while work, and in what way does it "not work"?
Both functions, loop and react are not pure. loop takes a call by name parameter and react a PartialFunction, both set variables on the raw actor. This is because an actor does not have a thread attached all the time. It will become active only when there is a message in it's messagebox. This is why a while(true) will lead to 100% cpu usage and the actor not responding.
I found an explanation that answers part a) of my question, in one of Haller and Odersky's papers on Actors (my emphasis below):
The central idea is as follows: An actor that waits in a receive
statement is not represented by a blocked thread but by a closure that
captures the rest of the actor's computation. The closure is executed
once a message is sent to the actor that matches one of the message
patterns specied in the receive. The execution of the closure is
\piggy-backed" on the thread of the sender.
If the receiving closure
terminates, control is returned to the sender as if a procedure
returns. If the receiving closure blocks in a second receive, control
is returned to the sender by throwing a special exception that unwinds
the receiver's call stack.
A necessary condition for the scheme to
work is that receivers never return normally to their enclosing actor.
In other words, no code in an actor can depend on the termination or
the result of a receive block...