Why does this function application generate a runtime error in purescript? - purescript

I have the following PureScript snippets; note parseXMLFromString is partially applied:
parseXMLFromString ∷ String → DOMParser → Effect Document
parseXMLFromString s d =
parseFromString "application/xml" s d
parseNoteDoc :: DOMParser -> Effect Document
parseNoteDoc = parseXMLFromString TD.noteXml
note <- parseNoteDoc domParser
The following code is generated:
// Generated by purs version 0.12.4
"use strict";
var Effect_Console = require("../Effect.Console/index.js");
var Test_Data = require("../Test.Data/index.js");
var Web_DOM_DOMParser = require("../Web.DOM.DOMParser/index.js");
var parseNoteDoc = Web_DOM_DOMParser.parseXMLFromString(Test_Data.noteXml);
var main = function __do() {
var v = Web_DOM_DOMParser.makeDOMParser();
var v1 = parseNoteDoc(v)();
return Effect_Console.log("TODO: You should add some tests.")();
};
module.exports = {
parseNoteDoc: parseNoteDoc,
main: main
};
The line var v1 = parseNoteDoc(v)(); gives the error TypeError: parseNoteDoc(...) is not a function.
I'm not sure where the extra () is coming from on parseNoteDoc but that is the issue. When I manually remove the () in the generated source, it works works as expected.
Update: Added the code to reproduce this on this branch. After the usual formalities, npm run testbrowser and open dist/index.html in a browser.

TL;DR: your FFI code is incorrect, you need to add an extra function().
Longer explanation:
The extra empty parens come from Effect.
This is how effectful computations are modeled in PureScript: an effectful computation is not a value, but a "promise" of a value that you can evaluate and get the value as a result. A "promise" of a value may be modeled as a function that returns a value, and this is exactly how it's modeled in PureScript.
For example, this:
a :: Effect Unit
is compiled to JavaScript as:
function a() { return {}; }
and similarly, this:
f :: String -> Effect Unit
is compiled to JavaScript as:
function f(s) { return function() { return {}; } }
So it takes a string as a parameter, and then returns Effect Unit, which is itself a parameterless function in JS.
In your FFI module, however, you are defining parseFromString as:
exports.parseFromString = function (documentType) {
return function (sourceString) {
return function (domParser) {
return domParser.parseFromString(sourceString, documentType);
};
};
};
Which would be equivalent to parseFromString :: String -> String -> DOMParser -> Document - i.e. it takes three parameters, one by one, and returns a parsed document.
But on the PureScript side you're defining it as parseFromString :: String -> String -> DOMParser -> Effect Document - which means that it should take three parameters, one by one, and then return an Effect Document - which should be, as described above, a parameterless function. And it is exactly this extra parameterless call that fails when you try to evaluate that Effect Unit, which in reality is not an Effect at all, but a Document.
So, in order to fix your FFI, you just need to insert an extra parameterless function, which will model the returned Effect:
exports.parseFromString = function (documentType) {
return function (sourceString) {
return function (domParser) {
return function() {
return domParser.parseFromString(sourceString, documentType);
}
};
};
};
(it is interesting to note that makeDOMParser :: Effect DOMParser is correctly modeled in your FFI module as a parameterless function)
But there is a better way
These pyramids of nested functions in JS do look quite ugly, you have to agree. So it's no surprise that there is an app for that - EffectFn1, runEffectFn1, and friends. These are wrappers that "translate" JavaScript-style functions (i.e. taking all parameters at once) into PureScript-style curried effectful functions (i.e. taking parameters one by one and returning effects).
You can declare your JS side as a normal JS function, then import it into PureScript as EffectFnX, and call it using runEffectFnX where needed:
// JavaScript:
exports.parseFromString = function (documentType, sourceString, domParser) {
return domParser.parseFromString(sourceString, documentType);
};
-- PureScript:
foreign import parseFromString ∷ EffectFn3 String String DOMParser Document
parseHTMLFromString ∷ String → DOMParser → Effect Document
parseHTMLFromString s d =
runEffectFn3 parseFromString "text/html" s d
P.S. People who purchased EffectFn1 also liked Fn1 and friends - same thing, but for pure (non-effectful) functions.

Related

How is new object instantiation handled in case of Datasets?

I have to following scenario
case class A(name:String)
class Eq { def isMe(s:String) = s == "ME" }
val a = List(A("ME")).toDS
a.filter(l => new Eq().isMe(l.name))
Does this create a new object Eq every time for each data point on each executor ?
Nice one! I didn't know there is a different filter method for a typed dataset.
In order to answer your question, I will do some deep dive into Spark internals.
filter on a typed Dtaset has the following signature:
def filter(func: T => Boolean): Dataset[T]
Note that func is parameterized with T, hence Spark needs to deserialize both your object A along with the function.
TypedFilter Main$$$Lambda$, class A, [StructField(name,StringType,true)], newInstance(class A)
where Main$$$Lambda$ is a randomly generated function name
During optimization phase it might be eliminated by the EliminateSerialization rule if the following condition is met:
ds.map(...).filter(...) can be optimized by this rule to save extra deserialization, but ds.map(...).as[AnotherType].filter(...) can not be optimized.
If the rule is applicable TypedFilter is replaced by Filter.
The catch here is a Filter's condition. In fact, it is another special expression named Invoke where:
targetObject is the filter function Main$$$Lambda$
functionName is apply since it is a regular Scala function.
Spark eventually runs in one of these two modes - generate code or interpreter. Let's concentrate on the first one as it is the default.
Here is a simplified stack trace of the methods invocation that will generate the code
SparkPlan.execute
//https://github.com/apache/spark/blob/03e30063127fd71bef8a14553381e805fe5b6679/sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala#L596
-> WholeStageCodegenExec.execute
[child: Filter]
-> child.execute
[condition Invoke]
-> Invoke.genCode
//https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala#L345
-> doGenCode
Simplified code after generation phase:
final class GeneratedIteratorForCodegenStage1 extends BufferedRowIterator {
private Object[] references;
private scala.collection.Iterator input;
private UnsafeRowWriter writer = new UnsafeRowWriter();
public GeneratedIteratorForCodegenStage1(Object[] references) {
this.references = references;
}
public void init(Iterator inputs) {
this.inputs = inputs;
}
protected void processNext() throws IOException {
while (input.hasNext() && !stopEarly()) {
InternalRow row = input.next();
do {
//Create A object
UTF8String value = row.getUTF8String(0));
A a = new A(value.toString)
//Filter by A's value
result = (scala.Function1) references[0].apply(a);
if (!result) continue;
writer.write(0, value)
append((writer.getRow());
}
if (shouldStop()) return;
}
}
}
We can see that projection is constructed with an array of objects passed in references variable. But where and how many times the references variable is instantiated?
It is created during WholeStageCodegenExec and instantiated only once per partition.
And this leads us to the answer that, however, filter function will be created only once per partition and not per data point, the Eq and A classes will be created per data point.
If you are curious about where it has been added to the code context:
It happens here
where javaType is scala.function1 .
and value is the implementation - Main$$$Lambda$

Does there exist REAL pass by reference in any language?

As far as I know there's no pass by reference in c and java essentially passes everything by value, there're dozens of stack overflow posts discussing about this.
Now I wonder that is there any example of REAL call by reference? Because during function call the value of all parameters (including pointers or mutable object identifiers) are always copied to local variables in callee's frame, in that sense everything surely passes by value.
Sure, there is. For example, C♯ has pass-by-reference. In order for pass-by-reference to occur, both the method parameter in the parameter list at the declaration site as well as the method call argument in the argument list at the call site must be annotated with the ref modifier. The same applies to Visual Basic.NET (here, the modifier is ByRef, I believe.)
C++ also has pass-by-reference, the modifier is &. PHP also has pass-by-reference and uses the same modifier. The same applies to E.
Rust also offers call-by-reference.
In contrast to all the languages listed above, where pass-by-value is the default and pass-by-reference has to be explicitly requested, Fortran II is a pass-by-reference language.
Now I wonder that is there any example of REAL call by reference? Because during function call the value of all parameters (including pointers or mutable object identifiers) are always copied to local variables in callee's frame, in that sense everything surely passes by value.
What you describe is pass-by-value. That's not pass-by-reference. With pass-by-reference, the reference itself is passed, not the value that is referenced.
Here is an example in C♯ that demonstrates pass-by-value of a value type, pass-by-value of a reference type, pass-by-reference of a value type, and pass-by-reference of a reference type:
struct MutableCell
{
public string value;
}
class Program
{
static void IsCSharpPassByValue(string[] foo, MutableCell bar, ref string baz, ref MutableCell qux)
{
foo[0] = "More precisely, for reference types it is call-by-object-sharing, which is a special case of pass-by-value.";
foo = new string[] { "C# is not pass-by-reference." };
bar.value = "For value types, it is *not* call-by-sharing.";
bar = new MutableCell { value = "And also not pass-by-reference." };
baz = "It also supports pass-by-reference if explicitly requested.";
qux = new MutableCell { value = "Pass-by-reference is supported for value types as well." };
}
static void Main(string[] args)
{
var quux = new string[] { "Yes, of course, C# *is* pass-by-value!" };
var corge = new MutableCell { value = "For value types it is pure pass-by-value." };
var grault = "This string will vanish because of pass-by-reference.";
var garply = new MutableCell { value = "This string will vanish because of pass-by-reference." };
IsCSharpPassByValue(quux, corge, ref grault, ref garply);
Console.WriteLine(quux[0]);
// More precisely, for reference types it is call-by-object-sharing, which is a special case of pass-by-value.
Console.WriteLine(corge.value);
// For value types it is pure pass-by-value.
Console.WriteLine(grault);
// It also supports pass-by-reference if explicitly requested.
Console.WriteLine(garply.value);
// Pass-by-reference is supported for value types as well.
}
}

Reusable Functions in Linq To Entites

I have 2 reusable functions that return lists. If the code from these functions is written directly into the linq to entities query all is good. However, separating these out into functions causes an error as it cannot be translated to a stored expression. I'm sure there must be a way of doing this though. Any ideas how to solve this problems. Ideally I want the reusable parts to be used outside of linq to entity queries also.
var activityBands = DbContext.ActivityBand
.OrderBy(x => x.ActivityBandDescription)
.Where(x => x.Active && x.ClientAccountId == clientAccountId)
.Select(x => new ActivityBandDdl
{
Name = x.ActivityBandDescription,
ActivityBandId = x.ActivityBandId,
ApplyAwr = x.ApplyAwr,
AssignmentLineTimeTypeIds = TimeTypesForActivityBand(x.DailyRate) ,
AssignmentTypeIds = AssTypesForActivityBand(x.StagePayment)
}).ToList();
public static Func<bool, List<int>> TimeTypesForActivityBand =
(dailyRate) => (new int[] { 1, 2, 3, 4 }).Where(t =>
((t != 1 && t != 2) || !dailyRate) //No Timed or NTS for daily rates
).ToList();
public static Func<bool, List<int>> AssTypesForActivityBand =
(stagePayment) => (new int[] { 2,3,4,5,6,7,8,9,10 }).Where(t =>
( t!=2 || !stagePayment) //Only stage pay ass have stage pay activity bands
).ToList();
TL;DR;
suggested solution for your problem:
get LinqKit ... have a look at it's Expand() function (in the docs: combining expressions)
https://github.com/scottksmith95/LINQKit#combining-expressions
the details:
the problem boils down to: what is the difference between the queries in both cases...
A LINQ query works with an expression tree ... in other words: just because the code you typed directly into the query and the code you typed into the static Func<...> looks the same, in fact, is the same, the resulting expression trees in both cases are not the same
what is an expression tree?
imagine a simpler query like ... someIQueryable.Where(x => x.a==1 && x.b=="foo")
the lamda that is passed to Where(...) can be seen as a straigt forward c# lambda expression that can be used as a Func
and it can also be seen as a Expression>
the later is a tree of objects that form the expression, in other words a description about the way, in that the passed in parameter can be evaluated to a bool without actually having the executable code, but just the description about what to do ... take the member a from the parameter, equality-compare it to the constant 1 ... take the boolean AND of the result with the result of: take the member b from the parameter, equality-compare it to the constant "foo" ... return the result of the boolean AND
why all of this?
it's the way LINQ works ... LINQ to entiteis takes the expression tree, looks at all the operations, finds the corresponding SQL, and builds an SQL statement which is executed in the end ...
when you have your extracted Func<...> there is a little problem ... at some point in the resulting expression tree there is something like ... take the parameter x and CALL the static Func ... the expression tree does no longer contain a description of whats happening inside the Func, but just a call to that ... as long as you want to compile that to a .net runtime executable function, it's all fun and games ... but when you try to parse it into SQL, LINQ to entiteis does not know a corresponding SQL for "call some c# function" ... therefore it tells you that this part of the expression tree can not be converted into a store expression

scala reassignment to val in Option Class

My code looks like:
case class SRecord(trialId: String, private var _max:Int) {
def max=_max
def max_=(value:Int):Unit=_max=value
}
Then later on I apply a function onto it:
def groupSummaryRecords(it:Iterator[Option[SRecord]], optionSummary:Option[SRecord]):Option[SRecord] = {
var max=0;
var sRecord1 : Option[SRecord] = None
var i=0
while(it.hasNext) {
var sRecord:Option[SRecord] = it.next();
if(i==0) {
sRecord1 = sRecord;
}
..
}
sRecord1.max=max; // getting 'reassignment to val' compilation error
..
}
Why am i getting this compilation error, and how to fix it ?
If I instead change sRecord and sRecord1 instances to be of type SRecord instead of Option[SRecord] as well as the method signature, it all works fine however.
But in some cases I may have a null SRecord hence the use of None/Some. I am new to Scala, using Option/Some all over feels like a real pain if you ask me, i am just thinking of removing all this Option nonsense and testing for 'null' in good ol' Java, at least my code would work ??!
With the line sRecord1.max=max you are trying to call the max method on an Option[SRecord], not an SRecord. You want to access the contained SRecord (if any) and call the method on that, which can be done using foreach:
sRecord1.foreach(_.max=max)
which is desugared to:
sRecord1.foreach( srec => srec.max=max )
(the actual name "srec" is made up, the compiler will assign some internal name, but you get the idea). If sRecord1 is None, this won't do anything, but if it is Some(srec), the method execution will be passed in to operate on the contained instance.

Implementing TypeScript interface with bare function signature plus other fields

How do I write a class that implements this TypeScript interface (and keeps the TypeScript compiler happy):
interface MyInterface {
(): string;
text2(content: string);
}
I saw this related answer:
How to make a class implement a call signature in Typescript?
But that only works if the interface only has the bare function signature. It doesn't work if you have additional members (such as function text2) to be implemented.
A class cannot implement everything that is available in a typescript interface. Two prime examples are callable signatures and index operations e.g. : Implement an indexible interface
The reason is that an interface is primarily designed to describe anything that JavaScript objects can do. Therefore it needs to be really robust. A TypeScript class however is designed to represent specifically the prototype inheritance in a more OO conventional / easy to understand / easy to type way.
You can still create an object that follows that interface:
interface MyInterface {
(): string;
text2(content: string);
}
var MyType = ((): MyInterface=>{
var x:any = function():string { // Notice the any
return "Some string"; // Dummy implementation
}
x.text2 = function(content:string){
console.log(content); // Dummy implementation
}
return x;
}
);
There's an easy and type-safe way to do this with ES6's Object.assign:
const foo: MyInterface = Object.assign(
// Callable signature implementation
() => 'hi',
{
// Additional properties
text2(content) { /* ... */ }
}
)
Intersection types, which I don't think were available in TypeScript when this question was originally asked and answered, are the secret sauce to getting the typing right.
Here's an elaboration on the accepted answer.
As far as I know, the only way to implement a call-signature is to use a function/method. To implement the remaining members, just define them on this function. This might seem strange to developers coming from C# or Java, but I think it's normal in JavaScript.
In JavaScript, this would be simple because you can just define the function and then add the members. However, TypeScript's type system doesn't allow this because, in this example, Function doesn't define a text2 member.
So to achieve the result you want, you need to bypass the type system while you define the members on the function, and then you can cast the result to the interface type:
//A closure is used here to encapsulate the temporary untyped variable, "result".
var implementation = (() => {
//"any" type specified to bypass type system for next statement.
//Defines the implementation of the call signature.
var result: any = () => "Hello";
//Defines the implementation of the other member.
result.text2 = (content: string) => { };
//Converts the temporary variable to the interface type.
return <MyInterface>result;
})(); //Invokes the closure to produce the implementation
Note that you don't need to use a closure. You could just declare your temporary variable in the same scope as the resulting interface implementation. Another option is to name the closure function to improve readability.
Here's what I think is a more realistic example:
interface TextRetriever {
(): string;
Replace(text: string);
}
function makeInMemoryTextRetriever(initialText: string) {
var currentText = initialText;
var instance: any = () => currentText;
instance.Replace = (newText: string) => currentText = newText;
return <TextRetriever>instance;
}
var inMemoryTextRetriever = makeInMemoryTextRetriever("Hello");