I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4
I am new with Numba. I am trying to accelerate a pretty complicated solver. However, I keep getting an error such as
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend) Untyped global name 'isinstance': Cannot determine Numba type of <class 'builtin_function_or_method'>
I wrote a small example to reproduce the same error:
import numba
import numpy as np
from numba import types
from numpy import zeros_like, isfinite
from numpy.linalg import solve
from numpy.random import uniform
#numba.njit(parallel=True)
def foo(A_, b_, M1=None, M2=None):
x_ = zeros_like(b_)
r = b_ - A_.dot(x_)
flag = 1
if isinstance(M1, types.NoneType): # Error here
y = r
else:
y = solve(M1, r)
if not isfinite(y).any():
flag = 2
if isinstance(M2, types.NoneType):
z = y
else:
z = solve(M2, y)
if not isfinite(z).any():
flag = 2
return z, flag
N = 10
tmp = np.random.rand(N, N)
A = np.dot(tmp, tmp.T)
x = np.zeros((N, 1), dtype=np.float64)
b = np.vstack([uniform(0.0, 1.0) for i in range(N)])
X_1, info = foo(A, b)
Also if I change the decorator to generated_jit() I get the following error:
r = b_ - A_.dot(x_)
AttributeError: 'Array' object has no attribute 'dot'
Numba compiles the function and requires every variables to be statically typed. This means that each variable has only one unique type: one variable cannot be of both the type NoneType and something else as opposed to with CPython based on dynamic typing. Dynamic typing is also a major source of the slowdown of CPython. Thus, using isinstance in nopython JITed Numba functions does not make much sense. In fact, this built-in function is not supported.
That being said, Numba supports optional arguments by specifying optional(ArgumentType) in the signature (note that the resulting type of the variable is optional(ArgumentType) and not ArgumentType nor NoneType. You can then test if the argument is set using if yourArgument is None:. I do not know what is the type of M1 and M2 in your code but they need to be explicitly defined in the signature with optional argument.
Currently, I have within the same OCaml file,
blah.ml:
module type blah =
sig
val a : some-type
end
module type X =
sig
val x : some-type
end
module Y : X =
struct
let x = some-def
end
module Z : X =
struct
let x = some-other-def
end
blah.mli looks like this:
module type blah =
sig
val a
end
module type X =
sig
val x : some-type
end
module Y : X
module Z : X
I want X, Y, and Z to be in separate files with separate interfaces. How do I tell in Y.mli and Z.mli that Y and Z have type X?
Any readings for this would also be appreciated. There are a lot of resources talking about modules, interfaces, and functors, but they don't mention interface files for modules that have other modules as types.
You can create x.ml containing the sig, y.ml containing the module, and z.ml containing that module. You don't need to do anything special to tell the compiler that Y : X and Z : X. The compiler infers the module type automatically from the fact that the module conforms to the type i.e. it implements every binding that the module type needs. If Y or Z don't conform, the type error will be shown at the point of use.
If you want to restrict the module type at the point of definition that's also doable, by giving each module an interface file and includeing the required signature there. For example:
(* x.ml *)
module type S = sig
val x : some-type
end
(* y.mli *)
include X.S
(* y.ml *)
let x = some-def
...and so on.
However this often becomes too restrictive as the signature hides too much detail about the types. So in reality you may actually need to add type equality sharing constraints to avoid compile errors where you want to expose more type info. E.g.:
(* y.mli *)
include X.S with type t = int
Often it is easier to not have the explicit interface file at all unless you really need to make some parts of the module private.
I would like to print the function definition of any nested function when printing the definition of the outer function. Example:
g:{sin x}
f:{cos g x}
When I print f I get {cos g x} but I want to get {cos {sin x} x}
Thanks for the help
From what I am aware it is not possible to achieve that with in-build functions.
You can attempt to write your own function that does that but it will be a pain in the end. Something like this maybe:
q)m:string[v]!string value each v:value[f][3] except `
which creates a dictionary m :
q)m
,"g"| "{sin x}"
When given a function value returns a list containing (bytecode;parameters;locals(context;globals);constants[0];...;constants[n];definition)
However, if we pass a symbol to value it returns the value of that symbol (or function definition in this case).
You can then use ssr to replace the functions in f with the function definitions stored in your dictionary m.
q)ssr/[last value[f];key m;value m]
"{cos {sin x} x}"
but to ensure that your function is stable and adaptable to different functions would be very difficult.
For more details about how value have a look here: https://code.kx.com/q/ref/metadata/#value
For ssr check this link:
https://code.kx.com/q/ref/strings/#ssr
Hello fellow Overflowers. I am working on a group project to create a ray tracer that draws a 2D rendering of a 3D scene. The task I am currently on involves matrix transformation of objects (shapes), that need to be moved around, mirrored, sheared etc.
In working with shapes we have chosen to implement an interface that defines the type for a hit function. This hit function is defined in each shape, such as sphere, box, plane etc. When transforming a shape I need to transform the rays that hit the shape and the way to do that seems to be with a higher order function that alters the original hit function.
In order to do this I have implemented the function transformHitFunction, which seems to work, but the new type transformedShape, that implements the Shape interface, is giving me the error
No abstract property was found that corresponds to this override
which doesn't make any sense to me, as it works with other hit functions of the same type. Can anyone spot what's wrong?
I have tried to strip away all modules, namespaces and code that is not relevant to this issue.
type Transformation = Matrix of float [,]
type Vector =
| V of float * float * float
let mkVector x y z = V(x, y, z)
let vgetX (V(x,_,_)) = x
let vgetY (V(_,y,_)) = y
let vgetZ (V(_,_,z)) = z
type Point =
| P of float * float * float
let mkPoint x y z = P(x, y, z)
let pgetX (P(x,_,_)) = x
let pgetY (P(_,y,_)) = y
let pgetZ (P(_,_,z)) = z
type Material = Material
type Texture =
| T of (float -> float -> Material)
type Shape =
abstract member hit: Point * Vector -> (Texture*float*Vector) option
let transformPoint (p:Point) t =
match t with
| Matrix m -> mkPoint ((pgetX(p))*m.[0,0] + (pgetY(p))*m.[0,1] + (pgetZ(p))*m.[0,2] + m.[0,3])
((pgetX(p))*m.[1,0] + (pgetY(p))*m.[1,1] + (pgetZ(p))*m.[1,2] + m.[1,3])
((pgetX(p))*m.[2,0] + (pgetY(p))*m.[2,1] + (pgetZ(p))*m.[2,2] + m.[2,3])
let transformVector (v:Vector) t =
match t with
| Matrix m -> mkVector ((vgetX(v))*m.[0,0] + (vgetY(v))*m.[0,1] + (vgetZ(v))*m.[0,2] + m.[0,3])
((vgetX(v))*m.[1,0] + (vgetY(v))*m.[1,1] + (vgetZ(v))*m.[1,2] + m.[1,3])
((vgetX(v))*m.[2,0] + (vgetY(v))*m.[2,1] + (vgetZ(v))*m.[2,2] + m.[2,3])
let transformHitFunction fn (t:Transformation) =
fun (p:Point,v:Vector) ->
let tp = transformPoint p t
let tv = transformVector v t
match fn(tp,tv) with
| None -> None
| Some (tex:Texture, d:float, n) -> let tn = transformVector n t
Some (tex, d, tn)
type transformedShape (sh:Shape, t:Transformation) =
interface Shape with
member this.hit = transformHitFunction sh.hit t
Short answer
When having problems with implementing or overriding members, provide the argument list exactly as in the abstract or virtual member's definition. (Also, mind your parentheses, because additional parentheses can change the type of a member in subtle ways.)
E.g. in this case: member this.hit (arg1, arg2) = ...
Slightly longer answer
You're encountering a situation in which the difference between F#'s first-class functions and its support of object-oriented style methods is relevant.
For compatibility with the Common Language Infrastructure's (CLI's) object-oriented languages (and object-oriented programming style in F# programs), F# sometimes discriminates between not only functions and values, but even functions in the object-oriented and functional style.
F# uses very similar syntax for two things: the "classical" CLI methods that take an argument list (and also support overloading and optional parameters) versus F#'s own favorite function type FSharpFunc, which always takes one parameter but supports currying and may take multiple parameters via tuples. But the semantics of these two can be different.
The last line of the question tries to pass a function with tupled input to implement a method that takes two arguments the way a method in C# or VB.NET takes them: a CLI method's argument list. Directly assigning an F#-style first-class function won't work here, and nether would a single tuple argument; the compiler insists to get every argument explicitly. If you write the implementation with its complete method argument list, it will work. For example:
member this.hit (arg1, arg2) = transformHitFunction sh.hit t (arg1, arg2)
Another solution would be to declare hit as:
abstract member hit: (Point * Vector -> (Texture*float*Vector) option)
(Note the parentheses!) Now it's a property that contains a first-class function; you can implement it by returning such a function, but the type of the member subtly changed.
The latter is why even implementing the original interface as a single-argument function, e.g. like this:
member this.hit a = transformHitFunction sh.hit t a // error
will not work. More precisely, The compiler will refuse to see a as a tuple. The same issue applies to
member this.hit ((arg1, arg2)) = transformHitFunction sh.hit t (arg1, arg2) // error
What's wrong now? The outer parentheses define the argument list, but the inner parentheses use a tuple pattern to decompose a single argument! So the argument list still has only one argument, and compilation fails. The outermost parentheses and commas when writing methods are a different feature than the tuples used elsewhere, even though the compiler translates between the two in some cases.
At the moment, your transformedShape.hit is a non-indexed property. When invoked, it returns a function that you need to provide with a Point*Vector tuple, and you'll get the result you want. You'll be able to see that better if you add a helper binding: Hover over f here:
type transformedShape (sh:Shape, t:Transformation) =
interface Shape with
member this.hit =
let f = transformHitFunction sh.hit t
f
As others have remarked already, all you need to do is spell out the arguments explicitly, and you're good:
type transformedShape2 (sh:Shape, t:Transformation) =
interface Shape with
member this.hit(p, v) = transformHitFunction sh.hit t (p, v)