Missing type parameters for generic type "Callable" - callback

What is the correct way to add type hints to the following function?
from typing import Callable
def format_callback(f: Callable) -> Callable:
"""Function to wrap a function to use as a click callback.
Taken from https://stackoverflow.com/a/42110044/8056572
"""
return lambda _, __, x: f(x)
Now mypy is complaining with Missing type parameters for generic type "Callable"
The code needs to be compatible with both Python 3.9 and 3.10. I can use typing_extensions if needed.
Edit:
The following passes mypy but has too many Any's for my taste. Is there a better way?
from typing import Any
from typing import Callable
import click
def format_callback(f: Callable[[Any], Any]) -> Callable[[click.Context, dict[str, Any], Any], Any]:
"""Function to wrap a function to use as a click callback.
Taken from https://stackoverflow.com/a/42110044/8056572
"""
return lambda _, __, x: f(x)

Without looking at click, the immediate fix you can do is to provide type variables that match f(x):
from typing import Any, Callable, TypeVar
import click
ArgT = TypeVar("ArgT")
ReturnT = TypeVar("ReturnT")
def format_callback(
f: Callable[[ArgT], ReturnT]
) -> Callable[[click.Context, dict[str, Any], ArgT], ReturnT]:
return lambda _, __, x: f(x)
This will guard you against bad typing in the internal body of format_callback.
A brief scan of click seems to indicate that you want to pass the return value of format_callback to one of the following class constructors or their subclass constructors:
click.core.Parameter.__init__::callback:
callback: t.Optional[t.Callable[[Context, "Parameter", t.Any], t.Any]]
click.core.Command.__init__::callback
callback: t.Optional[t.Callable[..., t.Any]]
Now, based on the other answer that you linked to, which passes the keyword argument callback to #click.argument and #click.option, it seems like you'd actually want to use click.core.Parameter.__init__, because the decorators' fallback classes are ArgumentClass = click.core.Argument and OptionClass = click.core.Option, respectively, which are both subclasses of click.Core.Parameter. This means that the second argument to the return Callable type cannot be dict[str, Any], according to the type annotation for click.core.Parameter.__init__::callback. Then, you really should have this instead:
def format_callback(
f: Callable[[ArgT], ReturnT]
) -> Callable[[click.Context, click.Parameter, ArgT], ReturnT]:
return lambda _, __, x: f(x)
Since you're discarding click.Context and click.Parameter as _, __ in your lambda, providing these are only for documentation purposes, of course. They could easily be object, object instead.

Related

Partials functions as parameters and function parameter annotation

Suppose we have a function that receives a callable as parameter:
def foo(bar: Callable[[Any], Any]) -> None:
pass
According the annotation the callable we must pass as argument should receive only one argument and return anything, just like:
def third_party_callable(x: Any) -> None:
pass
But we have our own callable that receives 2 arguments so we decide to use functools.partial to match the foo signature:
def my_custom_callable(y: Any, x: Any) -> None:
pass
my_callable = partial(my_custom_callable, 'my_custom_y')
# And then ...
foo(my_callable)
This will result in a linter warning stating that: Expected type '(Any) -> None', got 'partial[None]' instead
Question 1: Is there some way to do this and not getting the linter warning?
Question 2: Should the returned type of partial be Callable[[...], ...]?
Question 3: If what I want is not a desired behavior could someone explain why?

What's wrong when use mypy to check type

from typing import Dict, List, Any, AnyStr, TypeVar
def abc(xyz: str) -> Dict[AnyStr, Any]:
return {"abc": 1}
And I use mypy to check this file. It's giving an Error.
Below is the Error message
"Dict entry 0 has incompatible type "str": "int"; expected "bytes":
"Any""
But I don't know why
The issue is that AnyStr is actually an alias for a typevar. This means your program is actually exactly equivalent to writing:
from typing import Dict, Any, AnyStr, TypeVar
T = TypeVar('T', str, bytes)
def abc(xyz: str) -> Dict[T, Any]:
return {"abc": 1}
This, however, presents us with a problem: how is mypy supposed to infer which of the two possible alternatives you wanted for T?
There are three possible fixes. You can either...
Find some way of using AnyStr at least two or more times within your type signature. For example, perhaps you decide this is really more what you meant?
def abc(xyz: AnyStr) -> Dict[AnyStr, Any]:
# etc
Use Union[str, bytes] instead of AnyStr:
from typing import Union, Dict, Any
def abc(xyz: str) -> Dict[Union[str, bytes], Any]:
# etc
If the type signature is starting to get uncomfortably long, you can shorten it by using a type alias:
from typing import Union, Dict, Any
# Note: this isn't a great type alias name, but whatever
UnionStr = Union[str, bytes]
def abc(xyz: str) -> Dict[UnionStr, Any]:
# etc

Scala understanding signatures

In Scala (Play Framework), I can't understand the type signatures even though I read through all the symbols in here.
For example:
/* matches if a == b */
def equalTo[T](t: => T): BeEqualTo
class BeEqualTo(t: => Any)
extends BeTypedEqualTo[Any]
What on earth do these type signatures even mean?
For example, what exactly is "a" and "b" in the documentation? All I see is "t". Is equalTo a function that takes in a function which returns a value of generic type T?
Another example...
Say I have this line...
status(home).must(equalTo(OK))
According to the IDE, OK is a pattern or symbol of type Int. So OK is an Int? You can monkey-patch an int to give it a function like "must", but how can an Int go inside of "equalTo", a function that takes in a function?
Now some of the type signatures make sense. For example...
def status(of: Future[Result])(implicit timeout: Timeout): Int
^ This is a curried function that takes in a future that returns something of type Result and that sucks in an implicit parameter of type Timeout from somewhere (Scala magic) and returns an Int. "home" is of type "Future[Result]", so it fits inside "status".
But other stuff...
contentType(home).must(beSome.which(_ == "text/html"))
^ My IDE says that beSome is of type
def beSome[T](check: ValueCheck[T]): SomeCheckedMatcher[T]
^ So "beSome" is a function. If that is the case, then how on earth can I appeand ".which()", another function, to it as if it were an object?
Another example...
def which[R : AsResult](f: (U) => R): OptionLikeCheckedMatcher[F, Nothing, U]
^ How on earth do you read these type signatures? "which" takes in "f: (U)", or a variable which we call "f" of type "(U)"? Why the unnecessary parenthesis around the "(U)"? Can't you just say "f: U" and make U be a String if R is a String?
"must" is like this...
def must(m: => Matcher[T]): MatchResult[T]
^ So "must" takes in a function that returns a Matcher. But "which" is passing in an OptionLikeCheckedMatcher. "m: => Matcher[T]" is a function that takes in an "m" and returns a "Matcher[T]". How is that the same as an object of type OptionLikeCheckedMatcher?
Can someone provide a definitive guide as to how to read Scala type signatures?
I think you're getting a little confused with the distinction between a function-valued parameter and a call-by-name parameter. They are similar but not quite the same. In the first example you posted, def equalTo[T](t: => T): BeEqualTo, the : => notation means that the parameter is not evaluated immediately (as it would be if it were declared with just a colon), but every time it is used in an expression. This link might clarify things a bit for you: https://tpolecat.github.io/2014/06/26/call-by-name.html
In another example you posted, def which[R: AsResult](f: (U) => R): OptionCheckedLikeMatcher[F, Nothing, U] takes in a function parameter called f, which accepts a U as its argument and returns a T. The brackets around the U are indeed unnecessary, but they would have been necessary if (for example) f were a function taking 2 parameters of type U - we would then have written it as f: (U, U) => R. The return value of which will be an OptionCheckedMatcher. Assuming that this is a subtype of Matcher (which seems sensible), this is the value which is passed to must.

Compose function with ONLY implicit parameter argument

Method with ONLY implicit parameter
scala> def test1 (implicit i:Int )= Option(i)
test1: (implicit i: Int)Option[Int]
In trying to convert test1 into a function as shown below throws following error. I must be missing something obvious here?
scala> def test2(implicit i:Int) = test1 _
<console>:8: error: type mismatch;
found : Option[Int]
required: ? => ?
def test2(implicit i:Int) = test1 _
When using implicit parameter you have one in the scope. Like this:
object test{
implicit var i = 10
def fun(implicit i: Int) = println(i)
}
test.fun
And if you want to make an option isntance of something you should use Some()
Because you have an implicit Int in scope when defining test2, it's applied to test1, so you really end up with test1(i) _, which makes no sense to the compiler.
If it did compile, test2 would return the same function corresponding to test1 independent of the argument. If that's what you actually want, you can fix it by being more explicit: test1(_: Int) or { x: Int => test1(x) } instead of test1 _.
There are several different possible answers here depending on what it is exactly you are trying to achieve. If you meant test2 to be a behaviorally identical method to test1 then there is no need to use the underscore (and indeed you cannot!).
def test3(implicit i: Int) = test1(i)
If you meant test2 to be a behaviorally identical function to test1, then you need to do the following:
val test4 = test1(_: Int)
Note that I am using a val instead of a def here, because when using the underscore, I am turning test1 which was originally a method into an instance of a class (specifically one which extends Function2, i.e. a function). Also note that test4 no longer has implicit parameters as it is a function (and to the best of my knowledge functions cannot have implicit parameters) whereas test1 was a method.
Why you need to do this as opposed to just test1 _ turns out to be pretty complex...
TLDR: Scala is finnicky about the difference between foo(_) and foo _, methods are different from functions, and implicit resolution takes higher precedence over eta-expansion
Turning a method into a function object by way of an anonymous function in Scala is known as "eta-expansion." In general eta-expansion comes from the lambda calculus and refers to turning a term into its equivalent anonymous function (e.g. thing becomes x => thing(x)) or in the language of the lambda calculus, adding an abstraction over a term.
Note that without eta-expansion, programming in Scala would be absolutely torturous because methods are not functions. Therefore they cannot be directly passed as arguments to other methods or other functions. Scala tries to convert methods into functions via eta-expansion whenever possible, so as to give the appearance that you can pass methods to other methods.
So why doesn't the following work?
val test5 = test1 _ // error: could not find implicit value for parameter i: Int
Well let's see what would happen if we tried test4 without a type signature.
val test6 = test1(_) // error: missing parameter type for expanded function ((x$1) => test1(x$1))
Ah ha, different errors!
This is because test1(_) and test1 _ are subtly different things. test1(_) is partially applying the method test1 while test1 _ is a direction to perform eta-expansion on test1 until it is fully applied.
For example,
math.pow(_) // error: not enough arguments for method pow: (x: Double, y: Double)Double.
math.pow _ // this is fine
But what about the case where we have just one argument? What's the difference between partial application and eta-expansion of just one abstraction? Not really all that much. One way to view it is that eta-expansion is one way of implementing partial application. Indeed the Scala Spec seems to be silent on the difference between foo(_) and foo _ for a single parameter method. The only difference that matters for us is that the expansion that occurs in foo(_) always "binds" tighter than foo _, indeed it seems to bind as closely as method application does.
That's why test1(_) gives us an error about types. Partial application is treated analogously to normal method application. Because normal method application must always take place before implicit resolution, otherwise we could never replace an implicit value with our own value, partial application takes place before implicit resolution. It just so happens that the way Scala implements partial application is via eta-expansion and therefore we get the expansion into the anonymous function which then complains about the lack of a type for its argument.
My reading of Section 6.26 of the Scala Spec suggests that there is an ordering to the resolution of various conversions. In particular it seems to list resolving implicits as coming before eta-expansion. Indeed, for fully-applied eta-expansion, it would seem that this would necessarily be the case since Scala functions cannot have implicit parameters (only its methods can).
Hence, in the case of test5 as #Alexey says, when we are explicitly telling Scala to eta-expand test1 with test1 _, the implicit resolution takes place first, and then eta-expansion tries to take place, which fails because after implicit resolution Scala's typechecker realizes we have an Option not a method.
So that's why we need test1(_) over test1 _. The final type annotation test1(_: Int) is needed because Scala's type inference isn't robust enough to determine that after eta-expanding test1, the only possible type you can give to the anonymous function is the same as the type signature for the method. In fact, if we gave the type system enough hints, we can get away with test1(_).
val test7: Int => Option[Int] = test1(_) // this works
val test8: Int => Option[Int] = test1 _ // this still doesn't

Demystifying a function definition

I am new to Scala, and I hope this question is not too basic. I couldn't find the answer to this question on the web (which might be because I don't know the relevant keywords).
I am trying to understand the following definition:
def functionName[T <: AnyRef](name: Symbol)(range: String*)(f: T => String)(implicit tag: ClassTag[T]): DiscreteAttribute[T] = {
val r = ....
new anotherFunctionName[T](name.toString, f, Some(r))
}
First , why is it defined as def functionName[...](...)(...)(...)(...)? Can't we define it as def functionName[...](..., ..., ..., ...)?
Second, how does range: String* from range: String?
Third, would it be a problem if implicit tag: ClassTag[T] did not exist?
First , why is it defined as def functionName...(...)(...)(...)? Can't we define it as def functionName[...](..., ..., ..., ...)?
One good reason to use currying is to support type inference. Consider these two functions:
def pred1[A](x: A, f: A => Boolean): Boolean = f(x)
def pred2[A](x: A)(f: A => Boolean): Boolean = f(x)
Since type information flows from left to right if you try to call pred1 like this:
pred1(1, x => x > 0)
type of the x => x > 0 cannot be determined yet and you'll get an error:
<console>:22: error: missing parameter type
pred1(1, x => x > 0)
^
To make it work you have to specify argument type of the anonymous function:
pred1(1, (x: Int) => x > 0)
pred2 from the other hand can be used without specifying argument type:
pred2(1)(x => x > 0)
or simply:
pred2(1)(_ > 0)
Second, how does range: String* from range: String?
It is a syntax for defining Repeated Parameters a.k.a varargs. Ignoring other differences it can be used only on the last position and is available as a scala.Seq (here scala.Seq[String]). Typical usage is apply method of the collections types which allows for syntax like SomeDummyCollection(1, 2, 3). For more see:
What does `:_*` (colon underscore star) do in Scala?
Scala variadic functions and Seq
Is there a difference in Scala between Seq[T] and T*?
Third, would it be a problem if implicit tag: ClassTag[T] did not exist?
As already stated by Aivean it shouldn't be the case here. ClassTags are automatically generated by the compiler and should be accessible as long as the class exists. In general case if implicit argument cannot be accessed you'll get an error:
scala> import scala.concurrent._
import scala.concurrent._
scala> val answer: Future[Int] = Future(42)
<console>:13: error: Cannot find an implicit ExecutionContext. You might pass
an (implicit ec: ExecutionContext) parameter to your method
or import scala.concurrent.ExecutionContext.Implicits.global.
val answer: Future[Int] = Future(42)
Multiple argument lists: this is called "currying", and enables you to call a function with only some of the arguments, yielding a function that takes the rest of the arguments and produces the result type (partial function application). Here is a link to Scala documentation that gives an example of using this. Further, any implicit arguments to a function must be specified together in one argument list, coming after any other argument lists. While defining functions this way is not necessary (apart from any implicit arguments), this style of function definition can sometimes make it clearer how the function is expected to be used, and/or make the syntax for partial application look more natural (f(x) rather than f(x, _)).
Arguments with an asterisk: "varargs". This syntax denotes that rather than a single argument being expected, a variable number of arguments can be passed in, which will be handled as (in this case) a Seq[String]. It is the equivalent of specifying (String... range) in Java.
the implicit ClassTag: this is often needed to ensure proper typing of the function result, where the type (T here) cannot be determined at compile time. Since Scala runs on the JVM, which does not retain type information beyond compile time, this is a work-around used in Scala to ensure information about the type(s) involved is still available at runtime.
Check currying:Methods may define multiple parameter lists. When a method is called with a fewer number of parameter lists, then this will yield a function taking the missing parameter lists as its arguments.
range:String* is the syntax for varargs
implicit TypeTag parameter in Scala is the alternative for Class<T> clazzparameter in Java. It will be always available if your class is defined in scope. Read more about type tags.