In C/C++ I'd normally do callbacks with a plain function pointer, maybe passing a void* userdata parameter too. Something like this:
typedef void (*Callback)();
class Processor
{
public:
void setCallback(Callback c)
{
mCallback = c;
}
void processEvents()
{
//...
mCallback();
}
private:
Callback mCallback;
};
What is the idiomatic way of doing this in Rust? Specifically, what types should my setCallback() function take, and what type should mCallback be? Should it take an Fn? Maybe FnMut? Do I save it Boxed? An example would be amazing.
Short answer: For maximum flexibility, you can store the callback as a boxed FnMut object, with the callback setter generic on callback type. The code for this is shown in the last example in the answer. For a more detailed explanation, read on.
"Function pointers": callbacks as fn
The closest equivalent of the C++ code in the question would be declaring callback as a fn type. fn encapsulates functions defined by the fn keyword, much like C++'s function pointers:
type Callback = fn();
struct Processor {
callback: Callback,
}
impl Processor {
fn set_callback(&mut self, c: Callback) {
self.callback = c;
}
fn process_events(&self) {
(self.callback)();
}
}
fn simple_callback() {
println!("hello world!");
}
fn main() {
let p = Processor {
callback: simple_callback,
};
p.process_events(); // hello world!
}
This code could be extended to include an Option<Box<Any>> to hold the "user data" associated with the function. Even so, it would not be idiomatic Rust. The Rust way to associate data with a function is to capture it in an anonymous closure, just like in modern C++. Since closures are not fn, set_callback will need to accept other kinds of function objects.
Callbacks as generic function objects
In both Rust and C++ closures with the same call signature come in different sizes to accommodate the different values they might capture. Additionally, each closure definition generates a unique anonymous type for the closure's value. Due to these constraints, the struct cannot name the type of its callback field, nor can it use an alias.
One way to embed a closure in the struct field without referring to a concrete type is by making the struct generic. The struct will automatically adapt its size and the type of callback for the concrete function or closure you pass to it:
struct Processor<CB>
where
CB: FnMut(),
{
callback: CB,
}
impl<CB> Processor<CB>
where
CB: FnMut(),
{
fn set_callback(&mut self, c: CB) {
self.callback = c;
}
fn process_events(&mut self) {
(self.callback)();
}
}
fn main() {
let s = "world!".to_string();
let callback = || println!("hello {}", s);
let mut p = Processor { callback };
p.process_events();
}
As before, set_callback() will accept functions defined with fn, but this one will also accept closures as || println!("hello world!"), as well as closures that capture values, such as || println!("{}", somevar). Because of this the processor doesn't need userdata to accompany the callback; the closure provided by the caller of set_callback will automatically capture the data it needs from its environment and have it available when invoked.
But what's the deal with the FnMut, why not just Fn? Since closures hold captured values, Rust's usual mutation rules must apply when calling the closure. Depending on what the closures do with the values they hold, they are grouped in three families, each marked with a trait:
Fn are closures that only read data, and may be safely called multiple times, possibly from multiple threads. Both above closures are Fn.
FnMut are closures that modify data, e.g. by writing to a captured mut variable. They may also be called multiple times, but not in parallel. (Calling a FnMut closure from multiple threads would lead to a data race, so it can only be done with the protection of a mutex.) The closure object must be declared mutable by the caller.
FnOnce are closures that consume some of the data they capture, e.g. by passing a captured value to a function that takes it by value. As the name implies, these may be called only once, and the caller must own them.
Somewhat counter-intuitively, when specifying a trait bound for the type of an object that accepts a closure, FnOnce is actually the most permissive one. Declaring that a generic callback type must satisfy the FnOnce trait means that it will accept literally any closure. But that comes with a price: it means the holder is only allowed to call it once. Since process_events() may opt to invoke the callback multiple times, and as the method itself may be called more than once, the next most permissive bound is FnMut. Note that we had to mark process_events as mutating self.
Non-generic callbacks: function trait objects
Even though the generic implementation of the callback is extremely efficient, it has serious interface limitations. It requires each Processor instance to be parameterized with a concrete callback type, which means that a single Processor can only deal with a single callback type. Given that each closure has a distinct type, the generic Processor cannot handle proc.set_callback(|| println!("hello")) followed by proc.set_callback(|| println!("world")). Extending the struct to support two callbacks fields would require the whole struct to be parameterized to two types, which would quickly become unwieldy as the number of callbacks grows. Adding more type parameters wouldn't work if the number of callbacks needed to be dynamic, e.g. to implement an add_callback function that maintains a vector of different callbacks.
To remove the type parameter, we can take advantage of trait objects, the feature of Rust that allows automatic creation of dynamic interfaces based on traits. This is sometimes referred to as type erasure and is a popular technique in C++[1][2], not to be confused with Java and FP languages' somewhat different use of the term. Readers familiar with C++ will recognize the distinction between a closure that implements Fn and an Fn trait object as equivalent to the distinction between general function objects and std::function values in C++.
A trait object is created by borrowing an object with the & operator and casting or coercing it to a reference to the specific trait. In this case, since Processor needs to own the callback object, we cannot use borrowing, but must store the callback in a heap-allocated Box<dyn Trait> (the Rust equivalent of std::unique_ptr), which is functionally equivalent to a trait object.
If Processor stores Box<dyn FnMut()>, it no longer needs to be generic, but the set_callback method now accepts a generic c via an impl Trait argument. As such, it can accept any kind of callable, including closures with state, and properly box it before storing it in the Processor. The generic argument to set_callback doesn't limit what kind of callback the processor accepts, as the type of the accepted callback is decoupled from the type stored in the Processor struct.
struct Processor {
callback: Box<dyn FnMut()>,
}
impl Processor {
fn set_callback(&mut self, c: impl FnMut() + 'static) {
self.callback = Box::new(c);
}
fn process_events(&mut self) {
(self.callback)();
}
}
fn simple_callback() {
println!("hello");
}
fn main() {
let mut p = Processor {
callback: Box::new(simple_callback),
};
p.process_events();
let s = "world!".to_string();
let callback2 = move || println!("hello {}", s);
p.set_callback(callback2);
p.process_events();
}
Lifetime of references inside boxed closures
The 'static lifetime bound on the type of the c argument accepted by set_callback is a simple way to convince the compiler that references contained in c, which might be a closure that refers to its environment, only refer to global values and will therefore remain valid throughout the use of the callback. But the static bound is also very heavy-handed: while it accepts closures that own objects just fine (which we've ensured above by making the closure move), it rejects closures that refer to local environment, even when they only refer to values that outlive the processor and would in fact be safe.
As we only need the callbacks alive as long as the processor is alive, we should try to tie their lifetime to that of the processor, which is a less strict bound than 'static. But if we just remove the 'static lifetime bound from set_callback, it no longer compiles. This is because set_callback creates a new box and assigns it to the callback field defined as Box<dyn FnMut()>. Since the definition doesn't specify a lifetime for the boxed trait object, 'static is implied, and the assignment would effectively widen the lifetime (from an unnamed arbitrary lifetime of the callback to 'static), which is disallowed. The fix is to provide an explicit lifetime for the processor and tie that lifetime to both the references in the box and the references in the callback received by set_callback:
struct Processor<'a> {
callback: Box<dyn FnMut() + 'a>,
}
impl<'a> Processor<'a> {
fn set_callback(&mut self, c: impl FnMut() + 'a) {
self.callback = Box::new(c);
}
// ...
}
With these lifetimes being made explicit, it is no longer necessary to use 'static. The closure can now refer to the local s object, i.e. no longer has to be move, provided that the definition of s is placed before the definition of p to ensure that the string outlives the processor.
If you are willing to deal with lifetime and cannot afford the heap allocation, then here is an implementation that uses reference to implement the callback:
use core::ffi::c_void;
use core::mem::transmute;
use core::ptr::null_mut;
use core::marker::PhantomData;
/// ErasedFnPointer can either points to a free function or associated one that
/// `&mut self`
struct ErasedFnPointer<'a, T, Ret> {
struct_pointer: *mut c_void,
fp: *const (),
// The `phantom_*` field is used so that the compiler won't complain about
// unused generic parameter.
phantom_sp: PhantomData<&'a ()>,
phantom_fp: PhantomData<fn(T) -> Ret>,
}
impl<'a, T, Ret> Copy for ErasedFnPointer<'a, T, Ret> {}
impl<'a, T, Ret> Clone for ErasedFnPointer<'a, T, Ret> {
fn clone(&self) -> Self {
*self
}
}
impl<'a, T, Ret> ErasedFnPointer<'a, T, Ret> {
pub fn from_associated<S>(struct_pointer: &'a mut S, fp: fn(&mut S, T) -> Ret)
-> ErasedFnPointer<'a, T, Ret>
{
ErasedFnPointer {
struct_pointer: struct_pointer as *mut _ as *mut c_void,
fp: fp as *const (),
phantom_sp: PhantomData,
phantom_fp: PhantomData,
}
}
pub fn from_free(fp: fn(T) -> Ret) -> ErasedFnPointer<'static, T, Ret> {
ErasedFnPointer {
struct_pointer: null_mut(),
fp: fp as *const (),
phantom_sp: PhantomData,
phantom_fp: PhantomData,
}
}
pub fn call(&self, param: T) -> Ret {
if self.struct_pointer.is_null() {
let fp = unsafe { transmute::<_, fn(T) -> Ret>(self.fp) };
fp(param)
} else {
let fp = unsafe { transmute::<_, fn(*mut c_void, T) -> Ret>(self.fp) };
fp(self.struct_pointer, param)
}
}
}
fn main() {
let erased_ptr = ErasedFnPointer::from_free(|x| {
println!("Hello, {}", x);
x
});
erased_ptr.call(2333);
println!("size_of_val(erased_ptr) = {}", core::mem::size_of_val(&erased_ptr));
ErasedFnPointer::from_associated(
&mut Test { x: 1},
Test::f
).call(1);
let mut x = None;
ErasedFnPointer::from_associated(&mut x, |x, param| {
*x = Some(param);
println!("{:#?}", x);
}).call(1);
}
struct Test {
x: i32
}
impl Test {
fn f(&mut self, y: i32) -> i32 {
let z = self.x + y;
println!("Hello from Test, {}", z);
z
}
}
For the type of scenario where a callback is used, you should consider the Promise alternative.
It is much easier to use than callbacks because it avoids nesting (callback hell).
Consider this:
fn main() {
let fut = do_async(&Calculation{ value: 12 });
let resp = fut().unwrap(); // call fut() to wait for the respbnse
println!("{}", resp);
}
For any calculation:
you define an struct whose fields are its inputs (name is not important).
you implement the Runner trait:
you choose what to return
you write the code of run(), which will be exected by a separate thread
struct Calculation { // <---- choose: name
value: i32 // <----- choose: inputs for your async work
}
impl Runner for Calculation {
type ReturnType = i32; // <--- choose: calculation return type
fn run(&self) -> Option<Self::ReturnType> { // <-- implement: code executed by a thread
println!("async calculation starts");
thread::sleep(Duration::from_millis(3000));
return Some(self.value * 2);
}
}
Finaly, this is the "magic":
trait Runner: Send + Sync {
type ReturnType: Send; // associated type
fn run(&self) -> Option<Self::ReturnType>;
}
fn do_async<TIn: Runner>(f: &'static TIn) -> impl FnOnce()-> Option<TIn::ReturnType> {
let (sender, receiver) = channel::<Option<TIn::ReturnType>>();
let hand = thread::spawn(move || {
sender.send(f.run()).unwrap();
});
let f = move || -> Option<TIn::ReturnType> {
let res = receiver.recv().unwrap();
hand.join().unwrap();
return res;
};
return f;
}
A simpler version of https://stackoverflow.com/a/70943671/286335 just for closures.
fn main() {
let n = 2;
let fut = do_async(move || {
thread::sleep(Duration::from_millis(3000));
return n * 1234;
});
let resp = fut(); // call fut() to wait for the response
println!("{}", resp);
} // ()
where do_async is
fn do_async<TOut, TFun>(foo: TFun) -> (impl FnOnce() -> TOut)
where
TOut: Send + Sync + 'static,
TFun: FnOnce() -> TOut + Send + Sync + 'static,
{
let (sender, receiver) = channel::<TOut>();
let hand = thread::spawn(move || {
sender.send(foo()).unwrap();
});
let f = move || -> TOut {
let res = receiver.recv().unwrap();
hand.join().unwrap();
return res;
};
return f;
} // ()
If we want to generalize applying #[derive(...)] to a struct, in some cases it would be useful to wrap this in a macro.
non-working example:
my_traits!(
pub struct MyType(u32),
MyType
);
Where my_traits could prefix the first argument with #[derive(...)], and use the second argument to declare impl SomeTrait for $t {...}.
Declaring implementations works without any problems, however I didn't manage to find a way to use a macro to prefix the struct declaration with attributes.
See this question for an example of what this could be used for:Possible to derive attributes *after* a struct declaration?
Putting #[derive(...)] into the macro seems to work fine:
#[derive(Eq,PartialEq,Debug)]
struct Foo(u32);
macro_rules! my_eq(
($name:ident) => {
#[derive(Eq,PartialEq,Debug)]
struct $name(u32);
};
);
my_eq!(Bar);
fn main() {
assert_eq!(Foo(3), Foo(3));
assert!(Foo(3) != Foo(4));
assert_eq!(Bar(3), Bar(3));
assert!(Bar(3) != Bar(4));
}
Playground link
Or if you want to pass the whole struct in:
macro_rules! my_eq(
($name:item) => {
#[derive(Eq,PartialEq,Debug)]
$name
};
);
my_eq!(struct Bar(u32););
Playground
Note that the macro takes an entire item, so the semicolon inside the macro call is needed (Foo{} structs don't need it, just like when written inline).
This can be done by passing in an item, answer thanks to #j_ey on IRC.
macro_rules! ideas {
($ty: item, $id: ident) => {
#[derive(Debug)]
$ty
impl $id {
fn doit(){}
}
}
}
ideas!(pub struct Foo(u32);, Foo);
fn main() {
let f = Foo(1);
println!("{:?}", f);
}
I want to write a macro like this:
macro_rules! a {
( $n:ident, $t:ty ) => {
struct $n {
x: $t
}
}
}
But $t should implement Add, Sub and Mul traits. How can I check it at compile-time?
First, solve the problem without macros. One solution is to create undocumented private functions that will fail compilation if your conditions aren't met:
struct MyType {
age: i32,
name: String,
}
const _: () = {
fn assert_send<T: Send>() {}
fn assert_sync<T: Sync>() {}
// RFC 2056
fn assert_all() {
assert_send::<MyType>();
assert_sync::<MyType>();
}
};
Then, modify the simple solution to use macros:
macro_rules! example {
($name:ident, $field:ty) => {
struct $name {
x: $field,
}
const _: () = {
fn assert_add<T: std::ops::Add<$field, Output = $field>>() {}
fn assert_mul<T: std::ops::Mul<$field, Output = $field>>() {}
// RFC 2056
fn assert_all() {
assert_add::<$field>();
assert_mul::<$field>();
}
};
};
}
example!(Moo, u8);
example!(Woof, bool);
In both cases, we create a dummy const value to scope the functions and their calls, avoiding name clashes.
I would then trust in the optimizer to remove the code at compile time, so I wouldn't expect any additional bloat.
Major thanks to Chris Morgan for providing a better version of this that supports non-object-safe traits.
It's worth highlighting RFC 2056 which will allow for "trivial" constraints in where clauses. Once implemented, clauses like this would be accepted:
impl Foo for Bar
where
i32: Iterator,
{}
This exact behavior has changed multiple times during Rust's history and RFC 2056 pins it down. To keep the behavior we want in this case, we need to call the assertion functions from another function which has no constraints (and thus must always be true).
How can we determine if protocol conforms to a specific subtype based on user provided instances, if it's not possible this way, any alternate solutions.
API
protocol Super {}
protocol Sub: Super {} //inherited by Super protocol
class Type1: Super {} //conforms to super protocol
class Type2: Type1, Sub {} //conforms to sub protocol
inside another API class
func store(closures: [() -> Super]) {
self.closures = closures
}
when it's time to call
func go() {
for closure in closures {
var instance = closure()
if instance is Super {
//do something - system will behave differently
} else { //it's Sub
//do something else - system will behave differently
}
}
}
users of the api
class Imp1: Type1 {}
class Imp2: Type2 {}
var closures: [() -> Super] = [ { Imp1() }, { Imp2() } ]
store(closures)
my current workaround within API
func go() {
for closure in closures {
var instance = closure()
var behavior = 0
if instance as? Type2 != nil { //not so cool, should be through protocols
behavior = 1 //instead of implementations
}
if behavior == 0 { //do something within the api,
} else { //do something else within the api
}
//instance overriden method will be called
//but not important here to show, polymorphism works in here
//more concerned how the api can do something different based on the types
}
}
You are jumping through a lot of hoops to manually recreate dynamic dispatch, i.e. one of the purposes of protocols and classes. Try actually using real runtime polymorphism to solve your problem.
Take this code:
if instance is Super {
//do something
} else { //it's Sub
//do something else
}
What you are saying is, if it’s a superclass, run the superclass method, else, run the subclass. This is a bit inverted – normally when you are a subclass you want to run the subclass code not the other way around. But assuming you turn it around to the more conventional order, you are essentially describing calling a protocol’s method and expecting the appropriate implementation to get called:
(the closures aren’t really related to the question in hand so ignoring them for now)
protocol Super { func doThing() }
protocol Sub: Super { } // super is actually a bit redundant here
class Type1: Super {
func doThing() {
println("I did a super thing!")
}
}
class Type2: Sub {
func doThing() {
println("I did a sub thing!")
}
}
func doSomething(s: Super) {
s.doThing()
}
let c: [Super] = [Type1(), Type2()]
for t in c {
doSomething(t)
}
// prints “I did a super thing!”, then “I did a sub thing!"
Alternatives to consider: eliminate Sub, and have Type2 inherit from Type1. Or, since there’s no class inheritance here, you could use structs rather than classes.
Almost any time you find yourself wanting to use is?, you probably meant to use an enum. Enums allow you use to the equivalent of is? without feeling bad about it (because it isn't a problem). The reason that is? is bad OO design is that it creates a function that is closed to subtyping, while OOP itself is always open to subtyping (you should think of final as a compiler optimization, not as a fundamental part of types).
Being closed to subtyping is not a problem or a bad thing. It just requires thinking in a functional paradigm rather than an object paradigm. Enums (which are the Swift implementation of a Sum type) are exactly the tool for this, and are very often a better tool than subclassing.
enum Thing {
case Type1(... some data object(s) ...)
case Type2(... some data object(s) ...)
}
Now in go(), instead of an is? check, you switch. Not only is this not a bad thing, it's required and fully type-checked by the compiler.
(Example removes the lazy closures since they're not really part of the question.)
func go(instances: [Thing]) {
for instance in instances {
switch instance {
case Type1(let ...) { ...Type1 behaviors... }
case Type2(let ...) { ...Type2 behaviors... }
}
}
}
If you have some shared behaviors, just pull those out into a function. You're free to let your "data objects" implement certain protocols or be of specific classes if that makes things easier to pass along to shared functions. It's fine if Type2 takes associated data that happens to be a subclass of Type1.
If you come along later and add a Type3, then the compiler will warn you about every switch that fails to consider this. That's why enums are safe while is? is not.
You need objects derived from the Objective-C world to do this:
#objc protocol Super {}
#objc protocol Sub: Super {}
class Parent: NSObject, Super {}
class Child: NSObject, Sub {}
func go( closures: [() -> Super]) {
for closure in closures {
let instance = closure()
if instance is Sub { // check for Sub first, check for Super is always true
//do something
} else {
//do something else
}
}
}
Edit: Version with different method implementations:
protocol Super {
func doSomething()
}
protocol Sub: Super {}
class Parent: Super {
func doSomething() {
// do something
}
}
class Child: Sub {
func doSomething() {
// do something else
}
}
func go( closures: [() -> Super]) {
for closure in closures {
let instance = closure()
instance.doSomething()
}
}
I'd like to implement a kind of thing which can be uniquely identified, besides, it contains no other fields. It 's kinds of like BasicObject in ruby, or Object in java.
I have added a PartialEq trait.
struct Thing;
impl PartialEq for Thing {
fn eq(&self, other: &Thing) -> bool {
unsafe {
self as *const Thing == other as *const Thing
}
}
}
Now, I want to allow that it can be hashed, and the question is how could I implement Hash trait for a unit struct which has no fileds ? Or, should I add some kind of object_id to it ?
You cannot rely on pointer address to determine for sure unicity of unit structs in Rust, for example the following code will print two times the same address:
struct Foo;
fn main() {
let x = Foo;
let y = Foo;
println!("{}", &x as *const _);
println!("{}", &y as *const _);
}
You need to have them occupy at least one byte in memory, for example by giving them a field that you never use :
struct Thing {
x: u8;
}
impl Thing {
fn new() -> Thing {
Thing { x: 0u8 }
}
}
Then, a simple way would be to continue on the same track as you are going, and using the address of your objects in memory as a base value for the Hash :
use std::hash::{hash,Hash};
use std::hash::sip::SipState;
struct Thing;
impl Hash for Thing {
fn hash(&self, state: &mut SipState) {
let addr = self as *const Thing as u64;
addr.hash(state);
}
}
fn main() {
let f = Thing;
println!("{}", hash(&f));
}
Note : casting &Thing to *const Thing (and then to u64) is not unsafe, so you don't need the unsafe{} block in your implementation of PartialEq.