Is it possible to let a macro expand to a struct field? - macros

I would like to do the following, but macros in that position don’t seem to work (I get error: expected `:`, found `!`. How can I pattern-match individual struct members and attach attributes to them based on the match?
use serde_derive::Serialize;
macro_rules! optional_param {
($name:ident : Option<$type:ty>) => { #[serde(skip_serializing_if = "Option::is_none")] pub $name: Option<$ty> };
($name:ident : Vec <$type:ty>) => { #[serde(skip_serializing_if = "Vec::is_empty" )] pub $name: Vec <$ty> };
($name:ident : bool ) => { #[serde(skip_serializing_if = "bool::not" )] pub $name: bool };
}
macro_rules! impl_extra {
( $name:ident { $( $param:ident : $type:ty ),* $(,)* } ) => (
#[derive(Default,Debug,Serialize)]
pub struct $name {
$( optional_param!($param : $type), )*
}
);
}
impl_extra!(MyStruct { member: Option<String> });
Link to the playground

Indeed, macro invocations are not valid in the middle of a struct definition. However, we can use metavariables there. The trick is to parse the parameters incrementally, building the tokens for the field definitions along the way, and when there's no more input to process, emit a struct definition with the field definitions coming from a metavariable.
As a first step, let's see what a macro that doesn't handle field types specifically looks like:
macro_rules! impl_extra {
( # $name:ident { } -> ($($result:tt)*) ) => (
#[derive(Default, Debug, Serialize)]
pub struct $name {
$($result)*
}
);
( # $name:ident { $param:ident : $type:ty, $($rest:tt)* } -> ($($result:tt)*) ) => (
impl_extra!(# $name { $($rest)* } -> (
$($result)*
pub $param : $type,
));
);
( $name:ident { $( $param:ident : $type:ty ),* $(,)* } ) => (
impl_extra!(# $name { $($param : $type,)* } -> ());
);
}
The only thing this macro does is add pub on each field and define a pub struct with a #[derive] attribute. The first rule handles the terminal case, i.e. when there are no more fields to process. The second rule handles the recursive case, and the third rule handles the macro's "public" syntax and transforms it into the "processing" syntax.
Note that I'm using an # as the initial token for internal rules to distinguish them from "public" rules. If this macro is not meant to be exported to other crates, then you could also move the internal rules to a different macro. If the macro is exported though, then the separate macro for the internal rules might have to be exported too.
Now, let's handle the various field types:
macro_rules! impl_extra {
( # $name:ident { } -> ($($result:tt)*) ) => (
#[derive(Default, Debug, Serialize)]
pub struct $name {
$($result)*
}
);
( # $name:ident { $param:ident : Option<$type:ty>, $($rest:tt)* } -> ($($result:tt)*) ) => (
impl_extra!(# $name { $($rest)* } -> (
$($result)*
#[serde(skip_serializing_if = "Option::is_none")]
pub $param : Option<$type>,
));
);
( # $name:ident { $param:ident : Vec<$type:ty>, $($rest:tt)* } -> ($($result:tt)*) ) => (
impl_extra!(# $name { $($rest)* } -> (
$($result)*
#[serde(skip_serializing_if = "Vec::is_empty")]
pub $param : Vec<$type>,
));
);
( # $name:ident { $param:ident : bool, $($rest:tt)* } -> ($($result:tt)*) ) => (
impl_extra!(# $name { $($rest)* } -> (
$($result)*
#[serde(skip_serializing_if = "bool::not")]
pub $param : bool,
));
);
( $name:ident { $( $param:ident : $($type:tt)* ),* $(,)* } ) => (
impl_extra!(# $name { $($param : $($type)*,)* } -> ());
);
}
Note that there's a difference in the last rule: instead of matching on a ty, we now match on a sequence of tt. That's because once the macro has parsed a ty, it can't be broken down, so when we make a recursive macro call, a ty cannot possibly match something like Option<$type:ty>.

Related

How to capture by reference in rust macro

I have a macro to generate match arms:
macro_rules! sort_by {
( $query:ident, $sort_by:expr, { $( $name:pat => $column:path,)+ } ) => {
match $sort_by.column {
$(
$name => if $sort_by.descending {
$query = $query.order_by($column.desc());
} else {
$query = $query.order_by($column.asc());
},
)+
}
}
}
and I want to call it like this:
sort_by!(query, sort_by.unwrap_or(Sort::desc("id")), {
"id" => table::id,
"customerName" => table::customer_name,
});
But I'm getting an error:
sort_by!(query, &sort_by.unwrap_or(Sort::desc("id")), {
^^^^^^^ value moved here in previous iteration of loop
So I have to call it like this:
let sort = sort_by.unwrap_or(Sort::desc("id"));
sort_by!(query, &sort, {
"id" => table::id,
"customerName" => table::customer_name,
});
What should I change to be able to use the expression directly in the macro invocation?
Using a macro is equivalent to substituting the code it expands to into its call site. This means if the macro expansion contains $sort_by multiple times, the code will evaluate the expression you pass in as $sort_by multiple times. If the expression consumes some variable, this will be invalid.
This is in contrast to how function calls work. If you pass an expression to a function, it will be evaluated before calling the function, and only the result is passed to the function.
If this is the source of your problem, you can fix it by assigning $sort_by to a local variable inside your macro expansion, and only access the local variable subsequently:
macro_rules! sort_by {
($query:ident, $sort_by:expr, { $($name:pat => $column:path,)+ }) => {
let sort_by = $sort_by;
match sort_by.column {
$(
$name => if sort_by.descending {
$query = $query.order_by($column.desc());
} else {
$query = $query.order_by($column.asc());
},
)+
}
}
}
(Note that I could not test this, since your example is incomplete.)

How can I use a macro to create an array of function names starting from a collection of function definitions?

I would like to create an array in a macro to transform something like:
let array = create_array!(
fn test() -> i32 { }
fn test1() { }
);
into
let array = [test, test1];
I tried this:
macro_rules! create_array {
() => {
};
(fn $func_name:ident () -> $return_type:ty $block:block $($rest:tt)*) => {
$func_name,
create_array!($($rest)*);
};
(fn $func_name:ident () $block:block $($rest:tt)*) => {
$func_name,
create_array!($($rest)*);
};
}
but it fails with the following error:
error: macro expansion ignores token `,` and any following
--> src/main.rs:11:19
|
11 | $func_name,
| ^
|
note: caused by the macro expansion here; the usage of `create_array!` is likely invalid in expression context
--> src/main.rs:27:18
|
27 | let array = [create_array!(
|
I also tried this:
macro_rules! create_array {
($(fn $func_name:ident () $( -> $return_type:ty )* $block:block)*) => {
[$($func_name),*]
};
}
but it fails with:
error: local ambiguity: multiple parsing options: built-in NTs block ('block') or 1 other option.
--> src/main.rs:22:19
|
22 | fn test() -> i32 { }
|
So how can I create an array in such a case?
The parsing ambiguity of -> vs $:block has been resolved as of Rust 1.20, so the second version you tried will now work as intended.
macro_rules! create_array {
($(fn $func_name:ident () $(-> $return_type:ty)* $block:block)*) => {
[$($func_name),*]
};
}
fn main() {
let test = "TEST";
let test1 = "TEST1";
let array = create_array! {
fn test() -> i32 {}
fn test1() {}
};
println!("{:?}", array);
}

Why does this macro invocation result in an unresolved name?

Here is a simplified version of a macro I am trying to implement for an RPC library I am working on:
#[macro_export]
macro_rules! msgpack_rpc {
(
$(
rpc $name:ident ( $( $arg:ident : $arg_ty:ty ),* ) -> $ret_ty:ty | $err_ty:ty;
)+
) => (
pub trait Service {
$(
fn $name ( &self, $( $arg : $arg_ty ),* ) -> Result<$ret_ty, $err_ty>;
)+
}
pub struct Server;
impl Server {
pub fn listen<S>(handle: &(), address: (), service: S)
-> ::std::io::Result<()>
where S: Service + Send + Sync + 'static {
let service = move |msg: &str| {
let result = match msg {
$(
stringify!($name) => {
service.$name($( $arg ),*)
.map(String::from)
.map_err(String::from)
}
),+,
_ => String::from("method not supported".into()),
};
};
Ok(())
}
}
)
}
msgpack_rpc! {
rpc echo(arg: i64) -> i64 | ();
}
The macro expansion fails to compile with this error:
error: unresolved name `arg` [--explain E0425]
--> <anon>:40:17
|>
40 |> rpc echo(arg: i64) -> i64 | ();
|> ^
<anon>:39:1: 41:2: note: in this expansion of msgpack_rpc! (defined in <anon>)
From reading similar questions, I know that macro_rules sometimes has problems expanding statements. However, I am confused as to why it is having trouble expanding items in this case.
Is there a workaround to fix the expansion?
You don't have any variable named arg in the context in which you call the function. This is the "unresolved arg" the compiler is complaining about.
stringify!($name) => {
$( let $arg = Default::default(); )*
service.$name($( $arg ),*)
.map(String::from)
.map_err(String::from)
}

Fixing "no rules expected the token" macro error

I'm trying to write a macro for destructuring BSON data which looks like this:
let bson: Document = ...;
let (id, hash, name, path, modification_time, size, metadata, commit_data) = bson_destructure! {
get id = from (bson), optional, name ("_id"), as ObjectId;
get hash = from (bson), as String, through (|s| ContentHash::from_str(&s));
get name = from (bson), as String;
get path = from (bson), as Bson, through (PathBuf::from_bson);
get modification_time = from (bson), as UtcDatetime, through (FileTime);
get size = from (bson), as I64, through (|n| n as u64);
get metadata = from (bson), as Document, through (Metadata::from_bson);
get commit_data = from (bson), optional, as Document, through (CommitData::from_bson);
ret (id, hash, name, path, modification_time, size, metadata, commit_data)
};
I've written the following macro (pretty large) for it:
macro_rules! bson_destructure {
// required field
(
#collect req,
[$target:ident, $source:expr, $field:expr, Bson, $f:expr],
[];
$($rest:tt)*
) => {{
let $target = try!(match $source.remove($field) {
Some(v) => $f(v),
None => Err(BsonDestructureError::MissingField {
field_name: $field,
expected: "Bson"
}),
});
bson_destructure!($($rest)*)
}};
(
#collect req,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[];
$($rest:tt)*
) => {{
let $target = try!(match $source.remove($field) {
Some(v) => match v {
::ejdb::bson::Bson::$variant(v) => $f(v),
v => Err(BsonDestructureError::InvalidType {
field_name: $field,
expected: stringify!($variant),
actual: v
})
},
None => Err(BsonDestructureError::MissingField {
field_name: $field,
expected: stringify!($variant)
}),
});
bson_destructure!($($rest)*)
}};
// optional field
(
#collect opt,
[$target:ident, $source:expr, $field:expr, Bson, $f:expr],
[];
$($rest:tt)*
) => {{
let $target = try!(match $source.remove($field) {
Some(v) => $f(v).map(Some),
None => Ok(None),
});
bson_destructure!($($rest)*)
}};
(
#collect opt,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[];
$($rest:tt)*
) => {{
let $target = try!(match $source.remove($field) {
Some(v) => match v {
::ejdb::bson::Bson::$variant(v) => $f(v).map(Some),
v => Err(BsonDestructureError::InvalidType {
field_name: $field,
expected: stringify!($variant),
actual: v
})
},
None => Ok(None),
});
bson_destructure!($($rest)*)
}};
// change variant name
(
#collect $k:tt,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[as $nv:ident, $($word:ident $arg:tt),*];
$($rest:tt)*
) => {
bson_destructure!(
#collect $k,
[$target, $source, $field, $nv, $f],
[$($word $arg),*];
$($rest)*
)
};
// change final mapping function
(
#collect $k:tt,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[through ($nf:expr), $($word:ident $arg:tt),*];
$($rest:tt)*
) => {
bson_destructure!(
#collect $k,
[$target, $source, $field, $variant, $nf],
[$($word $arg),*];
$($rest)*
)
};
// change field name
(
#collect $k:tt,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[name ($nn:expr), $($word:ident $arg:tt),*];
$($rest:tt)*
) => {
bson_destructure!(
#collect $k,
[$target, $source, $nn, $variant, $f],
[$($word $arg),*];
$($rest)*
)
};
// main forms
(get $target:ident = from ($source:expr), $($word:ident $arg:tt),*; $($rest:tt)*) => {
bson_destructure!(
#collect req,
[$target, $source, stringify!($target), Bson, Ok],
[$($word $arg),*];
$($rest)*
)
};
(get $target:ident = from ($source:expr), optional, $($word:ident $arg:tt),*; $($rest:tt)*) => {
bson_destructure!(
#collect opt,
[$target, $source, stringify!($target), Bson, Ok],
[$($word $arg),*];
$($rest)*
)
};
// final form
(ret $e:expr) => { $e }
}
However, the first example above results in the following compilation error:
src/db/data.rs:345:22: 345:25 error: no rules expected the token `opt`
src/db/data.rs:345 #collect opt,
^~~
I'm somewhat surprised that it doesn't show the error location as usual (that is, there is no indication where expansion happens), however, the error vanishes when I comment the piece of code which uses the macro out.
I can't see why it says that no rules expected this token because there is such a rule, but maybe I don't understand something.
I'm pretty sure that this is possible because that's roughly what quick_error crate does, but it seems that my macro writing skills are still lacking.
How should I fix the macro so it would work as I expect?
For completeness, the following is the definition of BsonDestructureError:
#[derive(Debug, Clone)]
pub enum BsonDestructureError {
InvalidType {
field_name: &'static str,
expected: &'static str,
actual: Bson
},
InvalidArrayItemType {
index: usize,
expected: &'static str,
actual: Bson
},
MissingField {
field_name: &'static str,
expected: &'static str
}
}
I'm also using bson crate reexported from ejdb crate. Here is a minimal example, runnable with cargo script on stable Rust.
Both cargo script, a recursive muncher, and my favourite internal rule syntax; how can I not?
First, the exact problem can be identified by running cargo rustc -- -Z trace-macros. This will output each rule as it gets expanded, giving us a "backtrace" which, after some manual reformatting, comes out looking like so:
bson_destructure! {
get id = from ( bson ) , optional , name ( "_id" ) , as ObjectId ;
get hash = from ( bson ) , as String ;
get name = from ( bson ) , as String ;
get path = from ( bson ) , as Bson ;
get modification_time = from ( bson ) , as UtcDatetime ;
get size = from ( bson ) , as I64 , through ( | n | n as u64 ) ;
get metadata = from ( bson ) , as Document ;
get commit_data = from ( bson ) , optional , as Document ;
ret ( id , hash , name , path , modification_time , size , metadata , commit_data )
}
bson_destructure! {
# collect opt ,
[ id , bson , stringify ! ( id ) , Bson , Ok ] ,
[ name ( "_id" ) , as ObjectId ] ;
get hash = from ( bson ) , as String ;
get name = from ( bson ) , as String ;
get path = from ( bson ) , as Bson ;
get modification_time = from ( bson ) , as UtcDatetime ;
get size = from ( bson ) , as I64 , through ( | n | n as u64 ) ;
get metadata = from ( bson ) , as Document ;
get commit_data = from ( bson ) , optional , as Document ;
ret ( id , hash , name , path , modification_time , size , metadata , commit_data )
}
bson_destructure! {
# collect opt ,
[ id , bson , "_id" , Bson , Ok ] , [ as ObjectId ] ;
get hash = from ( bson ) , as String ;
get name = from ( bson ) , as String ;
get path = from ( bson ) , as Bson ;
get modification_time = from ( bson ) , as UtcDatetime ;
get size = from ( bson ) , as I64 , through ( | n | n as u64 ) ;
get metadata = from ( bson ) , as Document ;
get commit_data = from ( bson ) , optional , as Document ;
ret ( id , hash , name , path , modification_time , size , metadata , commit_data )
}
A careful perusal of the rules in bson_destructure! shows the issue: there is no rule which matches the third expansion. macro_rules! is, frankly, rubbish at reporting sane error locations when it comes to recursive rules; that it's pointing to the opt token is irrelevant. The real problem is that it couldn't find a matching rule.
In particular, the offending rule is this one:
// change variant name
(
#collect $k:tt,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[as $nv:ident, $($word:ident $arg:tt),*];
$($rest:tt)*
) => {
...
};
Note the presence of a comma immediately after $nv:ident. Also note that there is no such comma in the input. This can be solved by moving the comma inside the repetition, like so:
// change field name
(
#collect $k:tt,
[$target:ident, $source:expr, $field:expr, $variant:ident, $f:expr],
[name ($nn:expr) $(, $word:ident $arg:tt)*];
$($rest:tt)*
) => {
...
};
Another alternative (and the one I ususally go with), is to simply mutate the input when it is first encountered to make sure there is always a trailing comma in place.
The code won't actually compile on my machine, due to a native dependency, but I did verify that making this change (both here, and to the other rules with a similar issue) allows it to complete macro expansion. You can check the output looks correct using cargo rustc -- -Z unstable-options --pretty=expanded.

String replace &amp with & in Perl

I have a script(Perl) which is taking a string from database through a variable and writing to a xml file. if the string contain "&" then while opening xml file it is giving parser error want to replace "&" with &amp through Perl script.
This is what I have tried
foreach my $ActiveLinkInfo ( #ActiveLinkInfos ) {
my ( $SubCID, $Subf, $Subt, $Subclosed, $SubCName ) = (
$ActiveLinkInfo->{'SubCID'},
$ActiveLinkInfo->{'Subf'},
$ActiveLinkInfo->{'Subt'},
$ActiveLinkInfo->{'Subclosed'},
$ActiveLinkInfo->{'SubCName'}
);
##$ActiveLinkInfo
if ( $Subf eq "HEADING" ) {
push( #menu, { "Name" => "$SubCName", "Dir" => "HEADING" } );
}
else {
my $res = GetChildren( "$path$SubCID\\", $SubCID, $VID );
$SubCName =~ s/ - .*//;
push( #menu, { "Name" => "$SubCName", "Dir" => "$SubCID/default.aspx" } );
}
}
If you're trying to convert text to HTML, use HTML::Entities.
You should take a look at the HTML::Escape module, which only translates in one direction -- from text to entities -- but does it very quickly