Using std::bind to capture a parameter pack "by move" - variadic-templates

I'm attempting to implement std::async from scratch, and have run into a hiccup with arguments of move-only type. The gist of it is, C++14 init-captures allow us to capture single variables "by move" or "by perfect forwarding", but they do not appear to let us capture parameter packs "by move" nor "by perfect forwarding", because you can't capture a parameter pack by init-capture — only by named capture.
I've found what appears to be a workaround, by using std::bind to capture the parameter pack "by move", and then using a wrapper to move the parameters out of the bind object's storage into the parameter slots of the function I really want to call. It even looks elegant, if you don't think too much about it. But I can't help thinking that there must be a better way — ideally one that doesn't rely on std::bind at all.
(Worst case, I'd like to know how much of std::bind I'd have to reimplement on my own in order to get away from it. Part of the point of this exercise is to show how things are implemented all the way down to the bottom, so having a dependency as complicated as std::bind really sucks.)
My questions are:
How do I make my code work, without using std::bind? (I.e., using only core language features. Generic lambdas are fair game.)
Is my std::bind workaround bulletproof? That is, can anybody show an example where the STL's std::async works and my Async fails?
Pointers to discussion and/or proposals to support parameter-pack capture in C++1z will be gratefully accepted.
Here's my code:
template<typename UniqueFunctionVoidVoid>
auto FireAndForget(UniqueFunctionVoidVoid&& uf)
{
std::thread(std::forward<UniqueFunctionVoidVoid>(uf)).detach();
}
template<typename Func, typename... Args>
auto Async(Func func, Args... args)
-> std::future<decltype(func(std::move(args)...))>
{
using R = decltype(func(std::move(args)...));
std::packaged_task<R(Args...)> task(std::move(func));
std::future<R> result = task.get_future();
#ifdef FAIL
// sadly this syntax is not supported
auto bound = [task = std::move(task), args = std::move(args)...]() { task(std::move(args)...) };
#else
// this appears to work
auto wrapper = [](std::packaged_task<R(Args...)>& task, Args&... args) { task(std::move(args)...); };
auto bound = std::bind(wrapper, std::move(task), std::move(args)...);
#endif
FireAndForget(std::move(bound));
return result;
}
int main()
{
auto f3 = [x = std::unique_ptr<int>{}](std::unique_ptr<int> y) -> bool { sleep(2); return x == y; };
std::future<bool> r3 = Async(std::move(f3), std::unique_ptr<int>{});
std::future<bool> r4 = Async(std::move(f3), std::unique_ptr<int>(new int));
assert(r3.get() == true);
assert(r4.get() == false);
}

It was suggested to me offline that another approach would be to capture the args pack in a std::tuple, and then re-expand that tuple into the argument list of task using something like std::experimental::apply (coming soon to a C++17 standard library near you!).
auto bound = [task = std::move(task), args = std::make_tuple(std::move(args)...)]() {
std::experimental::apply(task, args);
};
This is much cleaner. We've reduced the amount of library code involved, down from bind to "merely" tuple. But that's still a big dependency that I'd love to be able to get rid of!

Related

Protobuf without serialization

As the name suggests I was wondering if it makes sense to use Protobuf without the requirement of having to serialize the data in any form at the moment (might change in future). I mean to use them purely as data structures to pass Information from one function to the other, all executed in the same address space. Or do you feel it may be an Overkill and see other alternatives.
Backgroud:
I have to design a lib that implements certain interfaces. At the moment, my collegues have implemented it using several functions taking arguments ..
Example:
void readA(int iIP1, int iIP2, Result& oOP)
void readB(std::string iIP1, Result& oOP)
void readC(std::vector<int> iIP1, Result& oOP)
I want to change this and provide just one interface function:
void ReadFn(ReadMsg& ip, ReadResult& res);
And the data structures are defined in Protobuf as below ..
message ReadMsg {
enum ReadWhat {
A = 0;
B = 1;
C = 2;
}
message readA {
int32 iIP1 = 1;
int32 iIP2 = 2;
}
message readB {
string IP1 = 1;
}
message readC {
repeated int IP1 = 1;
}
oneof actRead {
readA rA = 1;
readB rB = 2;
readC rC = 3;
}
}
It offers many advantages over traditional interface design(using functions), with very Little effort from my side. And it will be future proof should these components be deployed as Services in different processes/machines (ofcourse with additional implementation). But given that Protocol Buffers strength is their serialization Features, which I do not make use of at the moment, would you choose to use them in such trivial Tasks ?
Thank you
It can make sense to group function arguments into a struct if there are many of them. And it can make sense to combine your readA, readB and readC functions into a single function if they share a lot of common parts.
What doesn't, however, make sense in my opinion is introducing a separate .proto file and a protobuf dependency if you are not going to use it for serialization. Similar features for grouping data into reusable structures already exist in most languages. And when you use the built-in features of the language, all the code remains in the same place and is easier to understand.

Why is generic instantiation syntax disallowed in Hack?

From the docs:
Note: HHVM allows syntax such as $x = Vector<int>{5,10};, but Hack
disallows the syntax in this situation, instead opting to infer
it.
Is there a specific reason for this? Isn't this a violation of the fail-fast rule?
There are some situations in which this would cause error to be deffered, which in turn leads to harder backtracing.
For example:
<?hh // strict
function main() : void {
$myVector = new Vector([]); // no generic syntax
$myVector->addAll(require 'some_external_source.php');
}
The above code causes no errors until it is used in a context where the statically-typed collection is actually in place:
class Foo
{
public ?Vector<int> $v;
}
$f = new Foo();
$f->v = $myVector;
Now there is an error if the vector contains something else then int. But one must trace back the error to the point where the flawed data was actually imported. This would not be necessary if one could instantiate the vector using generic syntax in the first place:
$myVector = new Vector<int>([]);
$myVector->addAll(require 'some_external_source.php'); // fail immediately
I work on the Hack type system and typechecker at Facebook. This question has been asked a few times internally at FB, and it's good to have a nice, externally-visible place to have an answer to it written down.
So first of all, your question is premised on the following code:
<?hh // strict
function main() : void {
$myVector = new Vector([]); // no generic syntax
$myVector->addAll(require 'some_external_source.php');
}
However, that code does not pass the typechecker due to the usage of require outside toplevel, and so any result of actually executing it on HHVM is undefined behavior, rendering this whole discussion moot for that code.
But it's still a legitimate question for other potential pieces of code that do actually typecheck, so let me go ahead and actually answer it. :)
The reason that it's unsupported is because the typechecker is actually able to infer the generic correctly, unlike many other languages, and so we made the judgement call that the syntax would get in the way, and decided to disallow it. It turns out that if you just don't worry about, we'll infer it right, and still give useful type errors. You can certainly come up with contrived code that doesn't "fail fast" in the way you want, but it's, well, contrived. Take for example this fixup of your example:
<?hh // strict
function main(): void {
$myVector = Vector {}; // I intend this to be a Vector<int>
$myVector[] = 0;
$myVector[] = 'oops'; // Oops! Now it's inferred to be a Vector<mixed>
}
You might argue that this is bad, because you intended to have a Vector<int> but actually have a Vector<mixed> with no type error; you would have liked to be able to express this when creating it, so that adding 'oops' into it would cause such an error.. But there is no type error only because you never actually tried to use $myVector! If you tried to pull out any of its values, or return it from the function, you'd get some sort of type compatibility error. For example:
<?hh // strict
function main(): Vector<int> {
$myVector = Vector {}; // I intend this to be a Vector<int>
$myVector[] = 0;
$myVector[] = 'oops'; // Oops! Now it's inferred to be a Vector<mixed>
return $myVector; // Type error!
}
The return statement will cause a type error, saying that the 'oops' is a string, incompatible with the int return type annotation -- exactly what you wanted. So the inference is good, it works, and you don't ever actually need to explicitly annotate the type of locals.
But why shouldn't you be able to if you really want? Because annotating only generics when instantiating new objects isn't really the right feature here. The core of what you're getting at with "but occasionally I really want to annotate Vector<int> {}" is actually "but occasionally I really want to annotate locals". So the right language feature is not to let you write $x = Vector<int> {}; but let you explicitly declare variables and write Vector<int> $x = Vector {}; -- which also allows things like int $x = 42;. Adding explicit variable declarations to the language is a much more general, reasonable addition than just annotating generics at object instantiation. (It's however not a feature being actively worked on, nor can I see it being such in the near to medium term future, so don't get your hopes up now. But leaving the option open is why we made this decision.)
Furthermore, allowing either of these syntaxes would be actively misleading at this point in time. Generics are only enforced by the static typechecker and are erased by the runtime. This means that if you get untyped values from PHP or Hack partial mode code, the runtime cannot possibly check the real type of the generic. Noting that untyped values are "trust the programmer" and so you can do anything with them in the static typechecker too, consider the following code, which includes the hypothetical syntax you propose:
<?hh // partial
function get_foo() /* unannotated */ {
return 'not an int';
}
<?hh // strict
function f(): void {
$v = Vector<int> {};
$v[] = 1; // OK
// $v[] = 'whoops'; // Error since explicitly annotated as Vector<int>
// No error from static typechecker since get_foo is unannotated
// No error from runtime since generics are erased
$v[] = get_foo();
}
Of course, you can't have unannotated values in 100% strict mode code, but we have to think about how it interacts with all potential usages, including untyped code in partial mode or even PHP.

How to support custom options with protobuf-net?

I use protobuf-net and ProtoGen.exe to parse following .proto file (given by another project)
enum RGBFlags { FLAG_RED = 1; FLAG_GREEN = 2; FLAG_BLUE = 4; }
message SomeMessage {
// Values from RGBFlags only allowed
optional int32 flags = 2;
}
My fellow programmers in C++ don't care about type safety and treat flags field as a plain integer. I wanted to be more strict and try to avoid such code:
SomeMessage foo = new SomeMessage();
foo.flags = (int)RGBFlags.FLAG_BLUE | (int)RGBFlags.FLAG_GREEN;
I thought that I could use protbuf custom options to amend proto code and modify XSLT transform of ProtoGet to generate necessary `[Flags]' annotations.
extend google.protobuf.EnumOptions {
optional bool generate_bit_field = 60000;
}
enum RGBFlags {
option (generate_bit_field) = true;
FLAG_RED = 1; FLAG_GREEN = 2; FLAG_BLUE = 4;
}
message SomeMessage {
// Values from RGBFlags only allowed
optional int32 flags = 2;
}
Problem is that all custom options appear as uninterpreted_option in the temporary file in ProtoGen.
Any idea what I could do to get [Flags] annotations in my code?
Re flags; the raw protobuf spec doesn't really have support for composite enum values, so in some ways I understand why they are doing it that way. And sadly there is no such this as partial enum, so you can't add the [Flags] in a separate code file.
Re custom options; it is an excellent question, and support for custom options has been raised before. It is definitely something I'd like to add, but relative to other features it simply isn't a massively demanded item, so (due to limited resource) it has not (yet) been investigated fully.
Therefore, I don't have a great answer for you; that feature isn't really there much right now. You could hard-code that one scenario in your xslt (for your specific types). Or wait until I get around to it (I don't have a specific timescale). Or take a peek yourself.

Entity Framework and Nested Lambda Expressions

I've just started using Lambda expressions, and really like the shortcut. I also like the fact that I have scope within the lambda of the encompassing method. One thing I am having trouble with is nesting lambdas. Here is what I am trying to do:
public void DoSomeWork()
{
MyContext context = new MyDomainContext();
context.GetDocumentTypeCount(ci.CustomerId, io =>
{
if (io.HasError)
{
// Handle error
}
// Do some work here
// ...
// make DB call to get data
EntityQuery<AppliedGlobalFilter> query =
from a in context.GetAppliedGlobalFiltersQuery()
where a.CustomerId == ci.CustomerId && a.FilterId == 1
select a;
context.Load<AppliedGlobalFilter>(query, lo =>
{
if (lo.HasError)
{
}
**// Do more work in this nested lambda.
// Get compile time error here**
}
}, null);
}, null);
}
The second lambda is where I get the following compile time error:
Cannot convert Lambda expression to type 'System.ServiceModel.DomainService.Client.LoadBehavior' because it is not a delegate type
The compiler is choosing the wrong overload for the Load method even though I am using the same override I did in the previous Lambda.
Is this because I am trying to nest? Or do I have something else wrong?
Thanks,
-Scott
Found the problem as described in my comment above. I'll head back to work now - red face and all....
I realize this is not the answer you want, but I suggest caution about lengthy and/or nested lambdas. They work, but they often make code harder to read / maintain by other developers. I try to limit my lambdas in length to three statements, with no nesting.

GWT: How to avoiding calls to dynamicCast and canCastUnsafe in generated JavaScript code?

I'm writing some special purpose data structures in Java, intended for use in the browser, (compiled to JavaScript with GWT).
I'm trying to match the performance of some of the built-in JDK classes I'm noticing things run reasonably fast, but when I compare my code trace to some of the emulated JDK code, mine has lots of calls to dynamicCast and canCastUnsafe, while the JDK emulated classes do not. And it just about accounts for the difference in performance too...
Any GWT gurus out there know how to avoid this? It's amounting to a 20% overhead :-(
Details:
Here's the profile output (captured in Firebug) for 10,000 insertions of random integers, between 0 and 100,000 into two different data structures:
Google's TreeMap implementation for java.util.TreeMap (a red-black tree):
Profile (4058.602ms, 687545 calls)
Function Calls Percent Own Time
$insert_1 129809 41.87% 1699.367ms
$compare_0 120290 16% 649.209ms
$isRed 231166 13.33% 540.838ms
compareTo_0 120290 8.96% 363.531ms
$put_2 10000 6.02% 244.493ms
wrapArray 10000 3.46% 140.478ms
createFromSeed 10000 2.91% 118.038ms
$TreeMap$Node 10000 2.38% 96.706ms
initDim 10000 1.92% 77.735ms
initValues 10000 1.49% 60.319ms
$rotateSingle 5990 0.73% 29.55ms
TreeMap$Node 10000 0.47% 18.92ms
My Code (An AVL tree):
Profile (5397.686ms, 898603 calls)
Function Calls Percent Own Time
$insert 120899 25.06% 1352.827ms
$compare 120899 17.94% 968.17ms
dynamicCast 120899 14.12% 762.307ms <--------
$balanceTree 120418 13.64% 736.096ms
$setHeight 126764 8.93% 482.018ms
compareTo_0 120899 7.76% 418.716ms
canCastUnsafe 120899 6.99% 377.518ms <--------
$put 10000 2.59% 139.936ms
$AVLTreeMap$Node 9519 1.04% 56.403ms
$moveLeft 2367 0.36% 19.602ms
AVLTreeMap$State 9999 0.36% 19.429ms
$moveRight 2378 0.34% 18.295ms
AVLTreeMap$Node 9519 0.34% 18.252ms
$swingRight 1605 0.26% 14.261ms
$swingLeft 1539 0.26% 13.856ms
Additional observations:
Same problem for another data structure I made (SkipList).
dynamicCast is being applied in the compare function:
cmp = dynamicCast(right.key, 4).compareTo$(key);
dynamicCast goes away if the class does not implement Map (ie: just removing " implements Map" from the class. Doesn't matter if it's accessed through the interface or directly. This results in the same line compiling to:
cmp = right.key.compareTo$(key);
This is the relevant section of Java source from SkipList:
private int compare(Node a, Object o) {
if (comparator != null)
return comparator.compare((K) a.key, (K) o);
return ((Comparable<K>) a.key).compareTo((K) o);
}
public V get(Object k) {
K key = (K) k;
Node<K, V> current = head;
for (int i = head.height - 1; i >= 0; i--) {
Node<K, V> right;
while ((right = current.right[i]) != null) {
int cmp = compare(right, key);
...
}
}
}
Unfortunately I'm still not exactly clear on the cause, but from my experience, it seems from explicit casts, like:
((Comparable) obj).compareTo(other)
The Javascript generated looks like:
dynamicCast(obj, 1).compareTo(other);
Where 1 is a generated typeId representing the target of the cast. dynamicCast in turn calls canCastUnsafe and if false, it throws a ClassCastException. The value of this has been debated, since this would already be caught in hosted mode.
It can be sidestepped with JSNI:
public static native int compare(Object a, Object b) /*-{
return a.#java.lang.Comparable::compareTo(Ljava/lang/Object;)(b);
}-*/;
Dunno if you've seen this thread in the GWT Contributor's forum...
Basically, it starts with the same problem you've identified, proposes some new compiler flags, and goes on to show how to use some JSNI to get around the casts.
Edit In the GWT trunk there's a new compiler flag. See the wiki...
An updated answer for GWT version 2.1 and later:
Since GWT 2.1 (at least that's the first mention), the GWT compiler has a new compiler argument called -XdisableCastChecking that disables all runtime checking of casts.
Note, this option is marked as experimental (probably because this would make class cast exceptions very hard to debug).
In my app dynamicCast was called thousands of times in a short profile run, and were the the 3rd most time consuming method in the Firebug profiler. Using this compiler argument significantly reduced the number of "Long Duration Events" messages in the Chrome Speed Tracer.
See GWT Compiler Options for this and other Compiler arguments.
It's definitely a compiler problem: I have the problem on the following line:
final DefaultIconedSuggestBox<SuggestValueProxy, IconedValueHolderItem<SuggestValueProxy>> fieldValueWidget = getCategoryWidget().getFieldValueWidget();
I don't really know how I can workaround it: this line happens in a moment I'm changing from a module to another (it is maybe related to the code splitter issue: even though I'm not using code split: I'm just loading another page with another module)
Does the use of java 1.5 generics and wildcards could avoid this ?