As the name suggests I was wondering if it makes sense to use Protobuf without the requirement of having to serialize the data in any form at the moment (might change in future). I mean to use them purely as data structures to pass Information from one function to the other, all executed in the same address space. Or do you feel it may be an Overkill and see other alternatives.
Backgroud:
I have to design a lib that implements certain interfaces. At the moment, my collegues have implemented it using several functions taking arguments ..
Example:
void readA(int iIP1, int iIP2, Result& oOP)
void readB(std::string iIP1, Result& oOP)
void readC(std::vector<int> iIP1, Result& oOP)
I want to change this and provide just one interface function:
void ReadFn(ReadMsg& ip, ReadResult& res);
And the data structures are defined in Protobuf as below ..
message ReadMsg {
enum ReadWhat {
A = 0;
B = 1;
C = 2;
}
message readA {
int32 iIP1 = 1;
int32 iIP2 = 2;
}
message readB {
string IP1 = 1;
}
message readC {
repeated int IP1 = 1;
}
oneof actRead {
readA rA = 1;
readB rB = 2;
readC rC = 3;
}
}
It offers many advantages over traditional interface design(using functions), with very Little effort from my side. And it will be future proof should these components be deployed as Services in different processes/machines (ofcourse with additional implementation). But given that Protocol Buffers strength is their serialization Features, which I do not make use of at the moment, would you choose to use them in such trivial Tasks ?
Thank you
It can make sense to group function arguments into a struct if there are many of them. And it can make sense to combine your readA, readB and readC functions into a single function if they share a lot of common parts.
What doesn't, however, make sense in my opinion is introducing a separate .proto file and a protobuf dependency if you are not going to use it for serialization. Similar features for grouping data into reusable structures already exist in most languages. And when you use the built-in features of the language, all the code remains in the same place and is easier to understand.
Related
I am working on moving some complex script logging from functions to classes. In the functional version I would use a string like this
"{header-[cf][0][:12]}_Started: 11:30:21"
{????}_ is the identifying info, here I have a header item header- (which has formatting implications), it will log to both console and file [cf], have no initial indentation [0], and will pad the following string to provide 12 characters before the : that follows Started [:12], so 5 spaces of padding. I had functions to increment the tab value, change the destination (console, file or both), etc. All driven with complicated RegularExpressions to extract the identifying info from the start of the string.
Now, as I move to classes, I have
Enum PxLogType {
blankLine
header
milieuCondition
}
class Px_LogItem {
[PxLogType]$type
[bool]$logToConsole
[bool]$logToFile
[int]$indent
[char]$alignToSymbol
[int]$alignSymbolLocation
[string]$string
# Constructor
Px_LogItem ([PxLogType]$type, [bool]$logToConsole, [bool]$logToFile, [int]$indent, [string]$string) {
$this.type = $type
$this.logToConsole = $logToConsole
$this.logToFile = $logToFile
$this.indent = $indent
$this.string = $string
}
Px_LogItem ([PxLogType]$type, [bool]$logToConsole, [bool]$logToFile, [int]$indent, [char]$alignToSymbol, [int]$alignSymbolLocation, [string]$string) {
$this.type = $type
$this.logToConsole = $logToConsole
$this.logToFile = $logToFile
$this.indent = $indent
$this.alignToSymbol = $alignToSymbol
$this.alignSymbolLocation = $alignSymbolLocation
$this.string = $string
}
}
Ultimately I will have more enumerations, a method to increase the tab indent, etc. This is working, but
$logItem = [Px_LogItem]::New([PxLogType]::header, $true, $true, 0, ':', 12, "Started: 11:30:21")
is not as readable to my eye. Especially the two instances of $true that replace [cf] to define the target for the log. And depending on where I go, I can see having a few more constructor variations, which starts to get messy. At least, messy in my eyes that are as yet not so familiar with the way constructors are so... specific.
So, I wonder if I am on the right track, and just not familiar enough yet with how class based code looks, or am I off track? I imagine I could do another enum for the log target, with valid values of c, f & cf. Or even have a version that doesn't take those arguments, and defaults both to true to simplify things. That would require method chaining for the constructors, which might actually be a good idea, but how many constructors is considered the threshold for that change? I realize this is all pretty non specific and basically just opinion, but I hope specific enough for some folks to weigh in with their opinions, since I'm too ignorant to have a valid opinion yet.
I'm attempting to implement std::async from scratch, and have run into a hiccup with arguments of move-only type. The gist of it is, C++14 init-captures allow us to capture single variables "by move" or "by perfect forwarding", but they do not appear to let us capture parameter packs "by move" nor "by perfect forwarding", because you can't capture a parameter pack by init-capture — only by named capture.
I've found what appears to be a workaround, by using std::bind to capture the parameter pack "by move", and then using a wrapper to move the parameters out of the bind object's storage into the parameter slots of the function I really want to call. It even looks elegant, if you don't think too much about it. But I can't help thinking that there must be a better way — ideally one that doesn't rely on std::bind at all.
(Worst case, I'd like to know how much of std::bind I'd have to reimplement on my own in order to get away from it. Part of the point of this exercise is to show how things are implemented all the way down to the bottom, so having a dependency as complicated as std::bind really sucks.)
My questions are:
How do I make my code work, without using std::bind? (I.e., using only core language features. Generic lambdas are fair game.)
Is my std::bind workaround bulletproof? That is, can anybody show an example where the STL's std::async works and my Async fails?
Pointers to discussion and/or proposals to support parameter-pack capture in C++1z will be gratefully accepted.
Here's my code:
template<typename UniqueFunctionVoidVoid>
auto FireAndForget(UniqueFunctionVoidVoid&& uf)
{
std::thread(std::forward<UniqueFunctionVoidVoid>(uf)).detach();
}
template<typename Func, typename... Args>
auto Async(Func func, Args... args)
-> std::future<decltype(func(std::move(args)...))>
{
using R = decltype(func(std::move(args)...));
std::packaged_task<R(Args...)> task(std::move(func));
std::future<R> result = task.get_future();
#ifdef FAIL
// sadly this syntax is not supported
auto bound = [task = std::move(task), args = std::move(args)...]() { task(std::move(args)...) };
#else
// this appears to work
auto wrapper = [](std::packaged_task<R(Args...)>& task, Args&... args) { task(std::move(args)...); };
auto bound = std::bind(wrapper, std::move(task), std::move(args)...);
#endif
FireAndForget(std::move(bound));
return result;
}
int main()
{
auto f3 = [x = std::unique_ptr<int>{}](std::unique_ptr<int> y) -> bool { sleep(2); return x == y; };
std::future<bool> r3 = Async(std::move(f3), std::unique_ptr<int>{});
std::future<bool> r4 = Async(std::move(f3), std::unique_ptr<int>(new int));
assert(r3.get() == true);
assert(r4.get() == false);
}
It was suggested to me offline that another approach would be to capture the args pack in a std::tuple, and then re-expand that tuple into the argument list of task using something like std::experimental::apply (coming soon to a C++17 standard library near you!).
auto bound = [task = std::move(task), args = std::make_tuple(std::move(args)...)]() {
std::experimental::apply(task, args);
};
This is much cleaner. We've reduced the amount of library code involved, down from bind to "merely" tuple. But that's still a big dependency that I'd love to be able to get rid of!
With dapper, I can do batch execute for Stored Procedures, something similar to:
connection.Execute(#"
exec sp1 #i = #one, #y = #two
exec sp2 #i = #three",
new { one = 1, two = 2, three = 3 });
However, the only means of retrieving data that I have seen till now is by using
results.Read<Type>()
What if the results don't map to an object? For instance, I am writing "generic" code to execute any SP with variable in/out parameters & result sets.
Thanks
What API do you want? If you can process the grids separately: do that:
using(var multi = connection.QueryMultiple(...))
{
while(!multi.IsConsumed) {
// ...
}
}
where ... has access to:
Read() for dynamic rows - noting that each row also implements IDictionary<string,object>
Read<T>() for typed rows via generics
Read(Type) for typed rows without generics
Read<DapperRow>() (actually, this is just the T that Read<T>() uses to implement Read(), but perhaps more convenient), which provides slightly more access to metadata
If you want to drop to a raw IDataReader, do that:
using(var reader = connection.ExecuteReader(...)) {
// whatever you want
}
With regards to parameters: the DynamicParameters class provides much richer access to parameter control, including parameter-direction etc.
I use protobuf-net and ProtoGen.exe to parse following .proto file (given by another project)
enum RGBFlags { FLAG_RED = 1; FLAG_GREEN = 2; FLAG_BLUE = 4; }
message SomeMessage {
// Values from RGBFlags only allowed
optional int32 flags = 2;
}
My fellow programmers in C++ don't care about type safety and treat flags field as a plain integer. I wanted to be more strict and try to avoid such code:
SomeMessage foo = new SomeMessage();
foo.flags = (int)RGBFlags.FLAG_BLUE | (int)RGBFlags.FLAG_GREEN;
I thought that I could use protbuf custom options to amend proto code and modify XSLT transform of ProtoGet to generate necessary `[Flags]' annotations.
extend google.protobuf.EnumOptions {
optional bool generate_bit_field = 60000;
}
enum RGBFlags {
option (generate_bit_field) = true;
FLAG_RED = 1; FLAG_GREEN = 2; FLAG_BLUE = 4;
}
message SomeMessage {
// Values from RGBFlags only allowed
optional int32 flags = 2;
}
Problem is that all custom options appear as uninterpreted_option in the temporary file in ProtoGen.
Any idea what I could do to get [Flags] annotations in my code?
Re flags; the raw protobuf spec doesn't really have support for composite enum values, so in some ways I understand why they are doing it that way. And sadly there is no such this as partial enum, so you can't add the [Flags] in a separate code file.
Re custom options; it is an excellent question, and support for custom options has been raised before. It is definitely something I'd like to add, but relative to other features it simply isn't a massively demanded item, so (due to limited resource) it has not (yet) been investigated fully.
Therefore, I don't have a great answer for you; that feature isn't really there much right now. You could hard-code that one scenario in your xslt (for your specific types). Or wait until I get around to it (I don't have a specific timescale). Or take a peek yourself.
Recommendations for languages with native (so no FSM generation tools) support for state machine development and execution and passing of messages/signals. This is for telecoms, e.g implementation of FSMs of this level of complexity.
I have considered Erlang, but would love some feedback, suggestions, pointer to tutorials, alternatives, particularly Java based frameworks. Maybe Scala?
Open source only. I'm not looking for UML or regular expression related solutions.
As this is for the implementation of telecoms protocols the FSMs may be non-trivial. Many states, many transitions, signal based, input constraints/guards. Dynamic instantiation would be a plus. Switch statements are out of the question, it quickly nests to unusable. It's barely better that if/else.
I would prefer to not depend on graphical design; the format FSM description should be human readable/editable/manageable.
--
I have decided to focus on an Actor based solution for C++
For example, the Theron framework provides a starting point http://theron.ashtonmason.net/ and to avoid switch statements in the FSM based event handler this C++ FSM Template Framework looks useful http://satsky.spb.ru/articles/fsm/fsmEng.php
This particular application, telco protocol implementation, is what Erlang was built for. The initial applications of Erlang at Ericsson were telephone switches and the earliest commercial products were ATM switches supporting all manner of telco protocols.
OTP has a standard behaviour for implementing FSMs called gen_fsm. There's an example of its use in a non-trivial FSM in some of the OTP Documentation.
OSERL is an open souce SMPP implementation in Erlang and demonstrates how you can implement a telco protocol using gen_fsms. A good example to look at would be gen_esme_session.
While I can't point you to the code, I know there are quite a few Erlang companies selling telco oriented products: Corelatus, Synapse, Motivity among others.
I agree that switch statements should be out of the question... they eventually lead to maintenance nightmares. Can't you use the State Pattern to implement your FSM? Depending on your actual implementation, you could use actors (if you have multiple FSM collaborating - hm... is that possible?). The nice thing about actors is that the framework for passing messages is already there.
An example of using State would be:
trait State {
def changeState(message: Any): State
}
trait FSM extends Actor {
var state: State
def processMessage(message: Any) {
state = state.changeState(message)
}
override def act() {
loop {
react {
case m: Any => processMessage(m)
}
}
}
}
This is very basic code, but as I don't know more of the requirements, that's the most I can think of. The advantage of State is that every state is self-contained in one class.
I disagree that FSM are trivial to implement. This is very short-sighted, and shows either a lack of familiarity with the alternatives, or the lack of experience with complex state machines.
The fundamental problem is that a state machine graph is obvious, but FSM code is not. Once you get beyond a dozen states and a score of transitions, FSM code becomes ugly and difficult to follow.
There are tools whereby you draw the state machine, and generate Java code for it. I don't know of any open source tools for that, however.
Now, getting back to Erlang/Scala, Scala has Actors and message passing as well, and is based on the JVM, so it might be a better alternative than Erlang given your constraints.
There's a DFA/NFA library on Scala as well, though it is not particularly a good one. It supports conversion from arbitrary regular expressions (ie, the literals need not be characters) into DFA/NFA.
I'll post some code below using it. In this code, the idea is creating a FSM which will accept any sequential combination of arbitrary prefixes for a list of words, the idea being looking up menu options without predefined keybinds.
import scala.util.regexp._
import scala.util.automata._
// The goal of this object below is to create a class, MyChar, which will
// be the domain of the tokens used for transitions in the DFA. They could
// be integers, enumerations or even a set of case classes and objects. For
// this particular code, it's just Char.
object MyLang extends WordExp {
type _regexpT = RegExp
type _labelT = MyChar
case class MyChar(c:Char) extends Label
}
// We now need to import the types we defined, as well as any classes we
// created extending Label.
import MyLang._
// We also need an instance (singleton, in this case) of WordBerrySethi,
// which will convert the regular expression into an automatum. Notice the
// language being used is MyLang.
object MyBerrySethi extends WordBerrySethi {
override val lang = MyLang
}
// Last, a function which takes an input in the language we defined,
// and traverses the DFA, returning whether we are at a sink state or
// not. For other uses it will probably make more sense to test against
// both sink states and final states.
def matchDet(pat: DetWordAutom[MyChar], seq: Seq[Char]): Boolean =
!pat.isSink((0 /: seq) ((state, c) => pat.next(state, MyChar(c))))
// This converts a regular expression to a DFA, with using an intermediary NFA
def compile(pat: MyLang._regexpT) =
new SubsetConstruction(MyBerrySethi.automatonFrom(pat, 100000)).determinize
// Defines a "?" function, since it isn't provided by the library
def Quest(rs: _regexpT*) = Alt(Eps, Sequ(rs: _*)) // Quest(pat) = Eps|pat = (pat)?
// And now, the algorithm proper. It splits the string into words
// converts each character into Letter[MyChar[Char]],
// produce the regular expression desired for each word using Quest and Sequ,
// then the final regular expression by using Sequ with each subexpression.
def words(s : String) = s.split("\\W+")
def wordToRegex(w : String) : Seq[MyLang._regexpT] = w.map(c => Letter(MyChar(c)))
def wordRegex(w : String) = Quest(wordToRegex(w) reduceRight ((a,b) => Sequ(a, Quest(b))))
def phraseRegex(s : String) = Sequ(words(s).map(w => wordRegex(w)) : _*)
// This takes a list of strings, produce a DFA for each, and returns a list of
// of tuples formed by DFA and string.
def regexList(l : List[String]) = l.map(s => compile(phraseRegex(s)) -> s)
// The main function takes a list of strings, and returns a function that will
// traverse each DFA, and return all strings associated with DFAs that did not
// end up in a sink state.
def regexSearcher(l : List[String]) = {
val r = regexList(l)
(s : String) => r.filter(t => matchDet(t._1, s)).map(_._2)
}
I can hardly think of any language where implementing an FSM is non-trivial. Maybe this one.
...
if (currentState == STATE0 && event == EVENT0) return STATE1;
if (currentState == STATE1 && event == EVENT0) return STATE2;
...
The State pattern (using Java enums) is what we use in our telecom application, however we use small FSM's:
public class Controller{
private State itsState = State.IDLE;
public void setState(State aState){
itsState = aState;
}
public void action1(){
itsState.action1(this);
}
public void action2(){
itsState.action2(this);
}
public void doAction1(){
// code
}
public void doAction2(){
// code
}
}
public enum State{
IDLE{
#Override
public void action1(Controller aCtx){
aCtx.doAction1();
aCtx.setState(State.STATE1);
}
},
STATE1{
#Override
public void action2(Controller aCtx){
aCtx.doAction2();
aCtx.setState(State.IDLE);
}
},
public void action1(Controller aCtx){
throw new IllegalStateException();
}
public void action2(Controller aCtx){
throw new IllegalStateException();
}
}
FSM should be trivial to implement in any language that has a case statement.Your choice of language should be based on what that finite state machine needs to do.
For example, you state that you need to do this for telecom development and mention messages. I would look at systems/languages that support distributed message passing. Erlang does this, and I"m sure just about every other common language supports this through an API/library for the language.