I'd like to have a function interface that resolves which specific procedure is to be used depending on the size of an array argument. I could then, for example, have a program that handles vectors with less than or exactly N elements with one procedure and longer vectors with another one. As far as I know, Fortran only resolves using types, ranks and keywords of the arguments. Why is that? Aren't compilers intelligent enough to differentiate between arrays of different sizes or is it intrinsically impossible to write a compiler that does that?
Is there a workaround to achieve the desired functionality? I know of course that I could write a subroutine with an if-clause to sort out which procedure to use for which array size. Wouldn't that cost more CPU time though?
Resolution of specific procedures has been designed so that it is something which can be done at compile time. In the general case the size of an array is a run time concept.
If you know at compile time that a certain specific procedure is more suitable for certain input, then you can call that specific procedure directly.
Otherwise use IF to test and branch on the size (If the language had this sort of magic that's all that it would be doing behind the scenes anyway). That test and branch is likely to be substantially faster than calling reshape at runtime anyway.
Related
In the Optimization Experiment
AnyLogic allows the use of the top-level agent root on the requirements expression and does not allow the use of the top-level agent root on the constraints.
Although they mention in the AnyLogic help that root can be used in the constraints expression, the help is wrong, root can not be used. Please check the answer to this question:
Error - can not use root. in the constraints expression - AnyLogic
So, in this case, to avoid the root error: use requirements or change your constraints, so they do not need access to root.
If I change my constraints, so they do not need access to root, I will need to reduce the number of parameters (decision variables) which I'm trying to avoid as much as possible till I discover that there is no way else.
However, I'm afraid that if I used the requirements instead of the constraints, this would reduce the optimization performance. As you know, the constraints reduce the search space (this is mentioned in the help, too), but they did not mention if the requirement does the same (although they mentioned that requirements help in guiding to the solution):
"A requirement can also be a restriction on a response that requires its value to fall within a specified range."
Does this mean that the requirement is exactly the same as the constraint (in terms of reducing the search space)?
According to the above-mentioned, if I used requirement instead of constraints (because it is not allowed to use root in the constraints expression), what is the effect on the performance?
so let's review the concepts here...
CONSTRAINTS
First, the constraints are evaluated before the simulation run, and this is only used to check if your parameters fulfill certain conditions:
param1 and param2 can be evaluated between 0 and 10
but the constrain can be that the sum of both has to be below 10
This is effectively to reduce the search space since there would be no point to run the model for param1=8 and param2=8 if this is not within the search space
Requirements
Requirements are evaluated AT THE END of the simulation, that's why you can use root, so you can evaluate not only the parameters, but any variable in the simulation...
For instance if a variable ends up being above 10 when the system only allows its maximum value to be 8, then the solution is not feasible.
This means that requirements and constraints are very different, but they both find unfeasible solutions...
Other options
so from the optimization experiment side you only have these 2 options: evaluate the parameters before you run the simulation, or anything in your model after the simulation is run
Of course there's another option you can use, which is to define the restrictions inside the simulation. If your model is supposed to run for 1 day, but after 1 hour, a variable in question ends up being over 10 (which is not allowed) you can just use finishSimulation() in order to end the simulation early, and your Requirements will evaluate this variable after only 1 hour reducing the time it took to run that simulation and defining the results as unfeasible.
Conclusion
Obviously if you use requirements instead of constraints, you will have to run more simulations that you want, so the speed in which the optimization will find a solution will be much lower, so you shouldn't do that and there's no reason to do that.
Of course i have no idea what you are trying to optimize, but this is how all this works, and even though the help documentation may show an error, it wouldn't make sense to use root in the constraints.
I'm doing x86-64 binary obfuscation research and fundamentally one of the key challenges in the offense / defense cat and mouse game of executing a known-bad program and detecting it (even when obfuscated) is system call sequence analysis.
Put simply, obfuscation is just achieving the same effects on the system through a different sequence of instructions and memory states in order to minimize observable analysis channels. But at the end of the day, you need to execute certain system calls in a certain order to achieve certain input / output behaviors for a program.
Or do you? The question I want to study is this: Could the intended outcome of some or all system calls be achieved through different system calls? Let's say system call D, when executed 3 times consecutively, with certain parameters can be heuristically attributed to malicious behavior. If system calls A, B, and C could be found to achieve the same effect (perhaps in addition to other side-effects) desired from system call D, then it would be possible to evade kernel hooks designed to trace and heuristically analyze system call sequences.
To determine how often this system call outcome overlap exists in a given OS, I don't want to use documentation and manual analysis for a few reasons:
undocumented behavior
lots of work, repeated for every OS and even different versions
So rather, I'm interested in performing black-box analysis to fuzz system calls with various arguments and observing the effects. My problem is I'm not sure how to measure the effects. Once I execute a system call, what mechanism could I use to observe exactly which changes result from it? Is there any reliable way, aside from completely iterating over entire forensic snapshots of the machine before and after?
I am wondering if the built-in array structure for the contains function has any optimizations. If it does a linear search of contains every time I run it, it's at best O(n), which turns into O(n^2) because I'll be looping through another set of points to check against, however if it somehow behind the scenes sorts the array the first time 'contains' is run, then every subsequent 'contains' would be O(log(n)).
I have an array that gradually gets larger the more in depth the user gets into the application, and I am using the 'contains' a lot, so I am expecting it to slow down the application the longer the user is using the application.
If the array doesn't have any behind the scenes optimizations, then ought I build my own? (e.g. quicksort, and do an insert(newElement:, at:) every time I add to the array?)
Specifically, I'm using
[CGPoint],
CGPointArrayVariable.contains(newCGPoint) // run 100s - 10000s of times ideally every frame, (but realistically probably every second)
and when I add new CGPoints I'm using
CGPointArrayVariable += newCGPointSet.
So, the question: Am I ok continuing to use the built in .contains function (will it be fast enough?) or should I build my own structure optimizing for the contains, and keeping the array sorted? (maybe an insertion sort would be better to use opposed to a quicksort, if that's the direction recommended)
Doing something like that every frame will be VERY inefficient.
I would suggest re-thinking your design to avoid tracking that amount of information for every frame.
If it's absolutely necessary, building your own Type that uses a Dictionary rather than an Array should be more efficient.
Also, if it works with your use case using a Setmight be the best option.
I've got a piece of code that takes into account a given amount of features, where each feature is Boolean. I'm looking for the most efficient way to store a set of such features. My initial thought was to try and store these as a BitSet. But then, I realized that this implementation is meant to be used to store numbers in bit format rather than manipulate each bit, which is something I'd like to do (see the effect of switching any feature on and off). I then thought of using a Boolean array, but apparently the JVM uses much more memory for each Boolean element than the one bit it actually needs.
I'm therefore left with the question: What is the most efficient way to store a set of bits that I'd like to treat as independent bits rather than the building blocks of some number?
Please refer to this question: boolean[] vs. BitSet: Which is more efficient?
According to the answer of Peter Lawrey, boolean[] (not Boolean[]) is your way to go since its values can be manipulated and it takes only one byte of memory per bit to store. Consider that there is no way for a JVM application to store one bit in only one bit of memory and let it be directly (array-like) manipulated because it needs a pointer to find the address of the bit and the smallest addressable unit is a byte.
The site you referenced already states that the mutable BitSet is the same as the java.util.BitSet. There is nothing you can do in Java that you can't do in Scala. But since you are using Scala, you probably want a safe implementation which is probably meant to be even multithreaded. Mutable datatypes are not suitable for that. Therefore, I would simply use an immutable BitSet and accept the memory cost.
However, BitSets have their limits (deriving from the maximum number of int). If you need larger data sizes, you may use LongBitSets, which are basically Map<Long, BitSet>. If you need even more space, you may nest them in another map Map<Long, LongBitSet>, but in that case you need to use two or more identifiers (longs).
We're encouraged to use struct over class in Swift.
This is because
The compiler can do a lot of optimizations
Instances are created on the stack which is a lot more performant than malloc/free calls
The downside to struct variables is that they are copied each time when returning from or assigned to a function. Obviously, this can become a bottleneck too.
E.g. imagine a 4x4 matrix. 16 Float values would have to be copied on every assign/return which would be 1'024 bits on a 64 bit system.
One way you can avoid this is using inout when passing variables to functions, which is basically Swifts way of creating a pointer. But then we're also discouraged from using inout.
So to my question:
How should I handle large, immutable data structures in Swift?
Do I have to worry creating a large struct with many members?
If yes, when am I crossing the line?
This accepted answer is not entirely answering the question you had: Swift always copies structs. The trick that Array/Dictionary/String/etc do is that they are just wrappers around classes (which contain the actual stored properties). That way sizeof(Array) is just the size of the pointer to that class (MemoryLayout<Array<String>>.stride == MemoryLayout<UnsafeRawPointer>.stride)
If you have a really big struct, you might want to consider wrapping its stored properties in a class for efficient passing around as arguments, and checking isUniquelyReferenced before mutating to give COW semantics.
Structs have other efficiency benefits: they don't need reference-counting and can be decomposed by the optimiser.
In Swift, values keep a unique copy of their data. There are several
advantages to using value-types, like ensuring that values have
independent state. When we copy values (the effect of assignment,
initialization, and argument passing) the program will create a new
copy of the value. For some large values these copies could be time
consuming and hurt the performance of the program.
https://github.com/apple/swift/blob/master/docs/OptimizationTips.rst#the-cost-of-large-swift-values
Also the section on container types:
Keep in mind that there is a trade-off between using large value types
and using reference types. In certain cases, the overhead of copying
and moving around large value types will outweigh the cost of removing
the bridging and retain/release overhead.
From the very bottom of this page from the Swift Reference:
NOTE
The description above refers to the “copying” of strings, arrays, and dictionaries. The behavior you see in your code will always be as if a copy took place. However, Swift only performs an actual copy behind the scenes when it is absolutely necessary to do so. Swift manages all value copying to ensure optimal performance, and you should not avoid assignment to try to preempt this optimization.
I hope this answers your question, also if you want to be sure that an array doesn't get copied, you can always declare the parameter as inout, and pass it with &array into the function.
Also classes add a lot of overhead and should only be used if you really must have a reference to the same object.
Examples for structs:
Timezone
Latitude/Longitude
Size/Weight
Examples for classes:
Person
A View