Is there a way to calculate the square root function on Ciphertexts? I didn't see it see it in the evaluator method. If it's not possible with SEAL 2.3.1 will it be possible with future SEAL releases?
This functionality is not in SEAL at the moment. SEAL is a low level library and aims to provide only the most fundamental building blocks; including a high-level function such as sqrt in SEAL is unreasonable since how exactly it is best evaluated (on encrypted data) depends a lot on the context. Therefore, sqrt is unlikely to ever be provided by the SEAL API.
Related
Blocks and functions in Modelica have some similarities and differences. In blocks, output variables are most likely expressed in terms of input variables using equations, whereas in functions output variables are expressed in terms of input variables using assignments. Given a relationship y = f(u) that can be expressed using both notions, I am interested in knowing which notion shall you favour in which situation?
Personally,
Blocks can be better integrated in block diagrams using input/output connectors
Equations in blocks can be most likely better treated by compilers for symbolic manipulation, optimization, and evaluating analytical derivatives required for Jacobian evaluation. So I guess blocks are likely less sensitive to numerical errors in some boundary cases. For functions, derivatives are likely to be evaluated using finite difference methods, if they are not explicitly provided.
on the other hand a set of assignments in a function will be most likely treated as a single equation. The same set of assignments if expressed in terms of a larger set of equations in a block will result in a model of larger size probably leading to a decrease in runtime performance
although a block with an algorithmic section is kind of equivalent to a function with the same assignments set, the syntax of a function call is favored in couple of situations
One can establish hierarchies of blocks types and do all of sort of things of object oriented modelings. Functions are kind of limited. It is not possible to extend from a non-abstract function that contains an algorithm section. But it is possible to have (an) abstract function(s) that act(s) as (an) interface(s) out of which implemented functions can be established etc.
Some of the above arguments are dependent on the way a specific simulation environment treats a block or a function. These might be low-level details not necessarily known.
The list in your "question" is already a pretty good summary. Still there are some additional things that should be considered:
Regarding the differentiation of functions, the developer at least needs to define how often the assignments can be differentiated (here is a nice read on this), as e.g. Dymola will not do it automatically. Alternatively the differentiated function can be specified manually (here). By the way, a partial derivative can be defined as well, see Language Specification, Sec. 12.7.2.
When it is necessary to invert a function, it can be necessary to define it manually. This is described in the Language Specification, Sec. 12.8.
Also it could be important that code from a function can be inlined, which should overcome some of the issues mentioned above, see Language Specification, Sec. 18.3.
Generally I would go for blocks whenever there is no very strong reason for a function. Some that come to my mind are the need for procedural execution, or for-loops.
This is just my two cents - more opinions welcome...
You might be interested in the opposite: calling a block as if it was a function:
https://github.com/modelica/ModelicaSpecification/issues/1512
The advantage of using function syntax is that you don't need to declare + connect components:
Block b;
equation
connect(x, b.in1);
connect(y, b.in2);
connect(z, b.out1);
vs
z = Block(x, y);
Of course right now, this syntax does not exist yet. And you really want to use blocks when you can. Algorithmic blocks might as well be functions as they are shorter and easier to write and will introduce fewer trajectories in your result-file (good unless you want to debug what happens inside the function call I guess).
Learning about higher order functions that are available to collections in the Swift language has been exciting. I think the answer to my question is no because it seems higher order functions simplifies the code that needs to be written for the overall underlying process. However, I'd like a second opinion from the community just to be sure.
Using higher order functions has no effect on the time & space complexity. Higher order functions are simply a wrapper to replace for-loops (and rid of mutable states)
I believe (from doing some reading) that reading/writing data across the bus from CPU caches to main memory places a considerable constraint on how fast a computational task (which needs to move data across the bus) can complete - the Von Neumann bottleneck.
I have come across a few articles so far which mention that functional programming can be more performant than other paradigms like the imperative approach eg. OO (in certain models of computation).
Can someone please explain some of the ways that purely functional programming can reduce this bottleneck? ie. are any of the following points found (in general) to be true?
Using immutable data structures means generally less data is moving across that bus - less writes?
Using immutable data structures means that data is possibly more likely to be hanging around in CPU cache - because less updates to existing state means less flushing of objects from cache?
Is it possible that using immutable data structures means that we may often never even read the data back from main memory because we may create the object during computation and have it in local cache and then during same time slice create a new immutable object off of it (if there is a need for an update) and we then never use original object ie. we are working a lot more with objects that are sitting in local cache.
Oh man, that’s a classic. John Backus’ 1977 ACM Turing Award lecture is all about that: “Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs.” (The paper, “Lambda: The Ultimate Goto,” was presented at the same conference.)
I’m guessing that either you or whoever raised this question had that lecture in mind. What Backus called “the von Neumann bottleneck” was “a connecting tube that can transmit a single word between the CPU and the store (and send an address to the store).”
CPUs do still have a data bus, although in modern computers, it’s usually wide enough to hold a vector of words. Nor have we gotten away from the problem that we need to store and look up a lot of addresses, such as the links to daughter nodes of lists and trees.
But Backus was not just talking about physical architecture (emphasis added):
Not only is this tube a literal bottleneck for the data traffic of a problem, but, more importantly, it is an intellectual bottleneck that has kept us tied to word-at-a-time thinking instead of encouraging us to think in terms of the larger conceptual units of the task at hand. Thus programming is basically planning and detailing the enormous traffic of words through the von Neumann bottleneck, and much of that traffic concerns not significant data itself but where to find it.
In that sense, functional programming has been largely successful at getting people to write higher-level functions, such as maps and reductions, rather than “word-at-a-time thinking” such as the for loops of C. If you try to perform an operation on a lot of data in C, today, then just like in 1977, you need to write it as a sequential loop. Potentially, each iteration of the loop could do anything to any element of the array, or any other program state, or even muck around with the loop variable itself, and any pointer could potentially alias any of these variables. At the time, that was true of the DO loops of Backus’ first high-level language, Fortran, as well, except maybe the part about pointer aliasing. To get good performance today, you try to help the compiler figure out that, no, the loop doesn’t really need to run in the order you literally specified: this is an operation it can parallelize, like a reduction or a transformation of some other array or a pure function of the loop index alone.
But that’s no longer a good fit for the physical architecture of modern computers, which are all vectorized symmetric multiprocessors—like the Cray supercomputers of the late ’70s, but faster.
Indeed, the C++ Standard Template Library now has algorithms on containers that are totally independent of the implementation details or the internal representation of the data, and Backus’ own creation, Fortran, added FORALL and PURE in 1995.
When you look at today’s big data problems, you see that the tools we use to solve them resemble functional idioms a lot more than the imperative languages Backus designed in the ’50s and ’60s. You wouldn’t write a bunch of for loops to do machine learning in 2018; you’d define a model in something like Tensorflow and evaluate it. If you want to work with big data with a lot of processors at once, it’s extremely helpful to know that your operations are associative, and therefore can be grouped in any order and then combined, allowing for automatic parallelization and vectorization. Or that a data structure can be lock-free and wait-free because it is immutable. Or that a transformation on a vector is a map that can be implemented with SIMD instructions on another vector.
Examples
Last year, I wrote a couple short programs in several different languages to solve a problem that involved finding the coefficients that minimized a cubic polynomial. A brute-force approach in C11 looked, in relevant part, like this:
static variable_t ys[MOST_COEFFS];
// #pragma omp simd safelen(MOST_COEFFS)
for ( size_t j = 0; j < n; ++j )
ys[j] = ((a3s[j]*t + a2s[j])*t + a1s[j])*t + a0s[j];
variable_t result = ys[0];
// #pragma omp simd reduction(min:y)
for ( size_t j = 1; j < n; ++j ) {
const variable_t y = ys[j];
if (y < result)
result = y;
} // end for j
The corresponding section of the C++14 version looked like this:
const variable_t result =
(((a3s*t + a2s)*t + a1s)*t + a0s).min();
In this case, the coefficient vectors were std::valarray objects, a special type of object in the STL that have restrictions on how their components can be aliased, and whose member operations are limited, and a lot of the restrictions on what operations are safe to vectorize sound a lot like the restrictions on pure functions. The list of allowed reductions, like .min() at the end, is, not coincidentally, similar to the instances of Data.Semigroup. You’ll see a similar story these days if you look through <algorithm> in the STL.
Now, I’m not going to claim that C++ has become a functional language. As it happened, I made all the objects in the program immutable and automatically collected by RIIA, but that’s just because I’ve had a lot of exposure to functional programming and that’s how I like to code now. The language itself doesn’t impose such things as immutability, garbage collection or absence of side-effects. But when we look at what Backus in 1977 said was the real von Neumann bottleneck, “an intellectual bottleneck that has kept us tied to word-at-a-time thinking instead of encouraging us to think in terms of the larger conceptual units of the task at hand,” does that apply to the C++ version? The operations are linear algebra on coefficient vectors, not word-at-a-time. And the ideas C++ borrowed to do this—and the ideas behind expression templates even more so—are largely functional concepts. (Compare that snippet to how it would’ve looked in K&R C, and how Backus defined a functional program to compute inner product in section 5.2 of his Turing Award lecture in 1977.)
I also wrote a version in Haskell, but I don’t think it’s as good an example of escaping that kind of von Neumann bottleneck.
It’s absolutely possible to write functional code that meets all of Backus’ descriptions of the von Neumann bottleneck. Looking back on the code I wrote this week, I’ve done it myself. A fold or traversal that builds a list? They’re high-level abstractions, but they’re also defined as sequences of word-at-a-time operations, and half or more of the data passed through the bottleneck when you create and traverse a singly-linked list is the addresses of other data! They’re efficient ways to put data through the von Neumann bottleneck, and that’s basically why I did it: they’re great patterns for programming von Neumann machines.
If we’re interested in coding a different way, however, functional programming gives us tools to do so. (I’m not going to claim it’s the only thing that does.) Express a reduction as a foldMap, apply it to the right kind of vector, and the associativity of the monoidal operation lets you split up the problem into chunks of whatever size you want and combine the pieces later. Make an operation a map rather than a fold, on a data structure other than a singly-linked list, and it can be automatically parallelized or vectorized. Or transformed in other ways that produce the same result, since we’ve expressed the result at a higher level of abstraction, not a particular sequence of word-at-a-time operations.
My examples so far have been about parallel programming, but I’m sure quantum computing will shake up what programs look like a lot more fundamentally.
I have been reading about Finagle and trying to understand the code to figure out how Aperture's subset choice works.
I have seen that ApertureLeastLoaded has a "useDeterministicOrdering" and an "EndpointFactory" which I guess should be the key points to make the decision of which clients to take in the subset.
While reading the "deterministic subsetting" section of Google SRE's book, I understood that the best way to pick a subset of servers from the client point of view, is to know the total number of clients, and a unique sequential identifier of the current client, that can be used as seed of the subset generator.
In Finagle I can't understand how this process is done (I'm not super familiar with Scala) and the documentation both on the website and in the code, explain just how the aperture paradigm works, but not very clear how the initial subset is chosen
I hope somebody can enlighten me
One of the unique properties of Aperture is that its window is sized dynamically based on a clients offered load. That is, clients have a built in controller which can expand or shrink their window at runtime. This property is important as it allows clients to operate more efficiently and better adapt to a changing environment, but it does make it more complex to achieve a uniform load distribution across servers.
To contrast, the subsetting algorithm, as proposed by the Google SRE book, suggests that operators choose a static subset size which allows a uniform load distribution to be calculated analytically but introduces another static configuration that needs to be revisited as a system evolves.
Deterministic Aperture is, to the best of our knowledge, a novel algorithm for achieving a uniform load distribution while maintaining the dynamic properties of the window sizing mentioned above. From a high level, clients construct a topology of their peer cluster (which gives them a sense of ordering and proximity) and then derive a unique per-client permutation of the servers from the topology such that each server is uniformly represented across the permutations.
We are still in the early stages of testing this in production at Twitter, but early results look very promising. After we gather more empirical results, we hope to publish some more detailed content on how the algorithm works and its properties.
SciPy provides two functions for nonlinear least squares problems:
optimize.leastsq() uses the Levenberg-Marquardt algorithm only.
optimize.least_squares() allows us to choose the Levenberg-Marquardt, Trust Region Reflective, or Trust Region Dogleg algorithm.
Should we always use least_squares() instead of leastsq()?
If so, what purpose does the latter serve?
Short answer
Should we always use least_squares() instead of leastsq()?
Yes.
If so, what purpose does the latter serve?
Backward compatibility.
Explanation
The least_squares function is new in 0.17.1. Its documentation refers to leastsq as
A legacy wrapper for the MINPACK implementation of the Levenberg-Marquadt algorithm.
The original commit introducing least_squares actually called leastsq when the method was chosen to be 'lm'. But the contributor (Nikolay Mayorov) then decided that
least_squares might feel more solid and homogeneous if I write a new wrapper to MINPACK functions, instead of calling leastsq.
and so he did. So, leastsq is no longer required by least_squares, but I'd expect it to be kept at least for a while, to avoid breaking old code.