I'm currently setting up a server and successfully passing functions and test data to that server.
Is there an eloquent way to clear out all the functions, variables, tables, etc.? Since we are running KDB in a Docker container and accessing it via IP address, I prefer not to have to restart the q session on the server, but rather assign many/all values and functions to :: or null.
At present I'm assuming I have to reassign each function/variable/table to :: or similar to achieve this. It isn't a big issue but I expect to have numerous functions.
\p 5042
h:hopen `:XXX.XXX.XX.XX:5042
h "sq:{x*x}" //send sample function sq to server
h "sq: ::" //assign function sq to nothing (repeat for all variables/functions/tables etc)
hclose h
//check the IP address list of functions to confirm deletion http://XXX.XXX.XX.XX:5042/?\f
Would you be able to give us a bit more information as to why you want to clear out everything (and in particular, why do you want to set functions to null)?
Otherwise, you can do a few things. You can do a delete statement on a namespace to delete everything in it. To do delete all tables/variables/functions in the global namespace, you can do the following.
q)a: 1
q)b: 1 2 3
q)f: {1 + x}
q)value `.
a| 1
b| 1 2 3
f| {1 + x}
q)delete from `.
`.
q)value `.
q)f
'f
[0] f
^
q)
If you want to null them rather than delete them, you could use the system commands a, f and v to get lists of all the tables (a), functions (f) and varialbes (v) in the global (or other) name space, and then use set to set them all to null.
q)f: {1+x}
q)g: {2*x}
q)(system"f")set'(::)
`f`g
q)f
q)g
q)
Is this roughly what you were looking for?
(One obvious problem with this is that you might end up deleting other peoples variables.)
Few extra points to add to Matt's:
.Q.gc[] may be a good idea to run after doing this. This will return any memory to the OS that was being used by those variables in the root namespace e.g. if you had a large table defined before this.
Use an alternative namespace for functions you want to keep after this clearing e.g. .utils. You could even add a .utils.refresh function which clears the root namespace and runs .Q.gc[]
I wanted to test what happened if I replace a variable with a different data type:
clear
input id x0
1 1
2 13
3 .
end
list
save tabA, replace
clear
input id str5 x0
1 "1"
2 "23"
3 "33"
end
list
save tabB, replace
use tabA, clear
mmerge id using tabB, type(1:1) update replace
list
The result is:
+--------------------------------------------------+
| id x0 _merge |
|--------------------------------------------------|
1. | 1 1 in both, master agrees with using data |
2. | 2 13 in both, master agrees with using data |
3. | 3 . in both, master agrees with using data |
+--------------------------------------------------+
This seems very strange to me. I expected breakdown or disagreement. Is this a bug or am I missing something?
mmerge is user-written (Jeroen Weesie, SSC, 2002).
If you use the official merge in an up-to-date Stata, you will get what you expect.
. merge 1:1 id using tabB, update replace
x0 is str5 in using data
r(106);
I have not looked inside mmerge. My own guess is that what you see is a feature from the author's point of view, namely that it's not a problem if one variable is numeric and one variable is string so long as their contents agree. But why are you not using merge directly? There was a brief period several years ago when mmerge had some advantages over merge, but that's long past. BTW, I agree in wanting my merges to be very conservative and not indulgent on variable types.
If the bisect range includes multiple branches, how does hg bisect's search work. Does it effectively bisect each sub-branch (I would think that would be inefficient)?
For instance, borrowing, with gratitude, a diagram from an answer to this related question, what if the bisect got to changeset 7 on the "good" right-side branch first.
# 12:8ae1fff407c8:bad6
|
o 11:27edd4ba0a78:bad5
|
o 10:312ba3d6eb29:bad4
|\
| o 9:68ae20ea0c02:good33
| |
| o 8:916e977fa594:good32
| |
| o 7:b9d00094223f:good31
| |
o | 6:a7cab1800465:bad3
| |
o | 5:a84e45045a29:bad2
| |
o | 4:d0a381a67072:bad1
| |
o | 3:54349a6276cc:good4
|/
o 2:4588e394e325:good3
|
o 1:de79725cb39a:good2
|
o 0:2641cc78ce7a:good1
Will it then look only between 7 and 12, missing the real first-bad that we care about? (thus using "dumb" numerical order) or is it smart enough to use the full topography and to know that the first bad could be below 7 on the right-side branch, or could still be anywhere on the left-side branch.
The purpose of my question is both (a) just to understand the algorithm better, and (b) to understand whether I can liberally extend my initial bisect range without thinking hard about what branch I go to. I've been in high-branching bisect situations where it kept asking me after every test to extend beyond the next merge, so that the whole procedure was essentially O(n). I'm wondering if I can just throw the first "good" marker way back past some nest of merges without thinking about it much, and whether that would save time and give correct results.
To quote from Mercurial: The Definitive Guide:
The hg bisect command is aware of the “branchy” nature of a Mercurial
project's revision history, so it has no problems dealing with
branches, merges, or multiple heads in a repository. It can prune
entire branches of history with a single probe, which is how it
operates so efficiently.
The code that does the work is in hbisect.py and actually looks at the descendent and ancestor trees from each node where the state has been determined.
It looks to me like the changeset chosen to test is chosen by weighting "how central" it is in graph of those yet to test (i.e. bisecting by ancestors vs. non-ancestors, rather than chronology):
108 x = len(a) # number of ancestors
109 y = tot - x # number of non-ancestors
110 value = min(x, y) # how good is this test?
While answering this question, I came to realize that I was not sure whether Perl's map can be considered a loop or not?
On one hand, it quacks/walks like a loop (does O(n) work, can be easily re-written by an equivalent loop, and sort of fits the common definition = "a sequence of instructions that is continually repeated").
On the other hand, map is not usually listed among Perl's control structures, of which loops are a subset of. E.g. http://en.wikipedia.org/wiki/Perl_control_structures#Loops
So, what I'm looking for is a formal reason to be convinced of one side vs. the other. So far, the former (it is a loop) sounds a lot more convincing to me, but I'm bothered by the fact that I never saw "map" mentioned in a list of Perl loops.
map is a higher level concept than loops, borrowed from functional programming. It doesn't say "call this function on each of these items, one by one, from beginning to end," it says "call this function on all of these items." It might be implemented as a loop, but that's not the point -- it also might be implemented asynchronously -- it would still be map.
Additionally, it's not really a control structure in itself -- what if every perl function that used a loop in its implementation were listed under "loops?" Just because something is implemented using a loop, doesn't mean it should be considered its own type of loop.
No, it is not a loop, from my perspective.
Characteristic of (perl) loops is that they can be broken out of (last) or resumed (next, redo). map cannot:
map { last } qw(stack overflow); # ERROR! Can't "last" outside a loop block
The error message suggests that perl itself doesn't consider the evaluated block a loop block.
From an academic standpoint, a case can be made for both depending on how map is defined. If it always iterates in order, then a foreach loop could be emulated by map making the two equivalent. Some other definitions of map may allow out of order execution of the list for performance (dividing the work amongst threads or even separate computers). The same could be done with the foreach construct.
But as far as Perl 5 is concerned, map is always executed in order, making it equivalent to a loop. The internal structure of the expression map $_*2, 1, 2, 3 results in the following execution order opcodes which show that map is built internally as a while-like control structure:
OP enter
COP nextstate
OP pushmark
SVOP const IV 1
SVOP const IV 2
SVOP const IV 3
LISTOP mapstart
LOGOP (0x2f96150) mapwhile <-- while still has items, shift one off into $_
PADOP gvsv GV *_
SVOP const IV 2 loop body
BINOP multiply
goto LOGOP (0x2f96150) <-- jump back to the top of the loop
LISTOP leave
The map function is not a loop in Perl. This can be clearly seen by the failure of next, redo, and last inside a map:
perl -le '#a = map { next if $_ %2; } 1 .. 5; print for #a'
Can't "next" outside a loop block at -e line 1.
To achieve the desired affect in a map, you must return an empty list:
perl -le '#a = map { $_ %2 ? () : $_ } 1 .. 5; print for #a'
2
4
I think transformation is better name for constructs like map. It transforms one list into another. A similar function to map is List::Util::reduce, but instead of transforming a list into another list, it transforms a list into a scalar value. By using the word transformation, we can talk about the common aspects of these two higher order functions.
That said, it works by visiting every member of the list. This means it behaves much like a loop, and depending on what your definition of "a loop" is it might qualify. Note, my definition means that there is no loop in this code either:
#!/usr/bin/perl
use strict;
use warnings;
my $i = 0;
FOO:
print "hello world!\n";
goto FOO unless ++$i == 5;
Perl actually does define the word loop in its documentation:
loop
A construct that performs something repeatedly, like a roller
coaster.
By this definition, map is a loop because it preforms its block repeatedly; however, it also defines "loop control statement" and "loop label":
loop control statement
Any statement within the body of a loop that can make a loop
prematurely stop looping or skip an "iteration". Generally you
shouldn't try this on roller coasters.
loop label
A kind of key or name attached to a loop (or roller coaster) so
that loop control statements can talk about which loop they want to
control.
I believe it is imprecise to call map a loop because next and its kin are defined as loop control statements and they cannot control map.
This is all just playing with words though. Describing map as like-a-loop is a perfectly valid way of introducing someone to it. Even the documentation for map uses a foreach loop as part of its example:
%hash = map { get_a_key_for($_) => $_ } #array;
is just a funny way to write
%hash = ();
foreach (#array) {
$hash{get_a_key_for($_)} = $_;
}
It all depends on the context though. It is useful to describe multiplication to someone as repeated addition when you are trying to get him or her to understand the concept, but you wouldn't want him or her to continue to think of it that way. You would want him or her to learn the rules of multiplication instead of always translating back to the rules of addition.
Your question turns on the issue of classification. At least under one interpretation, asking whether map is a loop is like asking whether map is a subset of "Loop". Framed in this way, I think the answer is no. Although map and Loop have many things in common, there are important differences.
Loop controls: Chas. Owens makes a strong case that Perl loops are subject to loop controls like next and last, while map is not.
Return values: the purpose of map is its return value; with loops, not so much.
We encounter relationships like this all the time in the real world -- things that have much in common with each other, but with neither being a perfect subset of the other.
-----------------------------------------
|Things that iterate? |
| |
| ------------------ |
| |map() | |
| | | |
| | --------|---------- |
| | | | | |
| | | | | |
| ------------------ | |
| | | |
| | Loop| |
| ------------------ |
| |
-----------------------------------------
map is a higher-order function. The same applies to grep. Book Higher-Order Perl explains the idea in full details.
It's sad to see that discussion moved towards implementation details, not the concept.
FM's and Dave Sherohman's answers are quite good, but let me add an additional way of looking at map.
map is a function which is guaranteed to look at every element of a structure exactly once. And it is not a control structure, as it (itself) is a pure function. In other words, the invariants that map preserves are very strong, much stronger than 'a loop'. So if you can use a map, that's great, because you then get all these invariants 'for free', while if you're using a (more general!) control structure, you'll have to establish all these invariants yourself if you want to be sure your code is right.
And that's really the beauty of a lot of these higher-order functions: you get many more invariants for free, so that you as a programmer can spend your valuable thinking time maintaining application-dependent invariants instead of worrying about low-level implementation-dependent issues.
map itself is generally implemented using a loop of some sort (to loop over iterators, typically), but since it is a higher-level structure, it's often not included in lists of lower-level control structures.
Here is a definition of map as a recurrence:
sub _map (&#) {
my $f = shift;
return unless #_;
return $f->( local $_ = shift #_ ),
_map( $f, #_ );
}
my #squares = _map { $_ ** 2 } 1..100;
"Loop" is more of a CS term rather than a language-specific one. You can be reasonably confident in calling something a loop if it exhibits these characteristics:
iterates over elements
does the same thing every time
is O(n)
map fits these pretty closely, but it's not a loop because it's a higher-level abstraction. It's okay to say it has the properties of a loop, even if it itself isn't a loop in the strictest, lowest-level sense.
I think map fits the definition of a Functor.
It all depends on how you look at it...
On the one hand, Perl's map can be considered a loop, if only because that's how it's implemented in (current versions of) Perl.
On the other, though, I view it as a functional map and choose to use it accordingly which, among other things, includes only making the assumption that all elements of the list will be visited, but not making any assumptions about the order in which they will be visited. Aside from the degree of functional purity this brings and giving map a reason to exist and be used instead of for, this also leaves me in good shape if some future version of Perl provides a parallelizable implementation of map. (Not that I have any expectation of that ever happening...)
I think of map as more akin to an operator, like multiplication. You could even think of integer multiplication as a loop of additions :). It's not a loop of course, even if it were stupidly implemented that way. I see map similarly.
A map in Perl is a higher order function that applies a given function to all elements of an array and returns the modified array.
Whether this is implemented using an iterative loop or by recursion or any other way is not relevant and unspecified.
So a map is not a loop, though it may be implemented using a loop.
Map only looks like a loop if you ignore the lvalue. You can't do this with a for loop:
print join ' ', map { $_ * $_ } (1 .. 5)
1 4 9 16 25
One of my pet peeves with debugging Perl code (in command line debbugger, perl -d) is the fact that mistakenly printing (via x command) the contents of a huge datastructure is guaranteed to freeze up your terminal for forever and a half while 100s of pages of data are printed. Epecially if that happens across slowish network.
As such, I'd like to be able to limit the amount of data that x prints.
I see two approaches - I'd be willing to try either if someone knows how to do.
Limit the amount of data any single command in debugger prints.
Better yet, somehow replace the built-in x command with a custom Perl method (which would calculate the "size" of the data structure, and refuse to print its contents without confirmation).
I'm specifically asking "how to replace x with custom code" - building a Good Enough "is the data structure too big" Perl method is something I can likely do on my own without too much effort although I see enough pitfalls preventing the "perfect" one from being a fairly frustrating endeavour. Heck, merely doing Data::Dumper->Dump and taking the length of the string might do the trick :)
Please note that I'm perfectly well aware of how to manually avoid the issue by recursively examining layers of datastructure (e.g. print the ref, print the count of keys/array elements, etc...)... the whole point is I want to be able to avoid thoughtlessly typing x $huge_pile_of_data without thinking - or stumbling on a bug populating said huge pile of data into what should be a scalar.
The x command takes an optional argument for the maximum depth to display. That's not quite the same as limiting the amount of data to N pages, but it's definitely useful to prevent overload.
DB<1> %h = (a => { b => { c => 1 } } )
DB<2> x %h
0 'a'
1 HASH(0x1d5ff44)
'b' => HASH(0x1d61424)
'c' => 1
DB<3> x 2 %h
0 'a'
1 HASH(0x1d5ff44)
'b' => HASH(0x1d61424)
You can specify the default depth to print via the o command, e.g.
DB<1>o dumpDepth=1
Add that to your .perldb file to apply it to all debugger sessions.
Otherwise, it looks like the x command invokes DB::dumpit() which is just a wrapper for dumpval.pl (or, more specifically, the main::dumpValue() sub it defines). You could modify/replace that script as you see fit. I'm not sure how you'd make it interactive, though.
The | command in the debugger pipes another command's output to your pager, e.g.
DB<1> |x %huge_datastructure