I was going thru few tutorials for Tree DS and I found this code which is really confusing to understand. Please explain
public func forEachDepthFirst(visit: (TreeNode) -> Void) {
visit(self) // 1
children.forEach { // 2
$0.forEachDepthFirst(visit: visit)
}
}
}
Why do we have visit(self) here?
I see explanation here https://forums.raywenderlich.com/t/help-understanding-the-recursion-for-depth-first-traversal/56552/2 but its still not clear
Any recusive method has
1- base case : which ends the run and here it's
children.forEach // when children property is empty meaning a leaf node
2- recusive case
$0.forEachDepthFirst(visit: visit) // call the same method with it's children
Your method takes a closure / completion that's be called for every node inside the main root node
So suppose You have root
0
- 1
- 1.1 , 1.2 , 1.3
- 2
- 2.1 , 2.2 , 2.3
Here 0 node is called then when runnign your function
visit(0)
children.forEach { // = 1,2
for 0 > 1
visit(1)
children.forEach { // = 1.1,1.2,1.3
for 0 > 2
visit(2)
children.forEach { // = 2.1,2.2,2.3
Inner case
for 0 > 1 > 1.1
visit(1.1)
children.forEach { // end here as there is no childrens ( leaf node)
so on for 1.2,1,3
for 0 > 2 > 2.1 / 2.2 / 2.3 same as above case
How to call
your method is an instance method inside the tree so every node can call it , if you want to traverse nodes of 0 then do this
zeroNode.forEachDepthFirst { (item) in
print(item.name) // suppose node object has a name
}
Then you will get
0 , 1 , 1.1 , 1.2 , 1.3 , 2.1 , 2.2 , 2.3
And that's as you called visit(NodeObject) for the main node and recursively all it's childrens
Why do we have visit(self) here?
Because if we didn't, we would never actually do anything to any of the nodes on the tree!
Consider this tree:
n1 -> n2 -> n3 -> n4
We now call our method forEachDepthFirst on n1. If we didn't have visit(self), we would immediately call forEachDepthFirst on n2, which would call it on n3, which would call it on n4. And then we'd stop. But at no time would we have called visit, so we would have looped through every node in the tree without doing anything to those nodes.
Related
I am attempting to run Daikon on a .decls and .dtrace file I generated from a CSV file using an open-source perl script. The .decls and .dtrace file will be provided below. The daikon.jar file is held within a directory, which has a sub-directory "scripts" where I keep the .dtrace and .decls.
I am attempting to call daikon using the following command from within the directory containing the daikon.jar file:
java -cp daikon.jar daikon.Daikon scripts/example.dtrace scripts/example.decls
The program response is the following:
Daikon version 5.8.10, released November 1, 2021; http://plse.cs.washington.edu/daikon.
(read 1 decls file)
Processing trace data; reading 1 dtrace file:
Error at line 1 in file scripts/example.dtrace: No declaration was provided for program point program.point:::POINT
I am confused as to why it can't find the declarations file I provided which contains the declaration for the program.point function. Below I have provided the contents of both the example.dtrace and the example.decls files.
example.dtrace
program.point:::POINT
a
1
1
b
1
1
c
2
1
d
2
1
e
4
1
aprogram.point:::POINT
a
3
1
b
3
1
c
4
1
d
4
1
e
5
1
example.decls
DECLARE
aprogram.point:::POINT
a
double
double
1
b
double
double
1
c
double
double
1
d
double
double
1
e
double
double
1
Your example.decls file declares a program point named aprogram.point:::POINT, which starts with an a. Your example.dtrace file contains samples for a program point named program.point:::POINT, which does not start with an a.
So, the message is right: there is no declaration for a program point named program.point:::POINT, though there is a declaration for a program point named aprogram.point:::POINT.
Making the program point names consistent between the two files should resolve your problem. By adding the character a to the beginning of your example.dtrace file, I was able to get Daikon to produce output:
Daikon version 5.8.11, released November 2, 2021; http://plse.cs.washington.edu/daikon.
(read 1 decls file)
Processing trace data; reading 1 dtrace file:
[2021-11-17T10:13:50.284232]: Finished reading example.dtrace
===========================================================================
aprogram.point:::POINT
a == b
c == d
a one of { 1.0, 3.0 }
c one of { 2.0, 4.0 }
e one of { 4.0, 5.0 }
Exiting Daikon.
Basically, I have a graph coloring program where each node with an edge to another node has to be different colors. Here, is my code:
node(1..4).
edge(1,2).
edge(2,3).
edge(3,4).
edge(4,1).
edge(2,4).
color(1..3).
{ assign(N,C) : color(C) } = 1 :- node(N).
1 { assign(N,1) : color(1) } 1 :- node(N). %line in question
:- edge(N,M), assign(N,C), assign(M,C).
How would I tell the program to only assign color 1, once? The line labeled %line in question is the line giving me problems. Here is another solution I tried that didn't work:
node(1..4).
edge(1,2).
edge(2,3).
edge(3,4).
edge(4,1).
edge(2,4).
color(1..3).
{ assign(N,C) : color(C) } = 1 :- node(N).
:- edge(N,M), assign(N,C), assign(M,C).
vtx(Node, Color) :- node(Node), color(Color).
1 { vtx(N, 1) : color(1) } 1 :- node(N).
#show vtx/2.
If anyone could help me out it would be much appreciated.
In this simple case of restricting a single color to be used once, you can write the a single constraint
:- assign(N, 1), assign(M, 1), node(N), node(M), N!=M.
Actually, the line you marked as in question :
1 { assign(N,1) : color(1) } 1 :- node(N). %line in question
can be translated as
If N is a node, we will (and we must) assign color(1) to node(N) and only assign once, i.e. If node(i) is true, we will have exactly one node(i, 1).
Therefore, with this rule and your facts node(1..4), you will immediately get assign(1,1), assign(2,1), assign(3,1), assign(4,1). This is defninitely unsatisfiable under color problem (with the last constraint).
Back to your requirement:
How would I tell the program to only assign color 1, once?
The problem here is the constraint you set in the line: "color 1 is assigned only once" applies to each node(i), i=1,2,3,4 instead of all nodes.
To make it clearer, you might as well consider that this line would be instantiated as:
1 { assign(1,1) : color(1) } 1 :- node(1).
1 { assign(2,1) : color(1) } 1 :- node(2).
1 { assign(3,1) : color(1) } 1 :- node(3).
1 { assign(4,1) : color(1) } 1 :- node(4).
With node(1..4) all true, we will have assign(1,1), assign(2,1), assign(3,1), assign(4,1).
What you want is assign(N, 1) appears once and only once in the answer, thus in your rule, this should be true with no premiere condition.
Therefore, change the problem line into:
{ assign(N,1): node(N), color(1) } = 1. %problem line changed
You will get the proper assignment:
clingo version 5.4.0
Reading from test.lp
Solving...
Answer: 1
assign(2,2) assign(1,3) assign(3,3) assign(4,1)
Answer: 2
assign(1,2) assign(2,3) assign(3,2) assign(4,1)
Answer: 3
assign(2,1) assign(1,3) assign(3,3) assign(4,2)
Answer: 4
assign(2,1) assign(1,2) assign(3,2) assign(4,3)
SATISFIABLE
Intuitively, this line means the assign(N, 1) should be in answer set under no condition, as long as N is a node. This will count all nodes instead of every single one.
My objective is to process a series of SequenceFile folders generated by calling org.apache.spark.rdd.RDD[_].saveAsObjectFile(...). My folder structure is similar to this:
\MyRootDirectory
\Batch0001
_SUCCESS
part-00000
part-00001
...
part-nnnnn
\Batch0002
_SUCCESS
part-00000
part-00001
...
part-nnnnn
...
\Batchnnnn
_SUCCESS
part-00000
part-00001
...
part-nnnnn
I need to extract some of the persisted data, however my collection - whether I use a ListBuffer, mutable.Map, or any other mutable type, loses scope and appears to be newed up on each iteration of sequenceFile(...).foreach
The following proof of concept generates a series of "Processing directory..." followed by "1 : 1" repeated and never increasing, as I expected counter and intList.size to do.
private def proofOfConcept(rootDirectoryName: String) = {
val intList = ListBuffer[Int]()
var counter: Int = 0
val config = new SparkConf().setAppName("local").setMaster("local[1]")
new File(rootDirectoryName).listFiles().map(_.toString).foreach { folderName =>
println(s"Processing directory $folderName...")
val sc = new SparkContext(config)
sc.setLogLevel("WARN")
sc.sequenceFile(folderName, classOf[NullWritable], classOf[BytesWritable]).foreach(f => {
counter += 1
intList += counter
println(s" $counter : ${intList.size}")
})
sc.stop()
}
}
Output:
"C:\Program Files\Java\jdk1.8.0_111\bin\java" ...
Processing directory C:\MyRootDirectory\Batch0001...
17/05/24 09:30:25.228 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[Stage 0:> (0 + 0) / 57] 1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
Processing directory C:\MyRootDirectory\Batch0002...
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
Processing directory C:\MyRootDirectory\Batch0003...
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
1 : 1
The function inside foreach is run in a spark worker JVM, not inside the client JVM, where the variable is defined. That worker gets a copy of that variable locally, increments it, and prints it. My guess is you are testing this locally? If you were running this in a production, distributed spark environment, you wouldn't even see the output of those prints.
More generally, pretty much any function you pass into one of RDD's methods will probably be actually executed remotely and will not have mutable access to any local variables or anything. It will get an essentially immutable snapshot of them.
If you want to move data from spark's distributed storage back to the client, use RDD's collect method. The reverse is done with sc.parallelize. But note that both of these are usually done very rarely, since they do not happen in parallel.
I'm trying to group my observable values into groups using windowCount, and for each value of each group send request.Then, concatenate those groups so that next group's requests would not start before current group's request are not completed.The problem is some values get skipped.Here's my code.(I'm not making actual ajax calls here, but Observable.timer should work for an example).
Observable.interval(300)
.take(12)
.windowCount(3)
.concatMap(obs => {
return obs.mergeMap(
v => Observable.timer(Math.random() * 1500).mapTo(v)
);
})
.do(v => console.log(v))
.finally(() => console.log('fin'))
.subscribe();
I tried to replace windowCount by creating the groups manually. And it works perfectly. No values are skipped.
Observable.interval(900)
.take(4)
.map(i => Observable.interval(300).take(3).map(j => j + i * 3))
.concatMap(obs => {
return obs.mergeMap(
v => Observable.timer(Math.random() * 1500).mapTo(v)
);
})
.do(v => console.log(v))
.finally(() => console.log('fin'))
.subscribe();
I was under impression that windowCount should group the emitted values the same way.But, apparently it does something else.
I would be really thankful for any explanation of its behavior. Thanks!
The missing values are a result of using a hot observable (Observable.interval(300)) that continues to output values that you are not storing for use.
Following is a slightly simplified version of your code that also logs the times that numbers are emitted. I replaced Math.random() with 1 so that the output is deterministic. I have also loaded the code in jsbin for you to try out:
https://jsbin.com/burocu/edit?js,console
Observable.interval(300)
.do(x => console.log(x + ") hot observable at: " + (x * 300 + 300)))
.take(12)
.windowCount(3)
.do(observe3 => {observe3.toArray()
.subscribe(x => console.log(x + " do window count at: " + (x[2] * 300 + 300)));})
.concatMap(obs => {
return obs.mergeMap(
v => Observable.timer(1 * 1500).mapTo(v)
)
.do(v => console.log(v + " merge map at: " + (v * 300 + 300 + 1500)));
})
.finally(() => console.log('fin windowCount'))
.subscribe();
It results in the output below. Notice that the hot observables march on while the other operators are still being processed.
This is what is giving you the impression that values are being dropped. You can see that windowCount(3) is doing what you thought but not when you thought.
"0) hot observable at: 300"
"1) hot observable at: 600"
"2) hot observable at: 900"
"0,1,2 do window count at: 900"
"3) hot observable at: 1200"
"4) hot observable at: 1500"
"5) hot observable at: 1800"
"3,4,5 do window count at: 1800"
"0 merge map at: 1800"
"6) hot observable at: 2100"
"1 merge map at: 2100"
"7) hot observable at: 2400"
"2 merge map at: 2400"
"8) hot observable at: 2700"
"6,7,8 do window count at: 2700"
"9) hot observable at: 3000"
"10) hot observable at: 3300"
"11) hot observable at: 3600"
"9,10,11 do window count at: 3600"
" do window count at: NaN"
"8 merge map at: 4200"
"fin windowCount"
Edit: further explanation...
After windowCount(3) there is a call to concatMap. concatMap is a combination of map and concatAll.
concatAll:
Joins every Observable emitted by the source (a higher-order
Observable), in a serial fashion. It subscribes to each inner
Observable only after the previous inner Observable has completed (emphasis added), and
merges all of their values into the returned observable.
So, looking at the output above we see that the first windowCount(3) values [0,1,2] are emitted between 1800 and 2400.
Notice that the second windowCount(3) values [3,4,5] are emitted at 1800. concatAll is not ready to subscribe when [3,4,5] is emitted because the previous inner Observable has not completed yet. So these values are effectively dropped.
Next, notice that the previous inner Observable [0,1,2] completes at 2400. concatAll subscribes at 2400.
The next value to appear is the value 8 at 2700 (300ms after the subscription started at 2400). The value 8 is then output by mergeMap at 4200 because of the interval delay of 300 from the subscription start point of 2400 and then a timer delay of 1500 (i.e 2400 + 300 + 1500 = 4200).
After this point the sequence is completed so no further values are emitted.
Please add a comment if more clarification is needed.
I've been struggling with this for a couple of days, and none of the solutions I've found work the way I'd like (I can be completely wrong, I've not used SNMP for a very long time, though).
This is existing code in my company, a perl application that connects to net-snmp agentx using POE::Component::NetSNMP::Agent. MIB defined for this application is defined, with base oid finished in .154. The MIB file defines 3 tables on it: status (.1), statistics (.2) and performance (.3). Everything works fine, registration with the agent goes fine, snmpwalk shows data being updated, etc.
But now a requirement has been implemented, allowing multiple (up to 32) instances of the application running in the same host. And monitoring shall be supported too, which brings the first issue: when connecting to agentX more than once, with the same OID, only one instance connects, and the others are refused.
I've though to make something like this:
.154
.1 (instance 1):
.1 (status table)
.2 (statistics table)
.3 (performance table)
.2 (instance 2):
.1 (status table)
.2 (statistics table)
.3 (performance table)
.3 (instance 3):
.1 (status table)
.2 (statistics table)
.3 (performance table)
[...]
.32 (instance 32):
.1 (status table)
.2 (statistics table)
.3 (performance table)
With this approach, each instance (they know their own id) can register to AgentX with no problems (nice!). Following the model above, tables for status, statistics and performance would be common to all instances.
querying to .154 would show the model above.
querying data for each specific instance by walking to .154.1, .154.2, etc would be possible too.
But I'm unable to get this running properly, as smlint, snmpwalk and iReasoning moan about different data types expected, data is not being shown the right way, etc.
What I've tried so far:
arrays: main index per instance, subindex on status, statistics and performance indexed with { main index, subindex}. Like this: SNMP: ASN.1 MIB Definitions. Referencing a table within a table.
Multiple definitions: re-define every table and component for the 32 instances, with different indices on names. It works moreover, but not exactly the way I was expecting: an snmpwalk of the parent does not show any child, so snmpwalk must be performed using . . . . .154.1, . . . . . .154.2, etc.
I've considered this solution as well: Monitoring multiple java processes on the same host via SNMP. But in my case does not work, as the instances connect to a common agent, they don't have their own agent running in a different port.
I have to admit I'm running out of ideas. Again, I could be completely wrong and could be facing the problem from a wrong perspective.
Is it possible to implement this the way I'm looking for? In SNMPv3 this could possibly be a good use for contexts, but they are not available in net-snmp to my knowledge.
edit
Solution number two, from my list above, is the one working better by far.
From parent MIB, 32 new child OIDs are defined:
sampleServer MODULE-IDENTITY
LAST-UPDATED "201210101200Z"
[...]
DESCRIPTION "Sample Server MIB module"
REVISION "201211191200Z" -- 19 November 2012
DESCRIPTION "Version 0.1"
::= { parentMibs 154 }
instance1 OBJECT IDENTIFIER ::= { sampleServer 1 }
instance2 OBJECT IDENTIFIER ::= { sampleServer 2 }
instance3 OBJECT IDENTIFIER ::= { sampleServer 3 }
instance4 OBJECT IDENTIFIER ::= { sampleServer 4 }
instance5 OBJECT IDENTIFIER ::= { sampleServer 5 }
instance6 OBJECT IDENTIFIER ::= { sampleServer 6 }
[...]
And tables are repeated for each instanceId, a python script wrote the big MIB file for this (I know,
-- the table contains static information for instance number 1
-- this includes version, start time etc
sampleStatusTable1 OBJECT-TYPE
SYNTAX SEQUENCE OF sampleStatusEntry1
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION "sample Server instance1 Status, table"
::= { instance1 1 }
[...]
-- this table contains statistics and sums that change constantly
-- please note that depending on sample_server configuraiton not all
-- of these will be filled in
sampleStatisticsTable1 OBJECT-TYPE
SYNTAX SEQUENCE OF sampleStatisticsEntry1
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION "sample Server Statistics, table"
::= { instance1 2 }
[...]
-- performance figures that reflect the current load of sample_server
samplePerformanceTable1 OBJECT-TYPE
SYNTAX SEQUENCE OF samplePerformanceEntry
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION "sample Server Performance, table"
::= { instance1 3 }
[...]
snmpwalk output for each instance:
snmpwalk -M +/opt/sample_server/docs/mibs -m +COMPANY-SAMPLE-MIB -v2c -cpublic localhost 1.3.6.1.1.1.2.154.1
COMPANY-SAMPLE-MIB::sampleStatusInstance1 = INTEGER: 1
COMPANY-SAMPLE-MIB::sampleStatusVersion1 = STRING: "3.58"
COMPANY-SAMPLE-MIB::sampleStatusStartTime1 = STRING: "2014-12-13T00:06:27+0000"
COMPANY-SAMPLE-MIB::sampleStatisticsInstance1 = INTEGER: 1
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPInputTransactions1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPInputErrors1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputTransactions1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputErrorsRecoverable1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputErrors1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsEntry1.7 = INTEGER: 0
COMPANY-SAMPLE-MIB::samplePerformanceInstance1 = INTEGER: 1
COMPANY-SAMPLE-MIB::samplePerformanceQueueLoad1 = INTEGER: 0
COMPANY-SAMPLE-MIB::samplePerformanceThroughput1 = INTEGER: 0
snmpwalk -M +/opt/sample_server/docs/mibs -m +COMPANY-SAMPLE-MIB -v2c -cpublic localhost 1.3.6.1.1.1.2.154.2
COMPANY-SAMPLE-MIB::sampleStatusInstance2 = INTEGER: 1
COMPANY-SAMPLE-MIB::sampleStatusVersion2 = STRING: "3.58"
COMPANY-SAMPLE-MIB::sampleStatusStartTime2 = STRING: "2014-12-13T00:06:27+0000"
COMPANY-SAMPLE-MIB::sampleStatisticsInstance2 = INTEGER: 1
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPInputTransactions2 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPInputErrors2 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputTransactions2 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputErrorsRecoverable2 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputErrors2 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsEntry2.7 = INTEGER: 0
COMPANY-SAMPLE-MIB::samplePerformanceInstance2 = INTEGER: 1
COMPANY-SAMPLE-MIB::samplePerformanceQueueLoad2 = INTEGER: 0
COMPANY-SAMPLE-MIB::samplePerformanceThroughput2 = INTEGER: 0
But the result it's not as good as I was expecting, because a snmpwalk to master .154 shows the status for .154.1, instead of showing for every instance. Not sure if this is the expected behavior.
snmpwalk -M +/opt/sample_server/docs/mibs -m +COMPANY-SAMPLE-MIB -v2c -cpublic localhost 1.3.6.1.1.1.2.154
COMPANY-SAMPLE-MIB::sampleStatusInstance1 = INTEGER: 1
COMPANY-SAMPLE-MIB::sampleStatusVersion1 = STRING: "3.58"
COMPANY-SAMPLE-MIB::sampleStatusStartTime1 = STRING: "2014-12-13T00:06:27+0000"
COMPANY-SAMPLE-MIB::sampleStatisticsInstance1 = INTEGER: 1
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPInputTransactions1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPInputErrors1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputTransactions1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputErrorsRecoverable1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsHTTPOutputErrors1 = INTEGER: 0
COMPANY-SAMPLE-MIB::sampleStatisticsEntry1.7 = INTEGER: 0
COMPANY-SAMPLE-MIB::samplePerformanceInstance1 = INTEGER: 1
COMPANY-SAMPLE-MIB::samplePerformanceQueueLoad1 = INTEGER: 0
COMPANY-SAMPLE-MIB::samplePerformanceThroughput1 = INTEGER: 0