i am trying to get into data analytics using Spark with Scala. My question is how do i get the triangles in a graph? And i mean not the Triangle Count that comes with graphx, but the actual nodes that consist the triangle.
Suppose we have a graph file, i was able to calculate the triangles in scala, but the same technique does not apply in spark since i have to use RDD operations.
The data i give to the function is a complex List consisting of the src and the List of the destinations of that source; ex. Adj(5, List(1,2,3)), Adj(4, List(9,8,7)), ...
My scala version is this :
(Paths: List[Adj])
Paths.flatMap(i=> Paths.map(j => Paths.map(k => {
if(i.src != j.src && i.src!= k.src && j.src!=k.src){
if(i.dst.contains(j.src) && j.dst.contains(k.src) && k.dst.contains(i.src)){
println(i.src,j.src,k.src) //3 nodes that make a triangle
}
else{
()
}
}
})))
And the output would be something like:
(1,2,3)
(4,5,6)
(2,5,6)
In conclusion i want the same output but in spark environment execution. In addition i am looking for a more efficient way to hold the information about Adjacencies like key mapping and then reducing by key or something. As the spark environment needs a quite different way to approach each problem (big data operations) i would be gratefull if you could explain the way of thinking and give me a small briefing about the functions you used.
Thank you.
tldr; How to mix two or more createApi endpoints results ?
So I'm using createApi from reduxToolkit and the problem I have quite simple but I'm kinda lost in this huge documentation of this beautiful tool.
The idea is that I have a view that will mix data coming from two different api endpoints , for example:
/users
/cars
That view will display an array mixing both results (for example the cars images are only in /cars).
A little bit like transformResponse but for 2 endpoints
What is the right way to do this mixing ? (doing that in the view doesn't seems the best and I don't want to that backend neither).
You may tell me use a reducer, but where a reducer/slice takes places in the createApi pattern ? that's what I don't get.
You can combine the result outside of rtk query.
const {data: data1} = useAQuery(...);
const {data: data2} = useBQuery(...);
const combined = useMemo(() => {...combine data1, data2}, [data1, data2]);
If it's needed in multiple components, you can create a custom hook useCarsAndUsers(...) to avoid code duplication.
I am using Scala spark streaming , and I need to push my results to a kafka topic. I am using .selectExpr("to_json(struct(Column1,Column2,Column3,Column4))as value") .
And I get the result:
{"Column1":"Value_Column1","Column4":"Value_Column4"}
{"Column1":"Value_Column1","Column2":"Value_Column2"}
{"Column1":"Value_Column1","Column3":"Value_COlumn3"}
How should I change the .selectExpr or what are the steps I need to take to get an output like this :
{"Column1":"Value_Column1","Column2":"Value_Column2","Column3":"Value_Column3","Column4":"Value_Column4"}
Thank you all in advance!
I want to make a list of currencies in alphabetical order with letters as headers. Please have a look at the pictures for better understanding.
This is what I want to achieve
The list I have been able to achieve so far (A simple one)
I searched for packages but I couldn't find anything as per my need. Your help would be much appreciated.
Thanks.
Sounds like: how to implement Alphabet scroll in flutter
another options is: You can make a Map for example
countriesDict = {"a" : {"Algeria, Andorra", "b" : {"Bolivia, Bosnia ...."}}}
and retrieve it it in your code. For example:
for (country in countriesDict) {
... Your code to sort it out
}
and then display it.
You can use this package to implement your desired result easily.
Grouped List
Rest of it depends on your design.
A processor 'a' takes care the header 'a' of a message 'a_b_c_d' and passes the payload 'b_c_d' to the another processor in the next level as following:
msg(a, b_c_d).
pro(a;b;c;d).
msg(b, c_d) :- pro(X), msg(X, b_c_d).
msg(c, d) :- pro(X), msg(X, c_d).
msg(d) :- pro(X), msg(X, d).
#hide. #show msg/2. #show msg/1.
How should I represent list 'a_b_c_d' in ASP, and change the above to general cases?
No, official way, but I think most people don't realize you can construct cons-cells in ASP.
For instance, here's how you can get items for all lists of length 5 from elements 1..6
element(1..6).
listLen(empty, 0).
listLen(cons(E, L), K + 1) :- element(E); listLen(L, K); K < 5.
is5List(L) :- listLen(L, 5).
#show is5List/1.
resulting in
is5List(cons(1,cons(1,cons(1,cons(1,cons(1,empty))))))
is5List(cons(1,cons(1,cons(1,cons(1,cons(2,empty))))))
is5List(cons(1,cons(1,cons(1,cons(1,cons(3,empty))))))
...
There is no 'official' way to handle lists in ASP as far as I know. But, DLV has built-in list handling similar to Prolog's.
The way you implement a list, the list itself cannot be used as a term and thus what if you want to bind between variables in the list and other elements of a rule? Perhaps you would like something such as p(t, [q(X), q(Y)]) :- X != Y.
You can try implementing a list as (a, b, c) and an append predicate but the problem is ASP requires grounding before computing answer-sets. Consequently a list defined in this way whilst more like lists in Prolog would mean the ground-program contains all ground-instances of all possible lists (explosion) regardless of whether they are used or not.
I therefore come back to my first point, try using DLV instead of Clingo if possible (for this task, at least).
By using index, I do have a way to walk a list, however, I do not know this is the official way to handle a list in ASP. Could someone has more experience in ASP give us a hand? Thanks.
index(3,a). index(2,b). index(1,c). index(0,d).
pro(a;b;c;d). msg(3,a).
msg(I-1,N) :- pro(P), msg(I,P), index(I,P), I>0, index(I-1,N).
#hide. #show msg/2.
You can use s(ASP) or s(CASP) ASP systems. Both of them support list operations like prolog. You might need to define the list built-in in ASP .