Filter multiple columns using sqlite.swift? - swift

let part = self.parts.filter(self.deleteStatus == 0).order(self.id.asc)
Above is the query I run to get all the records from local database. Now I want to add more filters like id==5 or name='product' in the filter. But the filter in sqlite.swift doesn't allow more than one expression in filter. How can I add more or filters in the filter method?


Filters can be chained with && (SQLite AND) and || (SQLite OR).
self.filter(deleteStatus == 0 && id == 5 && name == "product")
https://github.com/stephencelis/SQLite.swift/blob/master/Documentation/Index.md#filter-operators-and-functions


The filter in Sqlite.swift does allow more than one expression; you just chain them together like this:
users.filter(id == 1 %% location == "Office" %% supervisor == "John")

Related

Spark Scala : How to send more than 10 columns to a function

I am looking for pointers on ideal way to send multiple spark.sql.columns to a function (with out mentioning them individually) . Currently I am passing a single column to a function to derive a new column but I need to send multiple columns to the function. Any suggestion on how I can select specific columns say "Flag1", "Flag2" and so on and send as an struct or other way ? and be able to access the column Name and its value in the called function similar to how I am doing in the below function?
e.g.: just mentioning 3 columns here but I have around 25+ columns and need to almost 10 of them to the function.
var data = Seq(("10","value1","value2"), ("20","value2","value3"), ("30","value4","value5")).toDF("id","Flag1","Flag2")
data.withColumn("newCol",doSomething($"Flag1")).show(5)
def doSomething(flag:Column): Column ={
when(flag.isNotNull && flag === lit("value4"),"abc")
.when(flag.isNotNull && flag === lit("vaue5"),"xyz")
.otherwise(lit("unknown"))
}
Thanks in advance for your guidance.

Why don't you just have multiple arguments in the function definition?
e.g.
data.withColumn("newcolumn", doSomething(data.columns.map(col)))
def doSomething(flags: Array[Column]): Column = {
flag1 = flags(0)
flag2 = flags(1)
when(flag1.isNotNull && flag2 === lit("value4"),"abc")
.when(flag1.isNotNull && flag2 === lit("vaue5"),"xyz")
.otherwise(lit("unknown"))
}

GEN_NO_GENERATABLE_NOTIF

I want to choose index from list, so the element[index] complies my condition.
MyList[index].num==0
I tried the code bellow:
gen DescIdx2Choose keeping {
it < MyList.size();
MyList[it].num==0;//I tried a few way of read_only
};
How can I do it without using all_indices?
Thanks

Since you generating DescIdx2Choose then MyList will be input to the problem.
Therefore,
If seeking for the first index (if exists) then using random generation isn't required. Use the procedural code "first_index" as user3467290 suggested which is much more efficient:
var fidx := MyList.first_index(.num == 0);
if ( fidx != UNDEF ) {
DescIdx2Choose = fidx;
} else {
// error handling
};
If there are multiple indices and it is required to choose a random one, the most efficient way would be using "all_indices" as Thorsten suggested:
gen DescIdx2Choose keeping {
it in read_only( MyList.all_indices(.num == 0) );
};
The reason is the random generator doesn't need to read all possible values of "MyList.num" only a shorter list of valid indices.

This should do it, but MyList must fulfill the condition otherwise you get a contradiction. Pseudo-method first_index() is not bi-directional which is what we need here.
gen DescIdx2Choose keeping {
MyList.first_index(.num == 0) == it;
};

Maybe i missed something in the question, but If you always want the index of the first element that its num == 0, then why use constraints? can assign DescIdx2Choose == MyList.first_index(.num == 0).
To ensure that there is at least one such element, can constrain MyList.has(.num == 0).
Do you have additional constraints on DescIdx2Choose ?

Not equal operator in pyspark giving wrong results

I am looking for this particular record in an array and it find the rows as below:
xarr.filter(xarr["orderid"] == 27952740).count()
This gives 67,272 rows which is the right answer.
Then I do this. Assign all Non-zero values to another array:
xarr2 = xarr.filter(xarr["orderid"] != 0)
Now in the resulting array xarr2, I am trying to locate the record as follows:
xarr2.filter(xarr2["orderid"] == 27952740).count()
This one returns zero records. Why is this behavior? Any idea?

The data type of orderid is String. Changing != 0to != '0' gave the correct results.

Find RowNumber in Talend

How to find row number in Talend Open Studio 6.3?I want to insert rows in a order like
Row 1 to Row 10 in File1
Row 11 to Row 20 in File2
Row 21 to Row30 to File3
And then again from Row 31 to file1
How to achive that?I have generated a sequence column.now how to proceed?is it can be done using tsamplerow?
Suppose the Sourcefile is like this:-
EMPNO,EMPNAME,DEPTNO
10,A,1
11,B,2
12,C,3
13,D,4
14,E,1
15,F,1
16,G,2
17,H,3
18,I,4
19,J,2
20,K,3
21,L,1
22,M,2

You can get the current row number by defining an increment value in talend job using Numeric.sequence("s1",1,1).
Note: I have used OP's sample data and divided rows per file as 3. But in OP's scenario it is 10 rows
Below is the sample job which I tried out,
I am generating a sequence number to know the current row number like below
After getting the current row value, I have an another variable which will be incremented for every 3 rows (in my example) like below (in OP's example, it is for every 10 rows.)
This is the expression I am doing on SequenceRow context variable.
context.SequenceRow = (input_row.SequenceNumber > context.RowRangePerFile && input_row.SequenceNumber % context.RowRangePerFile == 1) ? context.SequenceRow+1 : context.SequenceRow;
Finally I am filtering the rows in tMap_2 based on the SequenceRow value, like below,
For out1, the filter condition is (out2.SequenceRow > context.TotalNoOfFiles && out2.SequenceRow % context.TotalNoOfFiles == 1) || out2.SequenceRow == 1
For out3, the filter condition is (out2.SequenceRow > context.TotalNoOfFiles && out2.SequenceRow % context.TotalNoOfFiles == 2) || out2.SequenceRow == 2
For out4, the filter condition is (out2.SequenceRow > context.TotalNoOfFiles && out2.SequenceRow % context.TotalNoOfFiles == 0) || out2.SequenceRow == 3
I have taken your sample data that you have provided in your question,
EMPNO,EMPNAME,DEPTNO
10,A,1
11,B,2
12,C,3
13,D,4
14,E,1
15,F,1
16,G,2
17,H,3
18,I,4
19,J,2
20,K,3
21,L,1
22,M,2
and I am writing every 3 rows in each file and the output I got is
Hope this may help you.

Spark Iterating RDD over another RDD with filter conditions Scala

I wants to iterate one BIG RDD with small RDD with some additional filter conditions . the below code is working fine but the process is running only with Driver and Not spread-ed across the nodes . So please suggest any other approach ?
val cross = titlesRDD.cartesian(brRDD).cache()
val matching = cross.filter{ case( x, br) =>
((br._1 == "0") &&
(((br._2 ==((x._4))) &&
((br._3 exists (x._5)) || ((br._3).head==""))
}
Thanks,
madhu

You probably don't want to cache cross. Not caching it will, I believe, let the cartesian product happen "on the fly" as needed for the filter, instead of instantiating the potentially large number of combinations resulting from the cartesian product in memory.
Also, you can do brRDD.filter(_._1 == "0") before doing the cartesian product with titlesRDD, e.g.
val cross = titlesRDD.cartesian(brRRD.filter(_._1 == "0"))
and then modify the filter used to create matching appropriately.