In Datastage, I have an INTEGER field from a Seq File 0, in a transformer i wanted to write a constraint that if the Source data from the seq file is <> 0 or <> Numeric (numbers) then it should be written in seq file 2 and other Numeric data and 0 into Seq File 1 sequential file.
Please help me out.
Seq File 0 ->Transformer(Constraint) -> Seq File 1 (0 or Other Numerics) and
-> Seq File 2 (Blanks and Non-numeric)
Check out the num Function or alpha function to check wether it is a number or alphanumeric value. You may have to handle the empty case separately.
Related
Need to write a DataFrame to a csv file with the file name changes according to some Iteration index (idx):
for (idx <- 1 to 3)
// do some operation and generate a df that depends on idx
...
df.coalesce(1).write.csv("/temp/path/file#.csv")
The # should vary as the idx changes (in other words, the file name should be sequentially file1.csv, file2.csv, file3.csv as the iteration goes). This seems to be a very common problem, but I have not found a clear solution yet in Scala. Thanks!
The classic way would be:
for (idx <- 1 to 3)
// do some operation and generate a df that depends on idx
...
df.coalesce(1).write.csv("/temp/path/file_" + idx + ".csv")
the fancier new way is
for (idx <- 1 to 3)
// do some operation and generate a df that depends on idx
...
df.coalesce(1).write.csv(s"/temp/path/file_${idx}.csv")
In q, I am trying to call a function f on an incrementing argument id while some condition is not met.
The function f creates a table of random length (between 1 and 5) with a column identifier which is dependent on the input id:
f:{[id] len:(1?1 2 3 4 5)0; ([] identifier:id+til len; c2:len?`a`b`c)}
Starting with id=0, f should be called while (count f[id])>1, i.e. so long until a table of length 1 is produced. The id should be incremented each step.
With the "repeat" adverb I can do the while condition and the starting value:
{(count x)>1} f/0
but how can I keep incrementing the id?
Not entirely sure if this will fix your issue but I was able to get it to work by incrementing id inside the function and returning it with each iteration:
q)g:{[id] len:(1?1 2 3 4 5)0; id[0]:([] identifier:id[1]+til len; c2:len?`a`b`c);#[id;1;1+]}
In this case id is a 2 element list where the first element is the table you are returning (initially ()) and the second item is the id. By amending the exit condition I was able to make it stop whenever the output table had a count of 1:
q)g/[{not 1=count x 0};(();0)]
+`identifier`c2!(,9;,`b)
10
If you just need the table you can run first on the output of the above expression:
q)first g/[{not 1=count x 0};(();0)]
identifier c2
-------------
3 a
The issue with the function f is that when using over and scan the output if each iteration becomes the input for the next. In your case your function is working on a numeric value put on the second pass it would get passed a table.
I noticed this interesting thing about the max() and min() functions in SV LRM (1800-2012) 7.12 (Array manipulation methods). I tried out the max() and min() functions in a dummy SV file
int a[3] = {0,5,5};
int q[$];
int b;
q = a.max(); // legal
b = a.max(); // illegal
The illegal statement error was
Incompatible complex type assignment
Type of source expression is incompatible with type of target expression.
Mismatching types cannot be used in assignments, initializations and
instantiations. The type of the target is 'int', while the type of the
source is 'int$[$]'.
So I commented out the illegal statement and tested it. It compiled and ran fine but I was hoping to get some more insight as to why the function returns a queue and not a single element - I printed out the contents of q and the size, but the size is still 1 and 5 is being printed just once. Kind of redundant then to make the max() and min() functions return a queue ?
The "SystemVerilog for Verification" book by Chris Spear and Greg Tumbush has a good explanation on this topic in Chapter 2.6.2, which I am quoting below:
"The array locator methods find data in an unpacked array. At first
you may wonder why these return a queue of values. After all, there
is only one maximum value in an array. However, SystemVerilog needs a
queue for the case when you ask for a value from an empty queue or
dynamic array."
It returns a queue to deal with empty queues and when the with () conditions have no matches. The the empty queue return is a a way to differentiate a true match from no matches.
Consider the below code. to find the minimum value of a that is greater than 5. a has data but none of its entries have above 5. b is empty, so it will return an empty. c will return 7.
int a[3] = '{0,5,5};
int b[$] = '{};
int c[4] = '{0,15,5,7};
int q[$];
q = a.min() with (item > 5); // no items >5, will return an empty queue
q = b.min(); // queue is empty, will return an empty queue
q = c.min() with (item > 5); // will return a queue of size 1 with value 7
I believe the example results as per Greg's answer is not correct.
As per System Verilog Language:
min() returns the element with the minimum value or whose expression evaluates to a minimum.
max() returns the element with the maximum value or whose expression evaluates to a maximum.
So, when with expression is evaluated, the resultant value will be:
a.min() with (item > 5); {0,0,0} -> Minimum is 0 and corresponding item is 5.
c.min() with (item > 5); {0,1,0,1}-> Minimum is 0 and corresponding item is 5.
Since, example demonstrates the usage of min, the result will be:
q = a.min() with (item > 5); // A queue of size 1 with value 5.
q = c.min() with (item > 5); //A queue of size 1 with value 5.
I have a String stored in a table with two logic gates which looks like 1*((CUBL>1)*((DIFL>1)*1)).
I have been able to replace the text with values from a different table i.e. 1*((1500>1)*((0>1)*1)). The issue i am having is being able to evaluate the equations between the brackets. The following is a list of steps that i believe are needed.
1*((CUBL>1)*((DIFL>1)*1)) -> 1*((1500>1)*((0>1)*1))
1*((1500>1)*((0>1)*1)) -> 1*(1*(0*1))
1*(1*(0*1)) -> 1*(1*0)
1*(1*0) -> 1*0 = 0
Native library has FNV-1 hash algorithm https://golang.org/pkg/hash/fnv/ that returns uint64 value (range: 0 through 18446744073709551615).
I need to store this value in PostgreSQL bigserial, but it's range is 1 to 9223372036854775807.
It is possible to change hash size to eg. 56?http://www.isthe.com/chongo/tech/comp/fnv/index.html#xor-fold
Can someone help to change native algorithm to produce 56 bit hashes?
https://golang.org/src/hash/fnv/fnv.go
Update
Did it myself using this doc http://www.isthe.com/chongo/tech/comp/fnv/index.html#xor-fold
package main
import (
"fmt"
"hash/fnv"
)
func main() {
const MASK uint64 = 1<<63 - 1
h := fnv.New64()
h.Write([]byte("1133"))
hash := h.Sum64()
fmt.Printf("%#x\n", MASK)
fmt.Println(hash)
hash = (hash >> 63) ^ (hash & MASK)
fmt.Println(hash)
}
http://play.golang.org/p/j7q3D73qqu
Is it correct?
Is it correct?
Yes, it's a correct XOR-folding to 63 bits. But there's a much easier way:
hash = hash % 9223372036854775808
The distribution of XOR-folding is dubious, probably proven somewhere but not immediately obvious. Modulo, however, is clearly a wrapping of the hash algo's distribution to a smaller codomain.