Use Spark when function without otherwise but keep column values - scala

I frequently find myself replacing values in columns using
when($"myCol".isNull,myCrazyFunction).otherwise($"myCol")
To me the .otherwise($"myCol") feels kind of redundant.
Is there a better way to replace some values under some condition and otherwise just leave everything as it is without using the otherwise?

I think you can use coalesce() for that.
select(coalesce($"myCol", myCrazyFunction))
Just remember that myCrazyFunction should return a Column type.

Related

Weak points while comparing columns in trigger as hstore-data

On some occasion I'd like before UPDATE to make sure which columns are changed. To make it as generic as possible, I don't want to use schema, table or column names in function. I found some solution here in SO and other places, and particularly liked idea to use hstore from this answer
Downside of hstore, as said widely, is it that this way I lose data types, everything is stringified.
But using it in context of trigger (while having no complex cols like json or hstore), where both NEW and OLD have same set of cols with according datatypes, I could think of just one problem: NULL and empty values will be not distinguishable.
What other problems I may be faced with, when I detect changes in trigger function like this:
changes := hstore(NEW) - hstore(OLD);
Alternative seems to use jsonb and then write some jsonb_diff function to discover changes. hstore's offered internal subtract-operation seems way more robust, but maybe I have not considered all weak points there.
I'm using Postgres 9.6.

Does PostgreSQL have the equivalent of an Oracle ArrayBind?

Oracle has the ability to do bulk inserts by passing arrays as bind variables. The database then does a separate row insert for each member of the array:
http://www.oracle.com/technetwork/issue-archive/2009/09-sep/o59odpnet-085168.html
Thus if I have an array:
string[] arr = { 1, 2, 3}
And I pass this as a bind to my SQL:
insert into my_table(my_col) values (:arr)
I end up with 3 rows in the table.
Is there a way to do this in PostgreSQL w/o modifying the SQL? (i.e. I don't want to use the copy command, an explicit multirow insert, etc)
Nearest that you can use is :
insert into my_table(my_col) SELECT unnest(:arr)
PgJDBC supports COPY, and that's about your best option. I know it's not what you want, and it's frustrating that you have to use a different row representation, but it's about the best you'll get.
That said, you will find that if you prepare a statement then addBatch and executeBatch, you'll get pretty solid performance. Sufficiently so that it's not usually worth caring about using COPY. See Statement.executeBatch. You can create "array bind" on top of that with a trivial function that's a few lines long. It's not as good as server-side array binding, but it'll do pretty well.
No, you cannot do that in PostgreSQL.
You'll either have to use a multi-row INSERT or a COPY statement.
I'm not sure which language you're targeting, but in Java, for example, this is possible using Connection.createArrayOf().
Related question / answer:
error setting java String[] to postgres prepared statement

ISNULL get the data from another column

If the email column is NULL, I want to have the phone number value in the PRIMARY_Contact column. I am using CASE expression, but could not do it properly.
You may be looking for COALESCE. It will return the first non-null value from a selection. It's actually just shorthand for CASE, but may be easier for you to implement.
In your example, perhaps that would look like this:
SELECT COALESCE(EMAIL,PRIMARY_CONTACT)
Check out https://msdn.microsoft.com/en-us/library/ms190349.aspx for more details.

Matlab changing variable type, checking not making sense

I might be doing something silly or just have a lack of understanding but I am currently working with a table of data and when I change the variable type of a column using the table.column notation when I check using the table(:,1) notation it returns a 0.
For example in my table the first column is Creditability (where table is the name of the table) so I changed the variable type using table.Creditability = logical(table.Creditability)
Then when I use islogical(table.Creditability) it returns 1
But when I use islogical(table(:,1)) it returns 0 yet when I type table(:,1) it returns a logical variable in true or false form.
I may just have a lack of understanding as I am new to this but any help would be appreciated.
Thanks
Of course it will return 0. This is because of basic point you are missing.
"table" is a structure variable in which a field named "Creditability" is created.
While "Creditability" is a logical array, it's parent "table" is still a structure.
Now, you are not getting error with statement table(:,1) although table is scalar. That is because, MATLAB treats everything as matrix. In this case, table is 1x1 matrix.
I hope it is clear now.

Execute statements for every record in a table

I have a temporary table (or, say, a function which returns a table of values).
I want to execute some statements for each record in the table.
Can this be done without using cursors?
I'm not opposed to cursors, but would like a more elegant syntax\way of doing it.
Something like this randomly made-up syntax:
for (select A,B from #temp) exec DoSomething A,B
I'm using Sql Server 2005.
I dont think what you want to to is that easy.
What i have found is that you can create a scalar function taking the arguments A and B and then from within the function execute an Extended Stored Procedure. This might achieve what you want to do, but it seems that this might make the code even more complex.
I think for readibility and maintainability, you should stick to the CURSOR implementation.
I would look into changing the stored proc so that it can work against a set of data rather than a single row input.
Would CROSS/OUTER APPLY do what you want if you need RBAR processing.
It's elegant, but depends on what processing you need to do