Can we set String column as partitionColumn? - scala

Table only has String column as primary column EMPLOYEE_ID how to partition it.
val destination = spark.read.options(options).jdbc(options("url"), options("dbtable"), "EMPLOYEE_ID", P00100001, P00100005000000, 10, new java.util.Properties()).rdd.map(_.mkString(","))
Is there any other way to Read JDBC table and process it.

It is not possible. Only integer columns can be used here. If your database supports some variant of rowid, which is integer or can be casted to integer, you can extract it in a query (pseudocode):
(SELECT CAST(rowid AS INTEGER), * FROM TABLE) AS tmp

Related

Need help inserting data into Postgres tables

I get an error trying to insert data into my tables ... but I don't know why?
Syntax is correct.
column "population" is of type integer but expression is of type record
create table states(name varchar(25), population int );
create table countries(name varchar(25), population int );
insert into states values (('tn',54945),('ap',2308));
select name from states;
insert into countries values (('india',3022),('america',30902));
select * from countries;
There are extra parentheses around the tuples of values to insert, which turns the whole thing to a single record of records.
Instead:
insert into countries(name, population) values ('india',3022),('america',30902);

COALESCE types character varying and numeric cannot be matched

I have a table named table1 with columns Gender varchar(10) and Team numeric.
create table table1 (
ID integer
Gender varchar(10),
Team numeric
);
insert into table1 (ID,Gender,Team) values
(1,'M',NULL),
(2,NULL,10),
(3,NULL,6),
(4,''F',NULL),
(5,NULL,3);
I will like to create a new column as Nxt that returns a row that is not null from any of the columns either a string or integer.
The column Nxt will look like: M,10,6,F,3
I tried this:
select coalesce(Gender,Team) as value from table1;
It returns this error:
COALESCE types character varying and numeric cannot be matched
Try to cast the column as text
select coalesce(Gender,Team::text) as value from table1;

index on composite primary key columns

I have table called
CREATE TABLE process (
batch_id Integer
,product_id Integer
,machine_id Integer
,created_date DATE
,updated_date DATE
,primary key(batch_id,product_id,machine_id)
)
But I generally use SQL like
SELECT *
FROM process
WHERE product_id = 123
AND machine_id = 1
When i check SQL plan for this id does not uses primary key index.
Do i need to create another index of both columns?
Database is DB2

Looking up values from many tables based on value in each column

I have several tables containing key value pairs for differint fields in my database. I also have a table that that contains the keys of these differint tables that should be selected as the value for that key. However, I can't figure out how to select these values from the multiple tables?
The tables
CREATE TABLE CHARACTERS(
ID INTEGER PRIMARY KEY,
NAME VARCHAR(64)
);
CREATE TABLE MEDIA(
ID INTEGER PRIMARY KEY,
NAME VARCHAR(64)
);
CREATE TABLE EPISODES(
ID INTEGER PRIMARY KEY,
MEDIAID INTEGER,
NAME VARCHAR(64)
);
-- Selecting from this table
CREATE TABLE APPS(
ID INTEGER PRIMARY KEY,
CHARID INTEGER,
EPISODEID INTEGER,
MEDIAID INTEGER
);
I am selecting from the APPS table, and I want to replace the value of the *ID columns with the value of the name in the accomping table's NAME column. I want this done for each row in the APPS table. Like so...
CHARID -> CHARACTERS.NAME
EPISODEID -> EPISODES.NAME
MEDIAID -> MEDIA.NAME
I have tried to use joins, but they don't do it for each row in the APPS table. I have 18 rows in the APPS table, but I only get back way less than I have in the table or way more than I have in the table. So how can I make it do it for each row in the APPS table?
You do by JOINing the tables together and selecting the desired columns from the individual tables:
SELECT c.name AS character_name, e.name AS episode, m.name AS media
FROM apps a
LEFT JOIN episodes e ON e.id = a.episodeid
LEFT JOIN media m ON m.id = a.mediaid
LEFT JOIN characters c ON c.id = a.charid;
If you want to present the rows in a specific order, you can specify that too as a final clause in the SELECT statement. You can use any field from the included tables; that field is not necessarily part of the columns selected:
ORDER BY a.id -- order by apps.id
or
ORDER BY e.id, c.name -- order first by episode id, then by character name
etc

updating a varchar column with multiple select stmts

I have to update a VARCHAR column of a table by concatenating values from SELECT queries from another table. I have build a query like
UPDATE url SET VALUE = (SELECT id FROM ids WHERE identifier='site')':'(SELECT id FROM cids WHERE identifier='cid')
WHERE name='SToken:CToken'
AND tokenvalue LIKE (SELECT id FROM ids WHERE identifier='site');
Here value is VARCHAR.
How should I do this?