The following cqlsh command adds a column to an already existing cassandra table.
cqlsh:demodb> ALTER TABLE users ADD coupon_code varchar;
How would I do the same with the scala spark-cassandra-connector?
I am not seeing reference in the documents.
ALSO: Is there a scaladoc for com.datastax.spark.connector?
You can use withSessionDo method of the CassandraConnector, like this:
import com.datastax.spark.connector.cql.CassandraConnector
CassandraConnector(conf).withSessionDo { session =>
session.execute("ALTER TABLE users ADD coupon_code varchar;")
}
See more examples in documentation for Spark Cassandra connector...
Related
I have two connections in my dbeaver namely preproduction and postproduction. Both the database have a table called profiles.
I want to upsert data from my postproduction.profiles into preproduction.profiles
I tried using the import feature but it is not providing conditional insert
What is the correct way to do so?
(My preproduction.profiles already have some records that's why I need upsert)
In DBeaver during export or import process you can change some settings params, one of them - replace method. This method has 3 values:
none
ON CONFLICT DO NOTHING
ON CONFLICT DO UPDATE
You can choose 2nd or 3rd for upsert.
For more detail information please visit these sites:
DBeaver Data Import and Replace
PostgreSQL Upsert Using INSERT ON CONFLICT statement
Is there a way to drop a BigQuery table from Spark by using Scala?
I only find ways to read and write BigQuery table from Spark by using Scala from the example here:
https://cloud.google.com/dataproc/docs/tutorials/bigquery-connector-spark-example
Can someone provide an example to drop a BigQuery table? For example, I can drop a table in BigQuery console using this statement "drop table if exists projectid1.dataset1.table1".
Please note that my purpose of removing the existing table is NOT to overwrite. I simply want to remove it. Please help. Thanks.
Please refer to the BigQuery API:
import com.google.cloud.spark.bigquery.repackaged.com.google.cloud.bigquery.{BigQueryOptions, TableId}
val bq = BigQueryOptions.getDefaultInstance().getService()
val table = bq.getTable(TableId.of("projectid1", "dataset1", "table1"))
if(table != null) {
table.delete()
}
Notice, this should work in Dataproc. In other cluster you will need to properly set the cresentials
How to create table and alter table dynamically in scala slick?
Is the way just only play SQL query?
Do you mean the Slick table objects or the schema in the database? In case of the former, you can call .column on your table object any time to request a column dynamcally. In case of the latter, Slick allows to create new schemas. For manipulation existing schemas use SQL or play evolutions. More info here: Play!: Does Slick's DDL replace Evolutions?
I am trying to rename a table in db2 like so
rename table schema1.mytable to schema2.mytable
but getting the following error message:
the name "mytable" has the wrong number of qualifiers.. SQLCODE=-108,SQLSTATE=42601
what is the problem here.... I am using the exact syntax from IBM publib documentation.
You cannot change the schema of a given object. You have to recreate it.
There are severals ways to do that:
If you have only one table, you can export and import/load the table. If you use the IDX format, the DDL will be included in the generated file. If using another format, the table has be created.
You can recreate the table by using:
Create table schema2.mytable like schema1.mytable
You can extract the DDL with the db2look tool
If you are changing the schema name for a schema given, you can use ADMIN_COPY_SCHEMA
These last two options only create the table structure, and you still need to import the data. After having create the table, you insert the data by different ways:
Inserting directly
insert into schema2.mytable select * from schema1.mytable
Via load from cursor
Via a Load or import from file (The file exported in the previous step)
The problem is the foreign relations, because they have to be recreated.
Finally, you can create an alias. It is easier, and you do not have to deal with relations.
You can easily rename a table with this statement:
RENAME TABLE SCHEMA.TABLENAME TO NEWTABLENAME;
You're not renaming table in provided example, you're trying to move to different schema, it's not the same thing. Look into db2move tool for this.
if you want to rename a table in the same schema, you can use like this.
RENAME TABLE schema.table_name TO "new_table_name";
Otherwise, you can use tools like DBeaver to rename or copy tables in a db2 db.
What if you leave it as is and create an alias with the new name and schema.
Renaming a table means to rename a table within same schema .To rename in other schema ,db2 call its ALIAS:
db2 create alias for
I wanted to prepare a load utility to load the data into DB2 table. The table has columns which contains GENERATEDALWAYS feature set.
So, I am not able to load an unloaded details from the table.
Is it possible to use import for tables having columns with GENERATEDALWAYS set?
Steps I did:
1. db2 "export to tbl.txt of del modified by coldel| select * from <schema.table> where col=value"
2. db2 "delete from <schema.table> where col=value"
3. db2 "import from tbl.txt of del modified by coldel| allow write access warningcount1 insert into <schema.table>"
The columns with "GENERATEDALWAYS" is having NEW Value after import. Is it possible to use import to populate GENERATEDALWAYS columns to have the old values?
Appreciate the assistance.
Thanks,
Mathew Liju
What you are asking is not possible. With IMPORT you can't override columns that have GENERATED ALWAYS. As #Peter Miehle suggests you could alter the table to specify that the column is GENERATED BY DEFAULT, but this may break other applications.
Your question's title implies that you don't want to use the LOAD utility (but you don't mention anything about it in the actual question). However, LOAD is the only way to write data into the table and maintain the values for the generated column as they exist in the file:
db2 "load from tbl.txt of del modified by generatedoverride insert into schema.table"
If you do this, be aware that:
DB2 does not check if there are conflicts with existing rows in the table. You would need to define a unique index on the column(s) in question to resolve this; this would cause DB2 to delete the rows that you just loaded in the DELETE phase of the load.
If your generated column(s) are using IDENTITY, make sure that you alter the column to ensure that future generated values do not conflict with the rows that you just inserted into the table.
maybe you can drop the "generation" from the column and add it after importing with the appropriate values again.
#Ian Bjorhovde has given you the options.
IMPORT actually does INSERTs in the background - ie, it first prepares a INSERT statement with parameter markers and uses the values in the input file for those markers.
In your SQL snapshot you will see INSERT statement that is used.
Anything that is not possible in an INSERT statement isn't possible with IMPORT (kind of .. )