when I create a Delta table I can set some columns to be NOT NULL
CREATE TABLE [db_name.]table_name
[(col_name1 col_type1 [NOT NULL], ...)]
USING DELTA
Is there any way to set non null columns with koalas.to_table?
It's not possible to do directly from koalas.to_table, but you can use ALTER TABLE statement to add the constraint after you wrote the data (full docs):
ALTER TABLE <table_name> CHANGE COLUMN <col_name> SET NOT NULL;
(for example, via spark.sql command)
Related
I need to add new column on my table would be uuid data type, here my code:
ALTER TABLE core.example add COLUMN newcolumn SET DATA TYPE UUID USING (uuid_generate_v4())
but show me this error:
ERROR: type modifier is not allowed for type "uuid"
LINE 1: ALTER TABLE core.example add COLUMN newsi UUID (uuid_genera...
I dont want to alter a column, would be to create a new column on my table. Any idea how to make this?
Regards
When adding a new column you don't use SET DATA TYPE. Your statement should look like:
ALTER TABLE core.example ADD COLUMN newcolumn UUID DEFAULT (uuid_generate_v4());
The DEFAULT clause will immediately fill the column with UUIDs.
Alternatively if you you just want to fill the column with initial data, you can drop the DEFAULT clause afterward:
ALTER TABLE core.example ALTER COLUMN newcolumn DROP DEFAULT;
Note that if you are using Postgres 13 and newer it is generally preferrable to use gen_random_uuid() since that method is built-in and does not rely on the uuid-ossp extension.
I have a column where I want to change the data type. I currently am using Redshift. I know I can use the alter table statement to change the datatype, but this would change the order of the columns.
Is there a way to change the datatype without changing the order of the column?
I would recommend creating a new table with the schema you want and copying it over from the old table using a insert into new_table (select * from old_table) statement (here you can also do any casting to the new data type), after which you can drop the old table and rename the new one:
drop table old_table;
alter table new_table rename to old_table;
Using ALTER TABLE table_name ALTER COLUMN column_name TYPE new_data_type will not change the order of the columns in your table.
Please note that this clause can only changes the size of a column defined as a VARCHAR data type.
There are also other limitations described in AWS documentation of ALTER TABLE
I am working with hive version 0.9 and I need delete columns of a hive table. I have searched in several manuals of hive commands but I only I have found commands to version 0.14. Is possible to delete a column of a hive table in hive version 0.9? What is the command?
Thanks.
We can’t simply drop a table column from a hive table using the below statement like sql.
ALTER TABLE tbl_name drop column column_name ---- it will not work.
So there is a shortcut to drop columns from a hive table.
Let’s say we have a hive table.
From this table I want to drop the column Dob. You can use the ALTER TABLE REPLACE statement to drop a column.
ALTER TABLE test_tbl REPLACE COLUMNS(ID STRING,NAME STRING,AGE STRING); you have to give the column names which you want to keep in the table
There isn't a drop column or delete column in Hive.
A SELECT statement can take regex-based column specification in Hive releases prior to 0.13.0, or in 0.13.0 and later releases if the configuration property hive.support.quoted.identifiers is set to none.
That being said you could create a new table or view using the following:
drop table if exists database.table_name;
create table if not exists database.table_name as
select `(column_to_remove_1|...|column_to_remove_N)?+.+`
from database.some_table
where
...
;
This will create a table that has all the columns from some_table except the columns named column_to_remove_1, ... , to column_to_remove_N. You can also choose to create a view instead.
ALTER TABLE table_name REPLACE COLUMNS ( c1 int, c2 String);
NOTE: eliminate column from column list. It will keep matched columns and removed unmentioned columns from table schema.
we can not delete column from hive table . But droping a table(if its external) in hive and the recreating table(with column excluded) ,wont delete ur data .
so what can u do is(if u dont have table structure) run this command :
show create table database_name.table_name;
Then you can copy it and edit it (with column eliminated).Afterwards you can do as per invoke the shell
table details are empid,name,dept,salary ,address. i want remove address column. Just write REPLACE COLUMNS like below query
jdbc:hive2://> alter table employee replace columns(empid int, name string,dept string,salary int);
As mentioned before, you can't drop table using an alter statement.
Alter - replace is not guaranteed to work in all the cases.
I found the best answer for this here:
https://stackoverflow.com/a/48921280/4385453
Assume I have a table named tracker with columns (issue_id,ingest_date,verb,priority)
I would like to add 50 columns to this table.
Columns being (string_ch_01,string_ch_02,.....,string_ch_50) of datatype varchar.
Is there any better way to add columns with single procedure rather than executing the following alter command 50 times?
ALTER TABLE tracker ADD COLUMN string_ch_01 varchar(1020);
Yes, a better way is to issue a single ALTER TABLE with all the columns at once:
ALTER TABLE tracker
ADD COLUMN string_ch_01 varchar(1020),
ADD COLUMN string_ch_02 varchar(1020),
...
ADD COLUMN string_ch_50 varchar(1020)
;
It's especially better when there are DEFAULT non-null clauses for the new columns, since each of them would rewrite the entire table, as opposed to rewriting it only once if they're grouped in a single ALTER TABLE.
Here's the caveat...
If you have a table that has a single column and that column happens to be an identity column with identity_insert turned OFF, is it still possible to write a T-SQL insert statement for this table?
sure
use insert TableName default values
example
create table Test (id int identity not null)
insert Test default values
select * from Test