I have a MANAGED table in delta format in databrciks and I wanted to change it to EXTERNAL to make sure dropping the table would not affect the data. However the following code did not change the table TYPE and just added a new table property. How can I correctly convert my managed table to an external table ?
%sql
alter table db_delta.table1 SET TBLPROPERTIES('EXTERNAL'='TRUE')
Describe Table:
# Detailed Table Information
Name
db_delta.table1
Location
dbfs:/user/hive/warehouse/db_delta.db/table1
Provider
delta
Type
MANAGED
Table Properties
[EXTERNAL=TRUE,overwriteSchema=true]
I found the following workaround for the above scenario.
1.Copy the Managed table location to external location
dbutils.fs.cp('dbfs:/user/hive/warehouse/amazon_data_agg','abfss://data#amazondata.dfs.core.windows.net/amzon_aggred/',True)
Now drop the managed table.
drop table amazon_data_agg;
Now create the external table by the schema of the already created table, if there is schema mismatch you will get error.
Now you can append and do all operation
df_agg.write.format('delta').mode('append').option('path','abfss://data#amazondata.dfs.core.windows.net/amzon_aggred/').saveAsTable('amazon_data_agg')
Related
Using Postgres database. I have an existing table, and several existing Views that query that table.
Call the table, 'contacts'.
I alter the table, changing the name to 'contacts_backup'. I then created a new table with the same name the older table used to have 'contacts'
Now it appears that if I query the existing views, the data is still retrieved from the renamed table, contacts_backup, and not the new table, 'contacts'.
Can this be? How can I update the Views to query the new table of the same name, and not the renamed contacts_backup?
My new table is actually a foreign table, but shouldn't the principle be the same? I was expecting the existing tables to query against the new table, not the old renamed one.
What is an efficient way to update the existing views to query from the new table?
This is because PostgreSQL does not store the view definition as an SQL string, but as a parsed query tree.
These parsed query trees don't contain the names of the referenced objects, but only their object identifier (oid), which does not change when you rename an object.
The same is true for table columns. This all holds for foreign tables as well.
When you examine the view definition, for example with pg_get_viewdef, the parse tree is rendered as text, so you will see the changed names.
If you want to change the table that a view is referring to, the only solution is to either DROP the view and CREATE it again, or you can use CREATE OR REPLACE VIEW.
In SQL stored procedures, we have an option of creating a temporary table "#temp" whose structure is as that of another table that it is referring to. Here we don't explicitly create and mention the structure of "#temp" table.
Do we have similar option is HQL Hive script to create a temp table during run time without actually creating the table structure. Thus I can dump data to temp table and use it. Below code shows an example of #temp table in SQL.
SELECT name, age, gender
INTO #MaleStudents
FROM student
WHERE gender = 'Male'
Hive has the concept of temporary tables, which are local to a user's session. These tables behave just like any other table, and can be created using CTAS commands too. Hive automatically deletes all temporary tables at the end of the Hive session in which they are created.
Read more about them here.
Hive Documentation
DWGEEK
You can create simple temporary table. On this table you can perform any operation.
Once you are done with work and log out of your session they will be deleted automatically.
Syntax for temporary table is :
CREATE TEMPORARY TABLE TABLE_NAME_HERE (key string, value string)
beginner at SQL here, specifically using PostgreSQL. I'm trying to do something for a class here, but I keep receiving an error message and I'm not sure why. I've created a new database called "games", and I'm trying to create a basic table. The code is below.
The error message I receive is, [0A000] ERROR: cross-database references are not implemented: "games.games_schema.player_data" Position: 14
I can make the table in the default DB with postgreSQL fine, but why am I having issues trying to specifically create this table within this new Database?
CREATE DATABASE games;
CREATE TABLE games.games_schema.Player_Data
(Player_ID int,
Player_Name VARCHAR(255),
Player_System VARCHAR(255));
I thought the way I have my create table statement set up, is telling the server to create a table at that location. (database --> DBschema --> table)
Thanks for any help.
You created the games database, but that does not create a games_schema within it. It'll only create the schema Public (unless the default template has been modified) The solution is to either create a "games_schema" in the "games" database, or create your DB objects in the public schema.
Option 1: Create a schema.
create schema games_schema;
create table games_schema.player_data( ... );
Option 2: Use the Public schema.
create table player_data( ... );
My choice would be option 1, as I never allow anything other than extensions and Postgres supplied objects in public.
Is there any way to create a postgis table with existing style from layer_styles table? Say for example i have so many styles stored in layer_styles table. I need to assign one of the style from layer_styles table to the new table which i am going to create. Can that be done using postgresql during table creation using sql command?
You need to identify the layer IDs of interest (or name, or any column you want) and to create the new table using this data+structure. However using the style in this secondary table may not be that easy
create table public.layer_styles_2 as
(select * from public.layer_styles where id in (2,3));
I am trying to rename a table in db2 like so
rename table schema1.mytable to schema2.mytable
but getting the following error message:
the name "mytable" has the wrong number of qualifiers.. SQLCODE=-108,SQLSTATE=42601
what is the problem here.... I am using the exact syntax from IBM publib documentation.
You cannot change the schema of a given object. You have to recreate it.
There are severals ways to do that:
If you have only one table, you can export and import/load the table. If you use the IDX format, the DDL will be included in the generated file. If using another format, the table has be created.
You can recreate the table by using:
Create table schema2.mytable like schema1.mytable
You can extract the DDL with the db2look tool
If you are changing the schema name for a schema given, you can use ADMIN_COPY_SCHEMA
These last two options only create the table structure, and you still need to import the data. After having create the table, you insert the data by different ways:
Inserting directly
insert into schema2.mytable select * from schema1.mytable
Via load from cursor
Via a Load or import from file (The file exported in the previous step)
The problem is the foreign relations, because they have to be recreated.
Finally, you can create an alias. It is easier, and you do not have to deal with relations.
You can easily rename a table with this statement:
RENAME TABLE SCHEMA.TABLENAME TO NEWTABLENAME;
You're not renaming table in provided example, you're trying to move to different schema, it's not the same thing. Look into db2move tool for this.
if you want to rename a table in the same schema, you can use like this.
RENAME TABLE schema.table_name TO "new_table_name";
Otherwise, you can use tools like DBeaver to rename or copy tables in a db2 db.
What if you leave it as is and create an alias with the new name and schema.
Renaming a table means to rename a table within same schema .To rename in other schema ,db2 call its ALIAS:
db2 create alias for