I have created a small sample of what I'm working with at this point. I am trying to properly quote the DEFAULT values, specifically #2 DEFAULT ONE value should be quoted as DEFAULT 'ONE':
#1 CREATE TABLE `table` (`column` int(10) unsigned DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
#2 ALTER TABLE `table` MODIFY COLUMN `column2` enum('ONE','TWO') NOT NULL DEFAULT ONE AFTER `column1`;
#3 ALTER TABLE `table` MODIFY COLUMN `column` varchar(64) NOT NULL DEFAULT '' FIRST;
I am currently using the following line against the above lines in order to fix line #2:
sed "s/DEFAULT \([a-zA-Z0-9_.]*\)/DEFAULT '\1'/g"
This is the output I'm getting:
#1 CREATE TABLE `table` (`column` int(10) unsigned DEFAULT 'NULL') ENGINE=InnoDB DEFAULT 'CHARSET'=utf8mb4 COLLATE=utf8mb4_unicode_ci;
#2 ALTER TABLE `table` MODIFY COLUMN `column2` enum('ONE','TWO') NOT NULL DEFAULT 'ONE' AFTER `column1`;
#3 ALTER TABLE `table` MODIFY COLUMN `column` varchar(64) NOT NULL DEFAULT '''' FIRST;
As you can see it fixes line #2 but #1 & #3 now have issues.
#1 DEFAULT 'NULL' (should remain DEFAULT NULL)
#1 DEFAULT 'CHARSET' (should remain DEFAULT CHARSET)
#2 DEFAULT 'ONE' (GOOD!)
#3 DEFAULT '''' (should remain DEFAULT '')
Is there a way to adjust the sed to ignore specific patterns such as DEFAULT NULL or DEFAULT CHARSET or DEFAULT ''?
sed "s/DEFAULT (not followed by NULL|CHARSET|'')\([a-zA-Z0-9_.]*\)/DEFAULT '\1'/g"
Or perhaps there is a better approach?
Thanks!
If Perl is your option, please try:
perl -pe "s/(DEFAULT) (?!(NULL|CHARSET|''))([a-zA-Z0-9_]+)/\1 '\3'/g" file
Output:
#1 CREATE TABLE `table` (`column` int(10) unsigned DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
#2 ALTER TABLE `table` MODIFY COLUMN `column2` enum('ONE','TWO') NOT NULL DEFAULT 'ONE' AFTER `column1`;
#3 ALTER TABLE `table` MODIFY COLUMN `column` varchar(64) NOT NULL DEFAULT '' FIRST;
Try:
sed -E "s/DEFAULT (NULL|CHARSET)/DEFAULT_\1/g; s/DEFAULT ([[:alnum:]_.]+)/DEFAULT '\1'/g; s/DEFAULT_(NULL|CHARSET)/DEFAULT \1/g" file
This works in three steps:
s/DEFAULT (NULL|CHARSET)/DEFAULT_\1/g
This hides the DEFAULT values that you don't want to change.
s/DEFAULT ([[:alnum:]_.]+)/DEFAULT '\1'/g
This changes the values that you do want to change.
Note that I changed the * to +. This means that empty strings will not be matched. This solves the problem in line #3.
I also changed [a-zA-Z0-9] to [:alnum:] so that the regex would match all alphanumeric characters in a unicode-safe way. (If this isn't what you wanted, just change this part back.)
s/DEFAULT_(NULL|CHARSET)/DEFAULT \1/g
This changes back the ones that you don't want to change.
This approach assumes that neither DEFAULT_NULL nor DEFAULT_CHARSET appear in your actual input. Based on what you have shown so far, this looks like a safe assumption.
Example
With your input file:
$ cat file
#1 CREATE TABLE `table` (`column` int(10) unsigned DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
#2 ALTER TABLE `table` MODIFY COLUMN `column2` enum('ONE','TWO') NOT NULL DEFAULT ONE AFTER `column1`;
#3 ALTER TABLE `table` MODIFY COLUMN `column` varchar(64) NOT NULL DEFAULT '' FIRST;
Our command produces:
$ sed -E "s/DEFAULT (NULL|CHARSET)/\n\1/g; s/DEFAULT ([[:alnum:]_.]+)/DEFAULT '\1'/g; s/\n(NULL|CHARSET)/DEFAULT \1/g" file
#1 CREATE TABLE `table` (`column` int(10) unsigned DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
#2 ALTER TABLE `table` MODIFY COLUMN `column2` enum('ONE','TWO') NOT NULL DEFAULT 'ONE' AFTER `column1`;
#3 ALTER TABLE `table` MODIFY COLUMN `column` varchar(64) NOT NULL DEFAULT '' FIRST;
With GNU awk for multi-char RS:
awk -v RS='DEFAULT \\w+' -v ORS= '
RT { split(RT,rt); if (rt[2] !~ /^(NULL|CHARSET)$/) RT=rt[1]" \047"rt[2]"\047" }
{ print $0 RT }
' file
#1 CREATE TABLE `table` (`column` int(10) unsigned DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
#2 ALTER TABLE `table` MODIFY COLUMN `column2` enum('ONE','TWO') NOT NULL DEFAULT 'ONE' AFTER `column1`;
#3 ALTER TABLE `table` MODIFY COLUMN `column` varchar(64) NOT NULL DEFAULT '' FIRST;
Related
I am using the COPY FROM command to load data from a file.
The table is defined with identity column, which is not part of the file.
CREATE TABLE APP2DBMAP (
FIELD_ID integer NOT NULL GENERATED BY DEFAULT AS IDENTITY,
FIELD_NAME varchar(128) ,
TABLE_NAME varchar(128) ,
COLUMN_NAME varchar(128) ,
CONSTRAINT PK_APP2DBMAP PRIMARY KEY ( FIELD_ID )
);
I executed the following COPY FROM command, the file contains 3 values in 1 row.
copy app2dbmap (field_name, table_name, column_name) from '/opt/NetMgr/data/templ_db.txt' DELIMITER ',' ;
And I got the following error:
ERROR: null value in column "field_id" violates not-null constraint
DETAIL: Failing row contains (null, 'aaa', 'bbb', 'ccc').
CONTEXT: COPY app2dbmap, line 1: "'aaa','bbb','ccc'"
I tried to change the column description of field_id to serial, and it did work fine.
I don't understand why it doesn't work with the original table definition.
The problem is you have specified the field_id to be a not null value and hence when the file is passing null as a value, your error is there.
If you want an auto increment id, Use,
CREATE TABLE APP2DBMAP (
FIELD_ID smallserial NOT NULL,
FIELD_NAME varchar(128) ,
TABLE_NAME varchar(128) ,
COLUMN_NAME varchar(128) ,
CONSTRAINT PK_APP2DBMAP PRIMARY KEY ( FIELD_ID )
);
You can also use bigserial(int4) instead of smallint(int8)
or you can give a default value,
CREATE TABLE APP2DBMAP (
FIELD_ID integer NOT NULL default 0,
FIELD_NAME varchar(128) ,
TABLE_NAME varchar(128) ,
COLUMN_NAME varchar(128) ,
CONSTRAINT PK_APP2DBMAP PRIMARY KEY ( FIELD_ID )
);
You will have to pass something, you cannot pass null in a not null column
I have a WKT data which looks like so:
GEOMETRYCOLLECTION(
POINT(-1763555.1165955865 310640.0829509564),
POINT(-1421117.229877997 -300856.1433304538)
)
The default projection is 'EPSG:3857'. In postgresql I created a parent table whith generic geometry column and several child tables with columns of specific geometry type. The schema looks like so:
# parent table with generic geometry column "geom"
CREATE TABLE "public"."layer_261_" (
"id" int4 DEFAULT nextval('layer_261__id_seq'::regclass) NOT NULL,
"feature_id" int4 DEFAULT 0 NOT NULL,
"feature_name" varchar(200),
"feature_type" varchar(50),
"geom" "public"."geometry",
"object_id" int4 DEFAULT 0 NOT NULL,
"row_id" int4 DEFAULT 0 NOT NULL
)
WITH (OIDS=FALSE);
ALTER TABLE "public"."layer_261_" ADD CHECK (st_srid(geom) = 3857);
ALTER TABLE "public"."layer_261_" ADD CHECK (st_ndims(geom) = 2);
ALTER TABLE "public"."layer_261_" ADD PRIMARY KEY ("id");
# child table which is supposed to contain only POINT data type
CREATE TABLE "public"."layer_261_points" (
"id" int4 DEFAULT nextval('layer_261__id_seq'::regclass) NOT NULL,
"feature_id" int4 DEFAULT 0 NOT NULL,
"feature_name" varchar(200),
"feature_type" varchar(50),
"geom" "public"."geometry",
"object_id" int4 DEFAULT 0 NOT NULL,
"row_id" int4 DEFAULT 0 NOT NULL
)
INHERITS ("public"."layer_261_")
WITH (OIDS=FALSE);
ALTER TABLE "public"."layer_261_points" ADD CHECK (st_ndims(geom) = 2);
ALTER TABLE "public"."layer_261_points" ADD CHECK (geometrytype(geom) = 'POINT'::text);
ALTER TABLE "public"."layer_261_points" ADD CHECK (st_srid(geom) = 3857);
So, how can I insert my data (two points to the database)? For example, I'm not sure whether I should convert points' coordinates to lat-lon. And besides, I'm not sure whether I should insert GEOMETRYCOLLECTION or all points one by one.
EDIT
I've just tried to execute a query with a real data point:
INSERT INTO layer_261_ (geom) VALUES (ST_Point(105177.3509204, -85609.471679397))
But as a result I got this error message:
new row for relation "layer_261_" violates check constraint
"enforce_srid_geom"
Does anybody know how to fix it?
EDIT
This query leads to the very same error message:
INSERT INTO layer_261_ (geom) VALUES (ST_SetSRID(ST_Point(105177.3509204, -85609.471679397),
4326))
You can only insert the WKT into the parent table because the point table won't accept a GEOMETRYCOLLECTION:
INSERT INTO "public"."layer_261_" ("geom", <other columns>)
VALUES (ST_GeomFromText(<your WKT>, 3857), <other values>);
Once you have the data in the parent table you can easily convert from the GEOMETRYCOLLECTION to separate POINTs using ST_Dump() and insert those in the point table:
INSERT INTO "public"."layer_261_points" ("geom", <other columns>)
SELECT p.geom, <other columns>
FROM "public"."layer_261_" m, ST_Dump("geom") p
WHERE ...;
You can of course also forget about the first step and do ST_Dump(ST_GeomFromText(<your WKT>, 3857)) in the second step but that is less intuitive and more prone to errors.
Note that ST_Dump() is a table function so it should be used in a FROM clause. It can then use columns from tables specified before the function.
The error you were getting from using ST_Point() is because you the geometry has a NULL SRID. You should set that explicitly with ST_SetSRID() (one of my great annoyances with PostGIS...).
I am Migrating my database from MySQL to PostgreSQL.While creating table I got an error which I can't resolve.My MySQL Query is like this.
MYSQL Query
CREATE TABLE `configuration` (
`Name` varchar(300) NOT NULL,
`Value` varchar(300) default NULL,
`CType` char(1) default NULL,
`Size` int(11) default NULL,
`CGroup` varchar(50) default NULL,
`RestartReq` char(1) NOT NULL default 'Y',
`Display` char(1) NOT NULL default 'Y',
PRIMARY KEY (`Name`),
KEY `CType` (`CType`),
CONSTRAINT `configuration_ibfk_1` FOREIGN KEY (`CType`) REFERENCES `conftype` (`CType`)
)ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_bin`
PostgreSQL Query
CREATE TABLE configuration (
Name varchar(300) PRIMARY KEY,
Value varchar(300) default NULL,
CType char(1) default NULL,
Size integer default NULL,
CGroup varchar(50) default NULL,
RestartReq char(1) NOT NULL default 'Y',
Display char(1) NOT NULL default 'Y',
KEY CType (CType),
CONSTRAINT `configuration_ibfk_1` FOREIGN KEY (CType) REFERENCES conftype (CType)
)
Running File with
psql -h localhost -p 5432 -U postgres -f ps.sql testdb
Error getting
psql:ps.sql:40: ERROR: syntax error at or near "(" at character 287
psql:ps.sql:40: LINE 9: KEY CType ('CType'),
From the MySQL documentation:
KEY is normally a synonym for INDEX.
In PostgreSQL you have to create the index separately from the table:
CREATE TABLE configuration (
name varchar(300) PRIMARY KEY,
value varchar(300),
ctype char(1),
size integer,
cgroup varchar(50),
restartreq boolean NOT NULL DEFAULT true,
display boolean NOT NULL DEFAULT true,
CONSTRAINT configuration_ibfk_1 FOREIGN KEY (ctype) REFERENCES conftype (ctype)
);
CREATE INDEX conf_key ON configuration(ctype);
A few other points:
PostgreSQL identifiers (mainly table and column names) are case-insensitive except when double-quoted. The standard approach is to put identifiers in lower case and keywords in upper case.
Using a varchar(300) as a PRIMARY KEY is usually not a good idea for performance reasons. Consider adding a serial type.
The default value of a column is NULL when nothing is specified, so no need to specify DEFAULT NULL.
PostgreSQL has a boolean data type.
I have a database about weather that updates every second.
It contains temperature and wind speed.
This is my database:
CREATE TABLE `new_table`.`test` (
`id` INT(10) NOT NULL,
`date` DATETIME() NOT NULL,
`temperature` VARCHAR(25) NOT NULL,
`wind_speed` INT(10) NOT NULL,
`humidity` FLOAT NOT NULL,
PRIMARY KEY (`id`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
I need to find the average temperature every hour.
This is my code:
Select SELECT AVG( temperature ), date
FROM new_table
GROUP BY HOUR ( date )
My coding is working but the problem is that I want to move the value and date of the average to another table.
This is the table:
CREATE TABLE `new_table.`table1` (
`idsea_state` INT(10) NOT NULL,
`dateavg` DATETIME() NOT NULL,
`avg_temperature` VARCHAR(25) NOT NULL,
PRIMARY KEY (`idsea_state`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
Is it possible? Can you give me the coding?
In order to insert new rows into a database based on data you have obtained from another table, you can do this by setting up an INSERT query targeting the destination table, then run a sub-query which will pull the data from the source table and then the result set returned from the sub-query will be used to provide the VALUES used for the INSERT command
Here is the basic structure, note that the VALUES keyword is not used:
INSERT INTO `table1`
(`dateavg`, `avg_temperature`)
SELECT `date` , avg(`temperature`)
FROM `test`;
Its also important to note that the position of the columns returned by result set will be sequentially matched to its respective position in the INSERT fields of the outer query
e.g. if you had a query
INSERT INTO table1 (`foo`, `bar`, `baz`)
SELECT (`a`, `y`, `g`) FROM table2
a would be inserted into foo
y would go into bar
g would go into baz
due to their respective positions
I have made a working demo - http://www.sqlfiddle.com/#!9/ff740/4
I made the below changes to simplify the example and just demonstrate the concept involved.
Here is the DDL changes I made to your original code
CREATE TABLE `test` (
`id` INT(10) NOT NULL AUTO_INCREMENT,
`date` DATETIME NOT NULL,
`temperature` FLOAT NOT NULL,
`wind_speed` INT(10),
`humidity` FLOAT ,
PRIMARY KEY (`id`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
CREATE TABLE `table1` (
`idsea_state` INT(10) NOT NULL AUTO_INCREMENT,
`dateavg` VARCHAR(55),
`avg_temperature` VARCHAR(25),
PRIMARY KEY (`idsea_state`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
INSERT INTO `test`
(`date`, `temperature`) VALUES
('2013-05-03', 7.5),
('2013-06-12', 17.5),
('2013-10-12', 37.5);
INSERT INTO `table1`
(`dateavg`, `avg_temperature`)
SELECT `date` , avg(`temperature`)
FROM `test`;
I use typo3 sql parser for creating tables. I want to create tables with default charset, but if I set default charset in my query:
CREATE TABLE `table1` (
`col1` varchar(64) NOT NULL DEFAULT '',
`col2` varchar(64) NOT NULL DEFAULT '',
`col3` text NOT NULL,
PRIMARY KEY (`col1`,`col2`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
parser returns
CREATE TABLE table1 (
col1 varchar(64) NOT NULL default '',
col2 varchar(64) NOT NULL default '',
col3 text NOT NULL,
PRIMARY KEY (col1,col2)
) ENGINE=InnoDB;
How to set default charset?
I guess it is not implemented in TYPO3 SQL Parser. But IMHO there is no need to. You should just use the DEFAULT CHARSET which is set for the DB. This should be UTF8 anyway, since TYPO3 does not support other charsets anymore (AFAIK).