I want to read CSV file thats on my desktop named "tripdata". I wrote a code but I always get this error:
ERROR: invalid input syntax for integer: "NULL"
CONTEXT: COPY tripdata, line 4, column birth_year: "NULL"
SQL state: 22P02
I do not know whats the problem. I read at the same way other CSV files.
CREATE TABLE public."tripdata" (tripduration integer,
starttime timestamp,
stoptime timestamp,
start_station_id integer,
start_station_name varchar(100),
start_station_latitude float,
start_station_longituder float,
end_station_id integer,
end_station_name varchar(100),
end_station_latitude float,
end_station_longituder float,
bikeid integer,
usertime varchar(100),
birth_year integer,
gender varchar(100));
SELECT * FROM public."tripdata";
COPY public."tripdata" FROM 'C:\Users\Pc\Desktop\tripdata.csv' DELIMITER ',' CSV HEADER;
select * from tripdata;
I believe you will have to tell COPY what NULL is.
https://www.postgresql.org/docs/10/sql-copy.html
NULL
Specifies the string that represents a null value. The default is \N (backslash-N) in text format, and an unquoted empty string in CSV
format.
So in your case:
COPY ... NULL AS 'NULL';
Related
i will import data csv to postgresql via pgAdmin 4. But, there are problem
ERROR: invalid input syntax for type integer: ""
CONTEXT: COPY films, line 1, column gross: ""
i understand about the error that is line 1 column gross there is null value and in some other columns there are also null values. My questions, how to import file csv but in the that file there is null value. I've been search in google but not found similar my case.
CREATE TABLE public.films
(
id int,
title varchar,
release_year float4,
country varchar,
duration float4,
language varchar,
certification varchar,
gross int,
budget int
);
And i try in this code below, but failed
CREATE TABLE public.films
(
id int,
title varchar,
release_year float4 null,
country varchar null,
duration float4 null,
language varchar null,
certification varchar null,
gross float4 null,
budget float4 null
);
error message in image
I've searched on google and on the stackoverflow forums. I hope that someone will help solve my problem
There is no difference between the two table definitions. A column accepts NULL by default.
The issue is not a NULL value but an empty string:
select ''::integer;
ERROR: invalid input syntax for type integer: ""
LINE 1: select ''::integer;
select null::integer;
int4
------
NULL
Create a staging table that has data type of varchar for the fields that are now integer. Load the data into that table. Then modify the empty string data that will be integer using something like:
update table set gross = nullif(trim(gross), '');
Then move the data to the production table.
This is not a pgAdmin4 issue it is a data issue. Working in psql because it is easier to follow:
CREATE TABLE public.films_text
(
id varchar,
title varchar,
release_year varchar,
country varchar,
duration varchar,
language varchar,
certification varchar,
gross varchar,
budget varchar
);
\copy films_text from '~/Downloads/films.csv' with csv
COPY 4968
CREATE TABLE public.films
(
id int,
title varchar,
release_year float4,
country varchar,
duration float4,
language varchar,
certification varchar,
gross int,
budget int
);
-- Below done because of this value 12215500000 in budget column
alter table films alter COLUMN budget type int8;
INSERT INTO films
SELECT
id::int,
title,
nullif (trim(release_year), '')::real, country, nullif(trim(duration), '')::real,
LANGUAGE,
certification,
nullif (trim(gross), '')::float, nullif(trim(budget), '')::float
FROM
films_text;
INSERT 0 4968
It worked for me:
https://learnsql.com/blog/how-to-import-csv-to-postgresql/
a small workaround but it works
I created a table
I added headers to csv file
Right click on the newly created table-> Import/export data, select csv file to upload, go to tab2 - select Header and it should work
Using the query editor in the pgAdmin4 app, I would like to import data from a csv file into a table. My code is as follows:
CREATE DATABASE gps_tracking_db
ENCODING = 'UTF8'
TEMPLATE = template0
LC_COLLATE = 'C'
LC_CTYPE = 'C';
CREATE SCHEMA main;
COMMENT ON SCHEMA main IS 'Schema that stores all the GPS tracking core data.';
CREATE TABLE main.gps_data(
gps_data_id serial,
gps_sensors_code character varying,
line_no integer,
utc_date date,
utc_time time without time zone,
lmt_date date,
lmt_time time without time zone,
ecef_x integer,
ecef_y integer,
ecef_z integer,
latitude double precision,
longitude double precision,
height double precision,
dop double precision,
nav character varying(2),
validated character varying(3),
sats_used integer,
ch01_sat_id integer,
ch01_sat_cnr integer,
ch02_sat_id integer,
ch02_sat_cnr integer,
ch03_sat_id integer,
ch03_sat_cnr integer,
ch04_sat_id integer,
ch04_sat_cnr integer,
ch05_sat_id integer,
ch05_sat_cnr integer,
ch06_sat_id integer,
ch06_sat_cnr integer,
ch07_sat_id integer,
ch07_sat_cnr integer,
ch08_sat_id integer,
ch08_sat_cnr integer,
ch09_sat_id integer,
ch09_sat_cnr integer,
ch10_sat_id integer,
ch10_sat_cnr integer,
ch11_sat_id integer,
ch11_sat_cnr integer,
ch12_sat_id integer,
ch12_sat_cnr integer,
main_vol double precision,
bu_vol double precision,
temp double precision,
easting integer,
northing integer,
remarks character varying
);
COMMENT ON TABLE main.gps_data
IS 'Table that stores raw data as they come from the sensors (plus the ID of
the sensor).';
ALTER TABLE main.gps_data
ADD CONSTRAINT gps_data_pkey
PRIMARY KEY(gps_data_id);
ALTER TABLE main.gps_data
ADD COLUMN insert_timestamp timestamp with time zone
DEFAULT now();
ALTER TABLE main.gps_data
ADD CONSTRAINT unique_gps_data_record
UNIQUE(gps_sensors_code, line_no); /*what does line_no mean?*/
COPY main.gps_data(
gps_sensors_code, line_no, utc_date, utc_time, lmt_date, lmt_time, ecef_x,
ecef_y, ecef_z, latitude, longitude, height, dop, nav, validated, sats_used,
ch01_sat_id, ch01_sat_cnr, ch02_sat_id, ch02_sat_cnr, ch03_sat_id,
ch03_sat_cnr, ch04_sat_id, ch04_sat_cnr, ch05_sat_id, ch05_sat_cnr,
ch06_sat_id, ch06_sat_cnr, ch07_sat_id, ch07_sat_cnr, ch08_sat_id,
ch08_sat_cnr, ch09_sat_id, ch09_sat_cnr, ch10_sat_id, ch10_sat_cnr,
ch11_sat_id, ch11_sat_cnr, ch12_sat_id, ch12_sat_cnr, main_vol, bu_vol,
temp, easting, northing, remarks)
FROM
'/Users/CDDEP/Downloads/Urbano 2014/tracking_db/data/sensors_data/GSM01438.csv'
WITH (FORMAT csv, HEADER, DELIMITER ';')
However, when I run the CREATE FROM command, the following error message is returned:
ERROR: could not open file "/Users/CDDEP/Downloads/Urbano 2014/tracking_db/data/sensors_data/GSM01438.csv" for reading: No such file or directory
HINT: COPY FROM instructs the PostgreSQL server process to read a file. You may want a client-side facility such as psql's \copy.
SQL state: 58P01
I wonder if the error is due to a formatting issue with the Mac filepath or something else.
Make sure the file /Users/CDDEP/Downloads/Urbano 2014/tracking_db/data/sensors_data/GSM01438.csv does exists
Replace COPY main.gps_data with \COPY main.gps_data to use the client side utility
I would like to have a generated column, which value will be the concated string from two other values:
CREATE TABLE public.some_data (
user_name varchar NULL,
domain_name serial NOT NULL,
email GENERATED ALWAYS AS (user_name ||'#'||domain_name) stored
);
But that gives SQL Error [42601]: ERROR: syntax error at or near "ALWAYS"
You need to provide the data type for the column as #Belayer commented.
And then you need to explicitly cast domain_name as text (or some varchar). Otherwise you'll get an error that the expression isn't immutable as #nbk commented. serial is translated to be basically an integer and for whatever reason implicit casts of an integer in concatenations are considered not immutable by the engine. We had that just recently here.
So overall, using the given types for the columns, you want something like:
CREATE TABLE public.some_data
(user_name varchar NULL,
domain_name serial NOT NULL,
email text GENERATED ALWAYS AS (user_name || '#' || domain_name::text) STORED);
But it's a little weird that a domain name is a serial? Shouldn't that be a text or similar? Then you wouldn't need the cast of course.
You need to create an IMMUTABLE function to achieve the generate column, for example:
CREATE OR REPLACE FUNCTION generate_email_concat(varchar,int) returns text as
$$
select $1 ||'#'||$2::text;
$$
LANGUAGE SQL IMMUTABLE ;
CREATE TABLE public.some_data (
user_name varchar NULL,
domain_name serial NOT NULL,
email text GENERATED ALWAYS AS (generate_email_concat(user_name,domain_name)) stored
);
INSERT into some_data(user_name) values ('hello');
You try to concatenate varchar and integer. You have to cast domain_name. This works for me
CREATE TABLE public.some_data (
user_name varchar NULL,
domain_name serial NOT NULL,
email varchar GENERATED ALWAYS AS (CASE WHEN user_name IS NULL THEN 'noname'||'#'||domain_name::text ELSE user_name ||'#'||domain_name::text END) STORED
);
I have three tables in PostgreSQL:
CREATE TABLE organization (id int, name text, parent_id int);
CREATE TABLE staff (id int, name text, family text, organization_id int);
CREATE TABLE clock(id int, staff_id int, Date date, Time time);
I need a function that gets all the fields of these tables as inputs (8 on total) and then inserts these inputs into appropriate fields of the tables
Here is my code:
CREATE FUNCTION insert_into_tables(org_name character varying(50), org_PID int, person_name character varying(50),_family character varying(50), org_id int, staff_id int,_date date, _time time without time zone)
RETURNS void AS $$
BEGIN
INSERT INTO "Org".organisation("Name", "PID")
VALUES ($1, $2);
INSERT INTO "Org".staff("Name", "Family", "Organization_id")
VALUES ($3, $4, $5);
INSERT INTO "Org"."Clock"("Staff_Id", "Date", "Time")
VALUES ($6, $7, $8);
END;
$$ LANGUAGE plpgsql;
select * from insert_into_tables('SASAD',9,'mamad','Imani',2,2,1397-10-22,'08:26:47')
But no data is inserted. I get the error:
ERROR: function insert_into_tables(unknown, integer, unknown, unknown, integer, integer, integer, unknown) does not exist
LINE 17: select * from insert_into_tables('SASAD',9,'mamad','Imani',2... ^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
Where did i go wrong?
That's because the 2nd last parameter is declared as date, not int. You forgot the single quotes:
select * from insert_into_tables('SASAD',9,'mamad','Imani',2,2,'1397-10-22','08:26:47');
Without single quotes, this is interpreted as subtraction between 3 integer constants, resulting in an integer: 1397-10-22 = 1365.
Also fix your identifiers: double-quoting preserves upper-case letters, so "Name" is distinct from name etc. See:
Are PostgreSQL column names case-sensitive?
I set up a table like this.
CREATE TABLE IF NOT EXISTS details (
CountyCode CHAR(3) NOT NULL,
VoterID CHAR(10) NOT NULL UNIQUE,
NameLast TEXT,
NameSuffix TEXT,
NameFirst TEXT,
NameMiddle TEXT,
PublicRecordExemption CHAR(1),
ResidenceAddressLine1 TEXT,
ResidenceAddressLine2 TEXT,
ResidenceCity TEXT,
ResidenceState TEXT,
ResidenceZipcode TEXT,
MailingAddressLine1 TEXT,
MailingAddressLine2 TEXT,
MailingAddressLine3 TEXT,
MailingCity TEXT,
MailingState CHAR(2),
MailingZipcode TEXT,
MailingCountry TEXT,
Gender CHAR(1),
Race CHAR(1),
BirthDate CHAR(10),
RegistrationDate CHAR(10),
PartyAffiliation CHAR(3),
Precinct CHAR(6),
PrecinctGroup CHAR(3),
PrecinctSplit CHAR(6),
PrecinctSuffix CHAR(3),
VoterStatus CHAR(3),
CongressionalDistrict CHAR(3),
HouseDistrict CHAR(3),
SenateDistrict CHAR(3),
CountyCommissionDistrict CHAR(3),
SchoolBoardDistrict CHAR(2),
DaytimeAreaCode CHAR(3),
DaytimePhoneNumber CHAR(7),
DaytimePhoneExtension CHAR(4),
Emailaddress TEXT
);
I ran this command to import data from tab-delimited file Detail.txt.
\copy details FROM Detail.txt;
After a few seconds, the command line console spits out this error.
ERROR: value too long for type character(2)
CONTEXT: COPY details, line 449121, column mailingstate: "273707216".
Here's line 449121 copied into a pastebin.
The error indicates PSQL tries to read the value 273707216 into the the mailingstate column, which is two characters in length. I thought PSQL would instead read NC into that column.
Why does PSQL read this line wrong?
I think the problem resides in these fields of that row.
Hand Dr \ Melbourne
The backslash in that data is immediately followed by a tab. To the copy function it looks like a \t, the double backslash might be treated as an escaped character and not a field delimiter, so that would "eat" up a column thus putting the mailingZipcode into the mailingState column.
Try to remove that backslash and try to re-import the row.