I'm trying to copy txt file to postgres database but I'm getting an error as follows:
[22P02] ERROR: invalid input syntax for integer: " "
Where: COPY eurotax_consumer, line 1, column tcomanmi: " "
But I'm using WITH NULL AS '' as the docs says:
COPY eurotax_consumer FROM 'path/to/file.txt' WITH NULL AS '';
What am I doing wrong?
You are trying to enter a string into an INT column. Can you set the destination column to a string type the do explicit conversion after insertion ?
Related
MySQL does not accept the \N value as null when I'm sure I've used it in the past for the same purpose.
Steps:
CREATE TABLE t (i INT NULL, name varchar(50));
Create a file called test.txt with tab separated fields:
1 John
\N Jim
In MySQL:
LOAD DATA INFILE '/var/lib/mysql-files/test.txt'
INTO TABLE t
Fields terminated BY "\t"
Escaped by ''
Lines terminated BY "\r\n";
Error:
ERROR 1366 (HY000): Incorrect integer value: '\N' for column 'i' at row 2
I should have done this:
LOAD DATA infile '/var/lib/mysql-files/test.txt'
INTO TABLE t
fields terminated BY "\t"
Escaped by '\\'
lines terminated BY "\r\n";
I'm trying to get a simple CSV file as source for a HSQL database.
I'm blocked at the date, getting the following error:
Exception in thread "main" java.sql.SQLException: bad TEXT table source file - line number: 1 org.hsqldb.HsqlException: data exception: invalid datetime format in statement [SET TABLE Patient SOURCE "queryDbTest.csv;fs=,;vs=,"]
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
I have tried many variations, all failed with the same error. The file is being read, and in debugging it I see it is failing at parsing the a date which is, so far as i understand, in the correct date format.
My code is as follows
System.setProperty("textdb.allow_full_path", "true");
Class.forName("org.hsqldb.jdbcDriver");
Connection conn = DriverManager.getConnection("jdbc:hsqldb:mem:mymemdb", "SA", "");
Statement st = conn.createStatement();
st.execute("CREATE TEXT TABLE Patient (" +
"HospitalNumber VARCHAR(100), " +
"NHS_Number VARCHAR(100), " +
"Name VARCHAR(100), " +
"other_names VARCHAR(100), " +
"dob Date, " +
"address VARCHAR(200))");
st.execute("SET TABLE Patient SOURCE \"" + "queryDbTest.csv" + ";all_quoted=true;fs=,;qc=\\quote\"");
and my CSV
'1234', 'N998877', 'Smith', 'Mary', '1999-12-01', 'an address 1'
Variations i have tried include:
Using 'DateTime' as a column type instead
Using the DateTime format in the CSV file (i..e, yyyy-MM-dd
HH:mm:ss)
Using a timestamp format in the CSV (i.e., yyyy-MM-dd
HH:mm:ss.SSSSS)
Using null as a value for date in the CSV.
With word 'DATE' before the date value in the csv file
In all cases, I get the same results.... a complaint about datetime format.
I'm using hsqldb-2.3.4
Any ideas?
In the TEXT table source file, the DATE column should be the data string, without the word DATE.
'1234', 'N998877', 'Smith', 'Mary', '1999-12-01', 'an address 1'
As you are using the single-quote as the quote character, you should specify it as the qc
st.execute("SET TABLE Patient SOURCE \"" + "queryDbTest.csv" + ";all_quoted=true;fs=,;qc=\\apos\"");
You can see the expected format by setting the table text source to a non-existent file and inserting and committing a single row of data using JDBC. The row will be saved in the new file with the format variables that you have set up.
I want to insert a query string into a Postgres database column in the following format
{"enrolled_time":'''SELECT DISTINCT enrolled_time AT TIME ZONE %s FROM alluser'''}
I try this:
UPDATE reports SET raw_query = {"enrolled_time":'''SELECT DISTINCT enrolled_time AT TIME ZONE %s FROM alluser'''} WHERE id=37;
It gives error like
ERROR: syntax error at or near "{"
LINE 1: UPDATE base_reports SET extra_query = {"enrolled_time":'''SE...
When I try using single quotes it throws error like following:
ERROR: syntax error at or near "SELECT"
LINE 1: ...DATE reports SET raw_query = '{"enrolled_time":'''SELECT DIS...
How can I overcome this situation
Use dollar quoting:
UPDATE reports
SET raw_query = $${"enrolled_time":'''SELECT DISTINCT enrolled_time AT TIME ZONE %s FROM alluser'''}$$
WHERE id = 37;
I want to write the content of a table from the database into CSV file using JDBC prepared statement. The PSQL query that I am using is:
COPY(select * from file where uploaded_at > ?) TO '/tmp/file_info.csv' With DELIMITER ',' CSV HEADER ";
My code is as follows:
private void copyData(Connection conn, Date date){
String sql = "COPY (select * from file where uploaded_at< ?) TO '/tmp/file_info.csv' With DELIMITER ',' CSV HEADER ";
PreparedStatement stmt = conn.prepareStatement(sql);
stmt.setTimestamp(1, new Timestamp(date.getTime()));
stmt.execute();
}
I get the following stack when I run this query:
org.postgresql.util.PSQLException: ERROR: could not determine data type of parameter $1
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:381)
at tests.com.paramatrix.pace.archival.file.TestFileArchival.main(TestFileArchival.java:63)
This exception is not generated while running a simple select query like:
select * from file where uploaded_at > ?
What could be the reason for it not working with COPY..TO statement?
You can't parameterize COPY statements. You'll have to use string interpolation or PgJDBC's CopyManager.
I'm doing a little application for the work and it includes a form. When the person using the form doesn't put a value in a data type "numeric" (lets say a PIN number) or even a date in one of my "date" Data Type fields (lets say his anniverssary), it returns me with the errors as follow:
Error Type – Type d'erreur
_Microsoft OLE DB Provider for ODBC Drivers (0x80004005)_
ERROR: invalid input syntax for type numeric: ""; Error while executing the query
Error Type – Type d'erreur
_Microsoft OLE DB Provider for ODBC Drivers (0x80004005)_
ERROR: syntax error at or near ")"; Error while executing the query
So it seems that when the person using the form enters nothing, it returns the string "empty" that is "".
Why can't the numeric type and data type read that as a NULL entry? These fields are not mandatory and so I need to have them be sometimes blank.
How can I make it possible? Is their a way to keep using "numeric" and "date" type so that when the user enters nothing in those fields, the table fills with a blank case insted of giving me this error?
Here is my sql statement :
trsConn.EXECUTE "INSERT INTO ""TRS"".employeetbl ( "& _
"""firstName"", " & _
"""lastName"" , " & _
"""detContract"", " & _
"sle, " & _
"""posNumber"" "& _
") VALUES (" & _
"'" & Request.Form("empFirst") & "', " & _**
"'" & Replace(Request.Form("empLast"), "'", "`") & "', " & _
"'" & Request.Form("dateContract") & "', " & _
"'" & Request.Form("sle") & "', " & _
"'" & Request.Form("posNum") & "');"
(The posNum and dateContract are both respectivly of type "numeric" and "date"
Thanks so much for the help. Looking forward for hearing what you geniuses have to say.
The concept of NULL in SQL is pretty muddled and inconsistent ... but it's very clear that '' is distinct from NULL.
'' isn't NULL, it's '', the empty string. You can't convert it to a date, number, etc:
regress=# SELECT CAST('' AS DATE);
ERROR: invalid input syntax for type date: ""
LINE 1: SELECT CAST('' AS DATE);
^
regress=# SELECT CAST('' AS NUMERIC);
ERROR: invalid input syntax for type numeric: ""
LINE 1: SELECT CAST('' AS NUMERIC);
^
Some products - notably Microsoft Access and old versions of MySQL - are confused about that matter. NULL is NULL, '' is the empty string; they aren't the same thing. You can't convert one to the other.
So it seems that when the person using the form enters nothing, it
returns the string "empty" that is "". Why can't the numeric type and
data type read that as a NULL entry? These fields are not mandatory
and so I need to have them be sometimes blank.
That's your application's job. When your app it sees the empty string come in on a form field for a numeric, date, or similar, it should send NULL to the database, not ''. That's normally a routine part of converting data from user input before it's supplied to the database. It is vital that you do such conversion; you should never just send values from the user straight to the database.
A quick search suggests that asp classic uses null or undefined as its null values; you should be able to pass them into your prepared statements when something is null.
The fact that you get a syntax error after the error about '' suggests that you're building your SQL statements as strings, not using prepared statements with placeholders. (Thanks JayC for the SO question ref). This is begging for SQL injection; in other words your application is critically insecure. Imagine what happens if the user enters the "date":
2012-01-01'); DROP SCHEMA public;--
and your app happily turns that into
INSERT INTO sometable (blah, blah, blah) VALUES (1, 2, DATE '2012-01-01'); DROP SCHEMA public;--');
The DROP SCHEMA then merrily executes and whoops, splat, there goes your database. That's just the dumbest, simplest kind of SQL injection attack too.