I am storing data which includes single and double quotes in a column which has a type of text.
I have a function that returns that column but it always returns the double quote character being doubled up. I tried to escape the character \" but that just returned \"". I have the same process working in MySQL well. I am trying to move to Postgres for greater scaling and stability but it seems it does not work well. I have noticed in pgAdmin when I look at the data output, it seems to show the data correctly so I am wondering what is being done there for it to work correctly?
Also I am getting a (" at the start of the returned value and ") at the end.
Can someone help please?
Use pg_escape_string to escape quote characters in your string.
Related
I am trying to save list of regex in mongodb.
But while saving the values '\' char present in value automatically gets removed.
e.g; "val":/55\sA\.?B\.?C\.?\s/
after insertion it becomes
"val":/5sA.?B.?C.?s/
You need to escape your characters with \. For example, if you're trying to add the ? character, you need to do something like ...\?B.\?c....
I hope my answer was helpful.
I've been working on an Express app that has a form designed to hold lines and quotes.
Some of the lines will have single quotes('), but overall it's able to store the info and I'm able to back it up and store it without any problems. Now, when I want do pg_dump and have the database put into an SQL file, the quotes seem to cause some things to appear a bit wonky in my text editor.
Would I have to create a method to change all the single quotation marks into double, or can I leave it as is and be able to upload it back to the database without causing major issues. I know people will continue to enter in lines that contain either single or double quotations, so I would like to know any solution or answer that would help greatly.
Single quotes in character data types are no problem at all. You just need to escape them properly in string literals.
To write data with INSERT you need to quote all string literals according to SQL syntax rules. There are tools to do that for you ...
Insert text with single quotes in PostgreSQL
However, pg_dump takes care of escaping automatically. The default mode produces text output to be re-imported with COPY (much faster than INSERT), and single quotes have no special meaning there. And in (non-default) csv mode, the default quote character is double-quote (") and configurable. The manual:
QUOTE
Specifies the quoting character to be used when a data value is quoted. The default is double-quote. This must be a single one-byte character. This option is allowed only when using CSV format.
The format is defined by rules for COPY and not by SQL syntax rules.
I have a large .csv file with 9 million rows. Some of these columns contain text with quotes or other special characters in them I would like to import from this .csv file into the database. For example I would like to import this row:
ID BH Units Name Type_building Year_cons
1 4 900.00 schoolgebouw "De Bolster Schoolgebouw 2014-01-01
As you can see there is a double quote in the fourth column. None of the values in the .csv file are quoted, but sometimes a double quote or backslash '\' appears in the text. When I try to upload the data using:
\COPY <tablename> FROM <path to file> WITH CSV DELIMITER ';' NULL '\N';
It gives an error message: ERROR value to long for type character varying(25).
Apparently it sees the double quote as the start of a string and it tries to combine everything after it in the .csv file (including the fifth and sixth column) into a single cell (so that cell will contain 'De Bolster Schoolgebouw 2014-01-01'), which doesnt fit because the 'Name' column allows max 25 characters.
I found a similar topic (Is it possible to turn off quote processing in the Postgres COPY command with CSV format?) in which this solution was presented:
\COPY <tablename> FROM <path to file> WITH CSV DELIMITER ';' QUOTE E'\b' NULL '\N';
I think what it does is sets the quote value (default is double quote) to something else, in this case a backspace, so it won't recognize a double quote as a quote anymore. However when I run this I get another error: INVALID input syntax for integer.
What has happened is that every value now is quoted, so ID with value '1' becomes value '"1"' and because ID is defined as an integer it won't accept quotes.
Do you have any idea how to import double quotes and other special characters from a .csv file into a postgres database?
Thanks in advance!!
Based on the error message, I'd be suspicious it has anything to do with double quoting or anything of the sort -- had it been so, it would have been a widely reported bug and fixed ages ago.
When it comes to Postgres, the error messages are almost always correct and helpful. As such, consider the very real possibility that there are more characters than meets the eye.
My own guess is that you've some trailing (or leading) spaces in there somewhere, and as such have pieces of data that look 24 characters long when viewed in a spreadsheet while being, in fact, longer.
If you don't, my second guess would be some kind of bizarro character sets conflicts or effects. Perhaps you've some double byte characters, or two single characters behaving as a single one due to a diacritic in there. These look fine in the viewer you're using for your data; but then when these get interpreted or viewed as utf8 they end up counting as two distinct characters. Unlikely imo, but possible (example).
Lastly and per Frank's suggestion, try removing the length constraint. It is only slowing you down as things stand, because it slows down inserts and is preventing you to move forward. Once done importing, re-add the constraint to the table's definition. You'll then be able to find the offending rows using the likes of:
select name from table where length(name) > 24;
... and upon fixing them, you'll be able to re-add your constraint if it serves any purpose. (Hint: it doesn't, or at the very least shouldn't have. There's a real person out there whose name is: "Kim-Jong Sexy Glorious Beast Divine Dick Father Lovely Iron Man Even Unique Poh Un Winn Charlie Ghora Khaos Mehan Hansa Kimmy Humbero Uno Master Over Dance Shake Bouti Bepop Rocksteady Shredder Kung Ulf Road House Gilgamesh Flap Guy Theo Arse Hole Im Yoda Funky Boy Slam Duck Chuck Jorma Jukka Pekka Ryan Super Air Ooy Rusell Salvador Alfons Molgan Akta Papa Long Nameh Ek.")
working on an iPhone app. I just imported some records into a SQL Lite database, and all my regular quote marks have been "doubled". An example:
Desired final format:
The song "ABC" will play at 3 PM.
The record is currently appearing in the database as:
The song ""ABC"" will play at 3 PM.
Does anyone know how to do a SQL update to change all "double-double" quotes to just regular quotation marks?
Just to clarify, I'm looking directly at the database, not via code. The code will just display these as "double-double" quotes just as they appear in the database, so I want to remove them. The "double-double" quotes are actually in the import file as well, but if I try to remove them, then the import fails. So I kept them there, and now that the records are successfully imported into the database, now I just want to correct the "double-double" quote thing with a mass SQL update if it's possible. Thanks in advance for any insight!
SQLite uses single quotes to escape string literals. It escapes single quotes by adding another single quote (likewise for double quotes). So technically as long as your SQL is well constructed, the import process should work properly. The strings should be enclosed in single quotes, and not double quotes. I suspect that your code may be constructing the SQL by hand instead of binding/properly escaping the values.
SQLite has a built in function to quote string's. It's called quote. Here are some sample inputs, and the corresponding output:
sqlite> SELECT quote("foo");
'foo'
sqlite> SELECT quote("foo ""bar""");
'foo "bar"'
sqlite> SELECT quote("foo 'bar'");
'foo ''bar'''
So you could remove the twice escaped double quote before it even goes to SQLite using NSString methods.
[#"badString\"\"" stringByReplacingOccurrencesOfString:#"\"\"" withString:#"\""];
If the database already contains bad values, then you could run the following update SQL to clean it up:
UPDATE table SET column = REPLACE(column, '""', '"');
In my application I am trying to read some data from sqlite database into UITextView.
This text has double quotes as well as single quotes, for ex:
she said : "Katie's purse has been
lost"
when I do this, I get strange characters in place of double and single quotes. Please help me with a solution to scrub these characters off.
Thanks.
If you get one strange character (or set of characters) replacing " and another replacing ', you can just scan the string and replace. If you aren't, then:
That's very strange, and
You should post your code.