I am trying to copy a table from on MySQL database on a remote machine to another MySQL database on my local machine. I noticed that after importing the dump to my local machine, there were characters like ’ instead of single quotes.
I assumed this was an encoding issue, so I went into both databases and ran show create table posts, near the end of both, I saw CHARSET=utf8. Also, I ran file -i on the dump file, both before and after scping it to my local machine, and they were both utf8.
However, when I import this file, I get this before:
attendees—policy makers,
and after:
attendees—policy makers,
I am not sure why this is happening, everything is using utf8, what am I missing?
EDIT: I am using mysql Ver 14.12 Distrib 5.0.75, for debian-linux-gnu (x86_64) remotely, and mysql Ver 14.14 Distrib 5.5.25a, for osx10.7 (i386) locally.
On both systems you must check that your connection encoding is correct:
SHOW VARIABLES LIKE 'character_set_%'
Usually seeing characters like that is the result of double-encoding. Make sure you can match up the connection and client encoding to be exactly the same. There is a number of command line options that can facilitate this, or if you're using a driver or client, something in there can tweak it.
Related
Using Windows 10 and PostgreSQL 9.3.
I have a batch file that executes a series of pg_dump commands to back up selected tables in a database, and a related file that uses psql to restore them into an empty database. For every call to pg_dump or psql, I have to enter a password. On customer's machines (usually some version of Windows Server), I have a pgpass.conf file to specify the password, and I don't have to enter it every time.
My pgpass.conf file contains localhost:5432:*:someuser:somepassword. I have tried putting it in %APPDATA%/postresql and putting it into a folder named c:\misc and creating an environment variable named PGPASSFILE containing c:\misc\pgpass.conf. I restarted the PostgreSQL service after I changed each one.
What am I missing?
I am afraid this is sort of a non-answer.
I figured I should try with a supported version of PostgreSQL, so I installed 9.6. I assigned it port 5435. I added a corresponding line to the file pointed to by my PGPASSFILE environment variable. It worked. I tried targeting my 9.3 database. It still worked. I removed the line for port 5435. It still worked. I copied the pgpass.conf file into the default %appdata%/postgresql folder, deleted the PGPASSFILE and it still worked!
So, I have no idea what was broken, nor do I have any idea what fixed it, but it's fixed.
I'm attempting to import an SQL dump in PgAdmin 4 using the psql client - However the error message returned is - The system cannnot find the file specified.
Here is a screenshot of my psql client -
The file films.sql is currently stored on my desktop, but I suspect the default location that the psql client accesses is not my desktop? Is there anyway to set the location that the client looks in order to resolve this?
The file SQL is viewable here: https://github.com/datacamp/courses-intro-to-sql/tree/master/datasets
I simply want to get the database on my local machine so that I don't need to store queries in an online learning platform. It would be best if this database is available locally to query and practice on.
I've attempted to execute the whole SQL file as a query on the films database but this does not seem to be working either and returns 'Asynchronous query execution/operation underway.
Query returned successfully in 388 msec.' - However it seems to be the case that the Asynchronous query never completes when I refresh the database.
Please can someone help?
Just give the path to your file:
psql -d my_database -f /path/to/the/file.sql
psql -d my_database -f C:/path/to/the/file.sql
Depending on whether you are on a unix/linux machine or Windows.
Oh, and if you aren't familiar with file paths you may want to take a step back and become more familiar with general computer terminology before diving into a RDBMS. Your learning will be much easier if you have a solid foundation to build upon.
I suspect this question might be moot for the asker at this point, but for anyone else stumbling upon it like I did: the interactive connection info prompts are provided by a batch script (in Windows, I'd guess there's an analogous shell script for Unix) called runpsql.bat, which then just passes your inputs as commandline arguments to the psql.exe executable. I was getting this error because I had migrated my Postgres installation and the batch script was calling a nonexistent path for psql.exe, hence The system cannot find the file specified. I edited runpsql.bat to point to the correct location of psql.exe and that resolved the issue. So for OP, I would look into PgAdmin4 and see where it's (presumably) calling runpsql.bat, then make sure that that calls psql.exe with the correct path.
I'm trying to connect to a MS SQL Server trough PHP 5.6 with an Ubuntu 16.04 server. I'm forced to use this version of PHP, in order to assure compatibility with an 'ancient' application.
I installed PHP 5.6 and its modules (pdo, pdo_mysql, readline, etc.) trough ondrej's PPA without any trouble, but I wasn't able to find and install the 'mssql.so' module package needed for the application I'm trying to make work.
That's why I decided I would use ODBC (and PDO_ODBC) drivers in order to connect to the database.
In order to do so, I installed freetds with unixodbc and configured my files like this :
'odbcinst.ini' file :
'odbc.ini' file :
and finally the 'freetds.conf' file :
Ok, I guess all my files are well configured, but the next part starts to be a bit weird, let me explain you : when I try to connect to the MS SQL database using tsql and not giving the password in option but typing it when asked, the connexion works :
But when I try to give the same password as an option (-P), it doesn't work (I tried it at least 10 times with the correct password) !!!
--> tsql failed to open a session for the user 'WIPSOS-PHP'
The same problem happens when I try to use isql with one of my connectors I configured in the 'odbc.ini' file :
It seems to be related to the password, but I can't find the problem, can you please help me ?
I found a partial answer, and it was effectively related to the password: it seems like special characters are not well interpreted when present in the password given for the -P option of the tsql command.
Therefore, depending on you shell, you have to escape these characters in the string which completes an option. In my case, using bash, I had to use the '\' to escape the special character :
tsql -S hostname -D Database -U User -P pa\$\$word
And it now works!
But when I try to escape the special characters with '\' in the 'odbc.ini' file, it still doesn't seem to work :
My next question is : 'How to escape a special character in a '.ini' file ?' or 'Is there something wrong in my configuration ?'
EDIT: I know it was a long time ago, but I had a configuration problem related to the hostname in the "freetds.conf" file. In fact, I was using the server's local network DNS name instead of its IP adress(which works).
I hope my issue will at least help some people, it isn't easy to deal with TSQL and FreeTDS...
Setup:
Ubuntu 12.04 Server installed via VMWare quick install
PostgreSQL 9.1
ElasticSearch 0.90
Mono 3.2.1
Rails 4
Nginx 1.4.2 + Passenger 4.0.16
I have a C# program that on start writes a new ElasticSearch index and points the alias that is used by the rails applications to it, the program then keeps going and watches a redis instance for things to update.
There is another C# program that scrapes data from web pages, once scraped they are put into Postgresql and the index writer above is notified via Redis. Those pages have varying encodings and are converted to UTF-8.
The first appearance of this bug was when I made a mistake and encoded data that was already UTF-8 as UTF-8 again.
Investigation
Now I thought that I obviously have some data corruption going on but the weird thing is: The umlauts are only corrupted when I start the indexing mono process from rails via nohup, if I kill this process and manually start it from the command line it works perfectly fine.
When I do a backup/restore of the database it works again from web interface but once the server is rebooted the umlauts are again replaced with ?? when starting the mono process from the web interface.
The first thing I did was to purge the affected rows from the database and scrape the data again (without encoding it twice), that didn't help and since the error only appears when running it as non-interactive via nohup from the rails application I assumed it is because of the locale setting so I changed that in both, /etc/defaults/locale and /etc/environment to en_US.UTF-8 and en_US:en but that did not help either.
I really have no idea what else I can do or what exactly causes this error, any help would be appreciated.
edit: I forgot to clarify the most important part, when umlauts are replaced with ?? ALL umlauts are replaced in every single document in the index.
Put this in the script that you use to start your process:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
The reason that your script only picks up the UTF-8 when you start things manually is that these things are not system wide. I've run into this with jruby and init.d scripts before and the solution is to not rely on defaults for this.
I've been using JDBC with a local Postgres DB to copy data from CSV files into the database with the Postgres COPY command. I use Java to parse the existing CSV file into a CSV format matches the tables in the DB. I then save this parsed CSV to my local disk. I then have JDBC execute a COPY command using the parsed CSV to my local DB. Everything works as expected.
Now I'm trying to perform the same process on a Postgres DB on a remote server using JDBC. However, when JDBC tries to execute the COPY I get
org.postgresql.util.PSQLException: ERROR: could not open file "C:\data\datafile.csv" for reading: No such file or directory
Am I correct in understanding that the COPY command tells the DB to look locally for this file? I.E. the remote server is looking on its C: drive (doesn't exist).
If this is the case, is there anyway to indicate to the copy command to look on my computer rather than "locally" on the remote machine? Reading through the copy documentation I didn't find anything that indicated this functionality.
If the functionality doesn't exist, I'm thinking of just populating the whole database locally then copying to database to the remote server but just wanted to check that I wasn't missing anything.
Thanks for your help.
Create your sql file as follows on your client machine
COPY testtable (column1, c2, c3) FROM STDIN WITH CSV;
1,2,3
4,5,6
\.
Then execute, on your client
psql -U postgres -f /mylocaldrive/copy.sql -h remoteserver.example.com
If you use JDBC, the best solution for you is to use the PostgreSQL COPY API
http://jdbc.postgresql.org/documentation/publicapi/org/postgresql/copy/CopyManager.html
Otherwise (as already noted by others) you can use \copy from psql which allows accessing the local files on the client machine
To my knowledge, the COPY command can only be used to read locally (either from stdin or from file) from the machine where the database is running.
You could make a shell script where you run you the java conversion, then use psql to do a \copy command, which reads from a file on the client machine.