I'm trying to restore tables from a dump file. It's illustrated by a footnote in the paper "VCCFinder-Finding Potential Vulnerabilities in Open-Source Projects to Assist Code Audits", that the dump file that the team created with pg_dump could be read with pg_restore. As it's shown in paper footnote with red line to emphasize. That's where I've started.
1. Use pg_restore command
By typing the command mentioned in your paper: VCCFinder: Finding Potential Vulnerabilities in Open-Source Projects to Assist Code Audits:
pg_restore -f vcc_base I:\OneDrive\PractiseProject\x_prjs\m_firmware_scan\m_firmware_scan.ref\vcc-database\vccfinder-database.dump
Windows CMD had returned an error message:
pg_restore: error: input file appears to be a text format dump. Please use psql.
As I had tried the operation in different version, including v14.4, v9.6, v9.4 and v9.3, the outcome is the same error message.
2.Use psql command
Then I turned to another direction: using psql. After typing command,
psql -v ON_ERROR_STOP=1 -U postgres < I:\OneDrive\PractiseProject\x_prjs\m_firmware_scan\m_firmware_scan.ref\vcc-database\vccfinder-database.dump
apart from postgreSQL 14.4 environment, the returned error message is:
psql: SCRAM authentication requires libpq version 10 or above
Under postgreSQL 14.4 environment, the returned message became:
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
If I remove the -v ON_ERROR_STOP=1 option, and returned message would be like this:
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
SET
SET
SET
ERROR: type "public.hstore" does not exist
LINE 27: patch_keywords public.hstore
^
ERROR: relation "cves" already exists
ERROR: relation "repositories" already exists
ERROR: relation "commits" does not exist
invalid command \n
invalid command \N
invalid command \N
...
(Solved) I have tried to solve the unreadable code problem shown in above error messages by typing chcp 65001, chcp 437 and etc to change character set into UTF8 or American English in Windows CMD, but it's not helpful. But after viewing the source code of the dump file in Visual Studio, it's not difficult to infer that those error messages were caused by psql commands in the dump file.
After the error messages became understandable, I focused on one particular error message:
ERROR: type "public.hstore" does not exist
LINE 27: patch_keywords public.hstore
So I manually created a "hstore" type below the "pulic SCHEMA", after that error messages turned into these:
SET
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
SET
SET
SET
ERROR: relation "commits" already exists
ERROR: relation "cves" already exists
ERROR: relation "repositories" already exists
ERROR: malformed record literal: ""do"=>"1", "if"=>"0", "asm"=>"41", "for"=>"5", "int"=>"13", "new"=>"0", "try"=>"0", "auto"=>"0", "bool"=>"0", "case"=>"0", "char"=>"1", "else"=>"0", "enum"=>"0", "free"=>"0", "goto"=>"0", "long"=>"15", "this"=>"0", "true"=>"0", "void"=>"49", "alloc"=>"0", "break"=>"0", "catch"=>"0", "class"=>"0", "const"=>"0", "false"=>"0", "float"=>"0", "short"=>"0", "throw"=>"0", "union"=>"0", "using"=>"0", "while"=>"1", "alloca"=>"0", "calloc"=>"0", "delete"=>"0", "double"=>"0", "extern"=>"4", "friend"=>"0", "inline"=>"18", "malloc"=>"0", "public"=>"0", "return"=>"4", "signed"=>"1", "sizeof"=>"0", "static"=>"32", "struct"=>"4", "switch"=>"0", "typeid"=>"0", "default"=>"0", "mutable"=>"0", "private"=>"0", "realloc"=>"0", "typedef"=>"0", "virtual"=>"0", "wchar_t"=>"0", "continue"=>"0", "explicit"=>"0", "operator"=>"0", "register"=>"0", "template"=>"0", "typename"=>"0", "unsigned"=>"23", "volatile"=>"23", "namespace"=>"0", "protected"=>"0", "const_cast"=>"0", "static_cast"=>"0", "dynamic_cast"=>"0", "reinterpret_cast"=>"0""
DETAIL: Missing left parenthesis.
CONTEXT: COPY commits, line 1, column patch_keywords: ""do"=>"1", "if"=>"0", "asm"=>"41", "for"=>"5", "int"=>"13", "new"=>"0", "try"=>"0", "auto"=>"0", "bo..."
ERROR: syntax error at or near "l022_save"
LINE 1: l022_save, pl022_load, s);
^
invalid command \n
invalid command \N
invalid command \N
...
Now the three tables have been created, but there is no content in them.
3. Install hstore
After searching for "hstore"hstore type does not exist with hstore installed postgresql, I realized that the "hstore" should be installed, but not be manually created. So I typed this in psql command line:
postgres=# create EXTENSION hstore; And there were new error messages:
SET
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
SET
SET
SET
CREATE TABLE
ERROR: relation "cves" already exists
ERROR: relation "repositories" already exists
ERROR: missing data for column "hunk_count"
CONTEXT: COPY commits, line 23201: "11388700 178 \N other_commit 1d6198c3b01619151f3227c6461b3d53eeb711e5\N blueswir1#c046a42c-6fe2-441..."
ERROR: syntax error at or near "l022_save"
LINE 1: l022_save, pl022_load, s);
^
invalid command \n
invalid command \N
invalid command \N
...
And still, there is no content in those three tables.
4. Generate and view tables
After looking into the source code of the dump file, and trying to fix the "hunk_count" problem but end up with failure. It occurs to me that the above error messages just caused by one paticular row of code. So I had deleted the row and the old error messages were gone but there were new error messages caused by another row. Evetually I have deleted 10 rows in total, comparing to the total row number: 351409, those deleted parts are negligible. And three tables weren't empty anymore, as it's shown in pgAdmin 4.
However, the pgADmin only demonstrated the structure of those tables, I still didn't know how to view the content in them. By refering to 2 Ways to View the Structure of a Table in PostgreSQL, I typed
SELECT
*
FROM
export.repositories/ export.cves/ export.commits
WHERE
TRUE
to generate and view corresponding tables in pgAdmin 4. For example, final cve table:
5. In the end
Looking back at these steps, these are all easy steps, but for a guy who was not familiar with the tools or operations, it could cost several days to search and type, step by step for one simple purpose. I wish this post could be useful to someone like me.
However, I am not so familiar with psql commands or anything about postgreSQL, as a matter of fact, I had never used them before. So I'm wondering if someone could point out some mistakes I may have made in those attempts, or provide some suggestions for my dilemma.
First , ensure your dump format.
Try to read header (first 5 chars) of dump file.
If it is signed as PGDMP then it is binary/custom dump else it is sql (human readable format).
- use pg_restore for binary dump import.
$ pg_restore -U postgres -d <dbname> file.dump
- use psql to import plain text sql dump.
$ psql -U postgres -d <dbname> < file.dump
Solved, as I've demonstrated above.
I imported my excel file into R Environment and saved the path by creating a new file in R scrip. However, when I tried to check my directory and load the dataset, I received the following message " Error: path does not exist: ‘MIS_655_RS_T3_Wholesale_Customers’
What am I doing wrong here?
Thanks
Have you missed the format of your dataset, eg. csv, xlsx.
I suggest you first set your file as working directory, then the following code might help you with it.
Dat_customers <- readxl::read_excel("MIS_655_RS_T3_Wholesale_Customers.xlsx")
Look at the following two sequences of two commands. They are the same up to the choice of variable a or z, but the version with a produces an error whereas the function with z does not.
First run (after a restart of maxima):
declare(a,constant); polynomialp(x^2+x+1/a,[x]);
Now replace a by z and there is no error when run (after a restart of maxima):
declare(z,constant); polynomialp(x^2+x+1/z,[x]);
The error reported in the case with variable a is:
define: in definition of dotproduct, found bad argument a
-- an error. To debug this try: debugmode(true);
Strangely enough, after running the second command of the first sequence again, the error does not recur.
Above code was run on Maxima versions 5.30 and 5.39.
I have moved my moodle database files from old server URLold= (https://example1.com) to new server URLnew=(https://example2.com). Now I want to replace URLold with URLnew in database tables using find and replace tool provided by moodle. But when i perform the operation I am getting this error. What should i do? Please help.
Error I am getting
Debug info: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'table = REPLACE(table, 'https://example1.com', 'https://example2.com')' at line 1
UPDATE mdl_pma_history SET table = REPLACE(table, ?, ?)
[array (
0 => 'https://example1.com',
1 => 'https://example2.com',
)]
Error code: dmlwriteexception
Stack trace:
line 426 of /lib/dml/moodle_database.php: dml_write_exception thrown
line 895 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
line 6787 of /lib/adminlib.php: call to mysqli_native_moodle_database->execute()
line 74 of /admin/tool/replace/index.php: call to db_replace()
So I got the answer on my own
Had to delete the mdl_pma_history table that was causing the error. The Steps i followed are as follows.
Exported the table to .sql file
Deleted the table because it was not allowing the script to run
Once the script (Find and replace) ran successfully imported the
table back
Done.
I am trying to update a column:
update t_references
set reference = 'Stöcker W, et al. Autoimmunity to Pancreatic Juice in Crohn’s Disease.
Results of an Autoantibody Screening in Patients With Chronic Inflammatory Bowel Disease. <i>Scand J Gastroenterol Suppl</i>. 1987;139:41-52.'
,index = 9
where reference_id = 161;
I got error:
The query could not be converted to the required encoding.
Please advise.
I had to login to the machine and then ran this from the command.
On experiencing the same error, I found I could recreate it when opening a .SQL file saved from PGAdmin in an editor(even Notepad) and then copying and pasting the contents into PGAdmin. When I opened the file directly with PGAdmin, there was no issue.
HTH