I got a TYPO3 8.7 instance running over a server whose database and tables collation is utf8mb4_unicode_ci. The character set is utf8mb4.
However, MySQL over the server shows the collation for the connection is latin1_swedish_ci and collation for the server is utf8_general_ci (I guess these are parameters that come pre-configured with the hosting):
+----------------------+--------------------+
| Variable_name | Value |
+----------------------+--------------------+
| collation_connection | latin1_swedish_ci |
| collation_database | utf8mb4_unicode_ci |
| collation_server | utf8_general_ci |
+----------------------+--------------------+
3 rows in set (0.00 sec)
I would like to reproduce all these settings over my DDEV instance. I got the last two variable names properly configured with these settings over my .ddev/mysql/no_utf8mb4.cnf file:
collation-server = utf8_general_ci
character-set-server = utf8
But I cannot get the collation_connection as latin1_swedish_ci. How could I achieve that?
I exported my database from the hosting and imported it into ddev, but with the current configuration, every time I login into the backend, I receive an error like this:
An exception occurred while executing 'INSERT INTO sys_log (userid, type, action, error, details_nr, details, log_data, tablename, recuid, IP, tstamp, event_pid, NEWid, workspace) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [1, 255, 1, 0, 1, "User %s logged in from %s (%s)", "a:3:{i:0;s:30:\"xxx#xxx.com\";i:1;s:10:\"172.18.0.6\";i:2;s:0:\"\";}", "", 0, "172.18.0.6", 1564691070, -1, "", -99]: Field 'request_id' doesn't have a default value
Has anyone experienced this issue?
My MariaDB version over my ddev instance is 10.1.37-MariaDB.
Related
I facing the issue while trying to export dashboard from server to another server of same version 1.5, but showing error: Integrity error. How to import dashboard?
ERROR: (while importing Dashboard)
sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.IntegrityError) UNIQUE constraint failed: tables.table_name
[SQL: INSERT INTO tables (uuid, created_on, changed_on, description, default_endpoint, is_featured, filter_select_enabled, "offset", cache_timeout, params, perm, schema_perm, is_managed_externally, external_url, table_name, main_dttm_col, database_id, fetch_values_predicate, schema, sql, is_sqllab_view, template_params, extra, created_by_fk, changed_by_fk) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: (<memory at 0x7f3b7c882640>, '2022-11-18 11:22:47.890965', '2022-11-18 11:22:47.890971', None, None, 0, 0, 0, None, '{"remote_id": 19, "database_name": "PostgreSQL", "import_time": 1668750767}', None, None, 0, None, 'storefrequency', None, '7', None, None, None, 0, None, None, 1, 1)]
(Background on this error at: http://sqlalche.me/e/13/gkpj) (Background on this error at: http://sqlalche.me/e/13/7s2a)
2022-11-18 11:22:47,893:INFO:werkzeug:127.0.0.1 - - [18/Nov/2022 11:22:47] "POST /superset/import_
I also changed the VERSIONED_EXPORT=True and DASHBOARD_RBAC=True but still it showing a error.
sql_insert_query = "insert into TABLE1 (building, course, description, course_type, course_type_desc, dual_credit)
VALUES (?, ?, ?, ?, ?, ?) ON DUPLICATE KEY UPDATE building = VALUES(building), course = VALUES(course), description = VALUES(description), course_type = VALUES(course_type), course_type_desc = VALUES(course_type_desc), dual_credit = VALUES(dual_credit);"
cursor.executemany(sql_insert_query, listCourse)
pyodbc.ProgrammingError: ('42000', u"[42000] [Microsoft][SQL Server
Native Client 11.0][SQL Server]Incorrect syntax near the keyword 'ON'.
(156) (SQLExecDirectW); [42000] [Microsoft][SQL Server Native Client
11.0][SQL Server]Statement(s) could not be prepared. (8180)")
This one below only works, but adds duplicate when running again.
"insert into TABLE1 (building, course, description, course_type, course_type_desc, dual_credit) VALUES (?, ?, ?, ?, ?, ?)"
I tied '%s' this did not work so I am using '?'
I have resolved it by tile with tuple
listCourse = numpy.tile(courses, 2)
listCourse = map(tuple,numpy.tile(courses, 2))
I recently started porting a SQLite database over to PostGreSQL for a Flask site built with SQLAlchemy. I have my schemas in PGSQL and even inserted the data into the database. However, I am unable to run my usual INSERT commands to add information to the database. Normally, I insert new records using SQL Alchemy by leaving the ID column to be NULL and then just setting the other columns. However, that results in the following error:
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null, 2017-07-24 20:40:37.787393+00, 2017-07-24 20:40:37.787393+00, episode_length_list = [52, 51, 49, 50, 83]
sum_length = 0
for ..., 0, f, 101, 1, 0, 0, , null).
[SQL: 'INSERT INTO submission (date_created, date_modified, code, status, correct, assignment_id, course_id, user_id, assignment_version, version, url) VALUES (CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, %(code)s, %(status)s, %(correct)s, %(assignment_id)s, %(course_id)s, %(user_id)s, %(assignment_version)s, %(version)s, %(url)s) RETURNING submission.id'] [parameters: {'code': 'episode_length_list = [52, 51, 49, 50, 83]\n\nsum_length = 0\n\nfor episode_length in episode_length_list:\n pass\n\nsum_length = sum_length + episode_length\n\nprint(sum_length)\n', 'status': 0, 'correct': False, 'assignment_id': 101, 'course_id': None, 'user_id': 1, 'assignment_version': 0, 'version': 0, 'url': ''}]
Here is my SQL Alchemy table declarations:
class Base(Model):
__abstract__ = True
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
def __repr__(self):
return str(self)
id = Column(Integer(), primary_key=True)
date_created = Column(DateTime, default=func.current_timestamp())
date_modified = Column(DateTime, default=func.current_timestamp(),
onupdate=func.current_timestamp())
class Submission(Base):
code = Column(Text(), default="")
status = Column(Integer(), default=0)
correct = Column(Boolean(), default=False)
assignment_id = Column(Integer(), ForeignKey('assignment.id'))
course_id = Column(Integer(), ForeignKey('course.id'))
user_id = Column(Integer(), ForeignKey('user.id'))
assignment_version = Column(Integer(), default=0)
version = Column(Integer(), default=0)
url = Column(Text(), default="")
I created the schema by calling db.create_all() in a script.
Checking the PostGreSQL side, we can see the constructed table:
Table "public.submission"
Column | Type | Modifiers | Storage | Stats target | Description
--------------------+--------------------------+-----------+----------+--------------+-------------
id | bigint | not null | plain | |
date_created | timestamp with time zone | | plain | |
date_modified | timestamp with time zone | | plain | |
code | text | | extended | |
status | bigint | | plain | |
correct | boolean | | plain | |
assignment_id | bigint | | plain | |
user_id | bigint | | plain | |
assignment_version | bigint | | plain | |
version | bigint | | plain | |
url | text | | extended | |
course_id | bigint | | plain | |
Indexes:
"idx_16881_submission_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"submission_course_id_fkey" FOREIGN KEY (course_id) REFERENCES course(id)
"submission_user_id_fkey" FOREIGN KEY (user_id) REFERENCES "user"(id)
Has OIDs: no
I'm still new to this, but shouldn't there be a sequence?
Any insight or suggestions on what to look for next would be super appreciated.
It is standard SQL that a PRIMARY KEY is UNIQUE and NOT NULL. PostgreSQL enforces the standard, and does not allow you to have any (not a single one) NULL on the table. Other databases allow you to have one NULL, therefore, the different behaviour.
PostgreSQL current documentation on Primary Keys clearly states it:
5.3.4. Primary Keys
A primary key constraint indicates that a column, or group of columns, can be used as a unique identifier for rows in the table. This requires that the values be both unique and not null.
If you want your PRIMARY KEY to be a synthetic (i.e.: not natural) sequence number, you should define it with type BIGSERIAL instead of BIGINT. I don't know the details on how this is achieved using SQLAlchemy, but look at the references.
When you then INSERT into your table, the id should NOT be in the INSERT column list (it should not be set to null, just not be there). I.e.:
This will generate a new id:
INSERT INTO public.submission (code) VALUES ('Some code') ;
will work.
This won't:
INSERT INTO public.submission (id, code) VALUES (NULL, 'Some code') ;
I guess SQLAlchemy should be smart enough to generate the proper SQL INSERT statements, once properly configured.
Reference:
Why isn't SQLAlchemy creating serial columns?
Ultimately, I discovered what went wrong, and it was definitely my fault. The process I used to load the old data into the database (pgloader) was doing more than just loading data - it was somehow overwriting parts of the table definitions! I was able to pg_dump the data out, reset the tables, and then load it back in - everything works as expected. Thanks for sanity checks!
I'm programming a script which queries some devices and writes the data to a PostgreSQL database.
The data includes a date which is formatted like 31.12.2015 13:45:00. The database uses the DateType "German" and the column is of the type Timestamp without timezone.
I always get this error message
DBD::Pg::st execute failed: ERROR: invalid input syntax for type timestamp:
"TO_TIMESTAMP('19.06.2015 11:24:20','DD.MM.YYYY HH24:MI:SS')::timestamp without time zone"
at temp_alcp2e_db.pl line 80, line 289.
I'm using this code, where $date_db has the date value:
$date_db = 'TO_TIMESTAMP(\'' . $date_db . '\',\'DD.MM.YYYY HH24:MI:SS\')::timestamp without time zone';
$stmt = $dbh->prepare("INSERT INTO rifutemp (\"USER_LINK_ID\", \"IP\", \"DATUM\", \"TEMPERATURE\") VALUES (?, ?, ?, ?)");
$stmt->execute($key_bgtr, $key_ip, $date_db, $temperatur) or die $DBI::errstr;
Hopefully, someone can show me what I did wrong.
The function can (and must) be part of the prepared statement.
Re-write your code as follows:
$stmt =$dbh->prepare(q{
INSERT INTO rifutemp ("USER_LINK_ID","IP","DATUM","TEMPERATURE")
VALUES (?, ?,
TO_TIMESTAMP(?, 'DD.MM.YYYY HH24:MI:SS')::timestamp without time zone,
?)
});
$stmt->execute($key_bgtr,$key_ip,$date_db,$temperatur) or die $DBI::errstr;
Ok, I've found my problem / the source of the errors:
To manage the Database visually I'm using the "EMS SQL Manager Lite for PostgreSQL", and this nice GUI tool always sets the DateStyle options to "ISO, MDY".
I've just changed my code like this:
$dbh->do("SET datestyle = 'German'");
$stmt =$dbh->prepare("INSERT INTO rifutemp (\"USER_LINK_ID\",\"IP\",\"DATUM\",\"TEMPERATURE\")
VALUES (?, ?, ?, ?)");
$stmt->execute($key_bgtr,$key_ip,$date_db,$temperatur) or die $DBI::errstr;
and set the $date_db variable to the correct format:
$date_db = Time::Piece->new->strftime('%d.%m.%Y %H:%M:%S');
and now everything works fine!
$q2 = "UPDATE `tasks` SET `title` = ?, task = ?, rules = ?, media = ?, type = ?, xp = ?, available = ?, timestamp = ? WHERE id = ?";
if ($stmt = $mysqli->prepare($q2)) {
$stmt->bind_param("sssssissi", $_POST["tasktitle"], $_POST["editor"], $_POST["rules"], serialize($_POST["media"]), $_POST["type"], $_POST["xp"], $a = 0, strtotime("now"), $_GET['id']);
$stmt->execute();
$stmt->close();
}
$r = $mysqli->query($q2) or die($mysqli->error);
I got this error msg:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?, task = ?, rules = ?, media = ?, type = ?, xp = ?, available = ?, timestamp = ' at line 1
What is problem, and how can i solve it?
I'm pretty certain it's coming from the call to $mysqli->query() which needs a properly escaped query (ie, none of that nice safe parameter stuff). That explains why it's complaining at the first ?.
Quick way to check is to actually comment out the entire if statement and se if the error still appears. If so, you know it's the query rather than the prepared statement execution.
My question to you is: why are you executing the prepared statement and then trying to run it again as a query?
I think you'll find the execute does your update quite well enough. Get rid of the call to query and you should be fine.