Error after upgrading Apache Superset - RuntimeError: Working outside of application context - upgrade

I just upgraded to newest version of Apache Superset.
When I start Apache Superset with the command
gunicorn \
-w 10 \
-k gevent \
--timeout 120 \
-b 0.0.0.0:8088 \
--limit-request-line 0 \
--limit-request-field_size 0 \
--statsd-host localhost:8088 \
superset:app
I get the error "Internal server error"
On the server it says:
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/superset/lib/python3.7/site-packages/werkzeug/local.py", line 376, in <lambda>
__call__ = lambda x, *a, **kw: x._get_current_object()(*a, **kw)
File "/usr/local/superset/lib/python3.7/site-packages/werkzeug/local.py", line 307, in _get_current_object
return self.__local()
File "/usr/local/superset/lib/python3.7/site-packages/flask/globals.py", line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
...
I found out that it I changed to "superset.app:create_app()" I could login, like this:
gunicorn \
-w 10 \
-k gevent \
--timeout 120 \
-b 0.0.0.0:8088 \
--limit-request-line 0 \
--limit-request-field_size 0 \
--statsd-host localhost:8088 \
"superset.app:create_app()"
But When I open a dashboard I now get this error:
File "/usr/local/superset/lib/python3.7/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1054, "Unknown column 'slices.schema_perm' in 'field list'")
[SQL: SELECT slices.created_on AS slices_created_on, slices.changed_on AS slices_changed_on, slices.id AS slices_id, slices.slice_name AS slices_slice_name, slices.datasource_id AS slices_datasource_id, slices.datasource_type AS slices_datasource_type, slices.datasource_name AS slices_datasource_name, slices.viz_type AS slices_viz_type, slices.params AS slices_params, slices.description AS slices_description, slices.cache_timeout AS slices_cache_timeout, slices.perm AS slices_perm, slices.schema_perm AS slices_schema_perm, slices.created_by_fk AS slices_created_by_fk, slices.changed_by_fk AS slices_changed_by_fk, tables_1.created_on AS tables_1_created_on, tables_1.changed_on AS tables_1_changed_on, tables_1.id AS tables_1_id, tables_1.description AS tables_1_description, tables_1.default_endpoint AS tables_1_default_endpoint, tables_1.is_featured AS tables_1_is_featured, tables_1.filter_select_enabled AS tables_1_filter_select_enabled, tables_1.offset AS tables_1_offset, tables_1.cache_timeout AS tables_1_cache_timeout, tables_1.params AS tables_1_params, tables_1.perm AS tables_1_perm, tables_1.schema_perm AS tables_1_schema_perm, tables_1.table_name AS tables_1_table_name, tables_1.main_dttm_col AS tables_1_main_dttm_col, tables_1.database_id AS tables_1_database_id, tables_1.fetch_values_predicate AS tables_1_fetch_values_predicate, tables_1.`schema` AS tables_1_schema, tables_1.`sql` AS tables_1_sql, tables_1.is_sqllab_view AS tables_1_is_sqllab_view, tables_1.template_params AS tables_1_template_params, tables_1.created_by_fk AS tables_1_created_by_fk, tables_1.changed_by_fk AS tables_1_changed_by_fk
FROM dashboard_slices, slices LEFT OUTER JOIN `tables` AS tables_1 ON slices.datasource_id = tables_1.id AND slices.datasource_type = %s
WHERE %s = dashboard_slices.dashboard_id AND slices.id = dashboard_slices.slice_id]
[parameters: ('table', 7)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
I have searched more for a solution. It seems that my MySQL-database is not corrected with the upgrade, when using the db upgrade command:
(superset) [root#host .superset]# superset db upgrade
logging was configured successfully
INFO:superset.utils.logging_configurator:logging was configured successfully
/usr/local/superset/lib/python3.7/site-packages/flask_caching/__init__.py:188: UserWarning: Flask-Cache: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn("Flask-Cache: CACHE_TYPE is set to null, "
WARNI [alembic.env] SQLite Database support for metadata databases will be removed in a future version of Superset.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
I dont see any errors?
When I call init I get this:
(superset) [root#host .superset]# superset init
logging was configured successfully
INFO:superset.utils.logging_configurator:logging was configured successfully
/usr/local/superset/lib/python3.7/site-packages/flask_caching/__init__.py:188: UserWarning: Flask-Cache: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn("Flask-Cache: CACHE_TYPE is set to null, "
Syncing role definition
INFO:superset.security.manager:Syncing role definition
Syncing Admin perms
INFO:superset.security.manager:Syncing Admin perms
Syncing Alpha perms
INFO:superset.security.manager:Syncing Alpha perms
Syncing Gamma perms
INFO:superset.security.manager:Syncing Gamma perms
Syncing granter perms
INFO:superset.security.manager:Syncing granter perms
Syncing sql_lab perms
INFO:superset.security.manager:Syncing sql_lab perms
Fetching a set of all perms to lookup which ones are missing
INFO:superset.security.manager:Fetching a set of all perms to lookup which ones are missing
Creating missing datasource permissions.
INFO:superset.security.manager:Creating missing datasource permissions.
Creating missing database permissions.
INFO:superset.security.manager:Creating missing database permissions.
Creating missing metrics permissions
INFO:superset.security.manager:Creating missing metrics permissions
Cleaning faulty perms
INFO:superset.security.manager:Cleaning faulty perms
Is it because superset needs some permission to MySQL to make the changes in tables?

I don't know why the command 'superset db upgrade' didn't update my MYSQL-database, but I then manually created these fields and tables and now it seems to be working:
slices.schema_perm
tables.schema_perm
dbs.encrypted_extra
dbs.allow_cvas
dbs.server_cert
saved_query.extra_json
And these tables with fields
CREATE TABLE `alerts` (
`id` int(11) NOT NULL,
`label` varchar(250) DEFAULT NULL,
`active` tinyint(2) DEFAULT NULL,
`crontab` varchar(250) DEFAULT NULL,
`sql` varchar(250) DEFAULT NULL,
`alert_type` varchar(250) DEFAULT NULL,
`recipients` varchar(250) DEFAULT NULL,
`log_retention` varchar(250) DEFAULT NULL,
`grace_period` varchar(250) DEFAULT NULL,
`slice_id` int(11) DEFAULT NULL,
`dashboard_id` int(11) DEFAULT NULL,
`database_id` int(11) DEFAULT NULL,
`last_eval_dttm` varchar(250) DEFAULT NULL,
`last_state` varchar(250) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
ALTER TABLE `alerts`
ADD PRIMARY KEY (`id`);
CREATE TABLE `tab_state` (
`created_on` datetime DEFAULT NULL,
`changed_on` datetime DEFAULT NULL,
`extra_json` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`id` int(11) NOT NULL,
`user_id` int(11) NOT NULL,
`label` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`active` tinyint(1) DEFAULT NULL,
`database_id` int(11) NOT NULL,
`schema` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`sql` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`query_limit` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`latest_query_id` int(11) NOT NULL,
`autorun` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`template_params` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_by_fk` int(11) NOT NULL,
`changed_by_fk` int(11) NOT NULL,
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
I don't know the field types, so maybe someone can help with the correct types.

Related

Why do I have an empty “Searchable attributes” after import with 'algolia/scout-extended'?

In a Laravel 9 application, I have algolia/scout-extended 2.0.
Running on my local OS command
php artisan scout:optimize
I got several files like config/scout-search-pages.php, and I modified one of them:
'searchableAttributes' => [
'page_title',
'page_slug',
'page_content',
'page_content_shortly',
'page_author_name',
'page_author_email',
'page_price',
'page_categories',
'page_created_at',
],
'customRanking' => ['asc(page_price)', 'desc(page_created_at)'],
And clearing the cache I run an import and after that I see a set of rows/columns I expected: https://prnt.sc/Ew0BdaH3dvON
But on the “Configuration” tab I see an empty “Searchable attributes” list, and I have to add fields manually.
I expected that these fields must be filled automatically from “searchableAttributes” option,
but not.
After I added more columns at “Add a Searchable Attribute” I see “unordered/unordered” option for fields.
How does it work and which option do I have to select?
The source table on MySQL 8 side has this structure:
CREATE TABLE `search_pages` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`page_title` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`page_slug` varchar(260) COLLATE utf8mb4_unicode_ci NOT NULL,
`page_content` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`page_content_shortly` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`page_author_name` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`page_author_email` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`page_price` decimal(9,2) NOT NULL DEFAULT '0.00',
`page_categories` json DEFAULT NULL,
`page_created_at` timestamp NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `search_pages_page_slug_unique` (`page_slug`),
KEY `search_pages_page_author_name_page_title_index` (`page_author_name`,`page_title`)
) ENGINE=InnoDB AUTO_INCREMENT=13 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

Getting error on Postgresql TABLESPACE and CREATE commands

I am completely new in Postgres commands.
I have the following DB2 commands to create tables and table spaces:
CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTOMATIC STORAGE#
CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE#
CREATE SEQUENCE REVISION AS BIGINT START WITH 1 INCREMENT BY 1 MAXVALUE 4611686018427387903 CYCLE CACHE 1000#
Now i want to run these commands in postgresql , I have tried with
my_db=# CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTOMATIC STORAGE;
ERROR: syntax error at or near "TABLESPACE"
LINE 1: CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTO...
my_db=# CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE;
ERROR: syntax error at or near "MANAGED"
LINE 1: CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE;
I also have this DB2 CREATE TABLE statement:
CREATE TABLE USER (
ID BIGINT GENERATED BY DEFAULT AS IDENTITY (START WITH 1 INCREMENT BY 1 NO MAXVALUE NO CYCLE CACHE 100),
E_VER BIGINT NOT NULL,
NAME VARCHAR(38) NOT NULL UNIQUE,
EMAIL_ADDRESS VARCHAR(255) NOT NULL,
PASSWORD VARCHAR(32) NOT NULL,
SUPER_ADMIN SMALLINT NOT NULL,
MAIN_ADMIN SMALLINT NOT NULL,
SERVER_ADMIN SMALLINT NOT NULL,
GROUP_ADMIN SMALLINT NOT NULL,
CLIENT_ADMIN SMALLINT NOT NULL,
ENABLED SMALLINT NOT NULL,
HIDDEN SMALLINT NOT NULL,
PRIMARY KEY (ID)
)#
and I have tried to convert this to Postgres:
CREATE SEQUENCE USER_seq START WITH 1 INCREMENT BY 1;
CREATE TABLE USER (
ID BIGINT DEFAULT NEXTVAL ('USER_seq'),
E_VER BIGINT NOT NULL,
NAME VARCHAR(38) NOT NULL UNIQUE,
EMAIL_ADDRESS VARCHAR(255) NOT NULL,
PASSWORD VARCHAR(32) NOT NULL,
SUPER_ADMIN SMALLINT NOT NULL,
MAIN_ADMIN SMALLINT NOT NULL,
SERVER_ADMIN SMALLINT NOT NULL,
GROUP_ADMIN SMALLINT NOT NULL,
CLIENT_ADMIN SMALLINT NOT NULL,
ENABLED SMALLINT NOT NULL,
HIDDEN SMALLINT NOT NULL,
PRIMARY KEY (ID)
)#
by http://www.sqlines.com/online this online site. After running this command i am getting error like
my_db=# CREATE SEQUENCE USER_seq START WITH 1 INCREMENT BY 1;
CREATE SEQUENCE
my_db=#
my_db=#
my_db=# CREATE TABLE USER (
my_db(# ID BIGINT DEFAULT NEXTVAL ('USER_seq'),
my_db(# E_VER BIGINT NOT NULL,
my_db(# NAME VARCHAR(38) NOT NULL UNIQUE,
my_db(# EMAIL_ADDRESS VARCHAR(255) NOT NULL,
my_db(# PASSWORD VARCHAR(32) NOT NULL,
my_db(# SUPER_ADMIN SMALLINT NOT NULL,
my_db(# MAIN_ADMIN SMALLINT NOT NULL,
my_db(# SERVER_ADMIN SMALLINT NOT NULL,
my_db(# GROUP_ADMIN SMALLINT NOT NULL,
my_db(# CLIENT_ADMIN SMALLINT NOT NULL,
my_db(# ENABLED SMALLINT NOT NULL,
my_db(# HIDDEN SMALLINT NOT NULL,
my_db(# PRIMARY KEY (ID)
my_db(# );
ERROR: syntax error at or near "USER"
LINE 1: CREATE TABLE USER (
^
Anything wrong this conversion? Any suggestion solve this error?
USER is a reserved word, you need to escape it as CREATE TABLE "USER".
As for your CREATE USER and CREATE TABLESPACE commands, that's just wrong syntax. There's no MANAGED BY in Postgres for example.

Error while creating table in PostgreSQL while migrating from MySQL

I am Migrating my database from MySQL to PostgreSQL.While creating table I got an error which I can't resolve.My MySQL Query is like this.
MYSQL Query
CREATE TABLE `configuration` (
`Name` varchar(300) NOT NULL,
`Value` varchar(300) default NULL,
`CType` char(1) default NULL,
`Size` int(11) default NULL,
`CGroup` varchar(50) default NULL,
`RestartReq` char(1) NOT NULL default 'Y',
`Display` char(1) NOT NULL default 'Y',
PRIMARY KEY (`Name`),
KEY `CType` (`CType`),
CONSTRAINT `configuration_ibfk_1` FOREIGN KEY (`CType`) REFERENCES `conftype` (`CType`)
)ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_bin`
PostgreSQL Query
CREATE TABLE configuration (
Name varchar(300) PRIMARY KEY,
Value varchar(300) default NULL,
CType char(1) default NULL,
Size integer default NULL,
CGroup varchar(50) default NULL,
RestartReq char(1) NOT NULL default 'Y',
Display char(1) NOT NULL default 'Y',
KEY CType (CType),
CONSTRAINT `configuration_ibfk_1` FOREIGN KEY (CType) REFERENCES conftype (CType)
)
Running File with
psql -h localhost -p 5432 -U postgres -f ps.sql testdb
Error getting
psql:ps.sql:40: ERROR: syntax error at or near "(" at character 287
psql:ps.sql:40: LINE 9: KEY CType ('CType'),
From the MySQL documentation:
KEY is normally a synonym for INDEX.
In PostgreSQL you have to create the index separately from the table:
CREATE TABLE configuration (
name varchar(300) PRIMARY KEY,
value varchar(300),
ctype char(1),
size integer,
cgroup varchar(50),
restartreq boolean NOT NULL DEFAULT true,
display boolean NOT NULL DEFAULT true,
CONSTRAINT configuration_ibfk_1 FOREIGN KEY (ctype) REFERENCES conftype (ctype)
);
CREATE INDEX conf_key ON configuration(ctype);
A few other points:
PostgreSQL identifiers (mainly table and column names) are case-insensitive except when double-quoted. The standard approach is to put identifiers in lower case and keywords in upper case.
Using a varchar(300) as a PRIMARY KEY is usually not a good idea for performance reasons. Consider adding a serial type.
The default value of a column is NULL when nothing is specified, so no need to specify DEFAULT NULL.
PostgreSQL has a boolean data type.

MYSQL import: Cannot get geometry object from data you send to the GEOMETRY field

I have recently upgraded to MySQL 5.7 and was trying to run a replication from 5.6 master. However the replication fails with the following error:
Error 'Cannot get geometry object from data you send to the GEOMETRY field' on query.
Turns out it also happens when I try to import data from the mysqldump. Table structure is as follows:
CREATE TABLE `locations` (
`location_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`country_id` int(10) unsigned NOT NULL,
`name` varchar(100) CHARACTER SET utf8 NOT NULL,
`locations_type_id` int(11) unsigned NOT NULL,
`parent_id` int(11) unsigned DEFAULT NULL,
`importance` decimal(3,2) NOT NULL DEFAULT '1.00',
`lat` decimal(10,7) DEFAULT NULL,
`lng` decimal(10,7) DEFAULT NULL,
`radius` decimal(6,3) DEFAULT NULL,
`polygon` polygon DEFAULT NULL,
PRIMARY KEY (`location_id`),
KEY `name` (`name`,`locations_type_id`,`parent_id`,`lat`,`lng`),
KEY `locations_type_id` (`locations_type_id`),
KEY `name_2` (`name`(8)),
KEY `country_id` (`country_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
It appears to me that the import is trying to insert some binary data into the polygon field but in fairness I have no idea how to make it work.
Any ideas?
If you can re-run mysqldump, try to add --hex-blob option to have all binary data exported as hex dump.

MariaDB -> Error: 1062, Duplicate entry '0' for key 'PRIMARY'

So I'm trying to import geoipcity data into my table like so:
mysqlimport --fields-terminated-by="," --fields-optionally-enclosed-by="\"" --lines-terminated-by="\n" --host=localhost --user=user --password=passw database_name /var/www/html/GeoLiteCity_20150804/geoip_city.csv
But I keep getting the error.
Error: 1062, Duplicate entry '0' for key 'PRIMARY'
Now I saw the question relating to this error has been asked before but I simply don't understand the answers. I'm not that much of a guru, I'm a volunteer IT guy and I have no idea how to resolve this. I tried using this instead:
LOAD DATA LOCAL INFILE '/var/www/html/GeoLiteCity_20150804/geoip_city_ips.csv' INTO TABLE geoip_city_ips;
But then it would simply fill the table with "NULL" in all the columns.
My table structure:
--
-- Table structure for table geoip_city
CREATE TABLE IF NOT EXISTS geoip_city (
locID int(10) unsigned NOT NULL,
country char(8) COLLATE utf8_unicode_ci DEFAULT NULL,
region char(8) COLLATE utf8_unicode_ci DEFAULT NULL,
city varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
postalCode char(32) CHARACTER SET latin1 DEFAULT NULL,
latitude double DEFAULT NULL,
longitude double DEFAULT NULL,
dmaCode char(8) CHARACTER SET latin1 DEFAULT NULL,
areaCode char(8) CHARACTER SET latin1 DEFAULT NULL,
PRIMARY KEY (locID),
KEY Index_Country (country)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci ROW_FORMAT=FIXED;
Some lines from geoip_city:
717543,"MX","32","Zacatecas","98051",22.7833,-102.5833,,
717544,"MX","26","Cananea","84624",30.9500,-110.3000,,
717545,"MX","07","Valles","79040",26.6667,-100.6833,,
717546,"DE","02","Berg","88276",47.9667,11.3500,,
717547,"DE","09","Schwalbach","65824",49.3000,6.8167,,
717548,"RU","48","Moscow","129233",55.7522,37.6156,,
717549,"MX","28","Reynosa","88520",26.0833,-98.2833,,
717550,"PH","40","San Jose","5100",12.4558,121.0459,,
717551,"ES","56","Tarragona","43070",41.1167,1.2500,,
717552,"GB","Z6","","",51.9167,-0.6500,,
Well I'm guessing this is a MariaDB issue then since nobody replied? Would going back to Debian solve the issue?