Special characters when creating a table via jdbc - pyspark

I am trying to export a table from cluster to oracle using jdbs:
df.write.mode('append').option("createTableColumnTypes", "S_ID INT, BREAK_$ FLOAT").jdbc(jdbc, ORACLE_TABLENAME, properties = properties)
I am getting the following error:
As I understand it, this is due to the fact that the $ symbol is a special character. How to solve this problem if you can not refuse such a column name? Tried to use 'BREAK_$', {BREAK_$} but it didn't work

Related

Why single quote escape cannot be used in QuestDB, Error: dangling expression

I'm trying to use Query Variables in Grafana, the panel query source is PostgreSQL for QuestDB.
I have added the variable without any issue, but I'm unable to use the variable in Panel query since the variable values contains the spaces (SENSOR01 ON_OFF), also I'm unable to figure-out how to add single quote escape.
Following are the scenarios I tried:
Scenario1: this indicates due to space in the Variable value, on_off considered as separate word
where sensor_name = $sensor
db query error: pq: unexpected token: on_off
.
.
Scenario2: tried to add single quotes explicitly for the variable value, but there is generic error from source DB (QuestDB)
where sensor_name = concat('''', $sensor, '''')
db query error: pq: dangling expression
When tried Scenario2 approach directly in query of Variable, getting the same error
..
Scenario3: Hard-coded the variable value with space and with single quotes, but this giving me error with first part of the variable, looks like the hard-coded single quotes not passed here!
Error (Scenario3):
Is there any way/workaround to tackle this issue?
Could you just add the quotes directly in the query?
where sensor_name = '$sensor'
I have a similar grafana panel querying a questDB database using a variable and it works for me. This is my query:
select device_type, avg(duration_ms) as avg_duration_ms, avg(speed) as avg_speed, avg(measure1) as avg_m1, avg(measure2) as avg_m2 from ilp_test
WHERE
$__timeFilter(timestamp) and device_type = '$deviceType'
A rather hacky workaround would be to do:
where sensor_name = concat(cast(cast('&' as int) + 1 as char), $sensor, cast(cast('&' as int) + 1 as char))
This should work, but I'm pretty sure there is a better solution. Let me find it and get back to you.
Update. We may support Postgres syntax (which is '' escaping for a single quote char) in one of upcoming versions. For now, you'd have to use the above workaround.

Cannot create user at runtime with Query with parameters

I am using C++ Builder 10.2.3 (Rad Studio Tokyo 10.2.3) with Interbase 2017
I need to create users at runtime for my users registration.
If I create the Query at runtime, in that case there is no parameter, it works. But this creates problems with MBCS characters I will explain later.
If I create the Query at design-time with parameters and try to set the parameters at runtime. I am getting the error message below:
[Application: ]
[Error] -104 335544569 Dynamic SQL Error
SQL error code = -104
Token unknown - line 2, char 14
?
The query I am using is below:
CREATE USER myuser
SET PASSWORD :mypass,
FIRST NAME :myfirstname,
LAST NAME :myname;
I replace the first line of the Query at runtime, so there is no character. And after all, Interbase cannot handle MBCS characters in USERNAME.
I need to use a Query with parameters because my application handles multi-bytes characters (MBCS), like Chinese and Japanese. And this is the only option to be sure of a proper conversion to UTF8 in Interbase. Because if the conversion of MBCS characters is not done, I cannot backup and restore my database. When I try to restore with MBCS characters in First and last name, I am getting an error message that Interbase cannot transliterate between character sets.
Base on the error message, it appears to me that it does not recognize the Query parameters.
I tried with both "TIBQuery" and "TIBSQL". Same issue. Impossible to use also Store procedures. Does not recognize the create word.
So, how to fix that ?

operator does not exist: # timestamp without time zone

In a parameterized query issued from c# code to PostgreSQL 10.14 via dotConnect 7.7.832 .NET connector, I select either a parameter value or the local timestamp, if the parameter is NULL:
using (var cmd = new PgSqlCommand("select COALESCE(#eventTime, LOCALTIMESTAMP)", connection)
When executed, this statement throws the error in subject. If I comment out the corresponding parameter
cmd.Parameters.Add("#eventTime", PgSqlType.TimeStamp).Value = DateTime.Now;
and hardcode
using (var cmd = new PgSqlCommand("select COALESCE('11/6/2020 2:36:58 PM', LOCALTIMESTAMP)", connection)
or if I cast the parameter
using (var cmd = new PgSqlCommand("select COALESCE(cast(#eventTime as timestamp without time zone), LOCALTIMESTAMP)", connection)
then it works. Can anyone explain what # operator in the error is referring to and why the error?
In the case that doesn't work, your .Net connection library seems to be passing an SQL command containing a literal # to the database, rather than substituting it. The database assumes you are trying to use # as a user defined operator, as it doesn't know what else it could possibly be. But no such operator has been defined.
Why is it doing that? I have no idea. That is a question about your .Net connection library, not about PostgreSQL itself, so you might want to add tag.
The error message you get from the database should include the text of the query it received (as opposed to the text you think it was sent) and it is often useful to see that in situations like this. If that text is not present in the client's error message (some connection libraries do not faithfully pass this info along) you should be able to pull it directly from the PostgreSQL server's log file.

Grafana (V7) adding variable in table name

I need to be able to use variables in table names - I basically have the same set of tables used for different types of data, so I would like to just have one dashboard and swapping between all types instead of always having to set up multiple identical dashboards.
My query is something like:
select * from table_$variable_name;
Where my list of possible variable is something like cat, dog, bird
I can seem to make this work, if I only put the variable as shown above I get the following error
Error 1146: Table 'table_$variable_name' doesn't exist
If I enclose it in curly brackets, I get this error instead.
Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '{bird}' at line 1
(i.e. with the selected variable actually being visible this time)
I'm not sure if the issue is having underscores in the table names, I tried putting underscores around my variables too to check and I had no luck with that.
Another thing I tried was gradually adding on to the table name, so e.g.
select * from table_$variable;
Still returns an error, but I can see the table name starting to form correctly
Error 1146: Table 'table_bird_' doesn't exist
However, as soon as I add another underscore, the variable is not picked up abymore
```Error 1146: Table 'table_$variable_' doesn't exist``
I'm sure it's something silly I am missing in the syntax of the query - anyone has any suggestions?
Using this https://grafana.com/docs/grafana/latest/variables/templates-and-variables/ for reference
As #arturomp suggests, use
${var:raw}
At least in my case, this was the solution that worked.
I found double square brackets work. e.g.
Rather than
select * from table_$variable_name;
use
select * from table_[[variable_name]];

pgAdmin argument formats can't be mixed

Background
Ubuntu 18.04
Postgresql 11.2 in Docker
pgAdmin4 3.5
Have a column named alias with type character varying[](64). Values have already been set on some rows before using psycopg2. Everything was alright then.
SQL = 'UPDATE public."mytable" SET alias=%s WHERE id=%s'
query = cursor.mogrify(SQL, ([values] , id))
cursor.execute(query)
conn.commit()
Recently, when I want to add more value using pgAdmin GUI as shown in the first figure, the error in the second figure happens, which says Argument formats can't be mixed:
Well, it turns out if insert the values using script such as psql or query tool in pgAdmin, the error does not happen, i.e., it only happens if using GUI of pgAdmin.
Example script:
UPDATE public."mytable" SET alias='{a, b}' WHERE id='myid'
But as the GUI is much easier to modify values, so really want to figure it out. Any idea?
It's a bug in pgAdmin 4.17.
It looks like it happens whenever you edit a char(n)[] or varchar(n)[] cell in a table (although char[] and varchar[] are unaffected).
It should be fixed in 4.18.
In the meantime, you can fix it yourself without much trouble. The pgAdmin4 backend is written in Python, so there's no need to rebuild anything; you can just dive in and change the source.
Find the directory where pgAdmin4 is installed, and open web/pgadmin/tools/sqleditor/__init__.py in an editor. Find the line:
typname = '%s(%s)[]'.format(
...and change it to:
typname = '{}({})[]'.format(
You'll need to restart the pgAdmin4 service for the change to take effect.
I wasn't able to get this working with the Character Varying data type but it worked once I converted the column data type to Text.