I have encountered an issue were it will not allow me to insert a string using update and returns. I'm running 6.5 (includes Apache Spark 2.4.5, Scala 2.11), but it is not working on 6.4 runtime as well.
I have a delta table with the following columns, partitioned by the created date
ID string
, addressLineOne string
, addressLineTwo string
, addressLineThree string
, addressLineFour string
, matchName string
, createdDate
And I'm running a process that hits an API and updates the matchName column.
Using Pyspark if it do this, just to test writing
deltaTable.update(col("ID") == "ABC123", {"matchName ": "example text"})
I get the following error:
Py4JJavaError: An error occurred while calling o1285.update.
: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object, tree: 'example
If I try this, change the string to 123, it updates without an issue
deltaTable.update(col("ID") == "ABC123", {"matchName ": "123"})
Yet if I use sql and do
UPDATE myTable SET matchName = "Some text" WHERE ID = "ABC123"
It inserts fine. I've searched and can't see a similar issue, Any suggestions? Have I missed something obvious?
Looks like you have an extra space after matchName in your python code
Related
Database used: postgreSQL
I have the following code:
#Query(value="select id, meta " + "from temp " +
"where meta #> \'{\"animal\": \"donkey\" }\'", nativeQuery = true)
List<classDemo> findByMeta();
The meta column has data in the following format:
{
"meta": { "animal":"dog", "type":"dirty"}
}
I would like to provide the value of animal as a parameter (e.g. donkey should be a parameter), so that I can extract all the records of the entered value during a get request. What can I do to the above code so that I can enter a parameter rather than a direct value?
Above code is jsonb format, present in meta column
If the meta column is json or jsonb, then you can use this for your query:
select id, meta
from temp
where meta->'meta'->>'animal' = 'donkey'
If meta is data type text, then this will work so long as all rows in the table have valid json stored in meta:
select id, meta
from temp
where (meta::jsonb)->'meta'->>'animal' = 'donkey'
It has been a long while since I have worked in Java and even longer when it comes to Spring, but you should be able to placeholder 'donkey' with a question mark and then setString(1, "donkey") on it.
I have the following SQL query using Kotlin Exposed to a Postgres server with a capitalized table name:
object Table: IntIdTable("Table") {
val tC = text("Text")
val vC = text("Value")
}
Database.connect("jdbc:postgresql://...", driver = "org.postgresql.Driver")
transaction {
logger.addLogger(StdOutSqlLogger)
val query = Table.select {
Table.id eq 5
}
query.forEach {
println( it[Table.tC] )
}
}
But I am getting back:
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: relation "table" does not exist
Usually I would simply be able to quote the table name "Table" to use the capitalized table names, but can't seem to do that with Kotlin Exposed; so is there a way to use the capitalized table name by preventing it from being lowercased?
I was able to resolve this by using escaped quotes within the table string, example for the above question would be as follows:
object Table : IntIdTable("\"Table\"") {
Could you provide the whole sample and point to the place where the exception is thrown? From the current code that's unclear who and how trying to create relation to the table.
I'm powering a search bar via AJAX that passes a selected filter (radio button) that relates to a database column and a search string for whatever is entered in the search bar. The scala/play/anorm code I am using is this:
def searchDB(searchString: String, filter: String): List[DatabaseResult] = {
DB.withConnection { implicit c =>
SQL(
"""
SELECT name, email, emailsecondary, picture, linkedin, title, company, companylink, companydesc, location, github, stackoverflow, twitter, blog
FROM mailinglistperson
WHERE {filter} LIKE '%{searchString}%'
""").on(
'filter -> filter,
'searchString -> searchString
).as(databaseResultParser.*)
}
}
When I run a query on the database (PostgreSQL) using psql that is isomorphic to the above anorm code, it returns 2 results, i.e.:
select id, name, email from mailinglistperson where company like '%kixer%';
But the anorm code returns 0 results when passed the exact same values (I've verified the values via println's)
EDIT: When I switch the anorm code to use String Interpolation I get:
[error] - play.core.server.netty.PlayDefaultUpstreamHandler - Cannot invoke the action
java.lang.RuntimeException: No parameter value for placeholder: 3
EDIT2: I also tried passing the '%...%' along with searchString into LIKE and still got 0 results.
There are two issues - the name of the column, and the filter value
As for the filter value: You have to omit the single ticks in the SQL command, and you should pass the placeholder "%" in the argument. The ticks are handled automatically in case of a string.
As for the column name: It's like a string parameter, so again ticks are handled automatically as well:
[debug] c.j.b.PreparedStatementHandle - select ... from ... where 'filter' like '%aaa%'
One solution: Use normal string interpolation s"""... $filter ...""".
All together:
SQL(
s"""
SELECT name, email, ...
FROM mailinglistperson
WHERE $filter LIKE {searchString}
""").on(
'searchString -> "%" + searchString + "%"
).as(databaseResultParser.*)
but that should be accompanied by a check before, something like
val validColumns = List("name", "email")
if (validColumns.contains(filter)) == false) {
throw new IllegalArgumentException("...")
}
to guard against SQL injection.
Update
As pointed out by cchantep: If Anorm >= 2.4 is used, one can use mixed interpolation (both for column names and values):
SQL"... WHERE #$filter LIKE $searchString"
In this case it's partially safe against SQL injection: that only covers the values, and not the column name.
Update 2
As for logging the SQL statements, see Where to see the logged sql statements in play2?
But as you are using PostgreSQL I suggest the definitive source: The PostgreSQL log: In postgresql.conf:
log_statement = 'all' # none, ddl, mod, all
then you will see in the PostgreSQL log file something like this:
LOG: select * from test50 where name like $1
DETAIL: Parameter: $1 = '%aaa'
I created a table with following syntax:
create table poll(poll_id string primary key,
poll_type_id integer,
poll_rating array(object as (rating_id integer,fk_user_id string, israted_image1 integer, israted_image2 integer, updatedDate timestamp, createdDate timestamp )),
poll_question string,
poll_image1 string,
poll_image2 string
)
And I inserted a record without "poll_rating" field which is actually an array of objects fields.
Now when I try to update a poll_rating with the following commands:
update poll set poll_rating = [{"rating_id":1,"fk_user_id":-1,"israted_image1":1,"israted_image2":0,"createddate":1400067339.0496}] where poll_id = "f748771d7c2e4616b1865f37b7913707";
I'm getting an error message like this:
"SQLParseException[line 1:31: no viable alternative at input '[']; nested: ParsingException[line 1:31: no viable alternative at input '[']; nested: NoViableAltException;"
Can anyone tell me why I get this error when I try to update the array of objects fields.
Defining arrays and objects directly in SQL statement is currently not supported by our SQL parser, please use parameter substitution using placeholders instead as described here:
https://crate.io/docs/current/sql/rest.html
Example using curl is as below:
curl -sSXPOST '127.0.0.1:4200/_sql?pretty' -d#- <<- EOF
{"stmt": "update poll set poll_rating = ? where poll_id = ?",
"args": [ [{"rating_id":1,"fk_user_id":-1,"israted_image1":1,"israted_image2":0,"createddate":1400067339.0496}], "f748771d7c2e4616b1865f37b7913707" ]
}
EOF
I'm trying to do an extremely simple active record query :
#row = Blazer.where(username: 'test').first
PostgreSQL generation :
Blazer.where(username: 'test').to_sql
outputs :
SELECT "blazer".* FROM "blazer" WHERE "username"."value" = 'test'
which causes an error :
ERROR -- : PG::UndefinedTable: ERROR: missing FROM-clause entry for table "username"
I expected the following PostgreSQL result :
SELECT "blazer".* FROM "blazer" WHERE "username" = 'test'
How can I fix this?
I'm using active record with Sinatra and the app runs on Heroku.
My Blazer class is the following :
class Blazer < ActiveRecord::Base
# phone:string username:string location:string
end
Maybe this is not the correct answer, but I was getting the same error.
I was looking for a Venue with a certain name
venue_name = params.require(:season).permit([:venue_name])
venue = Venue.find_by_name(venue_name)
I also tried with
venue = Venue.where(name: venue_name).first
Both failed saying because it was trying this query
SELECT "venues".* FROM "venues" WHERE "name"."venue_name" = 'venue name' LIMIT 1)
When I was trying this in the console everything looked OK. But when I checked what I was looking for was quite obvious
params.require(:season).permit([:venue_name])
Returns a Hash with only venue_name as key, so instead of looking for a key i was looking for a Hash.
I switched to params.require(:season).permit([:venue_name])[:venue_name] and the issue was fixed.