I am using SphinxQL to query Sphinxsearch engine. I want to simulate the SPH_MATCH_ANY which is implemented in the php API like this :
$cl->SetMatchMode(SPH_MATCH_ANY);
$cl->Query("test query", "index");
=> search for docs matching with "test" OR "query"
So, I have written a function (php) to replace spaces and other special chars with pipes (|) in order to use it with SphinxQL :
function formatQuery($str) {
return trim(preg_replace('/[^-_\'a-z0-9]+/', '|', $str), ' |');
}
$str = "test query";
$sql = "SELECT * FROM index WHERE MATCH('" . addslashes(formatQuery($str)) . "')";
=> SELECT * FROM index WHERE MATCH('test|query');
The problem is, for some characters like - (minus), it can break the query, example :
$str = "i-phone is great";
$sql = "SELECT * FROM index WHERE MATCH('" . addslashes(formatQuery($str)) . "')";
=> SELECT * FROM index WHERE MATCH('i-phone|is|great')
=> ok
$str = "i - phone is great";
$sql = "SELECT * FROM index WHERE MATCH('" . addslashes(formatQuery($str)) . "')";
=> SELECT * FROM index WHERE MATCH('i|-|phone|is|great')
=> broken query because of "|-|"
Do you know a better way to make SphinxQL queries work in SPH_MATCH_ANY mode? or a better regexp to make it works for all cases?
I know I could use a more restrictive regexp like this:
preg_replace('/[^a-z0-9]+/', '|', $str)
but it would split strings like "i-phone is great" in 'i|phone|is|great' and I don't want that...
Thank you,
Nico
One way might be to use quorom
$sql = "SELECT * FROM index WHERE MATCH('\"" . addslashes($str) . "\"/1')";
you will need to add - to your charset_table tho, so it becomes part of a word.
The other option is
$query = preg_replace('/(\w+?)[-\'](\w+?)/','$1~$2',$query);
$query = preg_replace('/[^\w\~]+/','|',$query);
$query = preg_replace('/(\w+~\w[\w~]*)/e','"\"".str_replace("~"," ","$1")."\""',$query);
To turn it into a phrase.
Related
Long story short, I have an admin section where the user can choose from multiple dropdown lists the tables and fields that must be queries in order to get some values. Therefore, the query in ZEND is performed by concatenating the strings
$query = "SELECT $fieldName1, $fieldName2 from $tableName where $fieldName1 = $value";
How can I escape the above using ZEND approach to avoid sql injection? I tried adding them all as ? and calling quoteinto but it seems this does not work on some of the variables (like table names or field names)
ZF has quoteIdentifier() specifically for this purpose:
$query = "SELECT ".$db->quoteIdentifier($fieldName1).","...
In your case you might (also) want to check against a white list of valid column names.
Use quoteInto() or Zend_db_Select::where() for the values, and for the table and column names, I would simply strip any non alpha characters and then wrap them in ` quotes prior to using them in your SQL.
Example:
// Strip non alpha and quote
$fieldName1 = '`' . preg_replace('/[^A-Za-z]/', '', $fieldName1) . '`';
$tableName = '`' . preg_replace('/[^A-Za-z]/', '', $tableName) . '`';
// ....
// Build the SQL using Zend Db Select
$db->select()->from($tableName, array($fieldName1, $fieldName2))
->where($fieldName1 . ' = ?', $value);
In SafeMysql you can make it as simple, as
$sql = "SELECT ?n, ?n from ?n where ?n = ?s";
$data = $db->getAll($sql,$fieldName1,$fieldName2, $tableName, $fieldName1, $value);
though I understand that you won't change your ZF to SafeMysql.
Nevertheless, there is one essential thing that is ought to be done manually:
I doubt you want to let users to browse users table or financial table or whatever.
So, you have to verify a passed table name against an allowed tables array.
like
$allowed = ('test1','test2');
if (!in_array($tableName, $allowed)) {
throw new _403();
}
In a Zend_Db_Select object, I am doing a join to get user information for some data records. For the join is on a userId, I would like to combine the user first and last name into a single column of name.
Basically I am looking to have something similar to this:
$table = array('u' => 'User');
$condition = 'u.id = t.id';
$columns = array('UserName' => 'u.FirstName + " " + u.LastName')
$select->joinLeft($table, $condition, $columns);
I have tried using a Zend_Db_Expr with no luck and the above does not work.
How would I go about doing this?
Zend_Db_Expr is the way to go, but you'll have better luck if you use the database's concatenation function. Assuming MySQL:
$columns = new Zend_Db_Expr("CONCAT(u.FirstName, ' ', u.LastName') AS name")
Is there any easier way to add char to the beginning and end of array elements in perl.
Ex:my $str ='table1,table2,table3'
my #arry = split (/,/,$str)
I like the output should look like
'table1','table2','table3'
So it can be used in sql query
Thanks.
join(',', map "'$_'", #arry)
is what you are asking for.
But it's much better to use placeholders:
my $str = 'table1,table2,table3';
my #arry = split(/,/, $str);
if (#arry) {
my $query = 'select * from tablename where colname in (' . join(',',('?') x #arry) . ')';
my $sth = $dbh->prepare($query);
$sth->execute(#arry);
...
}
If you are going to use DBI (https://metacpan.org/module/DBI) to work with your database, it can do it for you if you use placeholders in your queries, so that you don't have to quote values.
For example, if you want to insert something into a table:
$dbh->do("INSERT INTO table VALUES(?, ?, ?)", undef, #arry)
or die $dbh->errstr;
where #arry is an array containing exactly 3 values to be inserted into a table.
More on this can be found here https://metacpan.org/module/DBI#Placeholders-and-Bind-Values
I'm trying to use fetchAll on a query that has 2 variables. I can't figure out the syntax.
I can manage with only 1 variable:
$sql = "SELECT * FROM mytable WHERE field1 = ?";
$this->_db->fetchAll($sql,$value1); # that works
However I'm having some issues when query has multiple variables
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$this->_db->fetchAll($sql,$value1,$value2); # doesn't work
$this->_db->fetchAll($sql,array("field1"=>$value1,"field2"=>$value2)); # doesn't work either
The reason why I want to use ? instead of placing the variables directly into the query is that I've learned that using ? allows for the query to be compiled generically by the db engine and improves performances.
There are two types of parameter, named parameters and positional parameters. You're mixing the two types and that won't work.
Named parameters match a placeholder by name. Names are started with the : symbol. The parameter names are not the same as the names of the columns you happen to use them for. You supply parameter values in an associative array, using the parameter name (not the column name) as the array keys. For example:
$sql = "SELECT * FROM mytable WHERE field1 = :param1 AND field2 = :param2";
$this->_db->fetchAll($sql,array("param1"=>$value1,"param2"=>$value2));
Positional parameters use the ? symbol for the placeholder. You supply parameter values using a simple (non-associative) array, and the order of values in the array must match the order of parameter placeholders in your query. For example:
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$this->_db->fetchAll($sql,array($value1,$value2));
Most brands of SQL database natively support only one style or the other, but PDO attempts to support both, by rewriting the SQL if necessary before preparing the query. Since Zend_Db is modeled after PDO, Zend_Db also supports both parameter styles.
This question is a bit old, but I thought I'd just add to it for reference sake.
I would recommend starting to use Zend_Db_Select with Zend_Db. I've been doing a lot with Zend_Db lately. More from Zend_Db_Select reference guide.
Lets assume you have a Zend_Db adapter: $this->_db
# this will get the Zend_Db_Select object
$select = $this->_db->select();
# now you build up your query with Zend_Db_Select functions
$select->from('mytable');
$select->where('field1 = ?', $field1);
$select->where('field2 = ?', $field2);
[...]
# echo to see the SQL (helps in debugging)
# SELECT * FROM mytable WHERE field1 = ? AND field2 = ? [...]
echo '<p>My SQL: ' . $select . '</p>';
# Execute the SQL / Fetch results
$results = $select->query()->fetchAll();
That's the basics from your given example, but the Zend Framework reference guide on the select object has a lot of good information on how to build even more complex queries with JOINS, UNIONS, GROUP BY, LIMIT, HAVING, etc.
If you wanted to use an alias name for a table or parameters, you use an associative array with the alias name being the index value:
# SELECT p.* FROM products AS p
$select->from('p' => 'products');
If you want to return only selected fields, you add an array of field names as a second parameter:
# SELECT model FROM products
$select->from(products, array(model));
Actually, the above could should produce fully qualified SQL as:
SELECT 'products'.model FROM 'products'
but I wrote the above for brevity and clarity in the example.
One thing I just came across is using AND and OR in the WHERE condition.
# WHERE a = $a
$select->where('a = ?', $a);
# WHERE a = $a AND b = $b
$select->where('a = ?', $a);
$select->where('b = ?', $b);
# WHERE a = $a OR b = $b
$select->where('a = ?', $a);
$select->orWhere('b = ?', $b);
# WHERE a = $a AND b = $b
$select->orWhere('a = ?', $a);
$select->where('b = ?', $b);
Notice, that whatever the following "where" function you use, will combine with the previous statement as that operand. Ok, that sounded confusing.
If the second "where" is an "OR" it will be an "OR" conditional. If the second "where" is a "AND" the statement will be "AND".
In other words, the first WHERE function is ignored in terms of what condition it will use.
In fact, I just asked a question on Stack Overflow yesterday regarding doing a complex WHERE using select.
Hope that helps!
Cheers!
Try this:
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$statement = $this->_db->query($sql,array("field1"=>$value1,"field2"=>$value2));
$data = $statement->fetchAll();
$this->_db must be an instance of Db adapter.
Heres the actual Zend way to code for this.
$sql = "SELECT * FROM mytable WHERE field1 = :param1 AND field2 = :param2";
$this->_db->fetchAll($sql,array("param1"=>$value1,"param2"=>$value2));
$where = $this->_db->select()
->from('mytable')
->where('field1 = ?',$value1)
->where('field2 = ?',$value2);
$rowSet = $this->_db->fetchAll($where);
This works great for me
I'm writing a subroutine for DBI updates, and are having some trouble figuring out how to add placeholder and stuff...
I have this:
sub row_update {
my $table = shift;
my %updates = #_;
my $placeholders = ...
$dbh->do("UPDATE $table SET (foo) WHERE (bar)") etc...
}
Any ideas?
I just want a simple update function where I can send in x number of arguments (as a hash).
Thanks in advance!
Something like this might be Good Enough:
sub update {
my ($dbh, $args) = #_;
my $table = $args->{table} || die 'need table';
my $updates = $args->{updates} || die 'need updates';
my #cols = keys %$updates;
my $query = 'UPDATE $table SET '.
(join ', ', map { '$_ = ?' } #cols)
($args->{where} ? ' WHERE '. $args->{where} : '');
my $sth = $dbh->prepare($query);
$sth->execute(map { $updates->{$_} } #cols);
return $sth;
}
Use it like:
my $sth = update $dbh, {
table => 'foo',
updates => {
col1 => 'new_value',
col2 => 'another_value',
},
where => 'id=42',
};
Really, though, you want to look into using an ORM like
DBIx::Class. It will do a much better job of building queries than
string manipulation like this will.
(Rewriting the where clause to be parameterized is left as an exercise to the reader. You also need to quote the update keys and table name. See why people use ORMs?)
Edit: Thinking about this a bit more, you might like DBIx::Simple combined with SQL::Abstract. This will take less configuration effort than an ORM, but still give you many of the benefits.
If I understand the question correctly, it sounds like you're after SQL::Abstract. First, we create an SQL::Abstract object:
use SQL::Abstract;
my $sql = SQL::Abstract->new;
Now, as an example, we'll use it to insert some data into a table:
my %record = (
FirstName => 'Buffy',
LastName => 'Summers',
Address => '1630 Revello Drive',
City => 'Sunnydale',
State => 'California',
Occupation => 'Student',
Health => 'Alive',
);
my ($stmt, #bind) = $sql->insert(’staff’,\%record);
This results in:
$stmt = "INSERT INTO staff
(FirstName, LastName, Address, City,
State, Occupation, Health)
VALUES (?, ?, ?, ?, ?, ?, ?)";
#bind = ('Buffy','Summers','1630 Revello Drive',
'Sunnydale',’California','Student','Alive');
The nice thing about this is we can pass it directly to DBI:
$dbh->do($stmt, undef, #bind);
Of course, you want to be updating records, not just inserting them. Luckily, this is also quite easy:
my $table = 'People';
my %new_fields = (
Occupation => 'Slayer',
Health => 'Dead',
);
my %where = (
FirstName => 'Buffy',
LastName => 'Summers',
);
my ($stmt, #bind) = $sql->update($table, \%new_fields, \%where);
$dbh->do($stmt, undef, #bind);
This produces:
$stmt = 'UPDATE People SET Health = ?, Occupation = ?
WHERE ( FirstName = ? AND LastName = ? )';
#bind = ('Dead', 'Slayer', 'Buffy', 'Summers');
If you're after more information about SQL::Abstract, I recommend you look at its CPAN page. There's also a chapter in Perl Training Australia's Database Programming with Perl manual, which are freely available from our course notes page.
All the best,
Paul
Disclaimer: I'm managing director of Perl Training Australia, and therefore think that our course notes are pretty good.
Others suggested the usual "build a query with the right number of '?'s" approach.
For most queries like this the DBI->quote method is forgotten, but it can keep the code simpler and in most cases it's not any slower than the "proper" placeholder approach.
1) Use DBI->quote instead of place holders to build the query. For example for a simple select:
my $sql = "select foo from bar where baz in ("
. join(",", map { DBI->quote($_) } #bazs)
. ")";
my $data = $dbh->selectall_arrayref($sql);
2) As jrockway suggested - use an ORM to do this sort of low level stuff for you. DBIx::Class or Rose::DB::Object, for example.
The NUM_OF_PARAMS attribute may be of assistance:
"NUM_OF_PARAMS" (integer, read-only)
The number of parameters (placeholders) in the prepared
statement.
So for instance, I've got a script to run arbitrary SQL from the command line that uses this code:
my $s = $h->prepare($_);
for my $i (1..$s->{NUM_OF_PARAMS}){
my $param = shift #ARGV;
$s->bind_param($i, $param);
print LOG "Bind param $i using $param.\n"
or die "can't append to $opt{log}: $!";
}
$s->execute();
I can't say I've used the other modules that have been suggested, so they may do a better job.