I am using whoosh to index and search throught my documents. I developped a multi=field search, but I want to specify some "MUST" fields.
What I want is: I when I am searching for a book with a query q1, it search on title and summary, but I want to specify some filters like autor= 'name of author' and category= "books category".
The results must take into account the two 'MUST' field and search on the two others.
Thank you for your help
Searcher's search() has parameter filter:
https://whoosh.readthedocs.io/en/latest/searching.html#filtering-results
For filtering you can use a query like
'author:"Name of Author" AND category:Genre'
To build filtering queries automatically, let's assume we have sets authors and categories:
from whoosh.query import *
from whoosh.qparser import MultifieldParser
if categories:
# Filter the search by selected categories:
if len(categories) > 1:
# Must be: Category1 OR Category2 OR Category3...
cat_q = Or([Term('category', x) for x in categories])
else:
# If there's just 1 category in the set:
cat_q = Term('category', next(iter(categories)))
print('Query to filter categories:', cat_q)
if authors:
# Filter the search by authors:
if len(authors) > 1:
# Must be: Author1 OR Author2 OR Author3...
aut_q = Or([Term('author', x) for x in authors])
else:
# If there's just 1 author in the set:
aut_q = Term('author', next(iter(authors)))
print('Query to filter authors:', au_q)
# Now combine the two filters:
if categories:
# Both fields are used for filtering:
final_filter = And([cat_q, aut_q])
else:
# Only authors:
final_filter = aut_q
elif categories:
# Only categories:
final_filter = cat_q
else:
# None:
final_filter = None
print('final_filter:', final_filter)
# Now parse the user query for 2 fields:
parser = MultifieldParser(["title", "summary"], ix.schema)
query = parser.parse(q1)
if final_filter is None:
results = s.search(query)
else:
# Using the filter:
results = s.search(query, filter=final_filter)
You can use whooshMultifieldParser for this scenario
from whoosh.qparser import MultifieldParser
fields = ["title", "summary", "author", "category"]
query = MultifieldParser(fields, schema=idx.schema, group=qparser.OrGroup).parse(q1)
with idx.searcher() as searcher:
results = searcher.search(query, limit=limit)
...........
Above using Or Group which will search on all fields with or operator. According to your need you can customize them . more on operators here like and not etc.
Related
i have bigint column named as face in mysql. and this is my sphinx.conf
source src1
{
type = mysql
sql_host = localhost
sql_user = root
sql_pass = pass
sql_db = nums
sql_port = 3306 # optional, default is 3306
sql_query = SELECT id,id AS id_attr,tel,name,sex,face from tel
sql_attr_uint = id_attr
sql_attr_bigint = face
}
index num
{
rt_attr_bigint = face
rt_field = face
source = src1
path = C:/sphinx/bin/data/numaralar
}
i can make search by name and tel but not with face.
Fatal error: Uncaught exception 'Foolz\SphinxQL\Exception\DatabaseException' with message '[1064] index nums: query error: no field 'face' found in schema [ SELECT * FROM nums WHERE MATCH('(#face 123456)') LIMIT 0, 10 OPTION max_matches = 5000;SHOW META]' in ..
why may it be?
You are trying to use the value as an field. The # fulltext operator (and indeed the whole of MATCH() full text query, operates on fields ONLY.
You've instead defined face as an atribute. Attributes don't work in full-text queries.
Can
Make face a field instead (remove the sql_attr_bigint) or make it both an attribute and field. (to do that, would have to duplicate it like you've duplicated the id, one for field, one for attribute. or use sql_field_string, but that makes a string attribute)
or
Use filter by the attribute instead. Dont really know how to do that in Foolz. But the SphinxQL query would be something like
SELECT * FROM nums WHERE `face` = 123456 LIMIT 0, 10
Trying to experiment with Ecto embeds_many, and work great, until I need to query on some data in the embedded field.
So I have something like a product that embeds_many categories
schema "products" do
field :code, :string, null: false
embeds_many :categories, Category,
on_replace: :delete,
primary_key: {:id, :binary_id, autogenerate: false}
do
field :name, :string
end
end
def create_changeset(%Product{} = product, attrs) do
product
|> cast(attrs, [:code])
|> cast_embed(:categories, with: &attributes_changeset/2)
end
def attributes_changeset(%{} = product_attribute, attrs) do
product_attribute
|> cast(attrs, [:id, :name])
end
After creating products I end up with something like this in the postgres table
id code categories
1 11 {"{\"id\": \"dress\", \"name\": \"Dress\"},
"{\"id\": \"shirt\", \"name\": \"Shirt\"}}
2 22 {"{\"id\": \"dress\", \"name\": \"Dress\"}}
So now I want to query all products where id == "dress", and of course I would like to get the 2 results above.
I have experimented with something like this:
q = from p in Product, where: fragment("? #> ?", p.categories, '{"id": "dress"}') but transforms the array in integers: operator does not exist: jsonb[] #> integer[] ... WHERE (p0."categories" #> ARRAY[123,34,105,100,34,58,32,34,100,114,101,115,115,34,125])
or that:
q = from p in Product, where: fragment("? #> ?", p.categories, "[{\"id\": \"dress\"}]"), getting malformed array literal: "[{"id": "dress"}]"
What I hoped was something like:
q = from p in Product, where: fragment("? -> 'id' = ?", p.categories, "rochie") but not sure at all if that will work.
Since categories is a jsonb[] here, not a plain json, the operator #> won't work with it directly. You can use ANY and <# to do what you want:
where: fragment("? <# ANY(?)", ~s|{"id": "dress"}|, p.categories)
This will run '{"id": "dress"}' <# ? for each category in the categories array and return true if any of them match.
(~s|"a"| is just a cleaner way of writing "\"a\"".)
Below for each loop takes more time and i cant able to trace index usage using XREF as the table uses Oracle Schema.Please Help me.
Need to generate report for different Report Type ( Report type is the input parameter in my code ).Single report type may contain more than 50,000 records how to access all the record within minute.Index detail also mentioned below for each loop.
FOR EACH Report
FIELDS(EXTRACTDATE STATUS MailingType ReportType ReportNumber
RequestID CustID)
WHERE Report.EXTRACTDATE < Today
AND Report.ReportType = 'Customer Report'
AND Report.STATUS = 'Pending'
AND (Report.MailingType = "LETTER"
OR Report.MailingType = "Mail") NO-LOCK:
< Statements >
.
.
END.
**Index Detail**
CREATE INDEX "TYPE_IDX1" ON "Report" ("EXTRACTDATE" DESC, "ReportTYPE", "STATUS", "MailingTYPE", "PROGRESS_RECID")
CREATE INDEX "REQ_IDX1" ON "Report" ("REQUESTID", "PROGRESS_RECID")
CREATE INDEX "RTTYP_IDX1" ON "Report" ("ReportTYPE","PROGRESS_RECID")
The "OR" at the end will slow things down considerably - the AVM does better if you split it up into two sets of AND statements and OR the result, like so:
WHERE (Report.EXTRACTDATE < Today
AND Report.ReportType = 'Customer Report'
AND Report.STATUS = 'Pending'
AND Report.MailingType = "LETTER")
OR
(Report.EXTRACTDATE < Today
AND Report.ReportType = 'Customer Report'
AND Report.STATUS = 'Pending'
AND Report.MailingType = "Mail")
How can I re-write the following Formula from Crystal Reports in SSRS
select {spAR6100b_allcompanies;1.artyp}
case 'BCS','CIG','HUM','INA','INS','MNG','PHC','SEC','SPC','TRI','UHC','WMD' : 'Primary'
case 'COI','ICO','INP','MCO','COI' : 'Secondary'
default: ''
SSRS expressions generally don't work very well for IN comparisons.
There are a few options.
Use a Switch and a set of linked Or statements:
=Switch(Fields!arTyp.Value = "BCS" or Fields!arTyp.Value = "BCS" or Fields!arTyp.Value = "HUM", "Primary"
, Fields!arTyp.Value = "COI" or Fields!arTyp.Value = "ICO", "Secondary"
, True, "")
I've only included a few of your cases and you can see it's already long - you would need to add as many values as you need.
With string matches you can streamline this a bit by checking if the field value is in a string of match values:
=Switch(InStr(",BCS,CIG,HUM,", "," & Fields!arTyp.Value & ",") > 0, "Primary"
, InStr(",COI,ICO,INP,", "," & Fields!arTyp.Value & ",") > 0, "Secondary"
, True, "")
We wrap everything in commas to prevent false positives on part matches. Again, add all the required values into the string list of match values.
Finally, you could use custom code for this comparison and move the comparison to a VB.NET type function embedded in the report.
I'm trying to use fetchAll on a query that has 2 variables. I can't figure out the syntax.
I can manage with only 1 variable:
$sql = "SELECT * FROM mytable WHERE field1 = ?";
$this->_db->fetchAll($sql,$value1); # that works
However I'm having some issues when query has multiple variables
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$this->_db->fetchAll($sql,$value1,$value2); # doesn't work
$this->_db->fetchAll($sql,array("field1"=>$value1,"field2"=>$value2)); # doesn't work either
The reason why I want to use ? instead of placing the variables directly into the query is that I've learned that using ? allows for the query to be compiled generically by the db engine and improves performances.
There are two types of parameter, named parameters and positional parameters. You're mixing the two types and that won't work.
Named parameters match a placeholder by name. Names are started with the : symbol. The parameter names are not the same as the names of the columns you happen to use them for. You supply parameter values in an associative array, using the parameter name (not the column name) as the array keys. For example:
$sql = "SELECT * FROM mytable WHERE field1 = :param1 AND field2 = :param2";
$this->_db->fetchAll($sql,array("param1"=>$value1,"param2"=>$value2));
Positional parameters use the ? symbol for the placeholder. You supply parameter values using a simple (non-associative) array, and the order of values in the array must match the order of parameter placeholders in your query. For example:
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$this->_db->fetchAll($sql,array($value1,$value2));
Most brands of SQL database natively support only one style or the other, but PDO attempts to support both, by rewriting the SQL if necessary before preparing the query. Since Zend_Db is modeled after PDO, Zend_Db also supports both parameter styles.
This question is a bit old, but I thought I'd just add to it for reference sake.
I would recommend starting to use Zend_Db_Select with Zend_Db. I've been doing a lot with Zend_Db lately. More from Zend_Db_Select reference guide.
Lets assume you have a Zend_Db adapter: $this->_db
# this will get the Zend_Db_Select object
$select = $this->_db->select();
# now you build up your query with Zend_Db_Select functions
$select->from('mytable');
$select->where('field1 = ?', $field1);
$select->where('field2 = ?', $field2);
[...]
# echo to see the SQL (helps in debugging)
# SELECT * FROM mytable WHERE field1 = ? AND field2 = ? [...]
echo '<p>My SQL: ' . $select . '</p>';
# Execute the SQL / Fetch results
$results = $select->query()->fetchAll();
That's the basics from your given example, but the Zend Framework reference guide on the select object has a lot of good information on how to build even more complex queries with JOINS, UNIONS, GROUP BY, LIMIT, HAVING, etc.
If you wanted to use an alias name for a table or parameters, you use an associative array with the alias name being the index value:
# SELECT p.* FROM products AS p
$select->from('p' => 'products');
If you want to return only selected fields, you add an array of field names as a second parameter:
# SELECT model FROM products
$select->from(products, array(model));
Actually, the above could should produce fully qualified SQL as:
SELECT 'products'.model FROM 'products'
but I wrote the above for brevity and clarity in the example.
One thing I just came across is using AND and OR in the WHERE condition.
# WHERE a = $a
$select->where('a = ?', $a);
# WHERE a = $a AND b = $b
$select->where('a = ?', $a);
$select->where('b = ?', $b);
# WHERE a = $a OR b = $b
$select->where('a = ?', $a);
$select->orWhere('b = ?', $b);
# WHERE a = $a AND b = $b
$select->orWhere('a = ?', $a);
$select->where('b = ?', $b);
Notice, that whatever the following "where" function you use, will combine with the previous statement as that operand. Ok, that sounded confusing.
If the second "where" is an "OR" it will be an "OR" conditional. If the second "where" is a "AND" the statement will be "AND".
In other words, the first WHERE function is ignored in terms of what condition it will use.
In fact, I just asked a question on Stack Overflow yesterday regarding doing a complex WHERE using select.
Hope that helps!
Cheers!
Try this:
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$statement = $this->_db->query($sql,array("field1"=>$value1,"field2"=>$value2));
$data = $statement->fetchAll();
$this->_db must be an instance of Db adapter.
Heres the actual Zend way to code for this.
$sql = "SELECT * FROM mytable WHERE field1 = :param1 AND field2 = :param2";
$this->_db->fetchAll($sql,array("param1"=>$value1,"param2"=>$value2));
$where = $this->_db->select()
->from('mytable')
->where('field1 = ?',$value1)
->where('field2 = ?',$value2);
$rowSet = $this->_db->fetchAll($where);
This works great for me