Facing Issue in zend_search_lucene - zend-framework

I am using Zend Lucene Search:
......
$results = $test->fetchAll();
setlocale(LC_CTYPE, 'de_DE.iso-8859-1');
Zend_Search_Lucene_Analysis_Analyzer::setDefault(new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8());
foreach ($results as $result) {
$doc = new Zend_Search_Lucene_Document();
// add Fields
$doc->addField(
Zend_Search_Lucene_Field::Text('testid', $result->id));
$doc->addField(
Zend_Search_Lucene_Field::Keyword('testemail', strtolower(($result->email))));
$doc->addField(
Zend_Search_Lucene_Field::Text('testconfirmdate', $result->confirmdate));
$doc->addField(
Zend_Search_Lucene_Field::Text('testcreateddate', $result->createddate));
// Add document to the index
$index->addDocument($doc);
}
// Optimize index.
$index->optimize();
// Search by query
setlocale(LC_CTYPE, 'de_DE.iso-8859-1');
if(strlen($Data['name']) > 2){
//$query = Zend_Search_Lucene_Search_QueryParser::parse($Data['name'].'*');
$pattern = new Zend_Search_Lucene_Index_Term($Data['name'].'*');
$query = new Zend_Search_Lucene_Search_Query_Wildcard($pattern);
$this->view->hits = $index->find(strtolower($query));
}
else{
$query = $Data['name'];
$this->view->hits = $index->find($query);
}
............
Works fine here:
It works when I give complete word, first 3 character, case insensitive words
My issues are:
When I search for email, i got error like "Wildcard search is supported only for non-multiple word terms "
When I search for number/date like "1234" or 09/06/2011, I got error like "At least 3 non-wildcard characters are required at the beginning of pattern"
I want to search date, email, number here.

In file zend/search/Lucene/search/search/query/wildcard a parameter is set,
private static $_minPrefixLength = 3;
chnage it and it may work..!

Based on NaanuManu's suggestion, I did a little more digging to figure this out - I posted my answer on a related question here, but repeating for convenience:
Taken directly from the Zend Reference documentation, you can use:
Zend_Search_Lucene_Search_Query_Wildcard::getMinPrefixLength() to
query the minimum required prefix length and
use Zend_Search_Lucene_Search_Query_Wildcard::setMinPrefixLength() to
set it.
So my suggestion would be either of two things:
Set the prefixMinLength to 0 using Zend_Search_Lucene_Search_Query_Wildcard::setMinPrefixLength(0)
Validate all search queries using javascript or otherwise to ensure there is a minimum of Zend_Search_Lucene_Search_Query_Wildcard::getMinPrefixLength() before any wildcards used (I recommend querying that instead of assuming the default of "3" so the validation is flexible)

Related

Typoscript: how do I add a parameter to all links in the RTE?

I want to add a parameter to all links entered in the RTE by the user.
My initial idea was to do this:
lib.parseFunc_RTE.tags.link {
typolink.parameter.append = TEXT
typolink.parameter.append.value = ?flavor=lemon
}
So for example:
http://domain.com/mypage.php
becomes
http://domain.com/mypage.php?flavor=lemon
which sounds great -- as long as the link does not already have a query string!
In that case, I obviously end up with two question marks in the URL
So for example:
http://domain.com/prefs.php?id=1234&unit=moon&qty=300
becomes
http://domain.com/prefs.php?id=1234&unit=moon&qty=300?flavor=lemon
Is there any way to add my parameter with the correct syntax, depending on whether the URL already has a query string or not? Thanks!
That would be the solution:
lib.parseFunc_RTE.tags.link {
typolink.additionalParams = &flavor=lemon
}
Note that it has to start with an &, typo3 then generates a valid link. The parameter in the link also will be parsed with realURL if configured accordingly.
Edit: The above solution only works for internal links as described in the documentation https://docs.typo3.org/typo3cms/TyposcriptReference/Functions/Typolink/Index.html
The only solution that works for all links that I see is to use a userFunc
lib.parseFunc_RTE.tags.link {
typolink.userFunc = user_addAdditionalParams
}
Then you need to create a php script and include in your TS with:
includeLibs.rteScript = path/to/yourScript.php
Keep in mind that includeLibs is outdated, so if you are using TYPO3 8.x (and probably 7.3+) you will need to create a custom extension with just a few files
<?php
function user_addAdditionalParams($finalTagParts) {
// modify the url in $finalTagParts['url']
// $finalTagParts['TYPE'] is an indication of link-kind: mailto, url, file, page, you can use it to check if you need to append the new params
switch ($finalTagParts['TYPE']) {
case 'url':
case 'file':
$parts = explode('#', $finalTagParts['url']);
$finalTagParts['url'] = $parts[0]
. (strpos($parts[0], '?') === false ? '?' : '&')
. 'newParam=test&newParam=test2'
. ($parts[1] ? '#' . $parts[1] : '');
break;
}
return '<a href="' . $finalTagParts['url'] . '"' .
$finalTagParts['targetParams'] .
$finalTagParts['aTagParams'] . '>'
}
PS: i have not tested the actual php code, so it can have some errors. If you have troubles, try debugging the $finalTagParts variable
Test whether the "?" character is already in the URL and append either "?" or "&", then append your key-value pair. There's a CASE object available in the TypoScript Reference, with an example you can modify for your purpose.
For anyone interested, here's a solution that worked for me using the replacement function of Typoscript. Hope this helps.
lib.parseFunc_RTE.tags.link {
# Start by "replacing" the whole URL by itself + our string
# For example: http://domain.com/?id=100 becomes http://domain.com/?id=100?flavor=lemon
# For example: http://domain.com/index.html becomes http://domain.com/index.html?flavor=lemon
typolink.parameter.stdWrap.replacement.10 {
#this matches the whole URL
search = #^(.*)$#i
# this replaces it with itself (${1}) + our string
replace =${1}?flavor=lemon
# in this case we want to use regular expressions
useRegExp = 1
}
# After the first replacement is done, we simply replace
# the first '?' by '?' and all others by '&'
# the use of Option Split allow this
typolink.parameter.stdWrap.replacement.20 {
search = ?
replace = ? || & || &
useOptionSplitReplace = 1
}
}

Finding bad words from large list of email addressess using PHP -Mongodb

I have large list of email addressses from a file. It comes around 1 million email ids. I have list of bad words like spam,junk etc, it consist of 20,000+ bad words.
I need to validate email ids. If bad words is present any where in email id it will be marked as invalid.
For example;
testspam#gmail.com - invalid
newuser#desspam.com - invalid
I would like to know which will be fastest comparison method as array looping will take time.
I tried following methods
//$keyword_list- array of bad words;
//$check_key- the email id which need to validate
$arrays = array_chunk($keyword_list, 2000);
for($i=0;$i<count($arrays);$i++)
{
if (preg_match('/'.implode('|', $arrays[$i]).'/', $check_key, $matches)){
return 1;
}
}
The above method is taking more time when comparing 1 million data.
Next we tried with the following code and this also takes more time
//$contain = bad words separated by '|'
// $str - the email id which need to validate
if(stripos($contain,"|") !== false)
{
$s = preg_split('/[|]+/i',$contain);
$len = sizeof($s);
for($i=0;$i < $len;$i++)
{
if(stripos($str,$s[$i]) !== false)
{
return(true);
}
}
}
if(stripos($str,$contain) !== false)
{
return(true);
}
return(false);
Finally I had tried Mongodb Text Search. It works fast with the following issues
If 'Hell' is the word in my bad list and my email id is like
head#e-hellinglysussex.sch.uk, then the Mongodb Text Search wont matches it.
Here is the code I used;
$ret = $db->command( array("text" =>$section, "search" => $keyword_string, "limit"=>$cnt_finalnonmatch));
where $section = Collection name,
$keyword_string = Comparing keywords string separated by space, for eg "Hell Spam Junk" etc,
$cnt_finalnonmatch = total number of comparing email ids
Please help me to solve this issue.
I am not entirely sure, but I suspect that the problem is that 'Hell' is not equal to 'hell' when you search for text since mongodb is case sensitive.
The solution should be to force all the strings and word to be lowercase (or uppercase)
We have used Mongodb 'like' to solve this issue;
$keywords = $key['keyword']; // Keywords need to compare
$regexObj = new MongoRegex("/".$keywords."/i"); // MongoRegex function declration
$where = array($section => $regexObj); // $section is the collection name
$resultset = $info->find($where);

Copy range with conditional formatting

I have a range with Conditional Formatting in an existing Excel file. I used EPPlus to copy that range to a new sheet, then I found the conditional formatting was missing.
Is there any way to copy range with conditional formatting using EPPlus?
I found a solution for this. I did not test it on all formattingRuleTypes. (Only needed 2 of them for the moment)
In my application i have 1 template row for each sheet.
var formatList = fromSheet.ConditionalFormatting.ToList();
foreach (var cf in formatList)
{
// sourceRow is the row containing the formatting
if (cf.Address.Start.Row == sourceRow )
{
IExcelConditionalFormattingRule rule = null;
switch (cf.Type)
{
case OfficeOpenXml.ConditionalFormatting.eExcelConditionalFormattingRuleType.GreaterThan:
rule = dest.ConditionalFormatting.AddGreaterThan();
break;
case OfficeOpenXml.ConditionalFormatting.eExcelConditionalFormattingRuleType.GreaterThanOrEqual:
rule = dest.ConditionalFormatting.AddGreaterThanOrEqual();
break;
case OfficeOpenXml.ConditionalFormatting.eExcelConditionalFormattingRuleType.LessThan:
rule = dest.ConditionalFormatting.AddLessThan();
break;
case OfficeOpenXml.ConditionalFormatting.eExcelConditionalFormattingRuleType.LessThanOrEqual:
rule = dest.ConditionalFormatting.AddLessThanOrEqual();
break;
default:
break;
}
rule.Style.Fill = cf.Style.Fill;
rule.Style.Border = cf.Style.Border;
rule.Style.Font = cf.Style.Font;
rule.Style.NumberFormat = cf.Style.NumberFormat;
// I have no clue why the Formula property is not included in the IExcelConditionalFormattingRule interface. So I needed to cast this.
((ExcelConditionalFormattingRule)rule).Formula = ((ExcelConditionalFormattingRule)cf).Formula;
((ExcelConditionalFormattingRule)rule).Formula2 = ((ExcelConditionalFormattingRule)cf).Formula2;
// Calculate the new address for the formatting. This will be different in your case
var adr = new ExcelAddress( dest.Start.Row , cf.Address.Start.Column -1 , dest.Start.Row, cf.Address.Start.Column -1 + cf.Address.Columns -1 );
rule.Address = adr;
I have no clue why the Formula property is not included in the IExcelConditionalFormattingRule interface. So I needed to cast this.
To add to the answer of Luc Wuyts (I can't comment yet due to limited reputation):
// I have no clue why the Formula property is not included in the IExcelConditionalFormattingRule interface. So I needed to cast this.
((ExcelConditionalFormattingRule)rule).Formula = ((ExcelConditionalFormattingRule)cf).Formula;
((ExcelConditionalFormattingRule)rule).Formula2 = ((ExcelConditionalFormattingRule)cf).Formula2;
Some conditional formatting do not have the Formula-options. This cast will work, but applying the Formula properties to conditional formatting options which do not require it will have unforeseen results. Eg. the ConditionalFormatting.AddContainsBlanks() does not require Formula properties, and adding them might mess up the conditional formatting. A better approach is to check the type, and add the formula's only when required.
I had a similar problem, the only way I found to inspect, change or delete a conditional format of a cell or range is looking at the openxml specs. The conditional format is stored in the worksheet, with the range under the attribute sqref. So you can edit that range or add a new.
For example:
DIM p As New ExcelPackage(New FileInfo(ExlReportPath), True)
Dim ws As ExcelWorksheet = p.Workbook.Worksheets(ExlSheetName)
'--Find Node "worksheet" (1 in my case) , Find all Child Nodes "conditionalFormatting" (5 to 11 in my test)
Print.Debug(ws.WorksheetXml.ChildNodes(1).ChildNodes(5).Name)
'--You get: conditionalFormatting
'--Now you can inspect the range:
Print.Debug(ws.WorksheetXml.ChildNodes(1).ChildNodes(5).Attributes("sqref").Value)
'--Will give you the cell address that this formatting applies to example: "D11:D15"
'--you can change delete or add new range if you want, below I add F11:F15
ws.WorksheetXml.ChildNodes(1).ChildNodes(5).Attributes("sqref").Value="D11:D15 F11:F15"
'--You can inspect the rule itself in the InnerXml also...
If you need more details of the markup, google Wouter van Vugt, "Open XML The markup explained". I found it useful and the full document was online (free).
If you find an easier way please post it.
Regards

Help needed formatting Doctrine Query in Zend Framework

Can anyone tell me how to format the query below correctly in my controller.
Currently it gives me nothing in my FilteringSelect. However if I change it to >= I get back all the kennelIDs which is incorrect also but at least I'm getting something.
I've tested that the session variable is set and can confirm that there are kennels with the matching capacity.
// Create autocomplete selection for the service of this booking
public function servkennelAction()
{
$sessionKennelBooking = new Zend_Session_Namespace('sessionKennelBooking');
// disable layout and view rendering
$this->_helper->layout->disableLayout();
$this->getHelper('viewRenderer')->setNoRender(true);
// get list of grooming services for dogs from the table
$qry= Doctrine_Query::create()
->from('PetManager_Model_Kennels k');
//This should be set by default and narrows down the search criteria
if(isset($sessionKennelBooking->numPets)){
$b=(int)$sessionKennelBooking->numPets;
$qry->addWhere('k.capacity = ?','$b');
}
$result=$qry->fetchArray();
//generate and return JSON string using the primary key of the table
$data = new Zend_Dojo_Data('kennelID',$result);
echo $data->toJson();
}
Many thanks in Advance.
Graham
I think that addWhere condition is wrong. It has to be:
$qry->addWhere('k.capacity = ?', $b);
i.e. $b without quotes.

Youtube API - How to limit results for pagination?

I want to grab a user's uploads (ie: BBC) and limit the output to 10 per page.
Whilst I can use the following URL:
http://gdata.youtube.com/feeds/api/users/bbc/uploads/?start-index=1&max-results=10
The above works okay.
I want to use the query method instead:
The Zend Framework docs:
http://framework.zend.com/manual/en/zend.gdata.youtube.html
State that I can retrieve videos uploaded by a user, but ideally I want to use the query method to limit the results for a pagination.
The query method is on the Zend framework docs (same page as before under the title 'Searching for videos by metadata') and is similar to this:
$yt = new Zend_Gdata_YouTube();
$query = $yt->newVideoQuery();
$query->setTime('today');
$query->setMaxResults(10);
$videoFeed = $yt->getUserUploads( NULL, $query );
print '<ol>';
foreach($videoFeed as $video):
print '<li>' . $video->title . '</li>';
endforeach;
print '</ol>';
The problem is I can't do $query->setUser('bbc').
I tried setAuthor but this returns a totally different result.
Ideally, I want to use the query method to grab the results in a paginated fashion.
How do I use the $query method to set my limits for pagination?
Thanks.
I've decided just to use the user uploads feed as a way of getting pagination to work.
http://gdata.youtube.com/feeds/api/users/bbc/uploads/?start-index=1&max-results=10
If there is a way to use the query/search method to do a similar job would be interesting to explore.
I basically solved this in the same way as worchyld with a slight twist:
$username = 'ignite';
$limit = 30; // Youtube will throw an exception if > 50
$offset = 1; // First video is 1 (silly non-programmers!)
$videoFeed = null;
$uploadCount = 0;
try {
$yt = new Zend_Gdata_YouTube();
$yt->setMajorProtocolVersion(2);
$userProfile = $yt->getUserProfile($username);
$uploadCount = $userProfile->getFeedLink('http://gdata.youtube.com/schemas/2007#user.uploads')->countHint;
// The following code is a dirty hack to get pagination with the YouTube API without always starting from the first result
// The following code snippet was copied from Zend_Gdata_YouTube->getUserUploads();
$url = Zend_Gdata_YouTube::USER_URI .'/'. $username .'/'. Zend_Gdata_YouTube::UPLOADS_URI_SUFFIX;
$location = new Zend_Gdata_YouTube_VideoQuery($url);
$location->setStartIndex($offset);
$location->setMaxResults($limit);
$videoFeed = $yt->getVideoFeed($location);
} catch (Exception $e) {
// Exception handling goes here!
return;
}
The Zend YouTube API seems silly as the included getUserUploads method never returns the VideoQuery instance before it actually fetches the feed, and while you can pass a location object as a second parameter, it's an "either-or" situation - it'll only use the username parameter to construct a basic uri or only use the location, where you have to construct the whole thing yourself (as above).