EPiServer - how to search content in different EPiServer websites? - lucene.net

Greetings EPiServer gurus.
Picture this scenario:
A customer has got two different EPiServer-sites.
One internal and one external website.
The external is using EPiServer Find's REST API for search.
The internal is currently using a simple Search page which is based upon the Lucene indexer.
The customer wants to be able to search both the external and internal site's content INSIDE the internal site. They are not keen on the idea of having to buy another EPiServer Find license to apply on the internal. So basically they want to be able to search the content of the external site while inside of the internal.
What would be the proper approach in order to do this?
Any suggestions appreciated.
/ChrisRun

This is a tricky one. EPiServer Find support multi site setup but requires them to be hosted in the same solution. EPiServer constructed the indexing job in such way that it clears the entire Find index, this means that if you have the same Find index on two different machines they will erase each others indexes, effectively you'll only have the results from the most recently indexed site.
We've discussed this with EPiServer on changing this pattern to only allow an indexer to erase posts with siteId's available to the solution running the index job. However, no luck so far, instead we rely on hackish solutions :)
So, what you are asking is possible with a bit of coding, reflect the built-in indexer and ensure the ReindexTarget are scoped correctly (the code is easy to understand). When done this indexing job needs to be used on both the internal and external environment and the original job needs to be removed.
There's no need for filtering in your internal environment but in the external environment you'll have to ensure only external results are posted. If your results include anything else than pages you cannot filter on siteId since global items (like files and images) doesn't have any siteId. We've solved this with a url-filter like the one below.
private static FilterBuilder<ISearchContent> SiteFilterBuilder
{
get
{
var filter = SearchClient.Instance.BuildFilter<ISearchContent>();
filter = filter.Or(x => x.SearchHitUrl.Prefix(EPiServer.Web.SiteDefinition.Current.SiteUrl.AbsoluteUri));
return filter;
}
}
Implement
var query = SearchClient.Instance.UnifiedSearch(Language.Swedish)
.For(searchQuery.Query)
.AndInField(x => x.SearchCategories)
.UsingSynonyms()
.OrFilter(SiteFilterBuilder) // will scope to this site
.ApplyBestBets()
.Track()
.TermsFacetFor(x => x.SearchSection)
;

Off the top of my head, I can see multiple risks involved in adding the public Find index to the internal site - especially if you don't want it two-way (i.e. index the internal site in the same Find index).
One approach could be to add a search endpoint to the public website, which the internal website invokes to do searches.
Basically that endpoint (for example a controller action method) would perform a search using Find (this would happen inside the public web application) and then return the result to the internal website.
Technically, only the public website would use Find - but results would be available to the internal website.

Related

Change CKAN API Interface - are there limitations on the API?

I've looked around the site to see if there are any people who have changed the CKAN API interface so that instead of uploading documents and databases, they can directly type onto the site, but I haven't found any use cases.
Currently, we have a page where people upload data sets through excel forms that they've filled out, but we want to make it a bit more user friendly by changing the API so that they can fill out a form on the page rather than downloading the template, filling it out and then uploading it.
Does CKAN have the ability to support this? If so, are there any examples or use cases of websites that have use forms rather than uploads?
This is certainly possible.
I'm not aware of any existing extensions that provide that functionality, but you can check the official list of CKAN extensions if there's anything that fulfills your needs.
If there is no existing extension that suits you then you could write your own, see the extension guide for details on how to do that.
Adding an API function to CKAN's API is possible, but probably not what you want in this case: the web UI usually does not interact with CKAN via the API but via Flask/Pylons controllers. Hence, you would add your add controller which first serves your form and then processes the submitted inputs.
You can take a look at the ckanext-pages extension, which does exactly that (for editing static pages instead of datasets, but your code would be similar).

How to Implement Cache while using AEM Search API

We are using AEM 6.3 and we have need to implement Content search functionality in our project.We implemented it using Search API provided but issue is that Search API take only request parameter and hence we are not able to cache the search result page.
Did try to use selector or set request attributes (searchTerm and Tags)and than create Search Client instance and call getResult method but it doesn't return any results.
As we need to do content search across pages and mutilple properties can we use QueryBuilder API here and achieve the same result provided by Search API
Search API is highly performant and the caching is not the best strategy for using searches as you might get stale results. In practice, you end up reducing the cache lifetime and end up at the same problem.
You should look more into optimising your searches with proper indexes over targeted content etc.
However, if you really want to cache the search results you could look into 3rd party solutions but I would highly discourage it in the context of AEM as there are better solutions like:
Offloading searches to a dedicated publisher. You can do it via your LB or dispatcher rules.
Optimise searches by optimising indexes. Remember, indexes don't hit your repository.
Worst case if you really struggle with performance, look into AEM Solr integration as Solr has good caching. You can also achieve same with ElasticSearch or other DB. Just be warned that plumbing and TCO is not free for this.

Symfony design question - how can I share forms between apps?

I'm developing a site in Symfony, and I'm not sure what the best way is to handle this scenario.
I'm creating a party bookings system. Anyone can go to my frontend app and submit a new booking. Once they're finished, they'll just get a confirmation screen, they can't edit it. Easy.
Only certain users will be able to get to the admin app (it might be secured simply by being on an intranet, but that's not important, just assume it will be only accessible by admin users). They'll be able to view the list of submitted bookings. Easy.
My problem is around code re-use when allowing admin users to edit existing bookings. When you do generate-module in Symfony, the generated module (which as a newbie I'm assuming is a good example of structuring things) creates the form as a partial. I've had to customize this form a lot for my usage (lots of Javascript, etc), so of course I want to re-use this code, to be able to load an existing booking into this form. But there doesn't seem to be a way to share this partial between the apps (I've seen people mention making a plugin...but this seems complicated for this use).
I considered using an IFrame to load the form from the frontend and just passing an "id" parameter to load it in edit mode, but this would mean that the edit mode is not secure - anyone could go to the form on the frontend and pass this parameter to edit a booking.
I also considered putting all of the form display code (HTML, Javascript, etc) in a method on the form object, but this doesn't seem very MVC - all of the display code is then in the form. But this is only because I'm thinking of the form in the same way as a model - is that right?
I feel like this should be a common situation. You can share models and forms between apps, why can't you share this common form display code too?
Thanks!
You should reconsider having 2 applications in the first place. Not only you run into the code reuse problem, but also i18n, testings and other issues. I find it much easier to have 1 application with different bunch of modules for frontend and backend users. You can configure security per module. You can have one sign in form for all users and redirect them to appropriate module based on their credentials.
You can reuse partials between modules inside the same application, but you seem to be talking about two different applications (frontend and backend) so as far as i know the only way is to copy & paste the partial from one application to the other...

Adding pages "on the fly" with a CMS system

I am in the process of building a website content management system for one of my clients. It's a highly customized system, so I cannot use any "of the shelve" solution.
I need to allow my client to add pages to the website on the fly. I have two options here:
(1) Create a database driven page in the format of www.mycompany.com/page.aspx?catID=5&pageID=3 (query the database with the category and page ID's, grab the data and show it on the page) - or -
(2) Allow the management system to create static pages, something like www.mycompany.com/company/aboutus.aspx and www.mycompany.com/company/company_history.aspx , etc.
I believe that, while the former is much easier to implement, the latter is a better both for the user AND for Google.
My questions are (finally): (1) Would you agree that the latter is a better solution, and (2) What is the best way to implement such a solution? Should I create and update each file using the FileSystem (i.e. - the site's management system requires the user to supply a page/file name, page title and content, and creates the page on the fly based on these parameters)? Is there a better way?
Thank you!
It's entirely possible to have database driven pages with nice URLs. StackOverflow itself is a great example - this question's URL is http://stackoverflow.com/questions/1119274/adding-pages-on-the-fly-with-a-cms-system, but the page is built from the database, not static HTML.
I would use the first solution, but mask the addresses using a custom request handler. Basically, give each of your pages a unique string ID (such as about-us) and then, with your request handler that takes all requests, find this particular page in the database and render it.
See this article for some additional info (found it when googling for custom http handlers in ASP.NET.) In that article, it has the following handler added:
<add verb="*" path="*.piechart" type="PieChartHandler"/>
You would probably want to catch all paths (*), excluding certain media paths used for CSS, images and JavaScript.
More resources:
Custom HTTP Handler
HttpHandler in ASP.Net
I'd stay clear of static pages if I where you. Dynamic Data, MVC and some good planning should take you a long way!
What you need to do is to create some or many templates that each view/controller in mvc can use. Let whoever is responsible for the content handle it through dynamic data entities.
I would use the first idea, but work out a better URL scheme. If the system doesn't provide nice URLs (without ?), you'll have trouble getting the search engines to parse the whole site. Also using numbers instead of words make it hard on users to pass around URLs.
If you start to have performance problems you could add caching that would generate static pages from time to time. I would avoid doing that until you have to; caching can cause many headaches along the way to getting it right.
Although the existing advice is more-or-less sound, the commentators have failed to consider one factor which, admittedly, you haven't given much detail on. Are these pages that they'll edit once they're built, or a they one-shot creations? If the latter, your plan of generating static pages isn't quite so bad as they suggest. Why bother even having to think about database schemas and caching, when you can just serve flat content.
It will probably make for pretty lifeless, end-of-the-road pages, but if that's what you want ...

Jira RPC/SOAP GetCustomFields() can only be used by an administrator?

I'm currently using the Jira SOAP interface within a C# (I suppose the language used here isn't terribly important).
Basically, I'm creating an API and a Winform that wraps some of the functionality of the soap service so that our Devs can programmaticly add bugs when something goes wrong in our application.
As part of this, I need to know the custom field IDs that are in use in Jira, rather than hardcoding them (as they are still prone to the occasional change) I used the GetCustomFields() method in the jira-rpc api then filtered it, so that all the developer needs to know is the name of the field, then the ID is filled in for them automagically.
This all works fine, but with one quite important proviso: that you login to the SOAP/RPC service as a user with administrative privaliges.
The Jira documentation indicates that the soap/rpc service follows the usual workflows and security schemes, however I can't find anything anywhere that would appear to remove this restriction on enumerating custom fields (and quite why in any instance you would want someone to HAVE to be an administrator to gain this access, especially as the custom field id's tend to be in Jira's HTML source is beyond me)
Does anyone know if I've missed a setting somewhere? Or if there is some sort of work-around for this, short of hardcoding the custom field id's?
Or is this a case of having to delve in to Jira's RPC plugin and modifying the source for it in order to give me the functionality I require?
Cheers
Edit for the sake of google/posterity
Wow, all this time on, and it looks like Atlassian still haven't changed this behavior.
Worked around this by creating a custom dictionary that logs in as an administrative user, grabs the custom fields and then logs out. Not ideal, but it should work 'til atlassian change things
You're not missing anything - there's no way to get custom fields via standard SOAP API.
In JIRA Client, we learn about custom fields in two ways:
We download issues via RSS view of the issue navigator, or via XML representation of a specific issue. If a custom field is set for an issue, the XML will have its id, class and value (values).
From time to time we inspect the content of IssueNavigator search page - looking for searchers for the custom fields. Screen-scraping the HTML gives us not only ids of the custom fields but also possible values for enum fields.
This is hackery, of course, and it may go wrong, so a good API would have been a lot better.
In your case, I can suggest two solutions:
Create your own SOAP (or REST) remote API plugin that will give you just that info that you miss from the standard API. Since you're seemingly in control of your JIRA, you can install anything there.
Screen-scrape the "New Bug" page for the project and type of issue you need to submit. You'll get all the info - fields, options, default values, which field is required.