This is pretty wide question, but I'm hoping to get a push in the right direction (technologies and methodology).
Ok, I have an iphone app (which I am developing) that works with a web service (c#) through http requests. The web service connects to the underlying database, extracts the necessary data depending on the request and feeds it back to the application.
Now, I need to implement a search system in the app. The user searches for some words, and I need to provide the most relevant results. The search must be performed on different tables in the database. Each table can be searched in a number of columns. For example, when searching through the people table I need to search in the first name, lastname, company, and other fields. Other tables have other important columns.
I have so many questions that I don't even know where to start.
How do I make my sql queries to make the search, but still be fast enough. Do I need to make some extra tables with indexed content somehow?
How should I add relevance factor to the results so I can ultimately filter only the most relevant results? For example, if an user searches for Smith, maybe there is a person named Smith or even a Company. They should be displayed before any other content that can have smith in the description.
I know the question is a little vague/wide but I can explain more if somebody desires.
Thank you
This kind of depends on which language/rdbms you are using on your server. You might checkout various DB search solutions like Sphinx which will do all of that indexing for you and provide a simple Search API. Sphinx for example allows you to prioritize columns, define character mappings (ß->s, ä->a) etc.
In the end I have decided to use Lucene. It's a wonderful piece of technology and even if I had some doubts in the beginning, after reading 3/4 of the book called "Lucene in Action" it was clear to me that it had everything I needed (and much more).
I know it's not a fully-functional searching system (with all the elements needed), but merely a library handling the core of a search system. It will need some work to integrate it with my application/webservice/database. I will let you know how it goes :)
Thanks for your input!
Related
Hope it's okay to ask a question like this in here - I'm a little new to backend and am still finding my way.
I am building a kind of language-learning-themed social media program, but I'm a little lost as to how best to structure all the various bits of data. At the moment I am storing two types of objects in their own MongoDB collections: posts, with a nested array of all the replies, and users. At the moment that's working okay, however there's lots of things I'm intending to add down the line, and would quite like to get my data structured in a way where that isn't problematic later.
For instance, one feature I am intending to add is a phrasebook, where users can save messages or words to a personal list, that they can access later or export. Currently I'm planning to create a separate collection for these phrasebooks, with a string that links it to the user's ID, and then an array of phrase. But is that bad practice? Should I be trying to keep my number of collections to a minimum, or is it fine to have lots of different ones?
If anyone has suggestions of where I can read up more on best practice around this too, I'd appreciate that a lot! I've tried to google but haven't had much luck.
I've been updating and moving my massage business website to Wordpress. During SEO process I interested and decided to include some structured data but I'm bit confused how to do it properly. I'm going to test that stuff first on my current site.
I'm going to present information with JSON-LD and I've been reading alot of schema-org manuals and blog posts about the schemas, still they are bit vaque to me.
How much data should I provide?
I still would like to present list of services we provide and price range by currency/min/maxPrice and persons data who are working there (name, profession, phone).
Would it be wise to put that data in the <head>-section of every page?
Or just specific data to page that they relate to like staff info to "Contact Us" page and service list to "Services" page?
Is there any penalty or down sides to have all that data on every page?
How do I present personal courses that every person has taken or other studies?
How do I present those services?
Can business under that HealthAndBeautyBusiness handle 3 phone numbers with names or should I just put contact info under person's data?
Does it matter in which order I present that data?
The more data you provide, the better
Better to be specific, otherwise it could be interpreted as spam. The structured data should be closely related to the content of the page itself
You mean the employees? You could use the employee property and the alumniOf properties but that doesn't match it very well. I think such data is a bit too detailed to be described at the moment - I would omit it for the time being
List them as offers, see makesOffer property
I would limit it to 1 number
The order doesn't matter
In the future try to split your questions, would be much easier to answer them that way.
I'm going to present information with JSON-LD and I've been reading alot of schema-org manuals and blog posts about the schemas, still they are bit vaque to me.
In regards to this statement. If I were you, and I'm not, therefore I can only assume you are just learning about technologies such as json-ld and how they relate to the bigger picture that is the Semantic Web also known as Web 3.0.
It sounds like you are on the right track I would suggest additionally reading articles relating to api's as well as the http request life cycle.
-Happy Coding
I am creating an inventory app currently for iPhone using Parse for companies to keep track of all of their tools, supplies, inventory. Now I'd like to allow for the user/company when adding a new item to their database for them to have the option to search from a pre-made database of items such as for a construction company when adding a simple Dewalt Drill Battery to their inventory would search the pre-made database for "Dewalt #DC9096 18V XRP 2.4A Battery" or an office would search for pencils by brand/serial number/name. I am looking for a simple way to make a database or even a table containing multiple brands products including their prices, product specifications, website for ordering more, company website, warranty phone number, etc... I have considered parsing all of the retail websites for information but don't know the legalities behind it and if the websites change then I'd need to update code. If there is ANY (easier/better) way to do this then assistance or direction would be great!
Thanks always
I would not go down the route of trying to parse websites, that will be a huge pain in the neck and impossible to maintain unless you have extensive resources (and as you mention it probably violates most site's terms of service anyway). Your best bet would be to hook into existing product databases via an API, such as Google's Search API for shopping, or maybe Amazon's API. Here's where you can start if you wanted to use Google:
https://developers.google.com/shopping-search/
Hopefully that gets you going in the right direction.
Edit: Here's a list of a lot more shopping APIs that could be good options:
http://www.programmableweb.com/apis/directory/1?apicat=Shopping
If you did find yourself needing to parse many different vendor websites (we'd call this "screen scraping") and you have the legal right to do so, you should use a tool like SelectorGadget to get your XPaths, it's much faster, easier and less error-prone than doing it by hand.
If you're doing more than a couple websites, though, you'll probably find that you'll have to update the scraping rules pretty often, it definitely won't be a set-and-forget operation.
I'm new to iOS Development and am trying to make an application that essentially sorts through a list of 300 names or so. I've got the Drill-Down part of the application down, aside from the detailView, but am now faced with a challenge.
What I would like to do is have users select from 3 fields with a UIPickerView to come up with shorter lists for every time a user is looking for a person. I'd like to use a .plist, but I also have an XML feed of the information. Before I waste all of my time structuring these data sources, does anybody have a good overview as to how I should approach this?
Also, I've asked some this question before, and they tell me to read up on introductory iOS development topics. I understand the mechanics of development, I just can't ever figure out how to approach a task properly. (I'm working on it!)
Thanks in advance. I'd share an image to help clarify, but my rep isn't high enough.
Snip: It looks like I misread your intention which makes my earlier comments irrelevant, you want to have the user select one of 3 options to shrink the list, if I'm not mistaken.
Some more questions for you, so I take it that this XML feed is going to be potentially changing between times that the user loads up the app? Will it only ever grow or are those 300 or so names that are loaded once set for good? The reason I ask so that you can maybe see my train of thought is whether or not using Core Data might be useful. You could easily store your large list locally, save time having to reload this large list frequently, and also you can use the built fetchedObjectController to search your collection of names. I'll keep thinking about it and once you get a chance to answer these questions we can continue.
Ill check back for an edit or comment, and see if I can give you an approach. Also, maybe edit your question with any of your own approach ideas and we could also start from there and refine them if needed.
Edit 2: From the information in the comments this is one of the ways that I could see this being done that make sense to me:
Since you seem to be able to control the information you receive from the feed I would set it up to send you only the contacts that need to be added/removed. You could handle this a few ways depending on your deployment intentions but I would go with the following:
Find a way to signal a first time run of the application, and as a result all contacts would be new, and you could populate your list fully with a slightly longer first time setup. Then any further changes could be quickly handled by smaller edits made to the local list.
You would need to set up Core Data for your application, which should be fairly straightforward in your case, and after this you can use the built in NSFetchRequest to do your searches that will then quickly return a list of narrowed down contacts. As for the physical picker that is just a matter of building the UI which will require some design from your end as you are the only one that knows what you are going for in that regard. Depending on the complexity of your app and what functionality you will want to include you could get away with 1-2 views that simply do the displaying of the contacts in a table and then the picker just reloads when appropriate.
I'm not familiar with the implementation of XML Feeds and receiving data from them, but I have done XML Response parsing into Core Data from a SOAP service before and they shouldn't be terribly different.
Regarding resource to get you started should you need them, I would recommend the following:
eBooks:
http://www.techotopia.com/index.php/Objective-C_2.0_Essentials
http://www.techotopia.com/index.php/IPhone_iOS_4_Development_Essentials_Xcode_4_Edition
Tutorials:
http://www.raywenderlich.com/
The eBooks I have linked are both absolutely fantastic and one of the few xCode 4.0 books that I was able to find that seemed to be of an actual usable quality. They both contain easy to follow and clear tutorials on simple and more advanced aspects of programming for iOS.
Ray's site is an immensely helpful resource as it contains both a very active forum base for iOS programming in addition to a constantly growing tutorial collection as there are 4-5 people that constantly are creating new tutorials that the community votes on and suggests every week. It contains some more advanced topics than the above books and I would recommend looking at it after doing a few walk through/tutorials from the books.
I'll stick around if you have any further questions, otherwise you can send me a notification via these comments, or just post another question and someone is bound to help you out!
-Karoly
Since my last question was considered subjective :( , I'm trying to make it more specific.
I'm building an application in PHP where users can "friend" each other. This seems to be best suited to a graph datastore... For example, you can have this set of fields in a traditional RDBMs:
id | user1 | user2
and you have to deal with duplicate data (id = 1,user1 = Joe, user2 = Jeff, id=2, user1=Jeff, user2=Joe)...
You also have to search both columns for one user.
When performing certain friend of a friend searches, the recursion can be tricky indeed.
Do you agree a graph database is best?
If so, which one? and why is it best in your experience?
Since client already has MySQL, is it worth the overhead to obtain a graph store, or is there a good approach to the main issues with friending while keeping it in MySQL.
P.S. TO MODERATORS:
If you still have a problem with this post, I'd most appreciate if you could tell me if there's any particular way to ask this question and be considered a "constructive" post? gmail me (joedevon), tweet me (joedevon), add it in a comment. whatever method suits you best...
I just want some input from fellow programmers and I think the problem is common, filled w/ opportunities and issues, and interesting. Amazed that the original wasn't considered good for SO, but thems the rules...
Maybe it would be relevant to mention what language you use for building that app. If it is Java, I think there is no real competitor to neo4j. (my opinion)
Maybe you've already seen these neo4j presentation series, but in the 200. second of that presentation, you can see use case for Polyglot persistence, which means that you could utilize your existing data model or data model that doesn't need to be type free or isn't suitable for Graph data model, TOGETHER with the aspect of application (user bonding and relationships) that would use neo4j under the hood and its graph data model.
Spring Data project makes this really easy and I consider this a very perspective way to go.