I am about to run an online user experiment to compare different strategies of the recommender system.
I will compare 18 strategies and each strategy produces five recommendations, thus I have to ask subjects to evaluate 90 recommendations in total.
However, the strategies make duplicated recommendations (i.e., some recommendations appear in several strategies).
(i) In the current design, each page provides five recommendations made by one strategy.
After a user inputs evaluations for all of them, another page shows up, where five recommendations by another strategy are given.
Thus, a user sometimes sees recommendations which he already evaluated.
I think this design is widely used in previous works, although how to handle duplicated recommendations has not been discussed extensively.
(ii) Or is it reasonable to first extract all unique recommendations, split them into different pages, and show them in the random order?
I think this design makes the experiment shorter.
But this design has not been used, to the best of my knowledge.
If you have a suggestion or know a paper related to it, please give it to me.
If those algorithms are different and they generate the recommendations in a different way, then even if they generate some similar recommendations you should definitely keep those recommendations as if you don't keep them, then you changed the recommendation results for some algorithms and therefore, your experiment will lose its soundness. When they generate some similar recommendations, it means some items are just so favorable for some users that you cannot stop them from being recommended no matter what algorithm you are using. So you should keep them. You should somehow explain why this happens (something like what I explained here) to users that will evaluate those recommendations so that they don't mind seeing them again and again.
Related
I have built a recommender systems which has tens of thousands of items and their feature descriptions, but no user profiles as of now. I am looking for pointers to approaches that can help me bootstrap the system, so I can do some evaluation. I would appreciate any pointers to papers/applications that have addressed this problem.
How to deal with the cold-start problem depends a lot on your specific application.
An easy way of dealing with the user cold-start problem is to present the new user with random items, or the most popular items, or hand-selected items, and start learning from them.
Another way is to present users with a questionnaire, and then present items to them according to the results. Or you directly show them items/products and let them rate/select the ones they like.
Also note that in web-based system you usually know some things about your users: Which operating system/browser they use, where they (roughly) come from, which language they speak.
All this information can be used.
Papers:
see the Wikipedia article on the topic
My answer to another question on StackOverflow lists some papers for dealing with new items - most of the methods would also be applicable to new users.
Another approach is to select products/items that will help you most for learning about the user. Just out of my head, you can find them by querying Google Scholar for "recommendation" and the terms "decision trees", "active learning", "user cold-start", and so on.
We're currently researching different source control tools, and want to test each one with some light-weight but meaningful scenario to get a feel for the capabilities of each one.
Terminology and internal logic varying wildly between some tools, it would be nice to have the scenario expressed in terms of use cases ("We have to correct a bug on Release 1.3"), rather than in potentially tool-specific terms ("Create a branch named Release 1.3").
It's true that different things are important for different teams, but it would be interesting to have some kind of canonical test case from which different scenarios could be picked. Or am I being too optimistic?
Is anyone aware of something like this? Have you used a similar approach when investigating source control tools?
These are the requirements that Mozilla had when they set out evaluate version control systems for internal use in 2006. You might find a similar approach useful.
If you find scenarios that are specific to your company, perhaps you can translate them to requirements like the ones above.
You have some general criteria with Google DVCS analysis which can give some ideas.
But you need first to see if you want to evaluate:
CVCS (Central Version Control): update-merge-commit
DVCS (Distributed Version Control): commit-rebase/merge
For more on the different scenario to test (one answer for CVCS, one for DVCS), see the SO question:
" Describe your workflow of using version control (VCS or DVCS) "
You have to question things like: Do you have only a single line of Release/Development or do we create multiple releases in parallel? No only the mentioned scenario is needed you need to think about upon that one like merging back the changes into dev line or multiple other lines as well. This can influence the approach. The approach you selected sounds very good cause you try to understand the process and not using tool terms. I've done that for multiple times for customers of mine. In different teams/companies different things are handled different. So the problem is to figure out what the process of yours is (sometime people are not aware of this).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I've just picked up a contract to sort out a vipers-nest of e-commerce websites that a previous 'developer' left for one of my clients. There's about a couple of dozen of them using a custom shopping cart and CMS system that's too embedded to dump and works well enough, but desperately needs cleaning up, re-factoring, and bug fixing, so a reasonably substantial recoding job.
As part of this my client is desperate to ensure the best search engine placement he can get. Like many developers I've a nodding acquaintance with the idea, but no real knowledge, and it seems that it would be helpful to get up to speed on this so I can build appropriately into the code.
So can people advise on useful quality resources - books, websites, blogs etc? I do not wish to obsess over every last detail on this (he can use a specialist if he decides to pull every last ounce out - although I've always regarded such as little better than snake-oil peddlers), but I would like to build code and reconfigure templates in a manner that helps rather than hinders placement.
Look at Wikipedia with styles off. See how they order their content? See how they use correct tags to label the content? These are the keys to long term success.
The most important SEO advice is to create a semantic, logical site. The content comes first and is ordered by importance. Use the correct tags, don't do tables. Then apply styles. Then apply script to make it fancy. (Like Tomas said)
Know the difference between what is content (pictures of your products) and not (your logo tiled on the background). Basically do your best to present good content in a good way. You can't game the system for long term gains.
This will give you long term placement. Most SEO companies just do tricks or links farms or worry about keywords and meta tags, so they are temporary at best. I think it is good to assume that Google works just as hard to drop that crap from it's index as the SEO marketers do to include it.
According to Google's patent
Age of domain.
Important sites that link to you.
Content. Make it real and accurate.
http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=/netahtml/PTO/search-bool.html&r=1&f=G&l=50&co1=AND&d=PG01&s1=20050071741&OS=20050071741&RS=20050071741
A main concern when thinking SEO should always be usability. Make sure the mark-up is semantic, make sure the site works without javascript, css and images - in other words, make sure that 'low-level browsers' are able to read the contents of your page. Think of optimizing the site for a screen reader for the blind and visually impaired, and you will automatically cover the search engines, because that is exactly what they are - blind browsers.
A standard no-no is a table layout, mainly because it does not order the site content by relevance. To a human being reading your site, it is no problem looking two inches down instead of at the very top of your page, but the screenreader reads the code, not the visual view.
Thus, you should make sure that you have the important things - a high-level heading (preferrably <h1> or <h2>), the main menu (in a <ul> list) and the content in divs at the top of the source, and less important content (the logo, banners, quickmenus etc that aren't really crucially needed to be accessible) further down. You are always able to re-order where the stuff is actually shown in your css.
New tag to use called CANONICAL can now also be used, from Google, click HERE
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I'm working in a small company and weeks away from deploying a web-app that will be used a lot. Everyone at one location will have to learn to use it, and although I think it's pretty easy and intuitive I may be biased.
I've written a help guide with plenty of screenshots that's available on every page, but I'll still need to train everyone. What's the best way? How do you take a step back and explain code you've been working on for weeks?
First try to avoid the training:
Perform usability testing to ensure your web app is intuitive. Usability testing is a very important aspect of testing and it is often ignored. How you see your system will probably be very different as how a new user sees your system.
Also add contextual help as often as you can. For example when I hover over a tag in stack overflow, I know exactly what clicking it will do, because it tells me.
Also this may seem obvious, but make sure you link to your documentation from the site itself. People may not think of looking in your documentation unless its right in front of their eyes.
About training documentation:
Try to split up your material into how your users would use the system. I personally like the "trails" option that Sun created for their Java tutorials. In this tutorial you can do several things, and you can chose on which trail you'd like to go.
Support random reads in your help documentation. If they have a task to do in your web app, then they should be able to get help on that without reading a bunch of unrelated content.
Make sure your documentation is searchable.
About actual training sessions:
If you are actually performing training sessions, stay away from explaining anything related to your code at all. You don't need to know about the engine to drive a car.
Try to split up your training sessions into very focused aspects of your system. If you only have 1 training session available to you then just do one specialized use case of your system + the overall description of the system. Refer to the different parts of documentation where they can get help.
Letting the community help itself:
No matter how extensive your documentation is, you'll always have cases that you didn't cover. That's why it's a good idea to have a forum available to all users of the system. Allow them to ask each other questions.
You can review this forum and add content to your documentation as needed.
You could also open up a wiki for the documentation itself, but this is probably not desirable if your user base isn't very large.
Few ideas:
Do you have some canned walk-through scenarios? Don't know if it is applicable for your product, but I built a pretty substantial product a couple years ago and developed some training modules that they'd work through - nothing long, maybe 15 minutes tops for each one.
I put together a slide presentation that hit the highlights to talk about what it does. I would spend about 10 minutes going through the app's highlights to familiarize them with it before doing the hands-on stuff.
People don't tend to read stuff, unfortunately. You could put hours and hours into a help document, and still find that folks simply don't read it or skim over it. That can be frustrating. Expect that answers that are in your guide will be the topic of questions your users will have.
Break up any training you do into manageable chunks. I've been to a full-day training exercise before and the trainer broke it into short pieces and made it easy for me to get the training topic in my head. You don't want to data-dump on them because their eyes will gloss over and you'll lose them.
Ultimately, if your app is highly usable, it should be a piece of cake. If it isn't, you'll find out. You might want to have a few folks you know run through your training ahead of time and give you constructive criticism on it. Better to fix it before the big group is trained. You'll be more confident in the product and the training materials (whatever they are) and you'll likely have a better training experience.
If applicable, provide an online help/wiki/faq for them. Sometimes that is helpful.
Best of luck!
You should really have addressed this issue a lot earlier in the development cycle than you are doing.
In my view the ideal scenario for corporate software is one where the users design their own application and write their own documentation and I always try to strive for this. You should have identified key users early on and designed the system with them (I try to get my users to do basic screen designs and menu layouts in Excel or similar - then I implement that as static pages and review before writing a line of significant code, obviously they won't get the design right first time, but it's your job to guide them - and ideally in a way where they think they came up with the correct design decisions, not you :-) ).
These users should then write the user documentation from this design in parallel with you developing the system. I have never seen help documentation delivered by a IT department/software company used significantly in a corporate setting. Instead what happens is the users will create their own folder of notes and work-arounds and refer to this (in fact if you're ever doing system analysis to replace an existing system finding the 'user-bible' for the old system is a key strategy). Getting the users to write their own documentation up-front simply harnesses what will happen anyway - but this is vastly easier if the users feel they have ownership of the system because they designed it themselves in the first place.
Of course this approach needs commitment and time from your users, but generally it's not that hard a sell. It's trite, but working as a facilitator so the users can develop there own system rather than as a third party to give them a system pretty much guarantees user acceptance.
As you are where you are you're too late to implement all of this, but if you can identify a couple of keen, key, users and get time from them to write their own documentation then that would be a good move. If you can't get even that then you need to identify an evangelist who you can train to be the 'departmental' expert and give them 110% of your energy to support them.
The bottom line is that user acceptance is based on perception, and this does not necessarily correlate with how usable an system actually is. You have to focus on the group psychology of this as much as the reality of the system, which tends to be tricky for developers as we're much more factually based than most people.
I'll be looking into something like this too in the next few months.
In your case, hopefully the UI has already undergone user acceptance testing. You say you work in a small company. Is it possible to get the least tech-savvy person there to try it out? In fact, get them to try it out without any guidance from yourself except for questions they ask. Document the questions and make sure your user-guide answers them.
The main thing for me would be logic and consistency. If the app's workflow relates logically to the task it has been designed to accomplish and the UI is consistent you should be OK.
Create a wiki page to describe the use of your system. Giving edit rights to the users of your system lets the users:
update the documentation to correct any errors in the initial release of documentation,
share any tips on usage they may have found.
share unusual uses for the system that you may not have thought of.
request features.
provide any workarounds they've found while waiting for the new functionality to be implemented.
Try a few users first, one or two in a small company. Mostly watch, help as little as possible. This tells you what needs to be fixed, and it creates an experienced user base - so you are not the "training bottleneck" anymore.
Turn core requirements/use cases/storycards into HowTo / walkthroughs for your documentation.
For a public training, prepare a 10..15 minute presentation (just that, not more!) that covers key concepts that the users absolutely must understand, than show your core walkthroughs. Reserve extra time for questions about how to solve various tasks.
Think as a user, not as a techie: - noone cares if it's a SQL database and you spent a lot of time to get the locking mechanisms right. They do care about "does it slow me down" and "does something bad happen when two people do that at the same time". Our job is to make complicated things look easy.
It may help to put the documentation on the intranet in an editable form - page "comments", or wiki maybe. And/or put up a "error wiki" for error messages and blips - where you or your users can quickly add recomendations, workarounds and reasons for anything that does not go as expected.
Rather then train all those people I have chosen a few superusers (at least one person from each department) and trained them to teach the rest of the employees. It is of course vital that those super users are
well respected in their departments
able to teach
like the application
The easy way to ensure that they like the app is to have them to define the way it should work :-). Since they should work with this app each and every day they are the prime stakeholders, no matter what management states
I am interested in choosing a good structure for an online message board-type application. I will use SO as an example, as I think it's an example that we are all familiar with, but my question is more general; it is about how to achieve the right balance between organization and flexibility in online message boards.
The questions page is a load of random stuff. It moves quickly (some might say, too quickly) and contains a huge number of questions that I'm not interested in.
The idea, I imagine, is that we can use tags to find questions that we're interested in. However, I'm not sure that this works: you can't use tags negatively. I'm not interested in PHP or perl or web development. I want to exclude such posts. But with the tags, I can't.
Although discrete subforums are in a sense less flexible, as they generally force you to pick a category even if a question might fit into two (if SO had, say, areas for "Web Development", "Games development", "Computer Science", "Systems Programming", "Databases", etc. then sure, some people might want to post about developing of web-based games, for example) is it worth sacrificing some of that flexibility in order to make it easier to find the content that you are interested in, and hide the content that you are not interested in?
Is there any way with a pure tagging system to achieve the greater ease of use that subforums provide?
The real problem with subforums comes when you guess wrong about which topics have enough interest to get their own subforums. While some topics end up with their own vibrant subcommunities others end up as empty ghettos, with little activity or feeling of community. Topics that might flourish as occasional subjects in a larger forum end up fragmented among many subforums, none of which has the critical mass of people necessary to have an active, vibrant community.
Though I think that tagging is supperior to grouping, people tend to think hierarchically.
In general it depends on the target group for the forum.
Maybe you can go with a mixture: use tagging and later use tag groups to order to posts. Delicious uses this, for example, and I find it rather helpful.
If you're worried about the divide between specific forums and open tag-based systems, like Stack Overflow, consider making a query system that allows you to do a bit more complex queries than just the AND operator, like here on Stack Overflow.
I cannot make a query here that will give me all questions in .NET, SQL or C#, combined, and that is the biggest irritation I have with the tags. With such a query system, you can create virtual forums at least.
Other than that, I don't really have a good opinion. I like both, and I haven't yet decided which one is best.
The idea, I imagine, is that we can use tags to find questions that we're interested in. However, I'm not sure that this works: you can't use tags negatively. I'm not interested in PHP or perl or web development. I want to exclude such posts. But with the tags, I can't.
While it's currently the case that you can't use tags to hide content, it shouldn't be impossible. Using SO as an example again, there's no reason that a system similar to the ignore function on a forum couldn't be made for the tag system. By adding a right-click context menu or a small "X" link somewhere in the tag display, tags could be marked as ignored. This would also allow the current tag feature to function; Seeing everything (minus your ignore list), or clicking a tag to see only questions with that tag.
Ignored tags could be managed in your profile if you should later develop an interest in PHP or INTERCAL that you lacked before.
The real question is that of performance. In my head it's as simple as replacing a SELECT [stuff] WHERE Tag = 'buffer-overflow' with SELECT [stuff] WHERE Tag NOT IN ('php','offtopic','funny-hat-friday') but I've not put together any DB backed sites that get absolutely pounded on by thousands people.