Related
I am doing some research on potential options for building a chatbot. I am currently evaluating amazon Lex. The requirements for the bot are quite simple, a user can ask where to find something, the bot will tell them where in a document they will find the answer. All of these questions and answers have already been captured manually so we can easily have an excel sheet with question and answer.
Is there some way to input these pre-defined questions and responses into Lex? From my research I am having a hard time finding any info on something this basic. It won't really require any back and forth between the user and bot, (for ex. User: 'I need to order flowers' Bot: 'What kind of flowers?" etc.)
I have seen some info on incorporating Kendra, but I don't think the requirement is sophisticated enough to warrant using it
Ideally I would love to just hardcode it and say this is a question, and this is the response that should be given. Maybe this use case does not need something as powerful as Lex?
Lex can solve your problem at a fraction of the cost of Kendra.
Having said that, Kendra would be easier to work with when compared to Lex.
If you're got some Python capabilities I would recommend you take a look at the ExcelLexBot repo on GitHub. It is a Serverless Application that reads input from an Excel spreadsheet to build up a basic Lex bot for you.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have a serious question. Is it ever ethical to ignore the presence of a robots.txt file on a website? These are some of the considerations I've got in mind:
If someone puts a web site up they're expecting some visits. Granted, web crawlers are using bandwidth without clicking on ads that may support the site but the site owner is putting their site on the web, right, so how reasonable is it for them to expect that they'll never get visited by a bot?
Some sites apparently use a robots.txt exactly in order to keep their site from being crawled by Google or some other utility that might grab prices and therefore allow people to do price comparisons easily. They have private search engines on the site so they obviously want people to be able to search the site; apparently they just don't want people to be able to easily compare their information with other vendors.
As I said, I'm not trying to be argumentative; I would just like to know if anyone has ever come up with a case where it's ethically permissible to ignore the presence of a robots.txt file? I cannot think of a case where it's permissible to ignore the robots.txt mainly because people (or businesses) are paying money to put up their web sites so they should be able to tell the Googles/Yahoos/Other SE's of the world that they don't want to be on their indices.
To put this discussion in context, I'd like to create a price comparison website and one of the major vendors has a robots.txt that basically prevents anyone from grabbing their prices. I'd like to be able to get their information but, as I said, I can't justify simply ignoring the wishes of the site owner.
I have seen some very sharp discussion here and that's why I would like to hear the opinions of developers that follow Stack Overflow.
By the way, there is some discussion of this topic on a Hacker News question but they seem to mainly focus on the legal aspects of this.
Arguments:
A robots.txt file is an implied license, especially since you are aware of it. Thus, continuing to scrape their site could be seen as unauthorized access (i.e., hacking). Sucks, but arguments like this have been made in other legal cases recently (not directly related to robots.txt, but in relation to other "passive controls".)
Grabbing prices violates no copyright law, including DMCA, since copyright does not include factual information, only creative.
Ethically, you should not grab prices because the vendor should have the ability to change prices without worrying about being accused of a bait/switch by people coming from your site.
Have you taken the high road, explaining the site to them and saying you'd love to include them in your list of vendors? Maybe they will love the idea and actually expose the data in a way that is easy for you to consume and less resource-intensive for them to produce.
There are no laws written directly about robots.txt because netiquette is generally followed. Don't be one of the "bad guys."
Some people filter robots because they use URL links to perform "actions" like adding things to carts, and robots leave them with massive numbers of abandoned shopping carts in their database.
Some people filter robots because they have exclusive prices that they can't advertise openly based on agreements with their vendors. You could be putting them in a bad position by exposing those prices on your site.
In this economy, if a company doesn't want to do everything possible to advertise themselves, it's their own fault that you don't include them.
The other use of robots.txt is to help protect web spiders from themselves. It's relatively easy for a web spider to get mired in an infinitely deep forest of links, and a properly constructed robots.txt file will tell the spider that "you don't need to go here".
Many people have tried to build businesses off building "price comparison" engines that scraped major sites.
Once you start getting any sort of traffic/revenue to speak of, you will receive a cease and desist. It's happened to dozens, if not hundreds of projects. I even worked on a small project that received a C&D from Craigslist.
You know how they say "It's easier to ask forgiveness than it is to get permission"? It doesn't hold true with page scraping. Get permission, or you will be hearing from their lawyers.
If you're lucky, it'll be early on, when you've got nothing to lose. If it's late, you may lose your business and all your work overnight, with a single letter.
Getting permission shouldn't be hard. Unless you're doing something sneaky, you're likely going to drive them additional traffic. Hell, once your product takes off, sites may be begging you, or even paying you to add their data.
One reason we allow robots to dig through the web without complaint is that we have a way to stop them if we want to. Protects both sides.
Remember the uproar when Cuil's robots were accused of going over-the-top, apparently acting like a DoS attack in some cases and using up bandwidth allowances of some small sites?
If too many people violate robots.txt we might get something worse.
"No" means "no".
To answer the narrow question, for the price comparison website you're probably best grabbing the price in real time, rather then scrapping the database in advance. Hard to imagine that being a problem.
An interesting IRL version of story involving The Harvard Coop:
Coop Calls Cops On ISBN Copiers.
Short answer: No.
On the narrow issue: If a seller says that their prices are secret, I think you have to respect that. I'd contact them and ask if they really don't want price comparison engines like yours to include them, or if the "no trespassing" sign is for technical reasons. If the latter, perhaps they'll provide you with an alternative. If the former, then I'd say too bad, they don't get included, they lose some business, and it's their problem.
Tangential rant: Personally, I get pretty annoyed with companies that make me jump through hoops to find out the price of their products, places that make me call and talk to a salesman so he can give me a hard-sell pitch, or worse, make me give them my phone number so their salesman can call and harass me. I figure that if they're afraid to tell me the price, it probably means that it's too high.
In general: A robots.txt file is like a "No Trespassing" sign. It's the owner's right to say who is allowed on their property. If you think their reasons are dumb, you can politely suggest they take the sign down. But you don't have the right to disregard their wishes. If someone puts a No Trespassing sign on his yard, and I say, "Hey, I just want to take a quick short cut, what's the big deal?" -- Maybe I'm stepping on his prized Bulgarian violet bulbs and destroying a valuable investment. Maybe I'm crossing his people's sacred burial ground and offending their religious sensibilities. Or maybe he's just an ornery jerk. But it's still his property and his right. Oh, and if I fall into the dangerous sinkhole after ignoring the No Trespassing sign, who's to blame? (In America, I could probably still sue him for all he's worth despite the fact that he warned me, but is that right?)
I'm showing some ignorance here, but I always thought a bot was something only sent out by a search engine. Like Google or Yahoo.
Thus, if you wrote an application that searched content on the internet, I wouldn't consider that a search engine bot, which to my knowledge is what robots.txt is trying to block.
But this may just be selective ignorance, because I might do it until the webmaster of that site contacted me and asked me to stop :)
If people make it available to public access, they shouldn't try to put limits on it. Adding a robots.txt file to your site is the equivalent to putting a sign on your lawn that says "Please don't look at me."
As I'm starting to develop for the web, I'm noticing that having a document between the client and myself that clearly lays out what they want would be very helpful for both parties. After reading some of Joel's advice, doing anything without a spec is a headache, unless of course your billing hourly ;)
In those that have had experience,
what is a good way to extract all
the information possible from the
client about what they want their
website to do and how it looks? Good
ways to avoid feature creep?
What web specific requirements
should I be aware of? (graphic
design perhaps)
What do you use to write your specs in?
Any thing else one should know?
Thanks!
Ps: to "StackOverflow Purists" , if my question sucks, i'm open to feed back on how to improve it rather than votes down and "your question sucks" comments
Depends on the goal of the web-site. If it is a site to market a new product being released by the client, it is easier to narrow down the spec, if it's a general site, then it's a lot of back and forth.
Outline the following:
What is the goal of the site / re-design.
What is the expected raise in customer base?
What is the customer retainment goal?
What is the target demographic?
Outline from the start all the interactive elements - flash / movies / games.
Outline the IA, sit down with the client and outline all the sections they want. Think up of how to organize it and bring it back to them.
Get all changes in writing.
Do all spec preparation before starting development to avoid last minute changes.
Some general pointers
Be polite, but don't be too easy-going. If the client is asking for something impossible, let them know that in a polite way. Don't say YOU can't do it, say it is not possible to accomplish that in the allotted time and budget.
Avoid making comparisons between your ideas and big name company websites. Don't say your search function will be like Google, because you set a certain kind of standard for your program that the user is used to.
Follow standards in whatever area of work you are. This will make sure that the code is not only easy to maintain later but also avoid the chances of bugs.
Stress accessibility to yourself and the client, it is a big a thing.
More stuff:
Do not be afraid to voice your opinion. Of course, the client has the money and the decision at hand whether to work with you - so be polite. But don't be a push-over, you have been in the industry and you know how it works, so let them know what will work and what won't.
If the client stumbles on your technical explanations, don't assume they are stupid, they are just in another industry.
Steer the client away from cliches and buzz words. Avoid throwing words like 'ajax' and 'web 2.0' around, unless you have the exact functionality in mind.
Make sure to plan everything before you start work as I have said above. If the site is interactive, you have to make sure everything meshes together. When the site is thought up piece by piece, trust me it is noticeable.
One piece of advice that I've seen in many software design situations (not just web site design) relates to user expectations. Some people manage them well by giving the user something to see, while making sure that the user doesn't believe that the thing they're seeing can actually work.
Paper prototyping can help a lot for this type of situation: http://en.wikipedia.org/wiki/Paper_prototyping
I'm with the paper prototyping, but use iplotz.com for it, which is working out fine so far from us.
It makes you think about how the application should work in more detail, and thus makes it less likely to miss out on certain things you need to build, and it makes it much easier to explain to the client what you are thinking of.
You can also ask the client to use iplotz to explain the demands to you, or cooperate in it.
I also found looking for client questionnaires on google a good idea to help generate some more ideas:
Google: web client questionnaire,
There are dozens of pdfs and other forms to learn from
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I'm working in a small company and weeks away from deploying a web-app that will be used a lot. Everyone at one location will have to learn to use it, and although I think it's pretty easy and intuitive I may be biased.
I've written a help guide with plenty of screenshots that's available on every page, but I'll still need to train everyone. What's the best way? How do you take a step back and explain code you've been working on for weeks?
First try to avoid the training:
Perform usability testing to ensure your web app is intuitive. Usability testing is a very important aspect of testing and it is often ignored. How you see your system will probably be very different as how a new user sees your system.
Also add contextual help as often as you can. For example when I hover over a tag in stack overflow, I know exactly what clicking it will do, because it tells me.
Also this may seem obvious, but make sure you link to your documentation from the site itself. People may not think of looking in your documentation unless its right in front of their eyes.
About training documentation:
Try to split up your material into how your users would use the system. I personally like the "trails" option that Sun created for their Java tutorials. In this tutorial you can do several things, and you can chose on which trail you'd like to go.
Support random reads in your help documentation. If they have a task to do in your web app, then they should be able to get help on that without reading a bunch of unrelated content.
Make sure your documentation is searchable.
About actual training sessions:
If you are actually performing training sessions, stay away from explaining anything related to your code at all. You don't need to know about the engine to drive a car.
Try to split up your training sessions into very focused aspects of your system. If you only have 1 training session available to you then just do one specialized use case of your system + the overall description of the system. Refer to the different parts of documentation where they can get help.
Letting the community help itself:
No matter how extensive your documentation is, you'll always have cases that you didn't cover. That's why it's a good idea to have a forum available to all users of the system. Allow them to ask each other questions.
You can review this forum and add content to your documentation as needed.
You could also open up a wiki for the documentation itself, but this is probably not desirable if your user base isn't very large.
Few ideas:
Do you have some canned walk-through scenarios? Don't know if it is applicable for your product, but I built a pretty substantial product a couple years ago and developed some training modules that they'd work through - nothing long, maybe 15 minutes tops for each one.
I put together a slide presentation that hit the highlights to talk about what it does. I would spend about 10 minutes going through the app's highlights to familiarize them with it before doing the hands-on stuff.
People don't tend to read stuff, unfortunately. You could put hours and hours into a help document, and still find that folks simply don't read it or skim over it. That can be frustrating. Expect that answers that are in your guide will be the topic of questions your users will have.
Break up any training you do into manageable chunks. I've been to a full-day training exercise before and the trainer broke it into short pieces and made it easy for me to get the training topic in my head. You don't want to data-dump on them because their eyes will gloss over and you'll lose them.
Ultimately, if your app is highly usable, it should be a piece of cake. If it isn't, you'll find out. You might want to have a few folks you know run through your training ahead of time and give you constructive criticism on it. Better to fix it before the big group is trained. You'll be more confident in the product and the training materials (whatever they are) and you'll likely have a better training experience.
If applicable, provide an online help/wiki/faq for them. Sometimes that is helpful.
Best of luck!
You should really have addressed this issue a lot earlier in the development cycle than you are doing.
In my view the ideal scenario for corporate software is one where the users design their own application and write their own documentation and I always try to strive for this. You should have identified key users early on and designed the system with them (I try to get my users to do basic screen designs and menu layouts in Excel or similar - then I implement that as static pages and review before writing a line of significant code, obviously they won't get the design right first time, but it's your job to guide them - and ideally in a way where they think they came up with the correct design decisions, not you :-) ).
These users should then write the user documentation from this design in parallel with you developing the system. I have never seen help documentation delivered by a IT department/software company used significantly in a corporate setting. Instead what happens is the users will create their own folder of notes and work-arounds and refer to this (in fact if you're ever doing system analysis to replace an existing system finding the 'user-bible' for the old system is a key strategy). Getting the users to write their own documentation up-front simply harnesses what will happen anyway - but this is vastly easier if the users feel they have ownership of the system because they designed it themselves in the first place.
Of course this approach needs commitment and time from your users, but generally it's not that hard a sell. It's trite, but working as a facilitator so the users can develop there own system rather than as a third party to give them a system pretty much guarantees user acceptance.
As you are where you are you're too late to implement all of this, but if you can identify a couple of keen, key, users and get time from them to write their own documentation then that would be a good move. If you can't get even that then you need to identify an evangelist who you can train to be the 'departmental' expert and give them 110% of your energy to support them.
The bottom line is that user acceptance is based on perception, and this does not necessarily correlate with how usable an system actually is. You have to focus on the group psychology of this as much as the reality of the system, which tends to be tricky for developers as we're much more factually based than most people.
I'll be looking into something like this too in the next few months.
In your case, hopefully the UI has already undergone user acceptance testing. You say you work in a small company. Is it possible to get the least tech-savvy person there to try it out? In fact, get them to try it out without any guidance from yourself except for questions they ask. Document the questions and make sure your user-guide answers them.
The main thing for me would be logic and consistency. If the app's workflow relates logically to the task it has been designed to accomplish and the UI is consistent you should be OK.
Create a wiki page to describe the use of your system. Giving edit rights to the users of your system lets the users:
update the documentation to correct any errors in the initial release of documentation,
share any tips on usage they may have found.
share unusual uses for the system that you may not have thought of.
request features.
provide any workarounds they've found while waiting for the new functionality to be implemented.
Try a few users first, one or two in a small company. Mostly watch, help as little as possible. This tells you what needs to be fixed, and it creates an experienced user base - so you are not the "training bottleneck" anymore.
Turn core requirements/use cases/storycards into HowTo / walkthroughs for your documentation.
For a public training, prepare a 10..15 minute presentation (just that, not more!) that covers key concepts that the users absolutely must understand, than show your core walkthroughs. Reserve extra time for questions about how to solve various tasks.
Think as a user, not as a techie: - noone cares if it's a SQL database and you spent a lot of time to get the locking mechanisms right. They do care about "does it slow me down" and "does something bad happen when two people do that at the same time". Our job is to make complicated things look easy.
It may help to put the documentation on the intranet in an editable form - page "comments", or wiki maybe. And/or put up a "error wiki" for error messages and blips - where you or your users can quickly add recomendations, workarounds and reasons for anything that does not go as expected.
Rather then train all those people I have chosen a few superusers (at least one person from each department) and trained them to teach the rest of the employees. It is of course vital that those super users are
well respected in their departments
able to teach
like the application
The easy way to ensure that they like the app is to have them to define the way it should work :-). Since they should work with this app each and every day they are the prime stakeholders, no matter what management states
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I'm currently trying to build a personal website to create a presence on the web for myself. My plan is to include content such as my resume, any projects that I have done on my own and links to open source projects that I have contributed to, and so on. However, I'm not sure which approach would be better from a perspective of "advertising" myself, since that what this site does, especially since I am a software developer.
Should I use an out-of-the-box system and extend it as needed, with available modules and custom modules where needed or should I custom build a site and all of its features as I need them? Does a custom site look better in the eyes of a potential employer who might visit my site?
I've toyed with this idea in the past but I don't think it's really a good idea for a number of reasons. Firstly, there are a number of places that can take care of most of this without you needing to do the work or maintenance. Just signing up for a linkedIn account for example will allow you to get most of your needs catered for in this regard. You can create your resume there and bio information etc and make it publicly viewable. The other issue with your "own site" is that if you don't update it often, the information gets stale, and worse yet, people have no reason to go back because "nothing has changed" - and that's not much of an advert for you is it?
Now that I've said all that, I'll make another recommendation. Why not start a blog instead?! If you've got decent experience, why not share that. I'd be willing to bet that this will be the best advert for your skills because:
It's always updated (if you post often)
It's not like you're looking for work doing it - but your (future) employer, or their developers will check it out anyway to get a better insight into your character.
Putting something on your resume doesn't mean you can do it. I'm not saying that you'd lie about your skills :-), but there's no argument about your ability when you're writing articles about the stuff, getting comments and feedback, and better yet, learning EVEN MORE about your passions.
Best of all - you can run your blog from your chosen domain and also point to your resume that is stored in linkedIn. Just an idea...
That's my two pennys worth on that - hope it helps you come to a decision!
If you are a web-specific developer I would go with a custom site, but if you focus more on desktop applications or backend technologies, I think an out of the box system would be fine.
A nice looking, default, off the shelf, complete website could be more impressive than a poorly done, broken, tacked together, incomplete website. Perhaps start with something "off the shelf" but nice looking, keep it simple, professional, and then eventually add more custom functionality, style and content. Potential employers may like to see that you are capable of reusing tried and trued solutions instead of trying to create everything from scratch without a good reason. Or you could spend time combining great components into something even better than the sum of the parts, as Jeff Atwood talks about extensively in the Stack Overflow podcasts. Stack Overflow is a good example of writing lots of custom code, but combining that with some of the best Web 2.0 technologies/widgets/etc. into something coherent, instead of trying to prove that they could implement x/y/z from scratch.
(On the other hand, it's really fun to build your own login system, blog, or photo gallery. If you really enjoy it and you want to learn a lot or create something new and different, then go for it!)
Here's what I did (or am currently doing). First, use an out of the box solution to begin with. In my case, I used BlogEngine.NET, which was open source and easy to set up. This allows me to put content on my site as fast as possible. Now, I can continue to use BlogEngine.NET, and skin my site to give it more personality or I can start rolling out my own solution. However, I haven't found a requirement yet that would give me a reason to waste time building my own solution. Odds are you probably won't either.
I don't think it matters if your site is blatantly using a framework or other "generic" solution. The real question is "is it done well, with taste?" If you are using an out of the box solution, you should take the time and pay attention to details when customizing it as if you were creating it from scratch.
Alternatively, if you're looking for a great learning experience and something to spend a lot of your free time on -- write it yourself. But know that you are re-inventing the wheel, and embrace it.
edit
A recent post from 37Signals, Gearheads don't get it, really sums up a good point about not focusing on the technical details, but "content and community".
Reinventing the wheel is not such a great idea when you are building a personal site. Building your own CMS is fun, and to some degree is something to brag about, but not so much features you won't have the time to build and all the security holes that you won't have the time to fix.
It's much better to pick a good, well-established engine, build a custom theme, and contribute a module or two to it: you'll be writing code that you can show off as a code sample and at the same time creating something useful.
Knowing your way around an open source CMS is a good skill in just about any job: when your boss says - hey, we need a three pager site for client/product/person X in 10 hours, you can say - no problem.
For a simpler portfolio site, Wordpress might meet your needs.
You can set up 'static' Wordpress pages for contact information, various portfolios, a resume, etc. This would also give you a blog if you want to do this.
Wordpress does give you the flexibility to "hide" the blogging part of it and use it basically as a simpler CMS. For example, your root URL of example.com could point to a WP static page, while example.com/blog would be the actual blog pages.
If you self-host Wordpress on your own domain (which I really would recommend instead of going through wordpress.com), it should be trivial to set up a few subdomains for extra content. For example, downloads.example.com could host the actual downloads for projects you've developed linked from the Wordpress portfolio pages. Similarly, if you're doing a lot of web work, a subdomain like lab.example.com or samples.example.com could then host various static (or dynamic) pages where you show off sandboxed pages that are not under the control of Wordpress.
Keep in mind though that you'll want to make your page look good. A sloppy looking site can scare away potential clients, even if you are not looking to do any web work for them.
Putting your resume up online somewhere helps, I get a lot of recruitment emails from people who happened on my resume via googling. However I agree with ColinYounger in that you'll probably get more bang for your buck from LinkedIn.
My advice is this - if you want to take the time out to LEARN a CMS or something, to better yourself, then why not make your first project in one be your homepage?
Maybe enlighten us as to the "features" you want to have on a personal homepage? Outside of a link to an HTML resume and perhaps some links to things you like, not sure exactly what the features of a homepage would be...
It really depends on:
a) what services you provide
b) what your skill level is when it comes to web design/development
If you are primarily a web applications developer then running an off the shelf product or using blatantly using DreamWeaver to develop it may not be so smart -- or maybe your clients aren't adept enough to notice?
Likewise if you're primarily a web designer then it is probably a good idea to design your own website.
Just as a side question and following up on my 'ego trip' comment: why would you take anything on the web to be 'true'? IME printed submissions, while not necessarily accurate, tend to be slightly less, erm... exaggerated than web submissions.
Do those responding\viewing ever hire? I wouldn't google for a candidate. I might ego surf for a respondent, but would ignore CVs.
Rounding back to the OP, I would suggest that you need to SHOW what you're good at - participate in Open Source projects and POST on their forums, link to projects you can post details of and generally try to show what a Good Employee you could be. Just telling me that you're good at [insert latest trend here] means diddly.
I have come to see that the best way to advertise yourself is to put quality content out there. If you write about the technology that you have experience in, maybe create a few tutorials, and if you do all that often enough, that shows some authority in your chosen field of work.
This alone is one of the best advertisements. However, you also want to show passion. And online, that can be shown through how meticulously your site is done (it doesn't have to be a super great UI or something), but it should be neat, clean, and professional. It doesn't matter if its out of the box, or custom designed.
Either way, you will have to work hard to make it look good.