I've recently set up my robots.txt file.
Nothing special - I have an asp.net site, so in addition to my pages, web.config etc. in the root directory, there are links to /bin/ and some custom folders which are there for organisation rather than site structure.
At the moment I'm disallowing each directory individually, but I was wondering if it's possible to disallow all child directories from the root in one go.
I've taken a look at the documentation but not found the answer I want just yet - often it's not there because it's not possible.
Anyone advise as to whether this is do-able?
This can be done with wildcards:
User-agent: *
Disallow: /*/
Be aware that wildcards are an extension, and are not part of the original robots.txt standard. They are supported by all major search engines, but they are not supported by many special-purpose crawlers.
If you only care about major search engines, and you need to block hundreds or thousands of directories, then wildcards are a good solution. If you only need to block a handful of directories, then you are better off listing them individually.
Related
This is a query from a beginner wondering about best practice suggestions - I have a system for myself but am wondering what other more experienced people do.
My desired end result is a way to access a number of specified files which are in multiple different places within a working directory.
My specific example is with Django, where my HTML, CSS & JS files are spread in different directories (such as Template & Static). What I'm imagining is a way to quickly access/open all three files, even if they're in different locations - possibly creating a "ProjectFoo" folder, then within that folder are links/shortcuts/aliases to the required files.
I'm currently using the Favorites (https://marketplace.visualstudio.com/items?itemName=kdcro101.favorites) which covers the majority of my uses, and there are also these alternatives:
VSCode-Favorites: https://marketplace.visualstudio.com/items?itemName=howardzuo.vscode-favorites
Project Manager: https://marketplace.visualstudio.com/items?itemName=alefragnani.project-manager
...but am wondering aobut recommendations regarding best practice - what is the usual way people go about this, as it seems (to me at least) that it might be a common workflow issue.
Thanks!
Rich
There
Dokuwiki, an excellent wiki tool, but it fails to support filename which contains uppercase letter(s) when upload attachment.
Any idea to solve this?
DokuWiki aims to be as portable as possible. This means it allows you to move your existing wiki data between operating systems and file systems without problems. To achieve this (and cope with different file systems being case sensitive or not) it enforces some naming standards (all files being lower case being one of them).
When you upload the file through DokuWiki's media manager it will automatically be renamed to comply to the naming standards used within DokuWiki. If you upload via the file system you have to make sure your files comply manually.
I know that Eclipse offers a way to map custom folders on my HDD (for example, C:\Projects\Lol\ if working under Windows) as the folders of my project.
This can be achieved using the Link the folder in the file system option when performing New->Folder action.
However, I didn't find how to map the folders with relative (not absolute) paths. Is this possible?
I'm confused, because if mapping the folders with relative paths (say, ../Lol/Include) is impossible, this makes sharing projects with such folders among my team impossible.
How should I solve this?
Thank you.
I usally achieve this behavior by having a symbolic link on file-system level and adding it to the code repository. I don't know if it is THE way to go, but works perfectly well for me.
Any eclipse-only link is stored in the eclipse meta files and I doubt it's better checking them in because they will usually differ slightly from user to user.
hey everyone, we just added a nightly action to process the entire source tree with doxygen and place the output onto development webserver.
We also already have a sharepoint structure which holds design documents for various modules/projects. Currently, the level at which we are keeping this documentation is relatively high. We discuss structures of modules and talk about the major classes, but never go down to the individual method level. I wanted to bridge that gap by having hyperlinks in the SDS word documents that would point to doxygen output.
I noticed the links look like this:
http://example.com/docs/ProjectName/d4/d98/class_c_reader.html
http://example.com/docs/ProjectName/d4/d16/class_c_stream.html
The part that sketches me out a bit is "d4", "d98" and "d16" strings in the path. If I copy these links and create the hyperlinks, does anyone know if these URLs are guaranteed be preserved in the future. As I said, entire doxygen output gets regenerated nightly.
You can disable the d4/d98 subdirectories by disabling CREATE_SUBDIRS in the doxygen configuration.
Whether the name of the HTML files will stay the same I do not know for sure but from what I have seen when using doxygen it seems so. If you want to know for sure you can always look at the doxygen source.
Probably these links will not stay permanent.
Furthermore, Doxygen has a XML representation of the generated documentation but even this interface resp. the corresponding DSD has been changed with new releases of doxygen. This is quite frustrating, as we had used the XML representation for a similar application with the assumption that the structures would be kept identical with every new release.
What's are the best practices for versioning web sites?
Which revision control systems are well suited for such a job?
What special-purpose tools exist?
What other questions should I be asking?
Firstly you can - and should - use a revision control system, most will handle binary files although unlike text files you can't merge two different set of changes so you may want to set the system up to lock these files whilst they are being changed (assuming that that's not the default mode of operation for you rcs in the first place).
Where things get a bit more interesting for Websites is managing those files that are required for the site but don't actually form part of the site - the most obvious example being something like .psd files from which web graphics are produced but which don't get deployed.
We therefore have a tree for each site which has two folders: assets and site. Assets are things that aren't in the site, and site is - well the site.
What you have to watch with this is that designers tend to have their own "systems" for "versioning" graphic files (count the layers in the PSD). You don't need necessarily to stop them doing this but you do need to ensure that they commit each change too.
Other questions?
Deployment. We're still working on this one (-: But we're getting better (I'm happier now with what we do!)
Murph
In response to Christian Lescuyer's post, you also need to enable the "svn:keywords" property on the file with that line in it. Subversion won't bother looking in your files for keywords like $Revision$ unless that property is set.
Also, if using PHP like in his example, you may want to put $Revision$ inside a single-quoted string instead of a double quoted string to prevent PHP from trying to parse $Revision as a PHP variable and throwing a warning. :)
I use Subversion.
As an easy way to reference the website version (production, testing, development), I use a very simple trick. I add the revision number somewhere on the site (eg in the admin footer). Something like this:
<?php print("$Revision: 1 $"); ?>
Each time you checkout (development versions) or export (for production), the "1" will be replaced by the revision number in your repository, thus making it easy to setup the customer version on your test server, for example.