I want to exclude all subfolders named "ajax" in any folder from indexing by search engines.
Examples:
.com/a/ajax
.com/b/ajax
.com/c/ajax
Is this possible via robots.txt ?
It's only possible if you list out each folder explicitly. There is no wildcard support to accomplish the type of thing you want. The robots.txt exclusion standard is a little lacking in this respect.
Related
Visual Studio Code (as of version 1.41.1) is obviously very limited in regard of its file search. It seems to only allow to either search in folders recursively or in specific files, but it doesn't allow both.
Search in folders recursively
path/to/folder/ searches in any directories within subpaths matching path/to/folder including all subdirectories with no restriction in file names.
./path/to/folder/, ./path/to/another/folder searches in the directories with the paths path/to/folder and path/to/another/folder relative to the project's root directory.
Search in files
foo.bar searches in all files named foo.bar.
*.foo, *.bar searches in all files with the extensions foo or bar.
./path/to/folder/*/*.foo searches in all files with the extension foo that lie in a direct subdirectory of path/to/folder/ relative to the project's root directory.
Search in folders recursively and filter by file name
So, how to combine these two searches, i.e. filter the search by file names but search in specific directories with all their subdirectories?
In other editors like Eclipse you normally have two different fields for file names and folders, making it easy to specify them individually and avoid having to repeat yourself for multiple folders and file names. Therefore I have already created an enhancement request in the VSCode bug tracker asking to add a separate field for the folder.
In my testing, using the globstar does provide the functionality you desire.
https://github.com/isaacs/node-glob#glob-primer:
** If a "globstar" is alone in a path portion, then it matches zero
or more directories and subdirectories searching for matches. It does
not crawl symlinked directories.
So that ./path/to/folder/**/*.foo for example searches within all subdirectories of folder no matter how deep within files with the foo extension.
Same at https://code.visualstudio.com/docs/editor/codebasics#_advanced-search-options:
** to match any number of path segments, including none
I have mutiple folders I want to include in doxywizard. Any Idea how can I do that? Currently If I select folder with multiple subfolder in it and when I run doxygen, It is not showing me any output.
When having specified just folders in the INPUT tag the files here are handled but not the files in subdirectories. For the later ones one needs the RECURSIVE tag (from the documentation):
RECURSIVE
The RECURSIVE tag can be used to specify whether or not subdirectories should be searched for input files as well.
The default value is: NO.
When searching, VS Code has the ability to list files to include to scope the search. This is used by default when using the "find in folder" feature. For example, searching src results in ./src as the files to include.
Is there a syntax I can use to list multiple directories here? For example, I want to search ./src and ./lib in one search.
Did you try a comma like ./dir1, ./dir2? For me it seems to work
By the way, here is the documentation of 'files to include': https://code.visualstudio.com/Docs/editor/codebasics#_advanced-search-options
In particular, you can use glob notation. Also, VS Code will include/exclude certain directories or files by default, depending on your settings.json, in case anyone still sees unexpected behaviour.
is there a way to get a file list recursively based on one file collection that points to a directory in fileadmin?
Currently I only got it to work with files directly in that directory, not also with files in sub-directories of that directory.
So instead of setting lots of file collections for each (sub directory)
I'd like to set only the "top"level directory (here "Kurs77") and have the files, even from sub directories, displayed.
Reason is, editors may add an unknown amount of (sub)sdirectories, and I'd like to have the files automagically displayed in the file list in the front end -- without the need to create an increasing amount of file collections.
cheers,
Tom
it seems that this is a missing feature. Check out https://forge.typo3.org/issues/61238. It seems that the underlaying API is able to do that.
So one solution would be to use TypoScript to make that work.
To give the correct answer now: The recursive option is of course available but it is part of the sys_file_collection record.
In TYPO3 9 this is working out of the box. pity is not showing folder as title, but recursive works:
I know I can block search engines from accessing types of files using a wild card like this:
Disallow: /*.gif$
That disallows access to gifs, or more like files ending in .gif.
But is there a way to prevent search engines from accessing for example all files starting with "_"?
Would something like this work?
Disallow: /_*.*$
Or at least perhaps this (if I absolutely need to set an extension)?
Disallow: /_*.php$
As per the "official" docs
Note also that globbing and regular expression are not supported in either the User-agent or Disallow lines.