I wanted to see how magics are defined from some sample source but did not find info from web.
Moreover, where should the IPython extensions reside?
To understand how to deploy your custom magic you need to read 2 sectinos of IPython's documentation:
How to write your own magic
How to write an extension
Then you will see that generally a magic is simply a class that define magics methods in Python module.
Here is for example a self contained IPython magic as an extension, the code is not the best, but just to show you it can be small.
As to where the IPython's builtin's one are defined, they all are in this folder, though I would not recommend to take them as an example, as there is a lot of historical baggage and can be quite complex.
A more recent full example and will appear in the docs once next version of IPython is released.
I need to use a PLT-scheme library that only runs on a very old version of DrScheme (DrScheme v103p1), which I think is from at least 15 years ago.
The problem I am having is that I can't figure out how to use the library in my code because aparently the "require" function from modern racket didn't exist back then. What do I have to do to use a library then? All I know for now is that the file containing the definitions I want resides on "C:\Program Files\PLT\collects\mzlib\spidey.ss".
I tried using (require "spidey.ss") and (require "spidey") but both failed with a reference to unidentified identifier: required error.
You can use require-library to load things from the standard library:
http://download.plt-scheme.org/doc/103p1/html/mzscheme/node157.htm
(require-library "spidey.ss")
You can also use the support facilities (load and friends) to load single files. You need to use absolute paths though or else its going to search relative to your current working directory.
http://download.plt-scheme.org/doc/103p1/html/mzscheme/node149.htm
(load "C:/Program Files/PLT/collects/mzlib/spidey.ss")
Since MrSpidey is integrated into DrScheme, open DrScheme and then follow these instructions:
http://download.plt-scheme.org/doc/103p1/html/mrspidey/node4.htm
I work on C++ projects, and I went through Alex Ott's guide to CEDET and other threads about tags in StackOverflow, but I am still confused about how Emacs interfaces with these different tag systems to facilitate autocompletion, the looking up of definitions, navigation of source code base or the previewing of doc-strings.
What is the difference (e.g. in terms of features) between etags, ebrowse, exuberant ctags, cscope, GNU Global and GTags? What do I need to do to use them in Emacs?
Do I need semantic/senator (CEDET) if I want to use tags to navigate/autocomplete symbols?
What does semantic bring to the table on top of these different tag utilities? How does it interface with these tools?
That's as a good question as I've recently read here, so I'll try explain the difference in more detail:
Point 1:
etags and ctags both generate an index (a.k.a. tag/TAGS) file of language objects found in source files that allows these items to be quickly and easily located by a text editor or other utility. A tag signifies a language object for which an index entry is available (or, alternatively, the index entry created for that object). The tags generated by ctags are richer in terms of metadata, but Emacs cannot interpret the additional data anyways, so you should consider them more or less the same (the main advantage of ctags would be its support for more languages). The primary use for the tags files is looking up class/method/function/constant/etc declaration/definitions.
cscope is much more powerful beast (at least as far as C/C++ and Java are concerned). While it operates on more or less the same principle (generating a file of useful metadata) it allows you do some fancier things like find all references to a symbol, see where a function is being invoked, etc (you can find definitions as well).
To sum it up:
ctags one allows you to navigate to symbol declaration/definitions (what some would call a one-way lookup). ctags is a general purpose tool useful for many languages.
On the other hand (as mentioned on the project's page) cscope allows you to:
Go to the declaration of a symbol
Show a selectable list of all references to a symbol
Search for any global definition
Functions called by a function
Functions calling a function
Search for a text string
Search for a regular expression pattern
Find a file
Find all files including a file
It should come as no surprise to anyone at this point, that when I deal with C/C++ projects I make heavy use of cscope and care very little about ctags. When dealing with other languages the situation would obviously be reversed.
Point 2.
To have intelligent autocompletion you need a true source code parser (like semantic), otherwise you won't know the types of the objects (for instance) in your applications and the methods that can be invoked on them. You can have an autocompletion based on many different sources, but to get the best results you'll ultimately need a parser. Same goes for syntax highlighting - currently syntax highlighting in Emacs major modes is based simply on regular expressions and that's very fragile and error prone. Hopefully with the inclusion of semantic in Emacs 23.2 (it used to be an external package before that) we'll start seeing more uses for it (like using it to analyse a buffer source code to properly highlight it)
Since Emacs 24.1 semantic is usable from the Emacs completion framework. The easiest way to test it is to open up a C source code file and typing M-TAB or C-M-i and watch as semantic automagically completes for you. For languages where semantic is not enabled by default, you can add it the following line to your major mode hook of choice:
(add-to-list 'completion-at-point-functions 'semantic-completion-at-point-function)
Point 3.
semantic brings true code awareness (for the few languages it currently supports) and closes the gap between IDEs and Emacs. It doesn't really interface with tools like etags and cscope, but it doesn't mean you cannot use them together.
Hopefully my explanations make sense and will be useful to you.
P.S. I'm not quite familiar with global and ebrowse, but if memory serves me they made use of etags.
I'll try to add some explanations to 1.
What is it?
Etags is a command to generate 'TAGS' file which is the tag file for Emacs. You can use the file with etags.el which is part of emacs package.
Ctags is a command to generate 'tags' file which is the tag file for vi. Universal Ctags, the successor of Exuberant Ctags, can generate 'TAGS' file by the -e option, supporting more than 41 programming languages.
Cscope is an all-in-one source code browsing tool for C language. It has own fine CUI (character user interface) and tag databases (cscope.in.out, cscope.out, cscope.po.out). You can use cscope from Emacs using xcscope.el which is part of cscope package.
GNU GLOBAL is a source code tagging system. Though it is similar to above tools, it differs from them at the point of that it is dependent from any editor, and it has no user interface except for command line. Gtags is a command to generate tag files for GLOBAL (GTAGS, GRTAGS, GPATH). You can use GLOBAL from emacs using gtags.el which is part of GLOBAL package. In addition to this, there are many elisp libraries for it (xgtags.el, ggtags.el, anything-gtags.el, helm-gtags.el, etc).
Comparison
Ctags and etags treat only definitions. Cscope and GNU GLOBAL treat not only definitions but also references.
Ctags and etags use a flat text tag file. Cscope and GNU GLOBAL use key-value tag databases.
Cscope and GNU GLOBAL have a grep like search engine and incremental updating facility of tag files.
Combination
You can combine Universal Ctags's rich language support and GNU GLOBAL's database facility by using ctags as a plug-in parser of GLOBAL.
Try the following: (requires GLOBAL-6.5.3+ and Universal Ctags respectively)
Building GNU GLOBAL:
$ ./configure --with-universal-ctags=/usr/local/bin/ctags
$ sudo make install
Usage:
$ export GTAGSCONF=/usr/local/share/gtags/gtags.conf
$ export GTAGSLABEL=new-ctags
$ gtags # invokes Universal Ctags internally
$ emacs -f gtags-mode # load gtags.el
(However, you cannot treat references by this method, because ctags don't treat references.)
You can also use cscope as a client of GNU GLOBAL. GLOBAL package includes a command named 'gtags-cscope' which is a port of cscope, that is, it is cscope itself except that it use GLOBAL as a search engine instead of cscope's one.
$ gtags-cscope # this is GLOBAL version of cscope
With the combinations, you can use cscope for 41 languages.
Good luck!
TAGS files contain definitions
A TAGS file contains a list of where functions and classes are defined. It is usually placed in the root of a project and looks like this:
^L
configure,3945
as_fn_success () { as_fn_return 0; }^?as_fn_success^A180,5465
as_fn_failure () { as_fn_return 1; }^?as_fn_failure^A181,5502
as_fn_ret_success () { return 0; }^?as_fn_ret_success^A182,5539
as_fn_ret_failure () { return 1; }^?as_fn_ret_failure^A183,5574
This enables Emacs to find definitions. Basic navigation is built-in with find-tag, but etags-select provides a nicer UI when there are multiple matches.
You can also uses TAGS files for code completion. For example, company's etags backend uses TAGS files.
TAGS files can be built by different tools
ctags (formerly known as 'universal ctags' or 'exuberant ctags') can generate TAGS files and supports the widest range of languages. It is actively maintained on github.
Emacs ships with two programs that generate TAGS files, called etags and ctags. Emacs' ctags is just etags with the same CLI interface as universal ctags. To avoid confusion, many distros rename these programs (e.g. ctags.emacs24 on Debian).
There are also language specific tools for generating TAGS files, such as jsctags and hasktags.
Other file formats
ebrowse is a C program shipped with Emacs. It indexes C/C++ code and generates a BROWSE file. ebrowse.el provides the usual find definition and completion. You can also open the BROWSE file directly in Emacs to get an overview of the classes/function defined a codebase.
GNU Global has its own database format, which consists of a GTAGS, GRTAGS and GPATH file. You can generate these files with the gtags command, which parses C/C++ code. For other languages, GNU Global can read files generated by universal ctags.
GNU Global also provides a CLI interface for asking more sophisticated questions, like 'where is this symbol mentioned?'. It ships with an Emacs package gtags.el, but ggtags.el is also popular for accessing GNU Global databases.
Cscope is similar in spirit to GNU Global: it parses C/C++ into its own database format. It can also answer questions like 'find all callers/callees of this funciton'.
See also this HN discussion comparing global and cscope.
Client/Server projects
rtags parses and indexes C/C++ using a persistent server. It uses the clang parser, so it handles C++ really well. It ships with an Emacs package to query the server.
google-gtags was a project where a large TAGS file would be stored on a server. When you queried the server, it would provide a subset of the TAGS file that was relevant to your search.
Semantic (CEDET)
Semantic is a built-in Emacs package that contains a parser for C/C++, so it can find definitions too. It can also import data from TAGS files, csope databases, and other sources. CEDET also includes IDE style functionality that uses this data, e.g. generating UML diagrams of class hierarchies.
[answer updated from shigio's]
I'll try to add some explanations to part 1 of the question.
What is it?
Etags generates a TAGS file which is the tag file format for Emacs. You can use an Etags file with etags.el which is part of Emacs.
Ctags is the generic term for anything that can generate a tags file, which is the native tag file format for Vi. Universal Ctags (aka UCtags, formerly Exuberant Ctags) can also generate Etags with the -e option.
Cscope is an all-in-one source code browsing tool for C (with lesser support for C++ and Java), with its own tag databases (cscope.in.out, cscope.out, cscope.po.out) and TUI. Cscope support is built-in to Vim; you can use Cscope from Emacs using the xcscope.el package. There are also Cscope-based GUIs.
GNU GLOBAL (aka Gtags) is yet another source code tagging system (with significant differences--see next section), in that it also generates tag files.
Comparison
Ctags and Etags treat only definitions (of, e.g., variables and functions). Cscope and Gtags also treat references.
Ctags and Etags tag files are flat. Cscope and Gtags tagfiles are more powerful key-value databases, which allows (e.g.) incremental update.
Cscope and Gtags have a grep-like search engine.
Ctags has built-in support for more languages and data formats: see the current-in-repository list of Universal Ctags parsers. UCtags also has documented how to develop your own parser.
Cscope and Gtags are editor-independent.
Gtags does not provide its own user interface, but can currently (Oct 2016) be used from commandline (CLI), Emacs and relatives, Vi and relatives, less (pager), Doxygen, and any web browser.
Gtags provides gtags.el via the GLOBAL package, but there are also many other elisp extensions, including xgtags.el, ggtags.el, anything-gtags.el, helm-gtags.el.
Combination
You can combine Universal Ctags' rich language support with Gtags' database facility and numerous extensions by using Ctags as a GLOBAL plug-in parser:
# build GNU GLOBAL
./configure --with-exuberant-ctags=/usr/local/bin/ctags
sudo make install
# use it
export GTAGSCONF=/usr/local/share/gtags/gtags.conf
export GTAGSLABEL=ctags
gtags # invokes Universal Ctags internally
emacs -f gtags-mode # load gtags.el
Note again that if you use Ctags as the parser for your Gtags, you lose the ability to treat references (e.g., variable usage, function calls) which Gtags would otherwise provide. Essentially, you trade off Gtags' reference tracking for Ctags' greater built-in language support.
You can also use Cscope as a client of Gtags: gtags-cscope.
Good luck!
I haven't actually checked, but according to CEDET manual (http://www.randomsample.de/cedetdocs/common/cedet/CScope.html):
semantic can use CScope as a back end for database searches. To enable it, use:
(semanticdb-enable-cscope-databases)
This will enable the use of cscope for all C and C++ buffers.
CScope will then be used for project-wide searches as a backup when pre-existing semantic database searches may not have parsed all your files.
My emacs version is 23.2.1
Although I used the following to change the backend to use GTAGS.
(require 'semantic/db-global)
(semanticdb-enable-gnu-global-databases 'c-mode)
(semanticdb-enable-gnu-global-databases 'c++-mode)
I tried to generate a GTAGS file to be used as a backend of semantic.
However, everytime when I open a C file, Semantic is still parsing files without using the GTAGS file.
Is it possible to use the GTAGS file instead of the built-in parser of semantic? I found that the built-in parser is not very accurate.
Is it possible to use the GTAGS file without specifying the project scope? In my case, I tried to put GTAGS file in /usr/include which should be the standard include path of emacs. But Semantics is not using it.
Semantic doesn't use the GTAGS file for generating tags that it will use directly for features such as jumping or smart completion. The GNU Global backend to semantic db will use GTAGS instead as a giant name table. Thus if you need to find a symbol by name, GTAGS will tell semantic where it is, and Semantic will then parse those files itself more directly to get the details.
The reason GTAGS is not used directly as a replacement parser is because the information in GTAGS is insufficient for the kinds of operations Semantic needs, as it excludes datatype info, argument parsing, and local context parsing.
Using gtags in /usr/include is an interesting idea, but would probably not get used much. Semantic will only search header files actually used in your code instead of searching all include files. The GNU Global backend is specifically for scanning an entire project for a symbol, or symbol references. If you wanted to search all your includes for a symbol, then it would be useful, but there is no such feature in Semantic at this time.
It is possible to use ebrowse to do what you want, but I found that for C++, the parser was insufficient in subtle ways, and caused some problems.
If you think the Semantic parser is inaccurate, then you should post that as a bug on the cedet-devel mailing list.
Is there a way from WinDbg, without using the DbgEng API, to display the symbol server paths (i.e. PdbSig70 and PdbAge) for all loaded modules?
I know that
lml
does this for the modules whose symbols have loaded. I would like to know these paths for the symbols that did not load so as to diagnose the problem. Anyone know if this is possible without having to utilize the DbgEng API?
edited:
I also realize that you can use
!sym noisy
to get error messages about symbols loading. While this does have helpful output it is interleaved with other output that I want and is not simple and clear like 'lml'
!sym noisy and !sym quiet can turn on additional output for symbol loading, i.e.:
!sym noisy
.reload <dll>
X <some symbol in that DLL to cause a load>
!sym quiet
When the debugger attempts to load the PDB you will see every path that it tries to load and if PDB's weren't found or were rejected.
To my knowledge there's no ready solution in windbg.
Your options would be to either write a nifty script or an extension dependent on where you're the fittest.
It is pretty doable within windbg as a script. The information you're after is described in the PE debug directory.
Here's a link to the c++ sample code that goes into detail on extracting useful information (like the name of the symbol file in your case). Adapting it to windbg script should be no sweat.
Here's another useful pointer with tons of information on automating windbg. In particular, it talks about ways of passing arguments to windbg scripts (which is useful in your case as well, to have a common debug info extraction code which you can invoke from within the loaded modules iteration loop).
You can use the command
lme
to show the modules that did not have any symbols loaded.
http://ntcoder.com/bab/tag/lme/