how to extend existing LSP language server to derivative language? - visual-studio-code

VS Code has great Bash language support -- syntax highlighting (via shellscript), LSP support (via Bash-IDE), linting (via vscode-shellcheck), and a debugger (via bash debug). There is a Bash-based TAP-compliant unit test framework called BATS that also has syntax highlighting support for some new BATS-specific keywords (via bats). I'd primarily like to marry the Bash LSP with the BATS syntax language so that I can get individual tests listed in the VS Code outline panel, but I'd also be potentially interested in shellcheck linting as well (though that is a bigger project, I think).
There are pieces of all of this architecture that are extensively documented, but I haven't found anything that documents how all of these work together, let alone how to then derive off of them. A fundamental question therefor is: can I define a new language as an extension of an existing one such that I pick up all of this language support for free, albeit with different/added keyword definitions? I guess each piece has their own definition of the different syntax elements (e.g. how a "function" is defined), so perhaps I need to override each one of them individually somehow?

Related

Go to implementation instead of TypeScript declaration

When I click an imported variable while holding Cmd on MacOS in VSCode (or Ctrl on other platforms), I often end up looking at the TypeScript declaration of that variable.
Is there any way to have VSCode take me to the definition of it instead?
I don't use TypeScript myself, so the feature isn't helpful to me right now.
Try Go to source definition
This command will try to jump to the original JavaScript implementation of a function/symbol, even for code under node_modules.
JavaScript is a very dynamic language though, so we can't figure out the source location in every case. If you aren't getting results for a common library, please file an issue against TypeScript so we can investigate adding support
For faster and more accurate results, libraries can bundle declaration maps that map from .d.ts files back to source .ts (or .js) files. However many libraries currently do not include these
I found a simple solution for this after a lot of searching.
You just need to add "typescript.disableAutomaticTypeAcquisition": true to your project's settings.json (or vscode's global settings).
This will disable the automatic generation of TypeScript definitions and restore the original "Jump to" behaviour of going to the implementation.
Source:
https://ianwalter.dev/jump-to-source-definition-instead-of-typescript-definition-in-vs-code (archive.org link)
The author provided the wrong instructions though (false when it should have been true so be careful when you read the post. Re-installing node modules was also not needed.
VSCode was updated to include a new option Go to Source Definition. If the ts source is available and ts is upgraded to > 4.7 and VSCode to > 1.67 it should work.
Many library authors do not include the ts source code unfortunately. The package often only consists of the compiled *.js files and the *.d.ts definition files. That makes this new feature of VSCode useless for these packages unfortunately.
This is the original issue:
https://github.com/microsoft/TypeScript/issues/6209
And this is an issue for feedback on the new feature.
https://github.com/microsoft/TypeScript/issues/49003
Implementation is bundled and transpiled ro javascript and vscode is not able to take you there but instead of it will take you to interface. You can search for references in javascript file or you can clone or form the repo to see the implementation in typescript.
As other answers have already stated,
Regardless of any of your tsconfig and whether the package you are requiring/importing things from provides type declaration files or whether you installed a Definitely Typed package for it or not, you can use the TypeScript: Go to Source Definition command to go to the symbol definition in the JS file. This functionality is provided TypeScript and the vscode.typescript-language-features extension (which is built-into / ships out-of-box with VS Code).
I thought I'd try to give more interesting information that other answers haven't covered yet for fun and profit curiosity's sake (and also explain why this "often" happens to you, but not always):
You can bind that command to a keybinding. It's keybinding command ID is typescript.goToSourceDefinition.
If the package you require or import packages its own type declaration files or you installed a community-maintained type declaration file from the Definitely Typed project, then ctrl+clicking / cmd+clicking into the require/import argument or putting the caret on it and invoking whatever the editor.action.revealDefinition command or editor.action.goToTypeDefinition are bound to (F12 by default for editor.action.revealDefinition) will take you to the type declaration by default.
If the package you require or import doesn't package its own type declarations and you didn't install a types package from the Definitely Typed project, and you modify your tsconfig or jsconfig to set allowJs: true and maxNodeModuleJsDepth: <N>, then ctrl+clicking / cmd+clicking into the require/import argument or putting the caret on it and invoking whatever the editor.action.revealDefinition command or editor.action.goToTypeDefinition command are bound to (F12 by default for editor.action.revealDefinition) will take you to the symbol's definition in the JS file by default (unless you already performed this action at a point when a type declaration file declaration types for symbol was available and have not since reloaded/restarted VS Code or edited your tsconfig/jsconfig file, because it will cache that association in memory (smells like minor bug, but ¯\_( ツ )_/¯)).
The editor.action.revealDeclaration keybinding seems to do nothing here (at the time of this writing). I guess that keybinding is more for languages like C and C++.
Some loosely related release notes sections and user docs (non-exhaustive list (I don't get paid to do this)):
https://code.visualstudio.com/docs/editor/editingevolved#_go-to-definition
https://code.visualstudio.com/updates/v1_13#_go-to-implementation-and-go-to-type-definition-added-to-the-go-menu
https://code.visualstudio.com/updates/v1_35#_go-to-definition-improvements
https://code.visualstudio.com/updates/v1_67#_typescript-47-support
In TypeScript's GitHub repo: Go To Source Definition feedback thread #49003
https://code.visualstudio.com/updates/v1_68#_go-to-source-definition
Quoting from that last one:
One of VS Code's longest standing and most upvoted feature requests is to make VS Code navigate to the JavaScript implementation of functions and symbols from external libraries. Currently, Go to Definition jumps to the type definition file (the .d.ts file) that defines the types for the target function or symbol. This is useful if you need to inspect the types or the documentation for these symbols but hides the actual implementation of the code. The current behavior also confuses many JavaScript users who may not understand the TypeScript type from the .d.ts.
While changing Go to Definition to navigate to the JavaScript implementation of a symbol may sound simple, there's a reason why this feature request has been open for so long. JavaScript (and especially the compiled JavaScript shipped by many libraries) is much more difficult to analyze than a .d.ts. Trying to analyze all the JavaScript code under node_modules would be both slow and would also dramatically increase memory usage. There are also many JavaScript patterns that the VS Code IntelliSense engine is not able to understand.
That's where the new Go to Source Definition command comes in. When you run this command from either the editor context menu or from the Command Palette, TypeScript will attempt to track down the JavaScript implementation of the symbol and navigate to it. This may take a few seconds and we may not always get the correct result, but it should be useful in many cases.
See also: https://www.typescriptlang.org/docs/handbook/release-notes/typescript-4-7.html#go-to-source-definition.

Is it possible/easy to build a VS Code extension that does syntax highlighting with a lexer?

I am building an experimental lexer generator and I think it would be cool to output simple syntax highlighters for VS Code. The input grammar goes through the classic regular language -> NFA -> DFA transformation, then generates state machine code (it also has some unconventional features to support nested languages). Converting all this back into tmlanguage definitions is a complicated problem, and I'm starting to wonder if a VS Code extension is a better option. The question is:
Are VS Code syntax highlighting internals completely tied to the tmlanguage regex scanner, or would it be possible to write an extension that provides tokens / highlight ranges programmatically?
Is there an API that would make this reasonably straightforward, or would this project be a tour de force?
As of VSCode 1.15, you have to use textmate grammars for syntax highlighting. There's an feature request open that tracks what you are after: https://github.com/Microsoft/vscode/issues/1967

Filter Eclipse CDT textual search results by program-structural features

I want to use Eclipse CDT to search for all occurrences of regular expresion, say foo(_bar_)*baz - in the bodies/bodies and declarations of functions/methods meeting a certain criterion. For the example let's make it all functions/methods named ignore_me (but within all classes and namespaces).
Is that possible somehow?
I'm not aware of a way to do this sort of search in Eclipse CDT.
Clang's AST matchers provide a rich domain-specific language for expressing queries like yours (and many other kinds), but using them requires writing code (in a clang plugin or standalone clang-based tool). It may be relatively straightforward to write a simple tool that allows you to write a query and searches a codebase for matches. I'm not sure whether such a tool is already available.

Lisp native functions source codes [duplicate]

In C, if I want to see a function that how to work, I open the library which provides the function and analyze the code. How can be implementations of the lisp functions seen? For example, intersection function
You can also look at the source code of lisp functions.
For example, the source files for CLISP, one Common Lisp implementation, are available here: http://www.clisp.org/impnotes/src-files.html
If you want to examine the implementation of functions related to lists, you can look at the file: http://clisp.cvs.sourceforge.net/viewvc/clisp/clisp/src/list.d
The usual answer is "M-."
Assuming you have a properly configured IDE, and the source code of the function, clicking on its name and pressing M-. (that's Meta, or Alt or Option or Escape, and dot/period; or whatever key your IDE uses) should reveal its definition (or, for a generic function, definitions, plural; including any compiler macros that might optimize out some cases). Sometimes it's on a right-click or other mouse menu or toolbar.
If the source isn't available, you can often see the actual compiled form by evaluating (disassemble 'function)
Most IDE's, including perennial favourite Emacs+Slime, have other Inspection operations on the menu as well.
In a non-IDE environment, most compilers have reflection tools of their own (compiler-dependant) which are usually also mapped by the Swank library that Slime uses; one might find useful function in that package.
And this really should be documented in your IDE's manual.
I should postscript this that:
You really shouldn't care about the implementation of the core library functions; their contractual behavior is very well documented in the CLHS standard, which is available online and eg, Quicklisp has an utility to link it to Slime (C-c C-d h on a symbol in the COMMON-LISP package); for all well-written Lisp libraries, there should be documentation attached to functions, variables, classes, etc. accessible via the documentation function in the REPL or the IDE's menus and Inspection windows.
Core library functions are often highly optimized and far more complex than most user-level code should want to be, and often call down into compiler-specific "guts" that one should avoid doing in application code.

Xtext: grammar for language with significant/semantic whitespace

How can I use Xtext to parse languages with semantic whitespace? I'm trying to write a grammar for CoffeeScript and I can't find any good documentation on this.
Here's an example whitespace sensitive language in XText
AFAIK, you can't.
In case of parsing Python-like languages, you'd need the lexer to emit INDENT and DEDENT tokens. For that to happen, you'd need semantic predicates to be supported inside lexer rules (Xtext's terminal rules) that would first check if the current-position-in-line of the next character int the input equals 0 (the beginning of the line) and is a ' ' or '\t'.
But browsing through the documentation, I don't see this is supported by Xtext at the moment. Since Xtext 2.0, support has been added for semantic predicates in production rules (see: 6.2.8. Syntactic Predicates), but not in terminal rules.
The only way to do this with Xtext would be to let the lexer produce terminal spaces and line-breaks, but this would make an utter mess of your production rules.
If you want to parse such a language using Java (and a Java oriented parser generator) I'd recommend ANTLR, in which you can emit such INDENT and DEDENT tokens quite easily. But if you're keen on Eclipse integration, then I don't see how you'd be able to do this using Xtext, sorry.
Version 2.8 of Xtext comes with support for Whitespace-Aware Languages. This version ships with the "Home Automation Example" that you can use as a template.
For people interested in CoffeeScript, Adam Schmideg has an Eclipse plugin that uses XText.
For people interested in parsing Python-like DSL's in XText, Ralf Ebert's code for Todotext mentioned above is no longer available from Github but you can find it in the Eclipse test repository. See the original thread about this work and the Eclipse issue that was raised about it.
I have been playing with this code today and my conclusion is it no longer works in the current version of XText. When XText is used in Eclipse, I think it does "partial parsing". This is not compatible with the stateful lexer you need to process indentation sensative languages. So I suspect even if you patch the lexer, the Eclipse editor does not work. In the issue, it looks like Ralf proposed patches to address these issues, but looking into the XText source, these changes seem long gone? If I am wrong and someone can get it to work, I would be very interested?
There is a different implementation here but I cannot get that to work with the current version of XText either.
Instead I have switched to parboiled which does supports indentation based grammars out the box.