VSCode/TextMate syntax highlighting grammar -- matching code until end of line - visual-studio-code

Question

I am writing a TextMate grammar to implement syntax highlighting in VSCode for a custom flavor of Markdown. I would like everything on the same line after ##$ to be highlighted as Javascript.
This is what I came up with:
"majsdown_execute_statement": {
"begin": "(.*?)(##\\$)",
"name": "test",
"end": "(\\r\\n|\\r|\\n)",
"beginCaptures": {
"2": {
"name": "keyword.control.majsdown"
}
},
"patterns": [
{
"include": "source.js"
}
]
},
That almost works:
But I would like the ##$ part to always be highlighted as a keyword. Here's a mockup (edited image) of my desired result:
I've tried a lot of different combinations of "begin" and "end", and I've also tried many nested patterns like the following one:
"patterns": [
{
"begin": "\\s",
"while": "^(\\r\\n|\\r|\\n)",
"patterns": [
{
"include": "source.js"
}
]
}
]
Unfortunately, nothing provides the result I desire. How can I achieve my desired result?

Answer 1

To handle multiline constructs, I think you'll have to provide modified embedded language(JavaScript) definition as well. Easiest way to do it would be to make ##$ a comment in JavaScript, so it'll not mess up existing constructs.
I have no idea about VScode syntax highlighting. I'll try to demonstrate the idea using HighlightJs. It has very similar way of defining a language.
Demo: View in full page mode.
hljs.debugMode();
// default code for the demo
src.innerText = `
My custom markdown highlighted with custom rules.
Here is how javascript code looks:
##$ function test()
##$ {
##$ // TODO: stuff
##$ let rand = Math
##$ .random()
##$ .toFixed(2);
##$ }
##$ var string = "hello";
##$ const string2 = \`some long
##$ string\`;
Leave one empty line to get out of the code block.
Here is some more code:
##$ var rand = Math.random();
##$ console.log(rand);
We are out of the second code block now.
`;
// define our markup language, say 'mdown'
let langDef = {
name: 'Mdown',
aliases: ['mdown'],
case_insensitive: true,
contains: [
{
className: 'mscript',
begin: /##/,
end: /\$/,
keywords: { name: 'mscript' },
contains: [],
starts: {
end: /^\s*$/, // end on empty line
returnEnd: true,
subLanguage: ['js'], //embedded language
},
},
],
};
hljs.registerLanguage('mscript', () => langDef);
// patch javascript multiline structures
let js = hljs.getLanguage('javascript');
for (let c of js.contains) {
if (c.begin === "`") { // handle templet literals
c.contains.unshift({
begin: /^##/,
"relevance": 0,
end: /\$/,
contains: [],
scope: 'mscripttag'
})
}
}
// console.log(js);
// make '##$' a comment :)
// So it'll not mess existing styling
js.contains.push(hljs.COMMENT(/##/, /\$/, { scope: 'mscripttag', relevance: 10 }));
// for demo update highlighted code on user input
let handleChange = (event) => {
let html = hljs.highlight(src.innerText, { language: 'mscript', ignoreIllegals: true }).value;
code.innerHTML = html;
};
// javascript patching done
document.addEventListener('DOMContentLoaded', handleChange);
src.addEventListener('input', handleChange);
body { font-size: .8rem; }
.input {
width: 46%;
background-color: #eee;
display: inline-block;
overflow: auto;
}
.output {
width: 50%;
display: block;
float: right;
background-color: #ccc;
}
/* add extra theme for our tag ##$ */
.hljs { color: #bbb !important; }
.hljs-mscript,
.hljs-mscripttag { color: red; }
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.4.0/styles/base16/snazzy.min.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.4.0/highlight.min.js"></script>
<div>Edit below markdown code.</div>
<pre class="input"><div id="src" contenteditable></div></pre>
<pre class="output"><code id="code" class="javascript hljs">abcd</code></pre>
In case the library not available, here is how the output looks like:
In above code I have defined ##$ <jscontent> \n as a tag in our markdown language definition. And the content will be processed according to the embedded language JavaScript. Next I've modified the default JS language definition and added ##$ as a comment so it'll be harmless to exiting syntax. Now we just need to handle template literals which are multiline. In the literal definition I've added ##$ as a part of the literal, but with different scope/name/styleOption msscripttag.
I hope you'll be able to define similar syntax in TextMate for VSCode.

Answer 2

try using match instead...
"patterns": [
{
"name": "keyword.control.factory",
"match": "(\\w|-)[^(]*"
}
]
this code matches everything up to a "(" character
so your code look something like this:
"majsdown_execute_statement": {
"patterns": [
{
"name": "keyword.control.majsdown",
"match": "(?<=##\$)\\w*"
}
]
}
you can also try [^##$]*(\\w)
try it out here