A table in .md document included from external file - github

Is there an elegant way how to include table from an external file in markdown document rendered by GitHub?
Non elegant solutions I can think of:
include it as image (this one is really really ugly)
use Rmarkdown and print table using R (wont be rendered by default)
Just to give a bit of explanation. I am using a set of README.md files in my git repository (hosted by GitHub), so it is really clear to browse repo online, because GitHub renders automatically README.md file in every subdirectory.
I am algorithmically generating summary tables that should be included in those documents. It would be way more elegant if that table could be read from external file, because I do not want to write scripts that will modify README.md files directly.

There is no way to include files within markdown. So you need a "preprocessing" stage to generate the markdown which is then shown on Github (or rendered with normal markdown tools).
What Github supports is a basic table layout, which you'd need to render:
Solution 1: scripting
You could add something like this to your README.md:
<!-- TABLE_GENERATE_START -->
| First Header | Second Header |
| ------------- | ------------- |
| Content Cell | Content Cell |
| Content Cell | Content Cell |
<!-- TABLE_GENERATE_END -->
And then have a script which pulls in the external table, reformats it to match the github format and puts it between the two comment.
Solution 2: pandoc
Pandoc is a document converter framework with many possible inputs and outputs. If you're up to learn another tool you could
reformat you table into a markdown file using csv2table into table.md
create a readme_header.md and readme_footer.md with the markdown before/after the table
merge the three files with cat readme_header.md table.md readme_footer.md > REAME.md
Of course you can also do a mixture of both solutions, e.g. generate table.md using a script and merge using cat

Related

GitHub Wiki doesn't support HTML tables anymore?

Back in the days GitHub Wiki supported embedding HTML tables, but now all our tables are not rendered anymore. Is support for HTML tables officially dropped (can't find corresponding news or blog post)?
GitHub does support table tag but it's not as extensible as in any HTML file. It's pretty much limited to what one may need for a wiki. In my view, it's pointless to use <table> because they are tiring compared to other markdowns that GitHub wiki use.
Markdown -
| First Header | Second Header |
| ------------- | ------------- |
| Content Cell | Content Cell |
| Content Cell | Content Cell |
Preview -
Equivalent HTML -
<table>
<tr><th>First Header</th><th>Second Header</th></tr>
<tr><td>Content Cell</td><td>Content Cell</td></tr>
<tr><td>Content Cell</td><td>Content Cell</td></tr>
</table>
PS. Here is a post on tables for markdown.
2017:
Back in the days GitHub Wiki supported embedding HTML tables, but now all our tables are not rendered anymore.
2022: they should be rendered now:
Updates to Markdown pasting on GitHub
May 19, 2022
We've made some updates to how paste formatting works in Markdown-enabled fields on GitHub.
For example, in code editors and on gists, you'll now be able to paste URLs on selected texts that will render as Markdown links like [...](https://...) by using the keyboard shortcut cmd|ctl + v..
The following paste formatting changes have been made to pull requests, issue comments and wikis:
Spreadsheet cells and HTML tables will render as Markdown tables
Any copied text containing links will render the links in Markdown.
All of this formatting can be disabled when pasting using the keyboard shortcut: cmd|ctl + shift + v or cmd|ctl + shift + Alt + v.
According to GitHub's Markdown spec, the Markdown parser permits most any raw HTML. The important thing is that the tags must begin at the start of the line. There are also rules which change the behavior when the raw block contains blank lines in some situations. Prior to adopting the current spec, I'm not sure if they were as strict about that, but that could be a reason for the change (some example input would help narrow down the possibilities).
And as another answer suggests, the GitHub Flavored Markdown spec includes a tables extension, so you can create tables natively in Markdown. This eliminates the need to structure your own HTML to the whims of the Markdown parser.
However, that is only the beginning of GitHub's processing of your input. After passing your input through the Markdown parser, there are four additional steps of processing taken on the output generated by Markdown as documented in the github/markup project. The most likely culprit is step two, which sanitizes the HTML. If your raw HTML doesn't match the expectations of the very narrow sanitizer, then it will get stripped out. The specifics of the santitizer are not documented, but the code is available to review and pretty easy to follow (even for those of use who aren't very familiar with Ruby).
Interestingly, the Markdown parser is sure to output HTML that the santitizer allows through, and in fact, tables are allowed. However, if you are using raw HTML rather than Markdown tables for more flexibility, then it is likely the extras that Markdown doesn't give you are causing the sanitizer to eat your tables (for example, you only get limited attributes, and improperly nests tags are stripped). In other words, raw HTML tables can only be limited to the basic features you already get with Markdown tables. Given the simplicity of Markdown tables over raw HTML, most people just use the markdown tables. YMMV.

asciidoc: is there a way to create an anchor that will be visible in libreoffice writer?

Tl;dr;
What is the correct way to create an anchor in docbook? and is there a way that will make the anchor visible in writer?
Background
I am trying to split up documentation that was previously in single open office documents into smaller asciidoc documents which are both included in the main open office document and also converted to either or both of html & pdf.
I have this mostly working. I use asciidoctor to create html. asciidoctor-pdf to create pdf and a combination of asciidoctor and pandoc to create .odt files. I also tried the python implementation of asciidoc but found the interface less useable.
Round tripping between asciidoc and odt is obviously not possible. This is sort of a fusion where the master document is word processed but pieces of content that can be produced independently (think man pages - in fact that is one of several use cases) are included.
asciidoc to html:
asciidoctor -b html5 foo.adoc -o foo.html
asciidoc to pdf:
asciidoctor-pdf -b pdf foo.adoc -o foo.pdf
asciidoc to odt
asciidoctor -b docbook foo.adoc -o foo.docbook
pandoc --base-header-level=3 -V date:"" -V title:"" -f docbook foo.docbook -o foo.odt
With pandoc I have to nullify the date and title and set the header-level as desired for the section to be inserted as an extra complication.
I insert the resulting .odt into the main document using insert section inside open office.
Note that the main document is not a master document as I could not find a way of creating a master document without also automatically splitting the file on h1 boundaries.
I have two main problems to resolve with this set-up. I would like to add headings in the asciidoc document as cross references and also create entries for them in the alphabetical index (actually the first heading would be suffcient). Is there a way to do this?
Index markers in asciidoc do not result in entries in .odt file being created.
I am able to cross reference content in the inserted section using "insert reference/heading" and referencing the uniquely named header. However, whenever I use "update all" these cross references are invalidated. They are shown as "Error: Reference source not found".
[On a separate note I would also like a way to find broken cross references automatically]
I am currently using libreoffice - Version: 4.3.7.2
I am not adverse to switching version or flavours (i.e. apache) if one behaves better than the other.
I'm not sure if the answer is in the asciidoc or docbook parts of the chain. I would accept an answer which inserts a index entry at the start of the inserted section (top of the .adoc/docbook file) automatically.
I am also open to changing my toolchain to something that will work.
For example I tried the asciidoc-odt backend and fell foul of https://github.com/dagwieers/asciidoc-odf/issues/47 which does not inspire confidence.
Using asciidoc-odt I avoid the need to create an intermediate docbook file. However, I still can't get the anchor to appear.
I can get a macro to create an anchor but at present I haven't figured out how to run the macro from the command line.
To create an anchor in DocBook, make an inline anchor in the .adoc file. For example, giving this to asciidoctor:
[[X1]]Section1
---------------
produced this:
<title>
<anchor xml:id="X1" xreflabel="[X1]"/>
Section1
</title>
Conversely, putting this on separate lines did not create an anchor tag in my test:
[[X1]]
Section 1
Now for some bad news. From the Pandoc User's Guide:
Internal links are currently supported for HTML formats (including HTML slide shows and EPUB), LaTeX, and ConTeXt.
I interpret this to mean that currently, Pandoc does not create internal links in Writer. When I tried it, the link was ignored.
Note: It looks like I did not answer all of your questions. If you want to ask more about LibreOffice cross references and headings (the big bold paragraph towards the end of the question), maybe you could make a separate question just for that part.

Difference between Github and npm markdown

I have an Node package up on Github (https://github.com/jrootham/argument-spec) and npm (https://www.npmjs.com/package/argument-spec). There is a specification table where the first cell contains the word 'undefined'. It shows up fine on Github but is blank on npm. Anybody have any idea what is going on. I searched for docs on npm markdown but none appeared.
If npm readme html pages are generated (as in this question) with evilstreak/markdown-js, then you have some table examples, like this one:
| First Header | Second Header |
| ------------- | ------------- |
| Content 1 | Content 2 |
| Content 3 | Content 4 |
There is issue 230 discussing about the proper supoprt for table with this package, but it should be able to render most tables correctly.
In your case, see if adding delimiters change anything:
####Specification meanings
|Specification|Valid argument|
-----------|----------
|undefined|anything|
|''|string|
The PR (Pull Request 480 mentions:
This updates newww to use marky-markdown to process readme content. Highlights include:
Human-readable code!
Lots of tests
Explicit HTML content policy with sanitize-html
Server-side syntax highlighting
Gravatar URL cleanup
GitHub relative link cleanup
Better tagging of badge elements
Forward-looking CDN image support
That project in turn uses markdown-it, which has a few issues around table.
#Jim Rootham is right about the process npm uses to display markdown: they are filtered through marky-markdown and then displayed. It looks like you've found a bug somewhere in this process; you should open an issue on the npm website repository, https://github.com/npm/newww

How do I create some kind of table of content in GitHub wiki?

If you look here: http://en.wikipedia.org/wiki/Stack_Overflow
You'll notice there's a little "Content" section, if you click on one of the links, it will send you to a specific section on the page.
How do I do this in GitHub wiki? With Markdown or whatever they use?
It is nicely demonstrated in the Table of Contents of the Markdown Cheatsheet.
##### Table of Contents
[Headers](#headers)
[Emphasis](#emphasis)
...snip...
<a name="headers"/>
## Headers
If you hover over a Header in a GitHub Markdown file, you'll see a little link sample to the left of it, you can also use that link. The format for that link is <project URL#<header name>. The <header name> must be all lower case.
Since github cannot use TOC directly, but we have other alternatives.
You can automatically generate TOC via Online tool:
Generate TOC Table of Contents from GitHub Markdown or Wiki Online
or via Local tool:
github-markdown-toc
Visual Studio Code
If you happen to use Visual Studio Code, there is easy-to-use extension called Markdown All in One that can make the TOC for any .md file in an instant.
Just open Command Palette (Ctrl-Shift-P) -> Markdown: Create Table of Contents
Auto-update messes your edited TOC?
As an additional tip, you might want to turn the "automatic TOC updates on save" OFF by using
"markdown.extension.toc.updateOnSave": false,
in your Visual Studio Settings (Command Palette -> Preferences: Open Settings (JSON)).
One possible (semi-automated) solution is Eugene Kalinin's github-markdown-toc. This tool essentially crunches through your README.md file and snarfs out #'s headings to create a TOC.
Download the script https://github.com/ekalinin/github-markdown-toc
Feed your README.md to the script (as noted in Eugene's README.md)
cat README.md | bash github-markdown-toc
Cut and paste generated TOC and place it at the top of your README.md file
Note that this bash implementation only works on Linux (from what I can tell).
As a side note, there is a golang implementation and is probably more of a hassle to get working.
Update Aug. 2021:
After ToC in README (see March 2021 below), you now have:
Table of content for Wikis
For Wikis we now automatically generate a table of contents based on the Markdown headings.
As illustrated here:
Do you wiki?
We just added an automatic table of contents to the sidebar to help with navigation
You can now (March 2021) check out what the CEO of GitHub Nat Friedman just announced
GitHub now automatically creates a table of contents for your http://README.md files from your headers.
After much consideration, we made this a feature of the viewer, not a concern of the editor: no special markdown to insert.
So... it does not modify your markdown (README.md or other .md files) to insert, or update your text: it only provides a menu which allows quick access to your test sections based on markdown headers.
That may, or may not, what you are after.
https://github.com/jonschlinkert/markdown-toc
git clone your-repo.wiki.git (add the .wiki right before .git to clone the wiki
npm i -g markdown-toc
Insert <!-- toc --> (case sensitive) in your wiki's markdown
markdown-toc -i my-wiki-markdown.md (-i will edit it in place)
Profit
Update: I think maybe https://github.com/thlorenz/doctoc is more popular now.
Currently it's not possible to do that using markdown syntax (.md). There is ongoing unofficial discussion about automatically generating table of contents TOC on rendered markdown files like README.md which lists some of the ideas.
However there are some other workarounds such as:
Use AsciiDoc instead as per suggestion from this comment. For example:
:toc: macro
:toc-title:
:toclevels: 99
# Title
toc::[]
## A
### A2
## B
### B2
Check the example at littlebits/react-popover (README.adoc).
Online Table Of Content Generator (raychenon/play-table-of-contents)
arthurhammer/github-toc - browser extension that adds a table of contents to GitHub repos
If you are not in the position to stick with Markdown, you can do as below:
on GitHub/wiki: switch Markdown to MediaWiki. Use __TOC__ Syntax. See sample.
on GitHub/repo: switch Markdown to AsciiDoc. Use :toc: Syntax. See demo.
on GitHub/repo: switch Markdown to reStructuredText. Use .. contents:: Syntax.
However, using Markdown files in GitHub/repo, you can get it on GitHub Pages like in Wikipedia
when Jekyll is activated, it generates GitHub Pages using Kramdown by default
Kramdown comes with Table Of Content. Use {:toc} Syntax. See the explanation.
You can choose the Edit mode "MediaWiki" which will generate a toc for the headers, e.g.
== First ==
== Second ==
Due to the fact that github has it's own way of generating id=".." attributes in h1, h2, h3, etc... headers in html version after processing Markdown (for example Bitbucket use little different pattern of sluggifying headers title to id="slug") it is handy to don't reinvent the wheel and use library that reverse engineered this process.
I found one quite good library for this task called markdown-toc.
For me it seems the best solution because I always have installed node on my machine.
Just execute npx markdown-toc -i file.md.
And it looks like it is one of more popular tools for this task - at least in node.js ecosystem.
ls
cat <<EOF >> test.md | tee
## Table of Contents
<!-- toc -->
- old toc 1
- old toc 2
- old toc 3
<!-- tocstop -->
## abc
This is a b c.
## xyz
This is x y z.
EOF
ls
cat test.md
npx markdown-toc -i test.md
cat test.md
output:
Pandoc
The swiss army knife of markup:
cat README.md | pandoc --from markdown --toc -s --to markdown -
You can use mdtoc (I am the author).
Once installed, simply run:
mdtoc path/to/file.md
One more TOC markdown related tool implemented on the top of Perl (which is shipped with Linux/Git-for-Windows always and with Cygwin optionally, and there are no dependencies on extra packages)
https://github.com/ildar-shaimordanov/git-markdown-toc
I guess my tool works similar or almost similar to ekalinin/git-markdown-toc mentioned above by other people. I have never compared tham because his tool is implemented as Go-Lang which doesn't exist on my system. The main goal of my script is to provide the good solution in creating TOC locally -- no any connection to any exteranl hosts and so on, only read a local file (README.md, by default) and create the TOC and embed it to the file.
Example:
[Go to Delete](#delete_lines)
#delete_lines
code here, will be pointed here
See: https://guides.github.com/features/mastering-markdown/
And, to make a nested outline:
* 1\. [Go to Delete](#delete_lines)
* 1.1\. item
* 1.2\. item
* 1.2\. item
* 2\. item
See: https://meta.stackexchange.com/questions/85474/how-to-write-nested-numbered-lists
And for more info and complex linking:
https://stackoverflow.com/questions/6695439/how-to-link-to-a-named-anchor-in-multimarkdown#:~:text=In%20standard%20Markdown%2C%20place%20an,%5Blink%20text%5D(%23abcd)%20.

Automatic TOC in github-flavoured-markdown

Is it possible to generate an automatic Table of Contents using Github Flavoured Markdown?
I created two options to generate a toc for github-flavored-markdown:
DocToc Command Line Tool (source) requires node.js
Installation:
npm install -g doctoc
Usage:
doctoc . to add table of contents to all markdown files in the current and all sub directories.
DocToc WebApp
If you want to try it online first, go to the doctoc site,
paste the link of the markdown page and it will generate a table of
content that you can insert at the top of your markdown file.
Github Wikis and Anchors
As Matthew Flaschen pointed out in the comments below, for its wiki pages GitHub previously didn't generate the anchors that doctoc depends on.
UPDATE: However, they fixed this issue.
GitHub Pages (which is basically a wrapper for Jekyll) appears to use kramdown, which implements all of Maruku, and therefore has support for an automatically generated table of contents via atoc attribute:
* auto-gen TOC:
{:toc}
The first line just starts an unordered list and is actually thrown away.
This results in a nested set of unordered lists, using the headers in the document.
Note: this should work for GitHub Pages, not GitHub Flavored Markdown (GFM) as used in comments or wiki pages. AFAIK a solution doesn't exist for that.
If you edit Markdown files with Vim, you can try this plugin vim-markdown-toc.
The usage is simple, just move your cursor to the place you want to append Table of Contents and run :GenTocGFM, done!
Screenshots:
Features:
Generate toc for Markdown files. (Support GitHub Flavored Markdown and Redcarpet)
Update existing toc.
Auto update toc on save.
Update March 2021: GitHub added an official workaround
READMEs now show a ToC like this as you scroll down into them:
demo: https://github.com/cirosantilli/test-git-web-interface/tree/master/d
It does not render inside the document as I wanted for better Ctrl + F, but it is better than nothing.
Also also works for non-README as well now, e.g.: https://github.com/cirosantilli/test-git-web-interface/blob/master/md.md
They also added a repository setting to enable disable that. It's so weird, who would ever want to disable it? Under https://github.com/cirosantilli/test-git-web-interface/settings Features:
Table of contents
Autogenerate table of contents for Markdown files in this repository. The table of contents will be displayed near the top of the file.
Original answer
It's not possible, except for the workarounds proposed.
I proposed Kramdown TOC extension and other possibilities to support#github.com and Steven! Ragnarök replied with the usual:
Thanks for the suggestion and links. I'll add it to our internal feature request list for the team to see.
Let's upvote this question until it happens.
Another workaround is to use Asciidoc instead of Markdown, which does render TOCs. I've moved to this approach for my content nowadays.
It's not automatic, but it uses Notepad++ regular expressions:
Replace all first by the second (removes all lines not having headers)
^##(#?)(#?)(.*?)$(.|\r|\n)*?(?=^##|\z)
-\1\2 [\3](#\3)\n
Then (converts headers III to spaces)
-##
-
Then (converts headers II to spaces)
-#
-
Then (remove unused chars at the beginning and at the end of link title)
\[ *((?:(?![ .:#!\?;]*\])[^#])*)[ #:!\?;]*\]
[\1]
Then (convert last tokens lowercase and dash instead of spaces)
\]([^ \r\n]*) ([^\r\n ]*)
]\L\1-\2
Remove unused final pounds and initial dashes:
(?:()[-:;!\?#]+$|(\]#)-)
\1\2
Remove useless chars in links:
(\].*?)(?:\(|\))
\1
And finally add parenthesis around final links:
\](?!\()(.*?)$
\]\(\1\)
And voilà! You can even put this in a global macro if you repeat it enough time.
Github Flavored Markdown uses RedCarpet as their Markdown engine.
From the RedCarpet repo:
:with_toc_data - add HTML anchors to each header in the output HTML,
to allow linking to each section.
It seems in that you'd need to get at the renderer level to set this flag, which isn't possible on Github obviously. However, the latest update to Github Pages, it seems that automatic anchoring is turned on for headers, creating linkable headings. Not exactly what you want, but it might help you create a TOC for your doc a bit easier (albeit manually).
A very convenient way to achieve a table of contents for a mardown file when working with Visual Studio Code is the extension Markdown-TOC.
It can add a toc to existing markdown files and even keep the toc up-to-date on saving.
It is possible to generate a webpage automatically with http://documentup.com/ from the README.md file. It's not creating a TOC, but for many it might solve the reason for wanting to create a TOC.
Another alternative to Documentup is Flatdoc: http://ricostacruz.com/flatdoc/
Gitdown is a markdown preprocessor for Github.
Using Gitdown you can:
Generate Table of Contents
Find dead URLs and Fragment Identifiers
Include variables
Include files
Get file size
Generate Badges
Print Date
Print information about the repository itself
Gitdown streamlines common tasks associated with maintaining a documentation page for a GitHub repository.
Using it is straightforward:
var Gitdown = require('gitdown');
Gitdown
// Gitdown flavored markdown.
.read('.gitdown/README.md')
// GitHub compatible markdown.
.write('README.md');
You can either have it as a separate script or have it as part of the build script routine (such as Gulp).
Use coryfklein/doctoc, a fork of thlorenz/doctoc that does not add "generated with DocToc" to every table of contents.
npm install -g coryfklein/doctoc
Majority of other answers require to install some tool.
I found a quick and easy online solution https://imthenachoman.github.io/nGitHubTOC.
For any markdown input it generates table of content output.
You can specify minimum and maximum heading level.
The source code is located at https://github.com/imthenachoman/nGitHubTOC
My colleague #schmiedc and I have created a GreaseMonkey script that installs a new TOC button left of the h1 button which uses the excellent markdown-js library to add/refresh a table of contents.
The advantage over solutions like doctoc is that it integrates into GitHub's wiki editor and does not need users to work on their command-line (and require users to install tools like node.js). In Chrome, it works by drag 'n dropping into the Extensions page, in Firefox you will need to install the GreaseMonkey extension.
It will work with plain markdown (i.e. it does not handle code blocks correctly, as that is a GitHub extension to markdown). Contributions welcome.
This is a not a direct answer to this question as so many people have provided workarounds. I don't think generating a TOC has been officially supported by Github yet to-date. If you want GitHub to render a Table of Contents on their GFM preview pages automatically, please participate the discussion on the official feature request issue.
Shameless "borrow" of this SO answer.
You can do this with Pandoc.
pandoc -s --toc input.md -o input_toc.md
Note: the order of the input and output files is important here.
Currently it's not possible using markdown syntax (see the ongoing discussion at GitHub), however you can use some external tools such as:
Online Table Of Content Generator (raychenon/play-table-of-contents)
arthurhammer/github-toc - browser extension that adds a table of contents to GitHub repos
Alternatively use AsciiDoc instead (e.g. README.adoc), e.g.
:toc: macro
:toc-title:
:toclevels: 99
# Title
## A
### A2
## B
### B2
as suggested in this comment. Check the demo here.
For Github's Texteditor Atom check out this awesome plugin (or "package" in Atom-lingo), which generates "TOC (table of contents) of headlines from parsed markdown" files:
markdown-toc
Once installed as Atom-package you can use the shortcut ctrl-alt-c to insert a TOC based on your markdown-doc-structure at the current cursor position...
Screenshots:
Atom Keybindings
markdown-toc gives you the following default key-bindings to control the plugin in Atom:
ctrl-alt-c => create TOC at cursor position
ctrl-alt-u => update TOC
ctrl-alt-r => delete TOC
Plugin Features (from the project's README)
Auto linking via anchor tags, e.g. # A 1 → #a-1
Depth control [1-6] with depthFrom:1 and depthTo:6
Enable or disable links with withLinks:1
Refresh list on save with updateOnSave:1
Use ordered list (1. ..., 2. ...) with orderedList:0
Here's a shell script I threw together today for this. Might need to tweak it for your needs, but it should be a good starting point.
cat README.md \
| sed -e '/```/ r pf' -e '/```/,/```/d' \
| grep "^#" \
| tail -n +2 \
| tr -d '`' \
| sed 's/# \([a-zA-Z0-9`. -]\+\)/- [\1](#\L\1)/' \
| awk -F'(' '{for(i=2;i<=NF;i++)if(i==2)gsub(" ","-",$i);}1' OFS='(' \
| sed 's/^####/ /' \
| sed 's/^###/ /' \
| sed 's/^##/ /' \
| sed 's/^#//'
If anyone knows a better way to do those final # replacements, please add a comment. I tried various things and wasn't happy with any, so I just brute forced it.
There's now a GitHub Action accomplishing this:
https://github.com/marketplace/actions/toc-generator
Specify location of TOC (option)
e.g. README.md
<!-- START doctoc -->
<!-- END doctoc -->
Setup workflow
e.g. .github/workflows/toc.yml
on: push
name: TOC Generator
jobs:
generateTOC:
name: TOC Generator
runs-on: ubuntu-latest
steps:
- uses: technote-space/toc-generator#v2
Update 2022-02
In VSCode, check out extension "Markdown All in One". It will generate and update the TOC of markdown automatically.
Install Extension.
Place cursor at where you want to insert TOC.
Run command "Markdown All in One: Create Table of Contents"
Enjoy!