Convert Stack Exchange Markdown to Github Markdown - github

Question

Has anyone documented the differences between Stack Exchange Markup and Github Markup?
I'm in the midst of a project to convert Stack Exchange Markdown to Github Markdown. It might be a little more complicated because Jekyll on Github Pages uses a Markdown derivative called "Kramdown".
I've already written some of the conversion in my Python program. For example old SE posts with #Header must be converted to # Header.
Another example are "> Block quote" lines have two spaces appended to the end of the line.
Now it's starting to get tricky (for me at least) where in an image in SE is specified as:
[![Ubuntu 5 DE.png][1]][1]
**Note:** Blah, blah, blah
[1]: https://i.stack.imgur.com/MoxHd.jpg
It has to be converted to Github image markdown format:
![Ubuntu 5 DE.png](https://i.stack.imgur.com/MoxHd.jpg)
**Note:** Blah, blah, blah
Another example of "footer hyper links" (for lack of a better noun) in Stack Exchange Markdown is:
- [Jack Master Volume?][1]
The simplest solution then is to install [JackMix][2]:
find listed [here][3].
[this script][4] is where you are heading:
[1]: https://discourse.ardour.org/t/jack-master-volume/84650
[2]: http://www.arnoldarts.de/jackmix/.
[3]: http://jackaudio.org/applications/
[4]: https://unix.stackexchange.com/questions/374085/lower-or-increase-pulseaudio-volume-on-all-outputs
Needs to be converted to Github Markdown format of:
- [Jack Master Volume?](https://discourse.ardour.org/t/jack-master-volume/84650)
The simplest solution then is to install [JackMix](http://www.arnoldarts.de/jackmix/.):
find listed [here](http://jackaudio.org/applications/).
[this script](https://unix.stackexchange.com/questions/374085/lower-or-increase-pulseaudio-volume-on-all-outputs) is where you are heading:
Finally tonight I discovered that in Stack Exchange you can have:
<!-- language: bash -->
#!/bin/bash
cat "$Filename.zip" | base64 > "$Filename64"
That needs reformatting to Github Markdown like this:
``` bash
#!/bin/bash
cat "$Filename.zip" | base64 > "$Filename64"
```
It gets even more complicated when SE Markdown has:
<!-- language-all: lang-bash -->
Or it has this:
<pre><code>Some lines of code
some more lines of
code </code></pre>
An existing Github Repo to convert would be awesome! If not then if someone has documented the differences between Stack Exchange Markup and Github Markup that would be great too.
If this question goes unanswered for a month then I guess I'll be answering it eventually after the trial-error-fix process is finished.

Answer 1

The self-answer seems to answer a different question, so I'll answer the one posed above.
Has anyone documented the differences between Stack Exchange Markup and Github Markup?
As of mid-2020, Stack Exchange uses CommonMark with support for a few custom features "like spoilers, MathJax, circuit diagrams, stack snippets, etc."
GitHub uses their own dialect, GitHub Flavored Markdown (GFM). The most notable extensions that GFM introduces are probably tables (supported on SE since late 2020), fenced code blocks (supported on SE since early 2019), and task lists (unsupported but also not really necessary on SE).
Most of the examples shown above work fine when rendered on Stack Exchange or by a compliant GFM implementation, but let's look at them in turn:
For example old SE posts with #Header must be converted to # Header.
This should have been cleaned up by the migration to CommonMark:
we will run a big migration across the network that will convert existing posts to use the new CommonMark format
Another example are "> Block quote" lines have two spaces appended to the end of the line.
Two spaces at the end of the line indicate a line break in Markdown, going all the way back to the original implementation.
Blockquotes do not require such line breaks, though they will rewrap without them. Line breaks and blockquotes are unrelated features.
Your image examples are interchangeable in both formats. Let's look at the SE one:
[![Ubuntu 5 DE.png][1]][1]
**Note:** Blah, blah, blah
[1]: https://i.stack.imgur.com/MoxHd.jpg
There is nothing specific to Stack Exchange here.
The [foo][1]… [1]: https://... syntax is a reference-style link which, again, comes from the original project. It is equivalent to the inline form [foo](https://...). Both forms work on both platforms. ![Ubuntu 5 DE.png][1]… [1]: https://i.stack.imgur.com/MoxHd.jpg is the same as ![Ubuntu 5 DE.png](https://i.stack.imgur.com/MoxHd.jpg).
On SE images are wrapped by a link by default, which introduces the wrapping [...][1]. But, again, this is not SE-specific. It's like comparing <img src="..."> to <img src="...">. This also works on both platforms.
Another example of "footer hyper links" (for lack of a better noun)
These are reference-style links already discussed and present in every version of Markdown I've ever seen, including the original project. They work in GFM as well as they do in CommonMark. No conversion is necessary.
Finally tonight I discovered that in Stack Exchange you can have:
<!-- language: bash -->
#!/bin/bash
cat "$Filename.zip" | base64 > "$Filename64"
That needs reformatting to Github Markdown like this:
``` bash
#!/bin/bash cat "$Filename.zip" | base64 > "$Filename64"
```
This is the only example above that requires any special behaviour at all. However, it mostly works out of the box. GFM supports indented code blocks and SE has supported fenced code blocks for over three years.
GFM doesn't understand legacy HTML-style SE language hints <!-- language: bash --> and <!-- language-all: ... -->, so it will render such code blocks without syntax highlighting. But they will still render as a code block.
The last example is just embedded HTML that both platforms (and the original) know how to render:
<pre><code>Some lines of code
some more lines of
code </code></pre>
With the exception of HTML-style language hints, every example you show above works out of the box on both Stack Exchange and GitHub Flavored Markdown. No conversion necessary.

Answer 2

Converting thousands of Stack Exchange Q&A in markdown format isn't as easy
as simply copying them over to GitHub Pages. The python program
stack-to-blog.py was used to convert Stack Exchange posts to
GitHub Pages Posts.
The full stack-to-blog.py program can be accessed on the
Pippim Website repo 🔗.
The program automatically:
Creates Jekyll front matter on posts and front matter totals for site.
Selects Stack Exchange Posts based on meeting minimum criteria such as up-votes or accepted answer status.
If self-answered question, the answer is included and not the question.
If self-answered question, the accepted answer alone doesn't qualify. Votes from other are the qualifier.
Initial testing allows selecting small set of random record numbers to convert.
Converts Stack Exchange Markdown formats to GitHub Pages Kramdown Markdown format.
Creates hyperlinks to original Answer in Stack Exchange and Kramdown in GitHub Pages.
Creates search word to URL indices excluding 50% of words like "a", "the", etc. to save space.
Selectively inserts Table of Contents based on minimum criteria settings.
Selectively inserts Section Navigation Buttons for: Top (Top of Page), ToS (Top of Section), ToC (Table of Contents) and Skip (Skip section).
Selectively inserts "Copy Code Block to System Clipboard" button based on lines of code.
Creates HTML with "Top Ten Answers" with the most votes.
Creates powerful nested expandable/collapsible detail/summary HTML for many thousands of tags by post.
Remaps hyperlinks in Stack Exchange Posts to {{ site.title }} website posts if they were converted.
Fixes old broken #header Stack Exchange Markdown.
Converts < block quote Stack Exchange Markdown into what works in Jekyll Kramdown.
Convert Stack Exchange <!-- language --> tags to fenced code block language.
When no fenced code block language is provided, uses shebang language first (if available).
Converts older four-space indented code blocks to fenced code blocks.
Converts Stack Exchange Hyperlinks where the website post title is implied and not explicit.
Prints list of self-answered questions that were not accepted after the mandatory two day wait period.
Prints list of Rouge Syntax Highlighting languages not supported in fenced code blocks.
Prints summary totals when finished.
Full documentation is provided here.
This is what program looks like when running: