diff options
Diffstat (limited to 'vendor/github.com/gomarkdown/markdown')
36 files changed, 8266 insertions, 0 deletions
diff --git a/vendor/github.com/gomarkdown/markdown/.gitignore b/vendor/github.com/gomarkdown/markdown/.gitignore new file mode 100644 index 00000000..8f15b677 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/.gitignore @@ -0,0 +1,13 @@ +*.out +*.swp +*.8 +*.6 +_obj +_test* +markdown +tags +fuzz-workdir/ +markdown-fuzz.zip +coverage.txt +testdata/*_got.md +testdata/*_ast.txt diff --git a/vendor/github.com/gomarkdown/markdown/.gitpod b/vendor/github.com/gomarkdown/markdown/.gitpod new file mode 100644 index 00000000..ad5feff6 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/.gitpod @@ -0,0 +1,7 @@ +checkoutLocation: "src/github.com/gomarkdown/markdown" +workspaceLocation: "." +tasks: + - command: > + cd /workspace/src/github.com/gomarkdown/markdown && + go get -v ./... && + go test -c diff --git a/vendor/github.com/gomarkdown/markdown/.travis.yml b/vendor/github.com/gomarkdown/markdown/.travis.yml new file mode 100644 index 00000000..4ec5d7b0 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/.travis.yml @@ -0,0 +1,17 @@ +dist: bionic +language: go + +go: + - "1.12.x" + +install: + - go build -v ./... + +script: + - go test -v ./... + - go test -run=^$ -bench=BenchmarkReference -benchmem + - ./s/test_with_codecoverage.sh + - ./s/ci_fuzzit.sh + +after_success: + - bash <(curl -s https://codecov.io/bash) diff --git a/vendor/github.com/gomarkdown/markdown/LICENSE.txt b/vendor/github.com/gomarkdown/markdown/LICENSE.txt new file mode 100644 index 00000000..68804610 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/LICENSE.txt @@ -0,0 +1,31 @@ +Markdown is distributed under the Simplified BSD License: + +Copyright © 2011 Russ Ross +Copyright © 2018 Krzysztof Kowalczyk +Copyright © 2018 Authors +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided with + the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN +ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. diff --git a/vendor/github.com/gomarkdown/markdown/README.md b/vendor/github.com/gomarkdown/markdown/README.md new file mode 100644 index 00000000..3bca71db --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/README.md @@ -0,0 +1,325 @@ +# Markdown Parser and HTML Renderer for Go + +[](https://godoc.org/github.com/gomarkdown/markdown) [](https://codecov.io/gh/gomarkdown/markdown) + +Package `github.com/gomarkdown/markdown` is a very fast Go library for parsing [Markdown](https://daringfireball.net/projects/markdown/) documents and rendering them to HTML. + +It's fast and supports common extensions. + +## Installation + + go get -u github.com/gomarkdown/markdown + +API Docs: + +- https://godoc.org/github.com/gomarkdown/markdown : top level package +- https://godoc.org/github.com/gomarkdown/markdown/ast : defines abstract syntax tree of parsed markdown document +- https://godoc.org/github.com/gomarkdown/markdown/parser : parser +- https://godoc.org/github.com/gomarkdown/markdown/html : html renderer + +## Usage + +To convert markdown text to HTML using reasonable defaults: + +```go +md := []byte("## markdown document") +output := markdown.ToHTML(md, nil, nil) +``` + +## Customizing markdown parser + +Markdown format is loosely specified and there are multiple extensions invented after original specification was created. + +The parser supports several [extensions](https://godoc.org/github.com/gomarkdown/markdown/parser#Extensions). + +Default parser uses most common `parser.CommonExtensions` but you can easily use parser with custom extension: + +```go +import ( + "github.com/gomarkdown/markdown" + "github.com/gomarkdown/markdown/parser" +) + +extensions := parser.CommonExtensions | parser.AutoHeadingIDs +parser := parser.NewWithExtensions(extensions) + +md := []byte("markdown text") +html := markdown.ToHTML(md, parser, nil) +``` + +## Customizing HTML renderer + +Similarly, HTML renderer can be configured with different [options](https://godoc.org/github.com/gomarkdown/markdown/html#RendererOptions) + +Here's how to use a custom renderer: + +```go +import ( + "github.com/gomarkdown/markdown" + "github.com/gomarkdown/markdown/html" +) + +htmlFlags := html.CommonFlags | html.HrefTargetBlank +opts := html.RendererOptions{Flags: htmlFlags} +renderer := html.NewRenderer(opts) + +md := []byte("markdown text") +html := markdown.ToHTML(md, nil, renderer) +``` + +HTML renderer also supports reusing most of the logic and overriding rendering of only specifc nodes. + +You can provide [RenderNodeFunc](https://godoc.org/github.com/gomarkdown/markdown/html#RenderNodeFunc) in [RendererOptions](https://godoc.org/github.com/gomarkdown/markdown/html#RendererOptions). + +The function is called for each node in AST, you can implement custom rendering logic and tell HTML renderer to skip rendering this node. + +Here's the simplest example that drops all code blocks from the output: + +````go +import ( + "github.com/gomarkdown/markdown" + "github.com/gomarkdown/markdown/ast" + "github.com/gomarkdown/markdown/html" +) + +// return (ast.GoToNext, true) to tell html renderer to skip rendering this node +// (because you've rendered it) +func renderHookDropCodeBlock(w io.Writer, node ast.Node, entering bool) (ast.WalkStatus, bool) { + // skip all nodes that are not CodeBlock nodes + if _, ok := node.(*ast.CodeBlock); !ok { + return ast.GoToNext, false + } + // custom rendering logic for ast.CodeBlock. By doing nothing it won't be + // present in the output + return ast.GoToNext, true +} + +opts := html.RendererOptions{ + Flags: html.CommonFlags, + RenderNodeHook: renderHookDropCodeBlock, +} +renderer := html.NewRenderer(opts) +md := "test\n```\nthis code block will be dropped from output\n```\ntext" +html := markdown.ToHTML([]byte(s), nil, renderer) +```` + +## Sanitize untrusted content + +We don't protect against malicious content. When dealing with user-provided +markdown, run renderer HTML through HTML sanitizer such as [Bluemonday](https://github.com/microcosm-cc/bluemonday). + +Here's an example of simple usage with Bluemonday: + +```go +import ( + "github.com/microcosm-cc/bluemonday" + "github.com/gomarkdown/markdown" +) + +// ... +maybeUnsafeHTML := markdown.ToHTML(md, nil, nil) +html := bluemonday.UGCPolicy().SanitizeBytes(maybeUnsafeHTML) +``` + +## mdtohtml command-line tool + +https://github.com/gomarkdown/mdtohtml is a command-line markdown to html +converter built using this library. + +You can also use it as an example of how to use the library. + +You can install it with: + + go get -u github.com/gomarkdown/mdtohtml + +To run: `mdtohtml input-file [output-file]` + +## Features + +- **Compatibility**. The Markdown v1.0.3 test suite passes with + the `--tidy` option. Without `--tidy`, the differences are + mostly in whitespace and entity escaping, where this package is + more consistent and cleaner. + +- **Common extensions**, including table support, fenced code + blocks, autolinks, strikethroughs, non-strict emphasis, etc. + +- **Safety**. Markdown is paranoid when parsing, making it safe + to feed untrusted user input without fear of bad things + happening. The test suite stress tests this and there are no + known inputs that make it crash. If you find one, please let me + know and send me the input that does it. + + NOTE: "safety" in this context means _runtime safety only_. In order to + protect yourself against JavaScript injection in untrusted content, see + [this example](https://github.com/gomarkdown/markdown#sanitize-untrusted-content). + +- **Fast**. It is fast enough to render on-demand in + most web applications without having to cache the output. + +- **Thread safety**. You can run multiple parsers in different + goroutines without ill effect. There is no dependence on global + shared state. + +- **Minimal dependencies**. Only depends on standard library packages in Go. + +- **Standards compliant**. Output successfully validates using the + W3C validation tool for HTML 4.01 and XHTML 1.0 Transitional. + +## Extensions + +In addition to the standard markdown syntax, this package +implements the following extensions: + +- **Intra-word emphasis supression**. The `_` character is + commonly used inside words when discussing code, so having + markdown interpret it as an emphasis command is usually the + wrong thing. We let you treat all emphasis markers as + normal characters when they occur inside a word. + +- **Tables**. Tables can be created by drawing them in the input + using a simple syntax: + + ``` + Name | Age + --------|------ + Bob | 27 + Alice | 23 + ``` + + Table footers are supported as well and can be added with equal signs (`=`): + + ``` + Name | Age + --------|------ + Bob | 27 + Alice | 23 + ========|====== + Total | 50 + ``` + +- **Fenced code blocks**. In addition to the normal 4-space + indentation to mark code blocks, you can explicitly mark them + and supply a language (to make syntax highlighting simple). Just + mark it like this: + + ```go + func getTrue() bool { + return true + } + ``` + + You can use 3 or more backticks to mark the beginning of the + block, and the same number to mark the end of the block. + +- **Definition lists**. A simple definition list is made of a single-line + term followed by a colon and the definition for that term. + + Cat + : Fluffy animal everyone likes + + Internet + : Vector of transmission for pictures of cats + + Terms must be separated from the previous definition by a blank line. + +- **Footnotes**. A marker in the text that will become a superscript number; + a footnote definition that will be placed in a list of footnotes at the + end of the document. A footnote looks like this: + + This is a footnote.[^1] + + [^1]: the footnote text. + +- **Autolinking**. We can find URLs that have not been + explicitly marked as links and turn them into links. + +- **Strikethrough**. Use two tildes (`~~`) to mark text that + should be crossed out. + +- **Hard line breaks**. With this extension enabled newlines in the input + translate into line breaks in the output. This extension is off by default. + +- **Non blocking space**. With this extension enabled spaces preceeded by an backslash n the input + translate non-blocking spaces in the output. This extension is off by default. + +- **Smart quotes**. Smartypants-style punctuation substitution is + supported, turning normal double- and single-quote marks into + curly quotes, etc. + +- **LaTeX-style dash parsing** is an additional option, where `--` + is translated into `–`, and `---` is translated into + `—`. This differs from most smartypants processors, which + turn a single hyphen into an ndash and a double hyphen into an + mdash. + +- **Smart fractions**, where anything that looks like a fraction + is translated into suitable HTML (instead of just a few special + cases like most smartypant processors). For example, `4/5` + becomes `<sup>4</sup>⁄<sub>5</sub>`, which renders as + <sup>4</sup>⁄<sub>5</sub>. + +- **MathJaX Support** is an additional feature which is supported by + many markdown editor. It translate inline math equation quoted by `$` + and display math block quoted by `$$` into MathJax compatible format. + hyphen `_` won't break LaTeX render within a math element any more. + + ``` + $$ + \left[ \begin{array}{a} a^l_1 \\ ⋮ \\ a^l_{d_l} \end{array}\right] + = \sigma( + \left[ \begin{matrix} + w^l_{1,1} & ⋯ & w^l_{1,d_{l-1}} \\ + ⋮ & ⋱ & ⋮ \\ + w^l_{d_l,1} & ⋯ & w^l_{d_l,d_{l-1}} \\ + \end{matrix}\right] · + \left[ \begin{array}{x} a^{l-1}_1 \\ ⋮ \\ ⋮ \\ a^{l-1}_{d_{l-1}} \end{array}\right] + + \left[ \begin{array}{b} b^l_1 \\ ⋮ \\ b^l_{d_l} \end{array}\right]) + $$ + ``` + +- **Ordered list start number**. With this extension enabled an ordered list will start with the + the number that was used to start it. + +- **Super and subscript**. With this extension enabled sequences between ^ will indicate + superscript and ~ will become a subscript. For example: H~2~O is a liquid, 2^10^ is 1024. + +- **Block level attributes**, allow setting attributes (ID, classes and key/value pairs) on block + level elements. The attribute must be enclosed with braces and be put on a line before the + element. + + ``` + {#id3 .myclass fontsize="tiny"} + # Header 1 + ``` + + Will convert into `<h1 id="id3" class="myclass" fontsize="tiny">Header 1</h1>`. + +- **Mmark support**, see <https://mmark.nl/syntax> for all new syntax elements this adds. + +## Todo + +- port https://github.com/russross/blackfriday/issues/348 +- port [LaTeX output](https://github.com/Ambrevar/Blackfriday-LaTeX): + renders output as LaTeX. +- port https://github.com/shurcooL/github_flavored_markdown to markdown +- port [markdownfmt](https://github.com/shurcooL/markdownfmt): like gofmt, + but for markdown. +- More unit testing +- Improve unicode support. It does not understand all unicode + rules (about what constitutes a letter, a punctuation symbol, + etc.), so it may fail to detect word boundaries correctly in + some instances. It is safe on all utf-8 input. + +## History + +markdown is a fork of v2 of https://github.com/russross/blackfriday that is: + +- actively maintained (sadly in Feb 2018 blackfriday was inactive for 5 months with many bugs and pull requests accumulated) +- refactored API (split into ast/parser/html sub-packages) + +Blackfriday itself was based on C implementation [sundown](https://github.com/vmg/sundown) which in turn was based on [libsoldout](http://fossil.instinctive.eu/libsoldout/home). + +## License + +[Simplified BSD License](LICENSE.txt) diff --git a/vendor/github.com/gomarkdown/markdown/ast/attribute.go b/vendor/github.com/gomarkdown/markdown/ast/attribute.go new file mode 100644 index 00000000..002c6a2e --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/ast/attribute.go @@ -0,0 +1,10 @@ +package ast + +// An attribute can be attached to block elements. They are specified as +// {#id .classs key="value"} where quotes for values are mandatory, multiple +// key/value pairs are separated by whitespace. +type Attribute struct { + ID []byte + Classes [][]byte + Attrs map[string][]byte +} diff --git a/vendor/github.com/gomarkdown/markdown/ast/doc.go b/vendor/github.com/gomarkdown/markdown/ast/doc.go new file mode 100644 index 00000000..376dc67c --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/ast/doc.go @@ -0,0 +1,4 @@ +/* +Package ast defines tree representation of a parsed markdown document. +*/ +package ast diff --git a/vendor/github.com/gomarkdown/markdown/ast/node.go b/vendor/github.com/gomarkdown/markdown/ast/node.go new file mode 100644 index 00000000..e6fcba9a --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/ast/node.go @@ -0,0 +1,559 @@ +package ast + +// ListType contains bitwise or'ed flags for list and list item objects. +type ListType int + +// These are the possible flag values for the ListItem renderer. +// Multiple flag values may be ORed together. +// These are mostly of interest if you are writing a new output format. +const ( + ListTypeOrdered ListType = 1 << iota + ListTypeDefinition + ListTypeTerm + + ListItemContainsBlock + ListItemBeginningOfList // TODO: figure out if this is of any use now + ListItemEndOfList +) + +// CellAlignFlags holds a type of alignment in a table cell. +type CellAlignFlags int + +// These are the possible flag values for the table cell renderer. +// Only a single one of these values will be used; they are not ORed together. +// These are mostly of interest if you are writing a new output format. +const ( + TableAlignmentLeft CellAlignFlags = 1 << iota + TableAlignmentRight + TableAlignmentCenter = (TableAlignmentLeft | TableAlignmentRight) +) + +func (a CellAlignFlags) String() string { + switch a { + case TableAlignmentLeft: + return "left" + case TableAlignmentRight: + return "right" + case TableAlignmentCenter: + return "center" + default: + return "" + } +} + +// DocumentMatters holds the type of a {front,main,back}matter in the document +type DocumentMatters int + +// These are all possible Document divisions. +const ( + DocumentMatterNone DocumentMatters = iota + DocumentMatterFront + DocumentMatterMain + DocumentMatterBack +) + +// CitationTypes holds the type of a citation, informative, normative or suppressed +type CitationTypes int + +const ( + CitationTypeNone CitationTypes = iota + CitationTypeSuppressed + CitationTypeInformative + CitationTypeNormative +) + +// Node defines an ast node +type Node interface { + AsContainer() *Container + AsLeaf() *Leaf + GetParent() Node + SetParent(newParent Node) + GetChildren() []Node + SetChildren(newChildren []Node) +} + +// Container is a type of node that can contain children +type Container struct { + Parent Node + Children []Node + + Literal []byte // Text contents of the leaf nodes + Content []byte // Markdown content of the block nodes + + *Attribute // Block level attribute +} + +// AsContainer returns itself as *Container +func (c *Container) AsContainer() *Container { + return c +} + +// AsLeaf returns nil +func (c *Container) AsLeaf() *Leaf { + return nil +} + +// GetParent returns parent node +func (c *Container) GetParent() Node { + return c.Parent +} + +// SetParent sets the parent node +func (c *Container) SetParent(newParent Node) { + c.Parent = newParent +} + +// GetChildren returns children nodes +func (c *Container) GetChildren() []Node { + return c.Children +} + +// SetChildren sets children node +func (c *Container) SetChildren(newChildren []Node) { + c.Children = newChildren +} + +// Leaf is a type of node that cannot have children +type Leaf struct { + Parent Node + + Literal []byte // Text contents of the leaf nodes + Content []byte // Markdown content of the block nodes + + *Attribute // Block level attribute +} + +// AsContainer returns nil +func (l *Leaf) AsContainer() *Container { + return nil +} + +// AsLeaf returns itself as *Leaf +func (l *Leaf) AsLeaf() *Leaf { + return l +} + +// GetParent returns parent node +func (l *Leaf) GetParent() Node { + return l.Parent +} + +// SetParent sets the parent nodd +func (l *Leaf) SetParent(newParent Node) { + l.Parent = newParent +} + +// GetChildren returns nil because Leaf cannot have children +func (l *Leaf) GetChildren() []Node { + return nil +} + +// SetChildren will panic becuase Leaf cannot have children +func (l *Leaf) SetChildren(newChildren []Node) { + panic("leaf node cannot have children") +} + +// Document represents markdown document node, a root of ast +type Document struct { + Container +} + +// DocumentMatter represents markdown node that signals a document +// division: frontmatter, mainmatter or backmatter. +type DocumentMatter struct { + Container + + Matter DocumentMatters +} + +// BlockQuote represents markdown block quote node +type BlockQuote struct { + Container +} + +// Aside represents an markdown aside node. +type Aside struct { + Container +} + +// List represents markdown list node +type List struct { + Container + + ListFlags ListType + Tight bool // Skip <p>s around list item data if true + BulletChar byte // '*', '+' or '-' in bullet lists + Delimiter byte // '.' or ')' after the number in ordered lists + Start int // for ordered lists this indicates the starting number if > 0 + RefLink []byte // If not nil, turns this list item into a footnote item and triggers different rendering + IsFootnotesList bool // This is a list of footnotes +} + +// ListItem represents markdown list item node +type ListItem struct { + Container + + ListFlags ListType + Tight bool // Skip <p>s around list item data if true + BulletChar byte // '*', '+' or '-' in bullet lists + Delimiter byte // '.' or ')' after the number in ordered lists + RefLink []byte // If not nil, turns this list item into a footnote item and triggers different rendering + IsFootnotesList bool // This is a list of footnotes +} + +// Paragraph represents markdown paragraph node +type Paragraph struct { + Container +} + +// Math represents markdown MathAjax inline node +type Math struct { + Leaf +} + +// MathBlock represents markdown MathAjax block node +type MathBlock struct { + Container +} + +// Heading represents markdown heading node +type Heading struct { + Container + + Level int // This holds the heading level number + HeadingID string // This might hold heading ID, if present + IsTitleblock bool // Specifies whether it's a title block + IsSpecial bool // We are a special heading (starts with .#) +} + +// HorizontalRule represents markdown horizontal rule node +type HorizontalRule struct { + Leaf +} + +// Emph represents markdown emphasis node +type Emph struct { + Container +} + +// Strong represents markdown strong node +type Strong struct { + Container +} + +// Del represents markdown del node +type Del struct { + Container +} + +// Link represents markdown link node +type Link struct { + Container + + Destination []byte // Destination is what goes into a href + Title []byte // Title is the tooltip thing that goes in a title attribute + NoteID int // NoteID contains a serial number of a footnote, zero if it's not a footnote + Footnote Node // If it's a footnote, this is a direct link to the footnote Node. Otherwise nil. + DeferredID []byte // If a deferred link this holds the original ID. +} + +// CrossReference is a reference node. +type CrossReference struct { + Container + + Destination []byte // Destination is where the reference points to +} + +// Citation is a citation node. +type Citation struct { + Leaf + + Destination [][]byte // Destination is where the citation points to. Multiple ones are allowed. + Type []CitationTypes // 1:1 mapping of destination and citation type + Suffix [][]byte // Potential citation suffix, i.e. [@!RFC1035, p. 144] +} + +// Image represents markdown image node +type Image struct { + Container + + Destination []byte // Destination is what goes into a href + Title []byte // Title is the tooltip thing that goes in a title attribute +} + +// Text represents markdown text node +type Text struct { + Leaf +} + +// HTMLBlock represents markdown html node +type HTMLBlock struct { + Leaf +} + +// CodeBlock represents markdown code block node +type CodeBlock struct { + Leaf + + IsFenced bool // Specifies whether it's a fenced code block or an indented one + Info []byte // This holds the info string + FenceChar byte + FenceLength int + FenceOffset int +} + +// Softbreak represents markdown softbreak node +// Note: not used currently +type Softbreak struct { + Leaf +} + +// Hardbreak represents markdown hard break node +type Hardbreak struct { + Leaf +} + +// NonBlockingSpace represents markdown non-blocking space node +type NonBlockingSpace struct { + Leaf +} + +// Code represents markdown code node +type Code struct { + Leaf +} + +// HTMLSpan represents markdown html span node +type HTMLSpan struct { + Leaf +} + +// Table represents markdown table node +type Table struct { + Container +} + +// TableCell represents markdown table cell node +type TableCell struct { + Container + + IsHeader bool // This tells if it's under the header row + Align CellAlignFlags // This holds the value for align attribute +} + +// TableHeader represents markdown table head node +type TableHeader struct { + Container +} + +// TableBody represents markdown table body node +type TableBody struct { + Container +} + +// TableRow represents markdown table row node +type TableRow struct { + Container +} + +// TableFooter represents markdown table foot node +type TableFooter struct { + Container +} + +// Caption represents a figure, code or quote caption +type Caption struct { + Container +} + +// CaptionFigure is a node (blockquote or codeblock) that has a caption +type CaptionFigure struct { + Container + + HeadingID string // This might hold heading ID, if present +} + +// Callout is a node that can exist both in text (where it is an actual node) and in a code block. +type Callout struct { + Leaf + + ID []byte // number of this callout +} + +// Index is a node that contains an Index item and an optional, subitem. +type Index struct { + Leaf + + Primary bool + Item []byte + Subitem []byte + ID string // ID of the index +} + +// Subscript is a subscript node +type Subscript struct { + Leaf +} + +// Subscript is a superscript node +type Superscript struct { + Leaf +} + +// Footnotes is a node that contains all footnotes +type Footnotes struct { + Container +} + +func removeNodeFromArray(a []Node, node Node) []Node { + n := len(a) + for i := 0; i < n; i++ { + if a[i] == node { + return append(a[:i], a[i+1:]...) + } + } + return nil +} + +// AppendChild appends child to children of parent +// It panics if either node is nil. +func AppendChild(parent Node, child Node) { + RemoveFromTree(child) + child.SetParent(parent) + newChildren := append(parent.GetChildren(), child) + parent.SetChildren(newChildren) +} + +// RemoveFromTree removes this node from tree +func RemoveFromTree(n Node) { + if n.GetParent() == nil { + return + } + // important: don't clear n.Children if n has no parent + // we're called from AppendChild and that might happen on a node + // that accumulated Children but hasn't been inserted into the tree + n.SetChildren(nil) + p := n.GetParent() + newChildren := removeNodeFromArray(p.GetChildren(), n) + if newChildren != nil { + p.SetChildren(newChildren) + } +} + +// GetLastChild returns last child of node n +// It's implemented as stand-alone function to keep Node interface small +func GetLastChild(n Node) Node { + a := n.GetChildren() + if len(a) > 0 { + return a[len(a)-1] + } + return nil +} + +// GetFirstChild returns first child of node n +// It's implemented as stand-alone function to keep Node interface small +func GetFirstChild(n Node) Node { + a := n.GetChildren() + if len(a) > 0 { + return a[0] + } + return nil +} + +// GetNextNode returns next sibling of node n (node after n) +// We can't make it part of Container or Leaf because we loose Node identity +func GetNextNode(n Node) Node { + parent := n.GetParent() + if parent == nil { + return nil + } + a := parent.GetChildren() + len := len(a) - 1 + for i := 0; i < len; i++ { + if a[i] == n { + return a[i+1] + } + } + return nil +} + +// GetPrevNode returns previous sibling of node n (node before n) +// We can't make it part of Container or Leaf because we loose Node identity +func GetPrevNode(n Node) Node { + parent := n.GetParent() + if parent == nil { + return nil + } + a := parent.GetChildren() + len := len(a) + for i := 1; i < len; i++ { + if a[i] == n { + return a[i-1] + } + } + return nil +} + +// WalkStatus allows NodeVisitor to have some control over the tree traversal. +// It is returned from NodeVisitor and different values allow Node.Walk to +// decide which node to go to next. +type WalkStatus int + +const ( + // GoToNext is the default traversal of every node. + GoToNext WalkStatus = iota + // SkipChildren tells walker to skip all children of current node. + SkipChildren + // Terminate tells walker to terminate the traversal. + Terminate +) + +// NodeVisitor is a callback to be called when traversing the syntax tree. +// Called twice for every node: once with entering=true when the branch is +// first visited, then with entering=false after all the children are done. +type NodeVisitor interface { + Visit(node Node, entering bool) WalkStatus +} + +// NodeVisitorFunc casts a function to match NodeVisitor interface +type NodeVisitorFunc func(node Node, entering bool) WalkStatus + +// Walk traverses tree recursively +func Walk(n Node, visitor NodeVisitor) WalkStatus { + isContainer := n.AsContainer() != nil + status := visitor.Visit(n, true) // entering + if status == Terminate { + // even if terminating, close container node + if isContainer { + visitor.Visit(n, false) + } + return status + } + if isContainer && status != SkipChildren { + children := n.GetChildren() + for _, n := range children { + status = Walk(n, visitor) + if status == Terminate { + return status + } + } + } + if isContainer { + status = visitor.Visit(n, false) // exiting + if status == Terminate { + return status + } + } + return GoToNext +} + +// Visit calls visitor function +func (f NodeVisitorFunc) Visit(node Node, entering bool) WalkStatus { + return f(node, entering) +} + +// WalkFunc is like Walk but accepts just a callback function +func WalkFunc(n Node, f NodeVisitorFunc) { + visitor := NodeVisitorFunc(f) + Walk(n, visitor) +} diff --git a/vendor/github.com/gomarkdown/markdown/ast/print.go b/vendor/github.com/gomarkdown/markdown/ast/print.go new file mode 100644 index 00000000..75daf911 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/ast/print.go @@ -0,0 +1,165 @@ +package ast + +import ( + "bytes" + "fmt" + "io" + "strings" +) + +// Print is for debugging. It prints a string representation of parsed +// markdown doc (result of parser.Parse()) to dst. +// +// To make output readable, it shortens text output. +func Print(dst io.Writer, doc Node) { + PrintWithPrefix(dst, doc, " ") +} + +// PrintWithPrefix is like Print but allows customizing prefix used for +// indentation. By default it's 2 spaces. You can change it to e.g. tab +// by passing "\t" +func PrintWithPrefix(w io.Writer, doc Node, prefix string) { + // for more compact output, don't print outer Document + if _, ok := doc.(*Document); ok { + for _, c := range doc.GetChildren() { + printRecur(w, c, prefix, 0) + } + } else { + printRecur(w, doc, prefix, 0) + } +} + +// ToString is like Dump but returns result as a string +func ToString(doc Node) string { + var buf bytes.Buffer + Print(&buf, doc) + return buf.String() +} + +func contentToString(d1 []byte, d2 []byte) string { + if d1 != nil { + return string(d1) + } + if d2 != nil { + return string(d2) + } + return "" +} + +func getContent(node Node) string { + if c := node.AsContainer(); c != nil { + return contentToString(c.Literal, c.Content) + } + leaf := node.AsLeaf() + return contentToString(leaf.Literal, leaf.Content) +} + +func shortenString(s string, maxLen int) string { + // for cleaner, one-line ouput, replace some white-space chars + // with their escaped version + s = strings.Replace(s, "\n", `\n`, -1) + s = strings.Replace(s, "\r", `\r`, -1) + s = strings.Replace(s, "\t", `\t`, -1) + if maxLen < 0 { + return s + } + if len(s) < maxLen { + return s + } + // add "..." to indicate truncation + return s[:maxLen-3] + "..." +} + +// get a short name of the type of v which excludes package name +// and strips "()" from the end +func getNodeType(node Node) string { + s := fmt.Sprintf("%T", node) + s = strings.TrimSuffix(s, "()") + if idx := strings.Index(s, "."); idx != -1 { + return s[idx+1:] + } + return s +} + +func printDefault(w io.Writer, indent string, typeName string, content string) { + content = strings.TrimSpace(content) + if len(content) > 0 { + fmt.Fprintf(w, "%s%s '%s'\n", indent, typeName, content) + } else { + fmt.Fprintf(w, "%s%s\n", indent, typeName) + } +} + +func getListFlags(f ListType) string { + var s string + if f&ListTypeOrdered != 0 { + s += "ordered " + } + if f&ListTypeDefinition != 0 { + s += "definition " + } + if f&ListTypeTerm != 0 { + s += "term " + } + if f&ListItemContainsBlock != 0 { + s += "has_block " + } + if f&ListItemBeginningOfList != 0 { + s += "start " + } + if f&ListItemEndOfList != 0 { + s += "end " + } + s = strings.TrimSpace(s) + return s +} + +func printRecur(w io.Writer, node Node, prefix string, depth int) { + if node == nil { + return + } + indent := strings.Repeat(prefix, depth) + + content := shortenString(getContent(node), 40) + typeName := getNodeType(node) + switch v := node.(type) { + case *Link: + content := "url=" + string(v.Destination) + printDefault(w, indent, typeName, content) + case *Image: + content := "url=" + string(v.Destination) + printDefault(w, indent, typeName, content) + case *List: + if v.Start > 1 { + content += fmt.Sprintf("start=%d ", v.Start) + } + if v.Tight { + content += "tight " + } + if v.IsFootnotesList { + content += "footnotes " + } + flags := getListFlags(v.ListFlags) + if len(flags) > 0 { + content += "flags=" + flags + " " + } + printDefault(w, indent, typeName, content) + case *ListItem: + if v.Tight { + content += "tight " + } + if v.IsFootnotesList { + content += "footnotes " + } + flags := getListFlags(v.ListFlags) + if len(flags) > 0 { + content += "flags=" + flags + " " + } + printDefault(w, indent, typeName, content) + default: + printDefault(w, indent, typeName, content) + } + for _, child := range node.GetChildren() { + printRecur(w, child, prefix, depth+1) + } +} diff --git a/vendor/github.com/gomarkdown/markdown/changes-from-blackfriday.md b/vendor/github.com/gomarkdown/markdown/changes-from-blackfriday.md new file mode 100644 index 00000000..b618dfef --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/changes-from-blackfriday.md @@ -0,0 +1,27 @@ +## Changes from blackfriday + +This library is derived from blackfriday library. Here's a list of changes. + +**Redesigned API** + +- split into 3 separate packages: ast, parser and html (for html renderer). This makes the API more manageable. It also separates e.g. parser option from renderer options +- changed how AST node is represented from union-like representation (manually keeping track of the type of the node) to using interface{} (which is a Go way to combine an arbitrary value with its type) + +**Allow re-using most of html renderer logic** + +You can implement your own renderer by implementing `Renderer` interface. + +Implementing a full renderer is a lot of work and often you just want to tweak html rendering of few node typs. + +I've added a way to hook `Renderer.Render` function in html renderer with a custom function that can take over rendering of specific nodes. + +I use it myself to do syntax-highlighting of code snippets. + +**Speed up go test** + +Running `go test` was really slow (17 secs) because it did a poor man's version of fuzzing by feeding the parser all subsets of test strings in order to find panics +due to incorrect parsing logic. + +I've moved that logic to `cmd/crashtest`, so that it can be run on CI but not slow down regular development. + +Now `go test` is blazing fast. diff --git a/vendor/github.com/gomarkdown/markdown/codecov.yml b/vendor/github.com/gomarkdown/markdown/codecov.yml new file mode 100644 index 00000000..f681ff11 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/codecov.yml @@ -0,0 +1,8 @@ +coverage: + status: + project: + default: + # basic + target: 60% + threshold: 2% + base: auto diff --git a/vendor/github.com/gomarkdown/markdown/doc.go b/vendor/github.com/gomarkdown/markdown/doc.go new file mode 100644 index 00000000..9fb77e02 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/doc.go @@ -0,0 +1,35 @@ +/* +Package markdown implements markdown parser and HTML renderer. + +It parses markdown into AST format which can be serialized to HTML +(using html.Renderer) or possibly other formats (using alternate renderers). + +Convert markdown to HTML + +The simplest way to convert markdown document to HTML + + md := []byte("## markdown document") + html := markdown.ToHTML(md, nil, nil) + +Customizing parsing and HTML rendering + +You can customize parser and HTML renderer: + + import ( + "github.com/gomarkdown/markdown/parser" + "github.com/gomarkdown/markdown/renderer" + "github.com/gomarkdown/markdown" + ) + extensions := parser.CommonExtensions | parser.AutoHeadingIDs + p := parser.NewWithExensions(extensions) + + htmlFlags := html.CommonFlags | html.HrefTargetBlank + opts := html.RendererOptions{Flags: htmlFlags} + renderer := html.NewRenderer(opts) + + md := []byte("markdown text") + html := markdown.ToHTML(md, p, renderer) + +For a cmd-line tool see https://github.com/gomarkdown/mdtohtml +*/ +package markdown diff --git a/vendor/github.com/gomarkdown/markdown/fuzz.go b/vendor/github.com/gomarkdown/markdown/fuzz.go new file mode 100644 index 00000000..704182b8 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/fuzz.go @@ -0,0 +1,9 @@ +// +build gofuzz + +package markdown + +// Fuzz is to be used by https://github.com/dvyukov/go-fuzz +func Fuzz(data []byte) int { + Parse(data, nil) + return 0 +} diff --git a/vendor/github.com/gomarkdown/markdown/go.mod b/vendor/github.com/gomarkdown/markdown/go.mod new file mode 100644 index 00000000..899e3237 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/go.mod @@ -0,0 +1,5 @@ +module github.com/gomarkdown/markdown + +go 1.12 + +require golang.org/dl v0.0.0-20190829154251-82a15e2f2ead // indirect diff --git a/vendor/github.com/gomarkdown/markdown/html/callouts.go b/vendor/github.com/gomarkdown/markdown/html/callouts.go new file mode 100644 index 00000000..e377af22 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/html/callouts.go @@ -0,0 +1,42 @@ +package html + +import ( + "bytes" + "io" + + "github.com/gomarkdown/markdown/ast" + "github.com/gomarkdown/markdown/parser" +) + +// EscapeHTMLCallouts writes html-escaped d to w. It escapes &, <, > and " characters, *but* +// expands callouts <<N>> with the callout HTML, i.e. by calling r.callout() with a newly created +// ast.Callout node. +func (r *Renderer) EscapeHTMLCallouts(w io.Writer, d []byte) { + ld := len(d) +Parse: + for i := 0; i < ld; i++ { + for _, comment := range r.opts.Comments { + if !bytes.HasPrefix(d[i:], comment) { + break + } + + lc := len(comment) + if i+lc < ld { + if id, consumed := parser.IsCallout(d[i+lc:]); consumed > 0 { + // We have seen a callout + callout := &ast.Callout{ID: id} + r.callout(w, callout) + i += consumed + lc - 1 + continue Parse + } + } + } + + escSeq := Escaper[d[i]] + if escSeq != nil { + w.Write(escSeq) + } else { + w.Write([]byte{d[i]}) + } + } +} diff --git a/vendor/github.com/gomarkdown/markdown/html/doc.go b/vendor/github.com/gomarkdown/markdown/html/doc.go new file mode 100644 index 00000000..f837c63d --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/html/doc.go @@ -0,0 +1,43 @@ +/* +Package html implements HTML renderer of parsed markdown document. + +Configuring and customizing a renderer + +A renderer can be configured with multiple options: + + import "github.com/gomarkdown/markdown/html" + + flags := html.CommonFlags | html.CompletePage | html.HrefTargetBlank + opts := html.RenderOptions{ + TItle: "A custom title", + Flags: flags, + } + renderer := html.NewRenderer(opts) + +You can also re-use most of the logic and customize rendering of selected nodes +by providing node render hook. +This is most useful for rendering nodes that allow for design choices, like +links or code blocks. + + import ( + "github.com/gomarkdown/markdown/html" + "github.com/gomarkdown/markdown/ast" + ) + + // a very dummy render hook that will output "code_replacements" instead of + // <code>${content}</code> emitted by html.Renderer + func renderHookCodeBlock(w io.Writer, node *ast.Node, entering bool) (ast.WalkStatus, bool) { + _, ok := node.Data.(*ast.CodeBlockData) + if !ok { + return ast.GoToNext, false + } + io.WriteString(w, "code_replacement") + return ast.GoToNext, true + } + + opts := html.RendererOptions{ + RenderNodeHook: renderHookCodeBlock, + } + renderer := html.NewRenderer(opts) +*/ +package html diff --git a/vendor/github.com/gomarkdown/markdown/html/esc.go b/vendor/github.com/gomarkdown/markdown/html/esc.go new file mode 100644 index 00000000..89ec9a27 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/html/esc.go @@ -0,0 +1,50 @@ +package html + +import ( + "html" + "io" +) + +var Escaper = [256][]byte{ + '&': []byte("&"), + '<': []byte("<"), + '>': []byte(">"), + '"': []byte("""), +} + +// EscapeHTML writes html-escaped d to w. It escapes &, <, > and " characters. +func EscapeHTML(w io.Writer, d []byte) { + var start, end int + n := len(d) + for end < n { + escSeq := Escaper[d[end]] + if escSeq != nil { + w.Write(d[start:end]) + w.Write(escSeq) + start = end + 1 + } + end++ + } + if start < n && end <= n { + w.Write(d[start:end]) + } +} + +func escLink(w io.Writer, text []byte) { + unesc := html.UnescapeString(string(text)) + EscapeHTML(w, []byte(unesc)) +} + +// Escape writes the text to w, but skips the escape character. +func Escape(w io.Writer, text []byte) { + esc := false + for i := 0; i < len(text); i++ { + if text[i] == '\\' { + esc = !esc + } + if esc && text[i] == '\\' { + continue + } + w.Write([]byte{text[i]}) + } +} diff --git a/vendor/github.com/gomarkdown/markdown/html/renderer.go b/vendor/github.com/gomarkdown/markdown/html/renderer.go new file mode 100644 index 00000000..367f7dfa --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/html/renderer.go @@ -0,0 +1,1318 @@ +package html + +import ( + "bytes" + "fmt" + "io" + "regexp" + "sort" + "strconv" + "strings" + + "github.com/gomarkdown/markdown/ast" +) + +// Flags control optional behavior of HTML renderer. +type Flags int + +// IDTag is the tag used for tag identification, it defaults to "id", some renderers +// may wish to override this and use e.g. "anchor". +var IDTag = "id" + +// HTML renderer configuration options. +const ( + FlagsNone Flags = 0 + SkipHTML Flags = 1 << iota // Skip preformatted HTML blocks + SkipImages // Skip embedded images + SkipLinks // Skip all links + Safelink // Only link to trusted protocols + NofollowLinks // Only link with rel="nofollow" + NoreferrerLinks // Only link with rel="noreferrer" + HrefTargetBlank // Add a blank target + CompletePage // Generate a complete HTML page + UseXHTML // Generate XHTML output instead of HTML + FootnoteReturnLinks // Generate a link at the end of a footnote to return to the source + FootnoteNoHRTag // Do not output an HR after starting a footnote list. + Smartypants // Enable smart punctuation substitutions + SmartypantsFractions // Enable smart fractions (with Smartypants) + SmartypantsDashes // Enable smart dashes (with Smartypants) + SmartypantsLatexDashes // Enable LaTeX-style dashes (with Smartypants) + SmartypantsAngledQuotes // Enable angled double quotes (with Smartypants) for double quotes rendering + SmartypantsQuotesNBSP // Enable « French guillemets » (with Smartypants) + TOC // Generate a table of contents + + CommonFlags Flags = Smartypants | SmartypantsFractions | SmartypantsDashes | SmartypantsLatexDashes +) + +var ( + htmlTagRe = regexp.MustCompile("(?i)^" + htmlTag) +) + +const ( + htmlTag = "(?:" + openTag + "|" + closeTag + "|" + htmlComment + "|" + + processingInstruction + "|" + declaration + "|" + cdata + ")" + closeTag = "</" + tagName + "\\s*[>]" + openTag = "<" + tagName + attribute + "*" + "\\s*/?>" + attribute = "(?:" + "\\s+" + attributeName + attributeValueSpec + "?)" + attributeValue = "(?:" + unquotedValue + "|" + singleQuotedValue + "|" + doubleQuotedValue + ")" + attributeValueSpec = "(?:" + "\\s*=" + "\\s*" + attributeValue + ")" + attributeName = "[a-zA-Z_:][a-zA-Z0-9:._-]*" + cdata = "<!\\[CDATA\\[[\\s\\S]*?\\]\\]>" + declaration = "<![A-Z]+" + "\\s+[^>]*>" + doubleQuotedValue = "\"[^\"]*\"" + htmlComment = "<!---->|<!--(?:-?[^>-])(?:-?[^-])*-->" + processingInstruction = "[<][?].*?[?][>]" + singleQuotedValue = "'[^']*'" + tagName = "[A-Za-z][A-Za-z0-9-]*" + unquotedValue = "[^\"'=<>`\\x00-\\x20]+" +) + +// RenderNodeFunc allows reusing most of Renderer logic and replacing +// rendering of some nodes. If it returns false, Renderer.RenderNode +// will execute its logic. If it returns true, Renderer.RenderNode will +// skip rendering this node and will return WalkStatus +type RenderNodeFunc func(w io.Writer, node ast.Node, entering bool) (ast.WalkStatus, bool) + +// RendererOptions is a collection of supplementary parameters tweaking +// the behavior of various parts of HTML renderer. +type RendererOptions struct { + // Prepend this text to each relative URL. + AbsolutePrefix string + // Add this text to each footnote anchor, to ensure uniqueness. + FootnoteAnchorPrefix string + // Show this text inside the <a> tag for a footnote return link, if the + // FootnoteReturnLinks flag is enabled. If blank, the string + // <sup>[return]</sup> is used. + FootnoteReturnLinkContents string + // CitationFormatString defines how a citation is rendered. If blnck, the string + // <sup>[%s]</sup> is used. Where %s will be substituted with the citation target. + CitationFormatString string + // If set, add this text to the front of each Heading ID, to ensure uniqueness. + HeadingIDPrefix string + // If set, add this text to the back of each Heading ID, to ensure uniqueness. + HeadingIDSuffix string + + Title string // Document title (used if CompletePage is set) + CSS string // Optional CSS file URL (used if CompletePage is set) + Icon string // Optional icon file URL (used if CompletePage is set) + Head []byte // Optional head data injected in the <head> section (used if CompletePage is set) + + Flags Flags // Flags allow customizing this renderer's behavior + + // if set, called at the start of RenderNode(). Allows replacing + // rendering of some nodes + RenderNodeHook RenderNodeFunc + + // Comments is a list of comments the renderer should detect when + // parsing code blocks and detecting callouts. + Comments [][]byte + + // Generator is a meta tag that is inserted in the generated HTML so show what rendered it. It should not include the closing tag. + // Defaults (note content quote is not closed) to ` <meta name="GENERATOR" content="github.com/gomarkdown/markdown markdown processor for Go` + Generator string +} + +// Renderer implements Renderer interface for HTML output. +// +// Do not create this directly, instead use the NewRenderer function. +type Renderer struct { + opts RendererOptions + + closeTag string // how to end singleton tags: either " />" or ">" + + // Track heading IDs to prevent ID collision in a single generation. + headingIDs map[string]int + + lastOutputLen int + disableTags int + + sr *SPRenderer + + documentMatter ast.DocumentMatters // keep track of front/main/back matter. +} + +// NewRenderer creates and configures an Renderer object, which +// satisfies the Renderer interface. +func NewRenderer(opts RendererOptions) *Renderer { + // configure the rendering engine + closeTag := ">" + if opts.Flags&UseXHTML != 0 { + closeTag = " />" + } + + if opts.FootnoteReturnLinkContents == "" { + opts.FootnoteReturnLinkContents = `<sup>[return]</sup>` + } + if opts.CitationFormatString == "" { + opts.CitationFormatString = `<sup>[%s]</sup>` + } + if opts.Generator == "" { + opts.Generator = ` <meta name="GENERATOR" content="github.com/gomarkdown/markdown markdown processor for Go` + } + + return &Renderer{ + opts: opts, + + closeTag: closeTag, + headingIDs: make(map[string]int), + + sr: NewSmartypantsRenderer(opts.Flags), + } +} + +func isHTMLTag(tag []byte, tagname string) bool { + found, _ := findHTMLTagPos(tag, tagname) + return found +} + +// Look for a character, but ignore it when it's in any kind of quotes, it +// might be JavaScript +func skipUntilCharIgnoreQuotes(html []byte, start int, char byte) int { + inSingleQuote := false + inDoubleQuote := false + inGraveQuote := false + i := start + for i < len(html) { + switch { + case html[i] == char && !inSingleQuote && !inDoubleQuote && !inGraveQuote: + return i + case html[i] == '\'': + inSingleQuote = !inSingleQuote + case html[i] == '"': + inDoubleQuote = !inDoubleQuote + case html[i] == '`': + inGraveQuote = !inGraveQuote + } + i++ + } + return start +} + +func findHTMLTagPos(tag []byte, tagname string) (bool, int) { + i := 0 + if i < len(tag) && tag[0] != '<' { + return false, -1 + } + i++ + i = skipSpace(tag, i) + + if i < len(tag) && tag[i] == '/' { + i++ + } + + i = skipSpace(tag, i) + j := 0 + for ; i < len(tag); i, j = i+1, j+1 { + if j >= len(tagname) { + break + } + + if strings.ToLower(string(tag[i]))[0] != tagname[j] { + return false, -1 + } + } + + if i == len(tag) { + return false, -1 + } + + rightAngle := skipUntilCharIgnoreQuotes(tag, i, '>') + if rightAngle >= i { + return true, rightAngle + } + + return false, -1 +} + +func isRelativeLink(link []byte) (yes bool) { + // a tag begin with '#' + if link[0] == '#' { + return true + } + + // link begin with '/' but not '//', the second maybe a protocol relative link + if len(link) >= 2 && link[0] == '/' && link[1] != '/' { + return true + } + + // only the root '/' + if len(link) == 1 && link[0] == '/' { + return true + } + + // current directory : begin with "./" + if bytes.HasPrefix(link, []byte("./")) { + return true + } + + // parent directory : begin with "../" + if bytes.HasPrefix(link, []byte("../")) { + return true + } + + return false +} + +func (r *Renderer) ensureUniqueHeadingID(id string) string { + for count, found := r.headingIDs[id]; found; count, found = r.headingIDs[id] { + tmp := fmt.Sprintf("%s-%d", id, count+1) + + if _, tmpFound := r.headingIDs[tmp]; !tmpFound { + r.headingIDs[id] = count + 1 + id = tmp + } else { + id = id + "-1" + } + } + + if _, found := r.headingIDs[id]; !found { + r.headingIDs[id] = 0 + } + + return id +} + +func (r *Renderer) addAbsPrefix(link []byte) []byte { + if r.opts.AbsolutePrefix != "" && isRelativeLink(link) && link[0] != '.' { + newDest := r.opts.AbsolutePrefix + if link[0] != '/' { + newDest += "/" + } + newDest += string(link) + return []byte(newDest) + } + return link +} + +func appendLinkAttrs(attrs []string, flags Flags, link []byte) []string { + if isRelativeLink(link) { + return attrs + } + var val []string + if flags&NofollowLinks != 0 { + val = append(val, "nofollow") + } + if flags&NoreferrerLinks != 0 { + val = append(val, "noreferrer") + } + if flags&HrefTargetBlank != 0 { + attrs = append(attrs, `target="_blank"`) + } + if len(val) == 0 { + return attrs + } + attr := fmt.Sprintf("rel=%q", strings.Join(val, " ")) + return append(attrs, attr) +} + +func isMailto(link []byte) bool { + return bytes.HasPrefix(link, []byte("mailto:")) +} + +func needSkipLink(flags Flags, dest []byte) bool { + if flags&SkipLinks != 0 { + return true + } + return flags&Safelink != 0 && !isSafeLink(dest) && !isMailto(dest) +} + +func isSmartypantable(node ast.Node) bool { + switch node.GetParent().(type) { + case *ast.Link, *ast.CodeBlock, *ast.Code: + return false + } + return true +} + +func appendLanguageAttr(attrs []string, info []byte) []string { + if len(info) == 0 { + return attrs + } + endOfLang := bytes.IndexAny(info, "\t ") + if endOfLang < 0 { + endOfLang = len(info) + } + s := `class="language-` + string(info[:endOfLang]) + `"` + return append(attrs, s) +} + +func (r *Renderer) outTag(w io.Writer, name string, attrs []string) { + s := name + if len(attrs) > 0 { + s += " " + strings.Join(attrs, " ") + } + io.WriteString(w, s+">") + r.lastOutputLen = 1 +} + +func footnoteRef(prefix string, node *ast.Link) string { + urlFrag := prefix + string(slugify(node.Destination)) + nStr := strconv.Itoa(node.NoteID) + anchor := `<a href="#fn:` + urlFrag + `">` + nStr + `</a>` + return `<sup class="footnote-ref" id="fnref:` + urlFrag + `">` + anchor + `</sup>` +} + +func footnoteItem(prefix string, slug []byte) string { + return `<li id="fn:` + prefix + string(slug) + `">` +} + +func footnoteReturnLink(prefix, returnLink string, slug []byte) string { + return ` <a class="footnote-return" href="#fnref:` + prefix + string(slug) + `">` + returnLink + `</a>` +} + +func listItemOpenCR(listItem *ast.ListItem) bool { + if ast.GetPrevNode(listItem) == nil { + return false + } + ld := listItem.Parent.(*ast.List) + return !ld.Tight && ld.ListFlags&ast.ListTypeDefinition == 0 +} + +func skipParagraphTags(para *ast.Paragraph) bool { + parent := para.Parent + grandparent := parent.GetParent() + if grandparent == nil || !isList(grandparent) { + return false + } + isParentTerm := isListItemTerm(parent) + grandparentListData := grandparent.(*ast.List) + tightOrTerm := grandparentListData.Tight || isParentTerm + return tightOrTerm +} + +func (r *Renderer) out(w io.Writer, d []byte) { + r.lastOutputLen = len(d) + if r.disableTags > 0 { + d = htmlTagRe.ReplaceAll(d, []byte{}) + } + w.Write(d) +} + +func (r *Renderer) outs(w io.Writer, s string) { + r.lastOutputLen = len(s) + if r.disableTags > 0 { + s = htmlTagRe.ReplaceAllString(s, "") + } + io.WriteString(w, s) +} + +func (r *Renderer) cr(w io.Writer) { + if r.lastOutputLen > 0 { + r.outs(w, "\n") + } +} + +var ( + openHTags = []string{"<h1", "<h2", "<h3", "<h4", "<h5"} + closeHTags = []string{"</h1>", "</h2>", "</h3>", "</h4>", "</h5>"} +) + +func headingOpenTagFromLevel(level int) string { + if level < 1 || level > 5 { + return "<h6" + } + return openHTags[level-1] +} + +func headingCloseTagFromLevel(level int) string { + if level < 1 || level > 5 { + return "</h6>" + } + return closeHTags[level-1] +} + +func (r *Renderer) outHRTag(w io.Writer, attrs []string) { + hr := tagWithAttributes("<hr", attrs) + r.outOneOf(w, r.opts.Flags&UseXHTML == 0, hr, "<hr />") +} + +func (r *Renderer) text(w io.Writer, text *ast.Text) { + if r.opts.Flags&Smartypants != 0 { + var tmp bytes.Buffer + EscapeHTML(&tmp, text.Literal) + r.sr.Process(w, tmp.Bytes()) + } else { + _, parentIsLink := text.Parent.(*ast.Link) + if parentIsLink { + escLink(w, text.Literal) + } else { + EscapeHTML(w, text.Literal) + } + } +} + +func (r *Renderer) hardBreak(w io.Writer, node *ast.Hardbreak) { + r.outOneOf(w, r.opts.Flags&UseXHTML == 0, "<br>", "<br />") + r.cr(w) +} + +func (r *Renderer) nonBlockingSpace(w io.Writer, node *ast.NonBlockingSpace) { + r.outs(w, " ") +} + +func (r *Renderer) outOneOf(w io.Writer, outFirst bool, first string, second string) { + if outFirst { + r.outs(w, first) + } else { + r.outs(w, second) + } +} + +func (r *Renderer) outOneOfCr(w io.Writer, outFirst bool, first string, second string) { + if outFirst { + r.cr(w) + r.outs(w, first) + } else { + r.outs(w, second) + r.cr(w) + } +} + +func (r *Renderer) htmlSpan(w io.Writer, span *ast.HTMLSpan) { + if r.opts.Flags&SkipHTML == 0 { + r.out(w, span.Literal) + } +} + +func (r *Renderer) linkEnter(w io.Writer, link *ast.Link) { + var attrs []string + dest := link.Destination + dest = r.addAbsPrefix(dest) + var hrefBuf bytes.Buffer + hrefBuf.WriteString("href=\"") + escLink(&hrefBuf, dest) + hrefBuf.WriteByte('"') + attrs = append(attrs, hrefBuf.String()) + if link.NoteID != 0 { + r.outs(w, footnoteRef(r.opts.FootnoteAnchorPrefix, link)) + return + } + + attrs = appendLinkAttrs(attrs, r.opts.Flags, dest) + if len(link.Title) > 0 { + var titleBuff bytes.Buffer + titleBuff.WriteString("title=\"") + EscapeHTML(&titleBuff, link.Title) + titleBuff.WriteByte('"') + attrs = append(attrs, titleBuff.String()) + } + r.outTag(w, "<a", attrs) +} + +func (r *Renderer) linkExit(w io.Writer, link *ast.Link) { + if link.NoteID == 0 { + r.outs(w, "</a>") + } +} + +func (r *Renderer) link(w io.Writer, link *ast.Link, entering bool) { + // mark it but don't link it if it is not a safe link: no smartypants + if needSkipLink(r.opts.Flags, link.Destination) { + r.outOneOf(w, entering, "<tt>", "</tt>") + return + } + + if entering { + r.linkEnter(w, link) + } else { + r.linkExit(w, link) + } +} + +func (r *Renderer) imageEnter(w io.Writer, image *ast.Image) { + dest := image.Destination + dest = r.addAbsPrefix(dest) + if r.disableTags == 0 { + //if options.safe && potentiallyUnsafe(dest) { + //out(w, `<img src="" alt="`) + //} else { + r.outs(w, `<img src="`) + escLink(w, dest) + r.outs(w, `" alt="`) + //} + } + r.disableTags++ +} + +func (r *Renderer) imageExit(w io.Writer, image *ast.Image) { + r.disableTags-- + if r.disableTags == 0 { + if image.Title != nil { + r.outs(w, `" title="`) + EscapeHTML(w, image.Title) + } + r.outs(w, `" />`) + } +} + +func (r *Renderer) paragraphEnter(w io.Writer, para *ast.Paragraph) { + // TODO: untangle this clusterfuck about when the newlines need + // to be added and when not. + prev := ast.GetPrevNode(para) + if prev != nil { + switch prev.(type) { + case *ast.HTMLBlock, *ast.List, *ast.Paragraph, *ast.Heading, *ast.CaptionFigure, *ast.CodeBlock, *ast.BlockQuote, *ast.Aside, *ast.HorizontalRule: + r.cr(w) + } + } + + if prev == nil { + _, isParentBlockQuote := para.Parent.(*ast.BlockQuote) + if isParentBlockQuote { + r.cr(w) + } + _, isParentAside := para.Parent.(*ast.Aside) + if isParentAside { + r.cr(w) + } + } + + tag := tagWithAttributes("<p", BlockAttrs(para)) + r.outs(w, tag) +} + +func (r *Renderer) paragraphExit(w io.Writer, para *ast.Paragraph) { + r.outs(w, "</p>") + if !(isListItem(para.Parent) && ast.GetNextNode(para) == nil) { + r.cr(w) + } +} + +func (r *Renderer) paragraph(w io.Writer, para *ast.Paragraph, entering bool) { + if skipParagraphTags(para) { + return + } + if entering { + r.paragraphEnter(w, para) + } else { + r.paragraphExit(w, para) + } +} +func (r *Renderer) image(w io.Writer, node *ast.Image, entering bool) { + if entering { + r.imageEnter(w, node) + } else { + r.imageExit(w, node) + } +} + +func (r *Renderer) code(w io.Writer, node *ast.Code) { + r.outs(w, "<code>") + EscapeHTML(w, node.Literal) + r.outs(w, "</code>") +} + +func (r *Renderer) htmlBlock(w io.Writer, node *ast.HTMLBlock) { + if r.opts.Flags&SkipHTML != 0 { + return + } + r.cr(w) + r.out(w, node.Literal) + r.cr(w) +} + +func (r *Renderer) headingEnter(w io.Writer, nodeData *ast.Heading) { + var attrs []string + var class string + // TODO(miek): add helper functions for coalescing these classes. + if nodeData.IsTitleblock { + class = "title" + } + if nodeData.IsSpecial { + if class != "" { + class += " special" + } else { + class = "special" + } + } + if class != "" { + attrs = []string{`class="` + class + `"`} + } + if nodeData.HeadingID != "" { + id := r.ensureUniqueHeadingID(nodeData.HeadingID) + if r.opts.HeadingIDPrefix != "" { + id = r.opts.HeadingIDPrefix + id + } + if r.opts.HeadingIDSuffix != "" { + id = id + r.opts.HeadingIDSuffix + } + attrID := `id="` + id + `"` + attrs = append(attrs, attrID) + } + attrs = append(attrs, BlockAttrs(nodeData)...) + r.cr(w) + r.outTag(w, headingOpenTagFromLevel(nodeData.Level), attrs) +} + +func (r *Renderer) headingExit(w io.Writer, heading *ast.Heading) { + r.outs(w, headingCloseTagFromLevel(heading.Level)) + if !(isListItem(heading.Parent) && ast.GetNextNode(heading) == nil) { + r.cr(w) + } +} + +func (r *Renderer) heading(w io.Writer, node *ast.Heading, entering bool) { + if entering { + r.headingEnter(w, node) + } else { + r.headingExit(w, node) + } +} + +func (r *Renderer) horizontalRule(w io.Writer, node *ast.HorizontalRule) { + r.cr(w) + r.outHRTag(w, BlockAttrs(node)) + r.cr(w) +} + +func (r *Renderer) listEnter(w io.Writer, nodeData *ast.List) { + // TODO: attrs don't seem to be set + var attrs []string + + if nodeData.IsFootnotesList { + r.outs(w, "\n<div class=\"footnotes\">\n\n") + if r.opts.Flags&FootnoteNoHRTag == 0 { + r.outHRTag(w, nil) + r.cr(w) + } + } + r.cr(w) + if isListItem(nodeData.Parent) { + grand := nodeData.Parent.GetParent() + if isListTight(grand) { + r.cr(w) + } + } + + openTag := "<ul" + if nodeData.ListFlags&ast.ListTypeOrdered != 0 { + if nodeData.Start > 0 { + attrs = append(attrs, fmt.Sprintf(`start="%d"`, nodeData.Start)) + } + openTag = "<ol" + } + if nodeData.ListFlags&ast.ListTypeDefinition != 0 { + openTag = "<dl" + } + attrs = append(attrs, BlockAttrs(nodeData)...) + r.outTag(w, openTag, attrs) + r.cr(w) +} + +func (r *Renderer) listExit(w io.Writer, list *ast.List) { + closeTag := "</ul>" + if list.ListFlags&ast.ListTypeOrdered != 0 { + closeTag = "</ol>" + } + if list.ListFlags&ast.ListTypeDefinition != 0 { + closeTag = "</dl>" + } + r.outs(w, closeTag) + + //cr(w) + //if node.parent.Type != Item { + // cr(w) + //} + parent := list.Parent + switch parent.(type) { + case *ast.ListItem: + if ast.GetNextNode(list) != nil { + r.cr(w) + } + case *ast.Document, *ast.BlockQuote, *ast.Aside: + r.cr(w) + } + + if list.IsFootnotesList { + r.outs(w, "\n</div>\n") + } +} + +func (r *Renderer) list(w io.Writer, list *ast.List, entering bool) { + if entering { + r.listEnter(w, list) + } else { + r.listExit(w, list) + } +} + +func (r *Renderer) listItemEnter(w io.Writer, listItem *ast.ListItem) { + if listItemOpenCR(listItem) { + r.cr(w) + } + if listItem.RefLink != nil { + slug := slugify(listItem.RefLink) + r.outs(w, footnoteItem(r.opts.FootnoteAnchorPrefix, slug)) + return + } + + openTag := "<li>" + if listItem.ListFlags&ast.ListTypeDefinition != 0 { + openTag = "<dd>" + } + if listItem.ListFlags&ast.ListTypeTerm != 0 { + openTag = "<dt>" + } + r.outs(w, openTag) +} + +func (r *Renderer) listItemExit(w io.Writer, listItem *ast.ListItem) { + if listItem.RefLink != nil && r.opts.Flags&FootnoteReturnLinks != 0 { + slug := slugify(listItem.RefLink) + prefix := r.opts.FootnoteAnchorPrefix + link := r.opts.FootnoteReturnLinkContents + s := footnoteReturnLink(prefix, link, slug) + r.outs(w, s) + } + + closeTag := "</li>" + if listItem.ListFlags&ast.ListTypeDefinition != 0 { + closeTag = "</dd>" + } + if listItem.ListFlags&ast.ListTypeTerm != 0 { + closeTag = "</dt>" + } + r.outs(w, closeTag) + r.cr(w) +} + +func (r *Renderer) listItem(w io.Writer, listItem *ast.ListItem, entering bool) { + if entering { + r.listItemEnter(w, listItem) + } else { + r.listItemExit(w, listItem) + } +} + +func (r *Renderer) codeBlock(w io.Writer, codeBlock *ast.CodeBlock) { + var attrs []string + // TODO(miek): this can add multiple class= attribute, they should be coalesced into one. + // This is probably true for some other elements as well + attrs = appendLanguageAttr(attrs, codeBlock.Info) + attrs = append(attrs, BlockAttrs(codeBlock)...) + r.cr(w) + + r.outs(w, "<pre>") + code := tagWithAttributes("<code", attrs) + r.outs(w, code) + if r.opts.Comments != nil { + r.EscapeHTMLCallouts(w, codeBlock.Literal) + } else { + EscapeHTML(w, codeBlock.Literal) + } + r.outs(w, "</code>") + r.outs(w, "</pre>") + if !isListItem(codeBlock.Parent) { + r.cr(w) + } +} + +func (r *Renderer) caption(w io.Writer, caption *ast.Caption, entering bool) { + if entering { + r.outs(w, "<figcaption>") + return + } + r.outs(w, "</figcaption>") +} + +func (r *Renderer) captionFigure(w io.Writer, figure *ast.CaptionFigure, entering bool) { + // TODO(miek): copy more generic ways of mmark over to here. + fig := "<figure" + if figure.HeadingID != "" { + fig += ` id="` + figure.HeadingID + `">` + } else { + fig += ">" + } + r.outOneOf(w, entering, fig, "\n</figure>\n") +} + +func (r *Renderer) tableCell(w io.Writer, tableCell *ast.TableCell, entering bool) { + if !entering { + r.outOneOf(w, tableCell.IsHeader, "</th>", "</td>") + r.cr(w) + return + } + + // entering + var attrs []string + openTag := "<td" + if tableCell.IsHeader { + openTag = "<th" + } + align := tableCell.Align.String() + if align != "" { + attrs = append(attrs, fmt.Sprintf(`align="%s"`, align)) + } + if ast.GetPrevNode(tableCell) == nil { + r.cr(w) + } + r.outTag(w, openTag, attrs) +} + +func (r *Renderer) tableBody(w io.Writer, node *ast.TableBody, entering bool) { + if entering { + r.cr(w) + r.outs(w, "<tbody>") + // XXX: this is to adhere to a rather silly test. Should fix test. + if ast.GetFirstChild(node) == nil { + r.cr(w) + } + } else { + r.outs(w, "</tbody>") + r.cr(w) + } +} + +func (r *Renderer) matter(w io.Writer, node *ast.DocumentMatter, entering bool) { + if !entering { + return + } + if r.documentMatter != ast.DocumentMatterNone { + r.outs(w, "</section>\n") + } + switch node.Matter { + case ast.DocumentMatterFront: + r.outs(w, `<section data-matter="front">`) + case ast.DocumentMatterMain: + r.outs(w, `<section data-matter="main">`) + case ast.DocumentMatterBack: + r.outs(w, `<section data-matter="back">`) + } + r.documentMatter = node.Matter +} + +func (r *Renderer) citation(w io.Writer, node *ast.Citation) { + for i, c := range node.Destination { + attr := []string{`class="none"`} + switch node.Type[i] { + case ast.CitationTypeNormative: + attr[0] = `class="normative"` + case ast.CitationTypeInformative: + attr[0] = `class="informative"` + case ast.CitationTypeSuppressed: + attr[0] = `class="suppressed"` + } + r.outTag(w, "<cite", attr) + r.outs(w, fmt.Sprintf(`<a href="#%s">`+r.opts.CitationFormatString+`</a>`, c, c)) + r.outs(w, "</cite>") + } +} + +func (r *Renderer) callout(w io.Writer, node *ast.Callout) { + attr := []string{`class="callout"`} + r.outTag(w, "<span", attr) + r.out(w, node.ID) + r.outs(w, "</span>") +} + +func (r *Renderer) index(w io.Writer, node *ast.Index) { + // there is no in-text representation. + attr := []string{`class="index"`, fmt.Sprintf(`id="%s"`, node.ID)} + r.outTag(w, "<span", attr) + r.outs(w, "</span>") +} + +// RenderNode renders a markdown node to HTML +func (r *Renderer) RenderNode(w io.Writer, node ast.Node, entering bool) ast.WalkStatus { + if r.opts.RenderNodeHook != nil { + status, didHandle := r.opts.RenderNodeHook(w, node, entering) + if didHandle { + return status + } + } + switch node := node.(type) { + case *ast.Text: + r.text(w, node) + case *ast.Softbreak: + r.cr(w) + // TODO: make it configurable via out(renderer.softbreak) + case *ast.Hardbreak: + r.hardBreak(w, node) + case *ast.NonBlockingSpace: + r.nonBlockingSpace(w, node) + case *ast.Emph: + r.outOneOf(w, entering, "<em>", "</em>") + case *ast.Strong: + r.outOneOf(w, entering, "<strong>", "</strong>") + case *ast.Del: + r.outOneOf(w, entering, "<del>", "</del>") + case *ast.BlockQuote: + tag := tagWithAttributes("<blockquote", BlockAttrs(node)) + r.outOneOfCr(w, entering, tag, "</blockquote>") + case *ast.Aside: + tag := tagWithAttributes("<aside", BlockAttrs(node)) + r.outOneOfCr(w, entering, tag, "</aside>") + case *ast.Link: + r.link(w, node, entering) + case *ast.CrossReference: + link := &ast.Link{Destination: append([]byte("#"), node.Destination...)} + r.link(w, link, entering) + case *ast.Citation: + r.citation(w, node) + case *ast.Image: + if r.opts.Flags&SkipImages != 0 { + return ast.SkipChildren + } + r.image(w, node, entering) + case *ast.Code: + r.code(w, node) + case *ast.CodeBlock: + r.codeBlock(w, node) + case *ast.Caption: + r.caption(w, node, entering) + case *ast.CaptionFigure: + r.captionFigure(w, node, entering) + case *ast.Document: + // do nothing + case *ast.Paragraph: + r.paragraph(w, node, entering) + case *ast.HTMLSpan: + r.htmlSpan(w, node) + case *ast.HTMLBlock: + r.htmlBlock(w, node) + case *ast.Heading: + r.heading(w, node, entering) + case *ast.HorizontalRule: + r.horizontalRule(w, node) + case *ast.List: + r.list(w, node, entering) + case *ast.ListItem: + r.listItem(w, node, entering) + case *ast.Table: + tag := tagWithAttributes("<table", BlockAttrs(node)) + r.outOneOfCr(w, entering, tag, "</table>") + case *ast.TableCell: + r.tableCell(w, node, entering) + case *ast.TableHeader: + r.outOneOfCr(w, entering, "<thead>", "</thead>") + case *ast.TableBody: + r.tableBody(w, node, entering) + case *ast.TableRow: + r.outOneOfCr(w, entering, "<tr>", "</tr>") + case *ast.TableFooter: + r.outOneOfCr(w, entering, "<tfoot>", "</tfoot>") + case *ast.Math: + r.outOneOf(w, true, `<span class="math inline">\(`, `\)</span>`) + EscapeHTML(w, node.Literal) + r.outOneOf(w, false, `<span class="math inline">\(`, `\)</span>`) + case *ast.MathBlock: + r.outOneOf(w, entering, `<p><span class="math display">\[`, `\]</span></p>`) + if entering { + EscapeHTML(w, node.Literal) + } + case *ast.DocumentMatter: + r.matter(w, node, entering) + case *ast.Callout: + r.callout(w, node) + case *ast.Index: + r.index(w, node) + case *ast.Subscript: + r.outOneOf(w, true, "<sub>", "</sub>") + if entering { + Escape(w, node.Literal) + } + r.outOneOf(w, false, "<sub>", "</sub>") + case *ast.Superscript: + r.outOneOf(w, true, "<sup>", "</sup>") + if entering { + Escape(w, node.Literal) + } + r.outOneOf(w, false, "<sup>", "</sup>") + case *ast.Footnotes: + // nothing by default; just output the list. + default: + panic(fmt.Sprintf("Unknown node %T", node)) + } + return ast.GoToNext +} + +// RenderHeader writes HTML document preamble and TOC if requested. +func (r *Renderer) RenderHeader(w io.Writer, ast ast.Node) { + r.writeDocumentHeader(w) + if r.opts.Flags&TOC != 0 { + r.writeTOC(w, ast) + } +} + +// RenderFooter writes HTML document footer. +func (r *Renderer) RenderFooter(w io.Writer, _ ast.Node) { + if r.documentMatter != ast.DocumentMatterNone { + r.outs(w, "</section>\n") + } + + if r.opts.Flags&CompletePage == 0 { + return + } + io.WriteString(w, "\n</body>\n</html>\n") +} + +func (r *Renderer) writeDocumentHeader(w io.Writer) { + if r.opts.Flags&CompletePage == 0 { + return + } + ending := "" + if r.opts.Flags&UseXHTML != 0 { + io.WriteString(w, "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" ") + io.WriteString(w, "\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n") + io.WriteString(w, "<html xmlns=\"http://www.w3.org/1999/xhtml\">\n") + ending = " /" + } else { + io.WriteString(w, "<!DOCTYPE html>\n") + io.WriteString(w, "<html>\n") + } + io.WriteString(w, "<head>\n") + io.WriteString(w, " <title>") + if r.opts.Flags&Smartypants != 0 { + r.sr.Process(w, []byte(r.opts.Title)) + } else { + EscapeHTML(w, []byte(r.opts.Title)) + } + io.WriteString(w, "</title>\n") + io.WriteString(w, r.opts.Generator) + io.WriteString(w, "\"") + io.WriteString(w, ending) + io.WriteString(w, ">\n") + io.WriteString(w, " <meta charset=\"utf-8\"") + io.WriteString(w, ending) + io.WriteString(w, ">\n") + if r.opts.CSS != "" { + io.WriteString(w, " <link rel=\"stylesheet\" type=\"text/css\" href=\"") + EscapeHTML(w, []byte(r.opts.CSS)) + io.WriteString(w, "\"") + io.WriteString(w, ending) + io.WriteString(w, ">\n") + } + if r.opts.Icon != "" { + io.WriteString(w, " <link rel=\"icon\" type=\"image/x-icon\" href=\"") + EscapeHTML(w, []byte(r.opts.Icon)) + io.WriteString(w, "\"") + io.WriteString(w, ending) + io.WriteString(w, ">\n") + } + if r.opts.Head != nil { + w.Write(r.opts.Head) + } + io.WriteString(w, "</head>\n") + io.WriteString(w, "<body>\n\n") +} + +func (r *Renderer) writeTOC(w io.Writer, doc ast.Node) { + buf := bytes.Buffer{} + + inHeading := false + tocLevel := 0 + headingCount := 0 + + ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus { + if nodeData, ok := node.(*ast.Heading); ok && !nodeData.IsTitleblock { + inHeading = entering + if !entering { + buf.WriteString("</a>") + return ast.GoToNext + } + nodeData.HeadingID = fmt.Sprintf("toc_%d", headingCount) + if nodeData.Level == tocLevel { + buf.WriteString("</li>\n\n<li>") + } else if nodeData.Level < tocLevel { + for nodeData.Level < tocLevel { + tocLevel-- + buf.WriteString("</li>\n</ul>") + } + buf.WriteString("</li>\n\n<li>") + } else { + for nodeData.Level > tocLevel { + tocLevel++ + buf.WriteString("\n<ul>\n<li>") + } + } + + fmt.Fprintf(&buf, `<a href="#toc_%d">`, headingCount) + headingCount++ + return ast.GoToNext + } + + if inHeading { + return r.RenderNode(&buf, node, entering) + } + + return ast.GoToNext + }) + + for ; tocLevel > 0; tocLevel-- { + buf.WriteString("</li>\n</ul>") + } + + if buf.Len() > 0 { + io.WriteString(w, "<nav>\n") + w.Write(buf.Bytes()) + io.WriteString(w, "\n\n</nav>\n") + } + r.lastOutputLen = buf.Len() +} + +func isList(node ast.Node) bool { + _, ok := node.(*ast.List) + return ok +} + +func isListTight(node ast.Node) bool { + if list, ok := node.(*ast.List); ok { + return list.Tight + } + return false +} + +func isListItem(node ast.Node) bool { + _, ok := node.(*ast.ListItem) + return ok +} + +func isListItemTerm(node ast.Node) bool { + data, ok := node.(*ast.ListItem) + return ok && data.ListFlags&ast.ListTypeTerm != 0 +} + +// TODO: move to internal package +func skipSpace(data []byte, i int) int { + n := len(data) + for i < n && isSpace(data[i]) { + i++ + } + return i +} + +// TODO: move to internal package +var validUris = [][]byte{[]byte("http://"), []byte("https://"), []byte("ftp://"), []byte("mailto://")} +var validPaths = [][]byte{[]byte("/"), []byte("./"), []byte("../")} + +func isSafeLink(link []byte) bool { + for _, path := range validPaths { + if len(link) >= len(path) && bytes.Equal(link[:len(path)], path) { + if len(link) == len(path) { + return true + } else if isAlnum(link[len(path)]) { + return true + } + } + } + + for _, prefix := range validUris { + // TODO: handle unicode here + // case-insensitive prefix test + if len(link) > len(prefix) && bytes.Equal(bytes.ToLower(link[:len(prefix)]), prefix) && isAlnum(link[len(prefix)]) { + return true + } + } + + return false +} + +// TODO: move to internal package +// Create a url-safe slug for fragments +func slugify(in []byte) []byte { + if len(in) == 0 { + return in + } + out := make([]byte, 0, len(in)) + sym := false + + for _, ch := range in { + if isAlnum(ch) { + sym = false + out = append(out, ch) + } else if sym { + continue + } else { + out = append(out, '-') + sym = true + } + } + var a, b int + var ch byte + for a, ch = range out { + if ch != '-' { + break + } + } + for b = len(out) - 1; b > 0; b-- { + if out[b] != '-' { + break + } + } + return out[a : b+1] +} + +// TODO: move to internal package +// isAlnum returns true if c is a digit or letter +// TODO: check when this is looking for ASCII alnum and when it should use unicode +func isAlnum(c byte) bool { + return (c >= '0' && c <= '9') || isLetter(c) +} + +// isSpace returns true if c is a white-space charactr +func isSpace(c byte) bool { + return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v' +} + +// isLetter returns true if c is ascii letter +func isLetter(c byte) bool { + return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') +} + +// isPunctuation returns true if c is a punctuation symbol. +func isPunctuation(c byte) bool { + for _, r := range []byte("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~") { + if c == r { + return true + } + } + return false +} + +// BlockAttrs takes a node and checks if it has block level attributes set. If so it +// will return a slice each containing a "key=value(s)" string. +func BlockAttrs(node ast.Node) []string { + var attr *ast.Attribute + if c := node.AsContainer(); c != nil && c.Attribute != nil { + attr = c.Attribute + } + if l := node.AsLeaf(); l != nil && l.Attribute != nil { + attr = l.Attribute + } + if attr == nil { + return nil + } + + var s []string + if attr.ID != nil { + s = append(s, fmt.Sprintf(`%s="%s"`, IDTag, attr.ID)) + } + + classes := "" + for _, c := range attr.Classes { + classes += " " + string(c) + } + if classes != "" { + s = append(s, fmt.Sprintf(`class="%s"`, classes[1:])) // skip space we added. + } + + // sort the attributes so it remain stable between runs + var keys = []string{} + for k, _ := range attr.Attrs { + keys = append(keys, k) + } + sort.Strings(keys) + for _, k := range keys { + s = append(s, fmt.Sprintf(`%s="%s"`, k, attr.Attrs[k])) + } + + return s +} + +func tagWithAttributes(name string, attrs []string) string { + s := name + if len(attrs) > 0 { + s += " " + strings.Join(attrs, " ") + } + return s + ">" +} diff --git a/vendor/github.com/gomarkdown/markdown/html/smartypants.go b/vendor/github.com/gomarkdown/markdown/html/smartypants.go new file mode 100644 index 00000000..a09866b0 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/html/smartypants.go @@ -0,0 +1,444 @@ +package html + +import ( + "bytes" + "io" +) + +// SmartyPants rendering + +// SPRenderer is a struct containing state of a Smartypants renderer. +type SPRenderer struct { + inSingleQuote bool + inDoubleQuote bool + callbacks [256]smartCallback +} + +func wordBoundary(c byte) bool { + return c == 0 || isSpace(c) || isPunctuation(c) +} + +func tolower(c byte) byte { + if c >= 'A' && c <= 'Z' { + return c - 'A' + 'a' + } + return c +} + +func isdigit(c byte) bool { + return c >= '0' && c <= '9' +} + +func smartQuoteHelper(out *bytes.Buffer, previousChar byte, nextChar byte, quote byte, isOpen *bool, addNBSP bool) bool { + // edge of the buffer is likely to be a tag that we don't get to see, + // so we treat it like text sometimes + + // enumerate all sixteen possibilities for (previousChar, nextChar) + // each can be one of {0, space, punct, other} + switch { + case previousChar == 0 && nextChar == 0: + // context is not any help here, so toggle + *isOpen = !*isOpen + case isSpace(previousChar) && nextChar == 0: + // [ "] might be [ "<code>foo...] + *isOpen = true + case isPunctuation(previousChar) && nextChar == 0: + // [!"] hmm... could be [Run!"] or [("<code>...] + *isOpen = false + case /* isnormal(previousChar) && */ nextChar == 0: + // [a"] is probably a close + *isOpen = false + case previousChar == 0 && isSpace(nextChar): + // [" ] might be [...foo</code>" ] + *isOpen = false + case isSpace(previousChar) && isSpace(nextChar): + // [ " ] context is not any help here, so toggle + *isOpen = !*isOpen + case isPunctuation(previousChar) && isSpace(nextChar): + // [!" ] is probably a close + *isOpen = false + case /* isnormal(previousChar) && */ isSpace(nextChar): + // [a" ] this is one of the easy cases + *isOpen = false + case previousChar == 0 && isPunctuation(nextChar): + // ["!] hmm... could be ["$1.95] or [</code>"!...] + *isOpen = false + case isSpace(previousChar) && isPunctuation(nextChar): + // [ "!] looks more like [ "$1.95] + *isOpen = true + case isPunctuation(previousChar) && isPunctuation(nextChar): + // [!"!] context is not any help here, so toggle + *isOpen = !*isOpen + case /* isnormal(previousChar) && */ isPunctuation(nextChar): + // [a"!] is probably a close + *isOpen = false + case previousChar == 0 /* && isnormal(nextChar) */ : + // ["a] is probably an open + *isOpen = true + case isSpace(previousChar) /* && isnormal(nextChar) */ : + // [ "a] this is one of the easy cases + *isOpen = true + case isPunctuation(previousChar) /* && isnormal(nextChar) */ : + // [!"a] is probably an open + *isOpen = true + default: + // [a'b] maybe a contraction? + *isOpen = false + } + + // Note that with the limited lookahead, this non-breaking + // space will also be appended to single double quotes. + if addNBSP && !*isOpen { + out.WriteString(" ") + } + + out.WriteByte('&') + if *isOpen { + out.WriteByte('l') + } else { + out.WriteByte('r') + } + out.WriteByte(quote) + out.WriteString("quo;") + + if addNBSP && *isOpen { + out.WriteString(" ") + } + + return true +} + +func (r *SPRenderer) smartSingleQuote(out *bytes.Buffer, previousChar byte, text []byte) int { + if len(text) >= 2 { + t1 := tolower(text[1]) + + if t1 == '\'' { + nextChar := byte(0) + if len(text) >= 3 { + nextChar = text[2] + } + if smartQuoteHelper(out, previousChar, nextChar, 'd', &r.inDoubleQuote, false) { + return 1 + } + } + + if (t1 == 's' || t1 == 't' || t1 == 'm' || t1 == 'd') && (len(text) < 3 || wordBoundary(text[2])) { + out.WriteString("’") + return 0 + } + + if len(text) >= 3 { + t2 := tolower(text[2]) + + if ((t1 == 'r' && t2 == 'e') || (t1 == 'l' && t2 == 'l') || (t1 == 'v' && t2 == 'e')) && + (len(text) < 4 || wordBoundary(text[3])) { + out.WriteString("’") + return 0 + } + } + } + + nextChar := byte(0) + if len(text) > 1 { + nextChar = text[1] + } + if smartQuoteHelper(out, previousChar, nextChar, 's', &r.inSingleQuote, false) { + return 0 + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartParens(out *bytes.Buffer, previousChar byte, text []byte) int { + if len(text) >= 3 { + t1 := tolower(text[1]) + t2 := tolower(text[2]) + + if t1 == 'c' && t2 == ')' { + out.WriteString("©") + return 2 + } + + if t1 == 'r' && t2 == ')' { + out.WriteString("®") + return 2 + } + + if len(text) >= 4 && t1 == 't' && t2 == 'm' && text[3] == ')' { + out.WriteString("™") + return 3 + } + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartDash(out *bytes.Buffer, previousChar byte, text []byte) int { + if len(text) >= 2 { + if text[1] == '-' { + out.WriteString("—") + return 1 + } + + if wordBoundary(previousChar) && wordBoundary(text[1]) { + out.WriteString("–") + return 0 + } + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartDashLatex(out *bytes.Buffer, previousChar byte, text []byte) int { + if len(text) >= 3 && text[1] == '-' && text[2] == '-' { + out.WriteString("—") + return 2 + } + if len(text) >= 2 && text[1] == '-' { + out.WriteString("–") + return 1 + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartAmpVariant(out *bytes.Buffer, previousChar byte, text []byte, quote byte, addNBSP bool) int { + if bytes.HasPrefix(text, []byte(""")) { + nextChar := byte(0) + if len(text) >= 7 { + nextChar = text[6] + } + if smartQuoteHelper(out, previousChar, nextChar, quote, &r.inDoubleQuote, addNBSP) { + return 5 + } + } + + if bytes.HasPrefix(text, []byte("�")) { + return 3 + } + + out.WriteByte('&') + return 0 +} + +func (r *SPRenderer) smartAmp(angledQuotes, addNBSP bool) func(*bytes.Buffer, byte, []byte) int { + var quote byte = 'd' + if angledQuotes { + quote = 'a' + } + + return func(out *bytes.Buffer, previousChar byte, text []byte) int { + return r.smartAmpVariant(out, previousChar, text, quote, addNBSP) + } +} + +func (r *SPRenderer) smartPeriod(out *bytes.Buffer, previousChar byte, text []byte) int { + if len(text) >= 3 && text[1] == '.' && text[2] == '.' { + out.WriteString("…") + return 2 + } + + if len(text) >= 5 && text[1] == ' ' && text[2] == '.' && text[3] == ' ' && text[4] == '.' { + out.WriteString("…") + return 4 + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartBacktick(out *bytes.Buffer, previousChar byte, text []byte) int { + if len(text) >= 2 && text[1] == '`' { + nextChar := byte(0) + if len(text) >= 3 { + nextChar = text[2] + } + if smartQuoteHelper(out, previousChar, nextChar, 'd', &r.inDoubleQuote, false) { + return 1 + } + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartNumberGeneric(out *bytes.Buffer, previousChar byte, text []byte) int { + if wordBoundary(previousChar) && previousChar != '/' && len(text) >= 3 { + // is it of the form digits/digits(word boundary)?, i.e., \d+/\d+\b + // note: check for regular slash (/) or fraction slash (⁄, 0x2044, or 0xe2 81 84 in utf-8) + // and avoid changing dates like 1/23/2005 into fractions. + numEnd := 0 + for len(text) > numEnd && isdigit(text[numEnd]) { + numEnd++ + } + if numEnd == 0 { + out.WriteByte(text[0]) + return 0 + } + denStart := numEnd + 1 + if len(text) > numEnd+3 && text[numEnd] == 0xe2 && text[numEnd+1] == 0x81 && text[numEnd+2] == 0x84 { + denStart = numEnd + 3 + } else if len(text) < numEnd+2 || text[numEnd] != '/' { + out.WriteByte(text[0]) + return 0 + } + denEnd := denStart + for len(text) > denEnd && isdigit(text[denEnd]) { + denEnd++ + } + if denEnd == denStart { + out.WriteByte(text[0]) + return 0 + } + if len(text) == denEnd || wordBoundary(text[denEnd]) && text[denEnd] != '/' { + out.WriteString("<sup>") + out.Write(text[:numEnd]) + out.WriteString("</sup>⁄<sub>") + out.Write(text[denStart:denEnd]) + out.WriteString("</sub>") + return denEnd - 1 + } + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartNumber(out *bytes.Buffer, previousChar byte, text []byte) int { + if wordBoundary(previousChar) && previousChar != '/' && len(text) >= 3 { + if text[0] == '1' && text[1] == '/' && text[2] == '2' { + if len(text) < 4 || wordBoundary(text[3]) && text[3] != '/' { + out.WriteString("½") + return 2 + } + } + + if text[0] == '1' && text[1] == '/' && text[2] == '4' { + if len(text) < 4 || wordBoundary(text[3]) && text[3] != '/' || (len(text) >= 5 && tolower(text[3]) == 't' && tolower(text[4]) == 'h') { + out.WriteString("¼") + return 2 + } + } + + if text[0] == '3' && text[1] == '/' && text[2] == '4' { + if len(text) < 4 || wordBoundary(text[3]) && text[3] != '/' || (len(text) >= 6 && tolower(text[3]) == 't' && tolower(text[4]) == 'h' && tolower(text[5]) == 's') { + out.WriteString("¾") + return 2 + } + } + } + + out.WriteByte(text[0]) + return 0 +} + +func (r *SPRenderer) smartDoubleQuoteVariant(out *bytes.Buffer, previousChar byte, text []byte, quote byte) int { + nextChar := byte(0) + if len(text) > 1 { + nextChar = text[1] + } + if !smartQuoteHelper(out, previousChar, nextChar, quote, &r.inDoubleQuote, false) { + out.WriteString(""") + } + + return 0 +} + +func (r *SPRenderer) smartDoubleQuote(out *bytes.Buffer, previousChar byte, text []byte) int { + return r.smartDoubleQuoteVariant(out, previousChar, text, 'd') +} + +func (r *SPRenderer) smartAngledDoubleQuote(out *bytes.Buffer, previousChar byte, text []byte) int { + return r.smartDoubleQuoteVariant(out, previousChar, text, 'a') +} + +func (r *SPRenderer) smartLeftAngle(out *bytes.Buffer, previousChar byte, text []byte) int { + i := 0 + + for i < len(text) && text[i] != '>' { + i++ + } + + out.Write(text[:i+1]) + return i +} + +type smartCallback func(out *bytes.Buffer, previousChar byte, text []byte) int + +// NewSmartypantsRenderer constructs a Smartypants renderer object. +func NewSmartypantsRenderer(flags Flags) *SPRenderer { + var ( + r SPRenderer + + smartAmpAngled = r.smartAmp(true, false) + smartAmpAngledNBSP = r.smartAmp(true, true) + smartAmpRegular = r.smartAmp(false, false) + smartAmpRegularNBSP = r.smartAmp(false, true) + + addNBSP = flags&SmartypantsQuotesNBSP != 0 + ) + + if flags&SmartypantsAngledQuotes == 0 { + r.callbacks['"'] = r.smartDoubleQuote + if !addNBSP { + r.callbacks['&'] = smartAmpRegular + } else { + r.callbacks['&'] = smartAmpRegularNBSP + } + } else { + r.callbacks['"'] = r.smartAngledDoubleQuote + if !addNBSP { + r.callbacks['&'] = smartAmpAngled + } else { + r.callbacks['&'] = smartAmpAngledNBSP + } + } + r.callbacks['\''] = r.smartSingleQuote + r.callbacks['('] = r.smartParens + if flags&SmartypantsDashes != 0 { + if flags&SmartypantsLatexDashes == 0 { + r.callbacks['-'] = r.smartDash + } else { + r.callbacks['-'] = r.smartDashLatex + } + } + r.callbacks['.'] = r.smartPeriod + if flags&SmartypantsFractions == 0 { + r.callbacks['1'] = r.smartNumber + r.callbacks['3'] = r.smartNumber + } else { + for ch := '1'; ch <= '9'; ch++ { + r.callbacks[ch] = r.smartNumberGeneric + } + } + r.callbacks['<'] = r.smartLeftAngle + r.callbacks['`'] = r.smartBacktick + return &r +} + +// Process is the entry point of the Smartypants renderer. +func (r *SPRenderer) Process(w io.Writer, text []byte) { + mark := 0 + for i := 0; i < len(text); i++ { + if action := r.callbacks[text[i]]; action != nil { + if i > mark { + w.Write(text[mark:i]) + } + previousChar := byte(0) + if i > 0 { + previousChar = text[i-1] + } + var tmp bytes.Buffer + i += action(&tmp, previousChar, text[i:]) + w.Write(tmp.Bytes()) + mark = i + 1 + } + } + if mark < len(text) { + w.Write(text[mark:]) + } +} diff --git a/vendor/github.com/gomarkdown/markdown/markdown.go b/vendor/github.com/gomarkdown/markdown/markdown.go new file mode 100644 index 00000000..fd5c1cfb --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/markdown.go @@ -0,0 +1,85 @@ +package markdown + +import ( + "bytes" + "io" + + "github.com/gomarkdown/markdown/ast" + "github.com/gomarkdown/markdown/html" + "github.com/gomarkdown/markdown/parser" +) + +// Renderer is an interface for implementing custom renderers. +type Renderer interface { + // RenderNode renders markdown node to w. + // It's called once for a leaf node. + // It's called twice for non-leaf nodes: + // * first with entering=true + // * then with entering=false + // + // Return value is a way to tell the calling walker to adjust its walk + // pattern: e.g. it can terminate the traversal by returning Terminate. Or it + // can ask the walker to skip a subtree of this node by returning SkipChildren. + // The typical behavior is to return GoToNext, which asks for the usual + // traversal to the next node. + RenderNode(w io.Writer, node ast.Node, entering bool) ast.WalkStatus + + // RenderHeader is a method that allows the renderer to produce some + // content preceding the main body of the output document. The header is + // understood in the broad sense here. For example, the default HTML + // renderer will write not only the HTML document preamble, but also the + // table of contents if it was requested. + // + // The method will be passed an entire document tree, in case a particular + // implementation needs to inspect it to produce output. + // + // The output should be written to the supplied writer w. If your + // implementation has no header to write, supply an empty implementation. + RenderHeader(w io.Writer, ast ast.Node) + + // RenderFooter is a symmetric counterpart of RenderHeader. + RenderFooter(w io.Writer, ast ast.Node) +} + +// Parse parsers a markdown document using provided parser. If parser is nil, +// we use parser configured with parser.CommonExtensions. +// +// It returns AST (abstract syntax tree) that can be converted to another +// format using Render function. +func Parse(markdown []byte, p *parser.Parser) ast.Node { + if p == nil { + p = parser.New() + } + return p.Parse(markdown) +} + +// Render uses renderer to convert parsed markdown document into a different format. +// +// To convert to HTML, pass html.Renderer +func Render(doc ast.Node, renderer Renderer) []byte { + var buf bytes.Buffer + renderer.RenderHeader(&buf, doc) + ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus { + return renderer.RenderNode(&buf, node, entering) + }) + renderer.RenderFooter(&buf, doc) + return buf.Bytes() +} + +// ToHTML converts markdownDoc to HTML. +// +// You can optionally pass a parser and renderer. This allows to customize +// a parser, use a customized html render or use completely custom renderer. +// +// If you pass nil for both, we use parser configured with parser.CommonExtensions +// and html.Renderer configured with html.CommonFlags. +func ToHTML(markdown []byte, p *parser.Parser, renderer Renderer) []byte { + doc := Parse(markdown, p) + if renderer == nil { + opts := html.RendererOptions{ + Flags: html.CommonFlags, + } + renderer = html.NewRenderer(opts) + } + return Render(doc, renderer) +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/aside.go b/vendor/github.com/gomarkdown/markdown/parser/aside.go new file mode 100644 index 00000000..96e25fe0 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/aside.go @@ -0,0 +1,73 @@ +package parser + +import ( + "bytes" + + "github.com/gomarkdown/markdown/ast" +) + +// returns aisde prefix length +func (p *Parser) asidePrefix(data []byte) int { + i := 0 + n := len(data) + for i < 3 && i < n && data[i] == ' ' { + i++ + } + if i+1 < n && data[i] == 'A' && data[i+1] == '>' { + if i+2 < n && data[i+2] == ' ' { + return i + 3 + } + return i + 2 + } + return 0 +} + +// aside ends with at least one blank line +// followed by something without a aside prefix +func (p *Parser) terminateAside(data []byte, beg, end int) bool { + if p.isEmpty(data[beg:]) <= 0 { + return false + } + if end >= len(data) { + return true + } + return p.asidePrefix(data[end:]) == 0 && p.isEmpty(data[end:]) == 0 +} + +// parse a aside fragment +func (p *Parser) aside(data []byte) int { + var raw bytes.Buffer + beg, end := 0, 0 + // identical to quote + for beg < len(data) { + end = beg + // Step over whole lines, collecting them. While doing that, check for + // fenced code and if one's found, incorporate it altogether, + // irregardless of any contents inside it + for end < len(data) && data[end] != '\n' { + if p.extensions&FencedCode != 0 { + if i := p.fencedCodeBlock(data[end:], false); i > 0 { + // -1 to compensate for the extra end++ after the loop: + end += i - 1 + break + } + } + end++ + } + end = skipCharN(data, end, '\n', 1) + if pre := p.asidePrefix(data[beg:]); pre > 0 { + // skip the prefix + beg += pre + } else if p.terminateAside(data, beg, end) { + break + } + // this line is part of the aside + raw.Write(data[beg:end]) + beg = end + } + + block := p.addBlock(&ast.Aside{}) + p.block(raw.Bytes()) + p.finalize(block) + return end +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/attribute.go b/vendor/github.com/gomarkdown/markdown/parser/attribute.go new file mode 100644 index 00000000..5fdb0709 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/attribute.go @@ -0,0 +1,116 @@ +package parser + +import ( + "bytes" + + "github.com/gomarkdown/markdown/ast" +) + +// attribute parses a (potential) block attribute and adds it to p. +func (p *Parser) attribute(data []byte) []byte { + if len(data) < 3 { + return data + } + i := 0 + if data[i] != '{' { + return data + } + i++ + + // last character must be a } otherwise it's not an attribute + end := skipUntilChar(data, i, '\n') + if data[end-1] != '}' { + return data + } + + i = skipSpace(data, i) + b := &ast.Attribute{Attrs: make(map[string][]byte)} + + esc := false + quote := false + trail := 0 +Loop: + for ; i < len(data); i++ { + switch data[i] { + case ' ', '\t', '\f', '\v': + if quote { + continue + } + chunk := data[trail+1 : i] + if len(chunk) == 0 { + trail = i + continue + } + switch { + case chunk[0] == '.': + b.Classes = append(b.Classes, chunk[1:]) + case chunk[0] == '#': + b.ID = chunk[1:] + default: + k, v := keyValue(chunk) + if k != nil && v != nil { + b.Attrs[string(k)] = v + } else { + // this is illegal in an attribute + return data + } + } + trail = i + case '"': + if esc { + esc = !esc + continue + } + quote = !quote + case '\\': + esc = !esc + case '}': + if esc { + esc = !esc + continue + } + chunk := data[trail+1 : i] + if len(chunk) == 0 { + return data + } + switch { + case chunk[0] == '.': + b.Classes = append(b.Classes, chunk[1:]) + case chunk[0] == '#': + b.ID = chunk[1:] + default: + k, v := keyValue(chunk) + if k != nil && v != nil { + b.Attrs[string(k)] = v + } else { + return data + } + } + i++ + break Loop + default: + esc = false + } + } + + p.attr = b + return data[i:] +} + +// key="value" quotes are mandatory. +func keyValue(data []byte) ([]byte, []byte) { + chunk := bytes.SplitN(data, []byte{'='}, 2) + if len(chunk) != 2 { + return nil, nil + } + key := chunk[0] + value := chunk[1] + + if len(value) < 3 || len(key) == 0 { + return nil, nil + } + if value[0] != '"' || value[len(value)-1] != '"' { + return key, nil + } + return key, value[1 : len(value)-1] +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/block.go b/vendor/github.com/gomarkdown/markdown/parser/block.go new file mode 100644 index 00000000..18a2dd89 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/block.go @@ -0,0 +1,1978 @@ +package parser + +import ( + "bytes" + "html" + "regexp" + "strconv" + "unicode" + + "github.com/gomarkdown/markdown/ast" +) + +// Parsing block-level elements. + +const ( + charEntity = "&(?:#x[a-f0-9]{1,8}|#[0-9]{1,8}|[a-z][a-z0-9]{1,31});" + escapable = "[!\"#$%&'()*+,./:;<=>?@[\\\\\\]^_`{|}~-]" +) + +var ( + reBackslashOrAmp = regexp.MustCompile("[\\&]") + reEntityOrEscapedChar = regexp.MustCompile("(?i)\\\\" + escapable + "|" + charEntity) + + // blockTags is a set of tags that are recognized as HTML block tags. + // Any of these can be included in markdown text without special escaping. + blockTags = map[string]struct{}{ + "blockquote": struct{}{}, + "del": struct{}{}, + "div": struct{}{}, + "dl": struct{}{}, + "fieldset": struct{}{}, + "form": struct{}{}, + "h1": struct{}{}, + "h2": struct{}{}, + "h3": struct{}{}, + "h4": struct{}{}, + "h5": struct{}{}, + "h6": struct{}{}, + "iframe": struct{}{}, + "ins": struct{}{}, + "math": struct{}{}, + "noscript": struct{}{}, + "ol": struct{}{}, + "pre": struct{}{}, + "p": struct{}{}, + "script": struct{}{}, + "style": struct{}{}, + "table": struct{}{}, + "ul": struct{}{}, + + // HTML5 + "address": struct{}{}, + "article": struct{}{}, + "aside": struct{}{}, + "canvas": struct{}{}, + "figcaption": struct{}{}, + "figure": struct{}{}, + "footer": struct{}{}, + "header": struct{}{}, + "hgroup": struct{}{}, + "main": struct{}{}, + "nav": struct{}{}, + "output": struct{}{}, + "progress": struct{}{}, + "section": struct{}{}, + "video": struct{}{}, + } +) + +// sanitizeAnchorName returns a sanitized anchor name for the given text. +// Taken from https://github.com/shurcooL/sanitized_anchor_name/blob/master/main.go#L14:1 +func sanitizeAnchorName(text string) string { + var anchorName []rune + var futureDash = false + for _, r := range text { + switch { + case unicode.IsLetter(r) || unicode.IsNumber(r): + if futureDash && len(anchorName) > 0 { + anchorName = append(anchorName, '-') + } + futureDash = false + anchorName = append(anchorName, unicode.ToLower(r)) + default: + futureDash = true + } + } + return string(anchorName) +} + +// Parse block-level data. +// Note: this function and many that it calls assume that +// the input buffer ends with a newline. +func (p *Parser) block(data []byte) { + // this is called recursively: enforce a maximum depth + if p.nesting >= p.maxNesting { + return + } + p.nesting++ + + // parse out one block-level construct at a time + for len(data) > 0 { + // attributes that can be specific before a block element: + // + // {#id .class1 .class2 key="value"} + if p.extensions&Attributes != 0 { + data = p.attribute(data) + } + + if p.extensions&Includes != 0 { + f := p.readInclude + path, address, consumed := p.isInclude(data) + if consumed == 0 { + path, address, consumed = p.isCodeInclude(data) + f = p.readCodeInclude + } + if consumed > 0 { + included := f(p.includeStack.Last(), path, address) + p.includeStack.Push(path) + p.block(included) + p.includeStack.Pop() + data = data[consumed:] + continue + } + } + + // user supplied parser function + if p.Opts.ParserHook != nil { + node, blockdata, consumed := p.Opts.ParserHook(data) + if consumed > 0 { + data = data[consumed:] + + if node != nil { + p.addBlock(node) + if blockdata != nil { + p.block(blockdata) + p.finalize(node) + } + } + continue + } + } + + // prefixed heading: + // + // # Heading 1 + // ## Heading 2 + // ... + // ###### Heading 6 + if p.isPrefixHeading(data) { + data = data[p.prefixHeading(data):] + continue + } + + // prefixed special heading: + // (there are no levels.) + // + // .# Abstract + if p.isPrefixSpecialHeading(data) { + data = data[p.prefixSpecialHeading(data):] + continue + } + + // block of preformatted HTML: + // + // <div> + // ... + // </div> + if data[0] == '<' { + if i := p.html(data, true); i > 0 { + data = data[i:] + continue + } + } + + // title block + // + // % stuff + // % more stuff + // % even more stuff + if p.extensions&Titleblock != 0 { + if data[0] == '%' { + if i := p.titleBlock(data, true); i > 0 { + data = data[i:] + continue + } + } + } + + // blank lines. note: returns the # of bytes to skip + if i := p.isEmpty(data); i > 0 { + data = data[i:] + continue + } + + // indented code block: + // + // func max(a, b int) int { + // if a > b { + // return a + // } + // return b + // } + if p.codePrefix(data) > 0 { + data = data[p.code(data):] + continue + } + + // fenced code block: + // + // ``` go + // func fact(n int) int { + // if n <= 1 { + // return n + // } + // return n * fact(n-1) + // } + // ``` + if p.extensions&FencedCode != 0 { + if i := p.fencedCodeBlock(data, true); i > 0 { + data = data[i:] + continue + } + } + + // horizontal rule: + // + // ------ + // or + // ****** + // or + // ______ + if p.isHRule(data) { + p.addBlock(&ast.HorizontalRule{}) + i := skipUntilChar(data, 0, '\n') + data = data[i:] + continue + } + + // block quote: + // + // > A big quote I found somewhere + // > on the web + if p.quotePrefix(data) > 0 { + data = data[p.quote(data):] + continue + } + + // aside: + // + // A> The proof is too large to fit + // A> in the margin. + if p.extensions&Mmark != 0 { + if p.asidePrefix(data) > 0 { + data = data[p.aside(data):] + continue + } + } + + // figure block: + // + // !--- + //  + //  + // !--- + if p.extensions&Mmark != 0 { + if i := p.figureBlock(data, true); i > 0 { + data = data[i:] + continue + } + } + + // table: + // + // Name | Age | Phone + // ------|-----|--------- + // Bob | 31 | 555-1234 + // Alice | 27 | 555-4321 + if p.extensions&Tables != 0 { + if i := p.table(data); i > 0 { + data = data[i:] + continue + } + } + + // an itemized/unordered list: + // + // * Item 1 + // * Item 2 + // + // also works with + or - + if p.uliPrefix(data) > 0 { + data = data[p.list(data, 0, 0):] + continue + } + + // a numbered/ordered list: + // + // 1. Item 1 + // 2. Item 2 + if i := p.oliPrefix(data); i > 0 { + start := 0 + if i > 2 && p.extensions&OrderedListStart != 0 { + s := string(data[:i-2]) + start, _ = strconv.Atoi(s) + if start == 1 { + start = 0 + } + } + data = data[p.list(data, ast.ListTypeOrdered, start):] + continue + } + + // definition lists: + // + // Term 1 + // : Definition a + // : Definition b + // + // Term 2 + // : Definition c + if p.extensions&DefinitionLists != 0 { + if p.dliPrefix(data) > 0 { + data = data[p.list(data, ast.ListTypeDefinition, 0):] + continue + } + } + + if p.extensions&MathJax != 0 { + if i := p.blockMath(data); i > 0 { + data = data[i:] + continue + } + } + + // document matters: + // + // {frontmatter}/{mainmatter}/{backmatter} + if p.extensions&Mmark != 0 { + if i := p.documentMatter(data); i > 0 { + data = data[i:] + continue + } + } + + // anything else must look like a normal paragraph + // note: this finds underlined headings, too + idx := p.paragraph(data) + data = data[idx:] + } + + p.nesting-- +} + +func (p *Parser) addBlock(n ast.Node) ast.Node { + p.closeUnmatchedBlocks() + + if p.attr != nil { + if c := n.AsContainer(); c != nil { + c.Attribute = p.attr + } + if l := n.AsLeaf(); l != nil { + l.Attribute = p.attr + } + p.attr = nil + } + return p.addChild(n) +} + +func (p *Parser) isPrefixHeading(data []byte) bool { + if data[0] != '#' { + return false + } + + if p.extensions&SpaceHeadings != 0 { + level := skipCharN(data, 0, '#', 6) + if level == len(data) || data[level] != ' ' { + return false + } + } + return true +} + +func (p *Parser) prefixHeading(data []byte) int { + level := skipCharN(data, 0, '#', 6) + i := skipChar(data, level, ' ') + end := skipUntilChar(data, i, '\n') + skip := end + id := "" + if p.extensions&HeadingIDs != 0 { + j, k := 0, 0 + // find start/end of heading id + for j = i; j < end-1 && (data[j] != '{' || data[j+1] != '#'); j++ { + } + for k = j + 1; k < end && data[k] != '}'; k++ { + } + // extract heading id iff found + if j < end && k < end { + id = string(data[j+2 : k]) + end = j + skip = k + 1 + for end > 0 && data[end-1] == ' ' { + end-- + } + } + } + for end > 0 && data[end-1] == '#' { + if isBackslashEscaped(data, end-1) { + break + } + end-- + } + for end > 0 && data[end-1] == ' ' { + end-- + } + if end > i { + if id == "" && p.extensions&AutoHeadingIDs != 0 { + id = sanitizeAnchorName(string(data[i:end])) + } + block := &ast.Heading{ + HeadingID: id, + Level: level, + } + block.Content = data[i:end] + p.addBlock(block) + } + return skip +} + +func (p *Parser) isPrefixSpecialHeading(data []byte) bool { + if p.extensions|Mmark == 0 { + return false + } + if len(data) < 4 { + return false + } + if data[0] != '.' { + return false + } + if data[1] != '#' { + return false + } + if data[2] == '#' { // we don't support level, so nack this. + return false + } + + if p.extensions&SpaceHeadings != 0 { + if data[2] != ' ' { + return false + } + } + return true +} + +func (p *Parser) prefixSpecialHeading(data []byte) int { + i := skipChar(data, 2, ' ') // ".#" skipped + end := skipUntilChar(data, i, '\n') + skip := end + id := "" + if p.extensions&HeadingIDs != 0 { + j, k := 0, 0 + // find start/end of heading id + for j = i; j < end-1 && (data[j] != '{' || data[j+1] != '#'); j++ { + } + for k = j + 1; k < end && data[k] != '}'; k++ { + } + // extract heading id iff found + if j < end && k < end { + id = string(data[j+2 : k]) + end = j + skip = k + 1 + for end > 0 && data[end-1] == ' ' { + end-- + } + } + } + for end > 0 && data[end-1] == '#' { + if isBackslashEscaped(data, end-1) { + break + } + end-- + } + for end > 0 && data[end-1] == ' ' { + end-- + } + if end > i { + if id == "" && p.extensions&AutoHeadingIDs != 0 { + id = sanitizeAnchorName(string(data[i:end])) + } + block := &ast.Heading{ + HeadingID: id, + IsSpecial: true, + Level: 1, // always level 1. + } + block.Literal = data[i:end] + block.Content = data[i:end] + p.addBlock(block) + } + return skip +} + +func (p *Parser) isUnderlinedHeading(data []byte) int { + // test of level 1 heading + if data[0] == '=' { + i := skipChar(data, 1, '=') + i = skipChar(data, i, ' ') + if i < len(data) && data[i] == '\n' { + return 1 + } + return 0 + } + + // test of level 2 heading + if data[0] == '-' { + i := skipChar(data, 1, '-') + i = skipChar(data, i, ' ') + if i < len(data) && data[i] == '\n' { + return 2 + } + return 0 + } + + return 0 +} + +func (p *Parser) titleBlock(data []byte, doRender bool) int { + if data[0] != '%' { + return 0 + } + splitData := bytes.Split(data, []byte("\n")) + var i int + for idx, b := range splitData { + if !bytes.HasPrefix(b, []byte("%")) { + i = idx // - 1 + break + } + } + + data = bytes.Join(splitData[0:i], []byte("\n")) + consumed := len(data) + data = bytes.TrimPrefix(data, []byte("% ")) + data = bytes.Replace(data, []byte("\n% "), []byte("\n"), -1) + block := &ast.Heading{ + Level: 1, + IsTitleblock: true, + } + block.Content = data + p.addBlock(block) + + return consumed +} + +func (p *Parser) html(data []byte, doRender bool) int { + var i, j int + + // identify the opening tag + if data[0] != '<' { + return 0 + } + curtag, tagfound := p.htmlFindTag(data[1:]) + + // handle special cases + if !tagfound { + // check for an HTML comment + if size := p.htmlComment(data, doRender); size > 0 { + return size + } + + // check for an <hr> tag + if size := p.htmlHr(data, doRender); size > 0 { + return size + } + + // no special case recognized + return 0 + } + + // look for an unindented matching closing tag + // followed by a blank line + found := false + /* + closetag := []byte("\n</" + curtag + ">") + j = len(curtag) + 1 + for !found { + // scan for a closing tag at the beginning of a line + if skip := bytes.Index(data[j:], closetag); skip >= 0 { + j += skip + len(closetag) + } else { + break + } + + // see if it is the only thing on the line + if skip := p.isEmpty(data[j:]); skip > 0 { + // see if it is followed by a blank line/eof + j += skip + if j >= len(data) { + found = true + i = j + } else { + if skip := p.isEmpty(data[j:]); skip > 0 { + j += skip + found = true + i = j + } + } + } + } + */ + + // if not found, try a second pass looking for indented match + // but not if tag is "ins" or "del" (following original Markdown.pl) + if !found && curtag != "ins" && curtag != "del" { + i = 1 + for i < len(data) { + i++ + for i < len(data) && !(data[i-1] == '<' && data[i] == '/') { + i++ + } + + if i+2+len(curtag) >= len(data) { + break + } + + j = p.htmlFindEnd(curtag, data[i-1:]) + + if j > 0 { + i += j - 1 + found = true + break + } + } + } + + if !found { + return 0 + } + + // the end of the block has been found + if doRender { + // trim newlines + end := backChar(data, i, '\n') + htmlBLock := &ast.HTMLBlock{ast.Leaf{Content: data[:end]}} + p.addBlock(htmlBLock) + finalizeHTMLBlock(htmlBLock) + } + + return i +} + +func finalizeHTMLBlock(block *ast.HTMLBlock) { + block.Literal = block.Content + block.Content = nil +} + +// HTML comment, lax form +func (p *Parser) htmlComment(data []byte, doRender bool) int { + i := p.inlineHTMLComment(data) + // needs to end with a blank line + if j := p.isEmpty(data[i:]); j > 0 { + size := i + j + if doRender { + // trim trailing newlines + end := backChar(data, size, '\n') + htmlBLock := &ast.HTMLBlock{ast.Leaf{Content: data[:end]}} + p.addBlock(htmlBLock) + finalizeHTMLBlock(htmlBLock) + } + return size + } + return 0 +} + +// HR, which is the only self-closing block tag considered +func (p *Parser) htmlHr(data []byte, doRender bool) int { + if len(data) < 4 { + return 0 + } + if data[0] != '<' || (data[1] != 'h' && data[1] != 'H') || (data[2] != 'r' && data[2] != 'R') { + return 0 + } + if data[3] != ' ' && data[3] != '/' && data[3] != '>' { + // not an <hr> tag after all; at least not a valid one + return 0 + } + i := 3 + for i < len(data) && data[i] != '>' && data[i] != '\n' { + i++ + } + if i < len(data) && data[i] == '>' { + i++ + if j := p.isEmpty(data[i:]); j > 0 { + size := i + j + if doRender { + // trim newlines + end := backChar(data, size, '\n') + htmlBlock := &ast.HTMLBlock{ast.Leaf{Content: data[:end]}} + p.addBlock(htmlBlock) + finalizeHTMLBlock(htmlBlock) + } + return size + } + } + return 0 +} + +func (p *Parser) htmlFindTag(data []byte) (string, bool) { + i := skipAlnum(data, 0) + key := string(data[:i]) + if _, ok := blockTags[key]; ok { + return key, true + } + return "", false +} + +func (p *Parser) htmlFindEnd(tag string, data []byte) int { + // assume data[0] == '<' && data[1] == '/' already tested + if tag == "hr" { + return 2 + } + // check if tag is a match + closetag := []byte("</" + tag + ">") + if !bytes.HasPrefix(data, closetag) { + return 0 + } + i := len(closetag) + + // check that the rest of the line is blank + skip := 0 + if skip = p.isEmpty(data[i:]); skip == 0 { + return 0 + } + i += skip + skip = 0 + + if i >= len(data) { + return i + } + + if p.extensions&LaxHTMLBlocks != 0 { + return i + } + if skip = p.isEmpty(data[i:]); skip == 0 { + // following line must be blank + return 0 + } + + return i + skip +} + +func (*Parser) isEmpty(data []byte) int { + // it is okay to call isEmpty on an empty buffer + if len(data) == 0 { + return 0 + } + + var i int + for i = 0; i < len(data) && data[i] != '\n'; i++ { + if data[i] != ' ' && data[i] != '\t' { + return 0 + } + } + i = skipCharN(data, i, '\n', 1) + return i +} + +func (*Parser) isHRule(data []byte) bool { + i := 0 + + // skip up to three spaces + for i < 3 && data[i] == ' ' { + i++ + } + + // look at the hrule char + if data[i] != '*' && data[i] != '-' && data[i] != '_' { + return false + } + c := data[i] + + // the whole line must be the char or whitespace + n := 0 + for i < len(data) && data[i] != '\n' { + switch { + case data[i] == c: + n++ + case data[i] != ' ': + return false + } + i++ + } + + return n >= 3 +} + +// isFenceLine checks if there's a fence line (e.g., ``` or ``` go) at the beginning of data, +// and returns the end index if so, or 0 otherwise. It also returns the marker found. +// If syntax is not nil, it gets set to the syntax specified in the fence line. +func isFenceLine(data []byte, syntax *string, oldmarker string) (end int, marker string) { + i, size := 0, 0 + + n := len(data) + // skip up to three spaces + for i < n && i < 3 && data[i] == ' ' { + i++ + } + + // check for the marker characters: ~ or ` + if i >= n { + return 0, "" + } + if data[i] != '~' && data[i] != '`' { + return 0, "" + } + + c := data[i] + + // the whole line must be the same char or whitespace + for i < n && data[i] == c { + size++ + i++ + } + + // the marker char must occur at least 3 times + if size < 3 { + return 0, "" + } + marker = string(data[i-size : i]) + + // if this is the end marker, it must match the beginning marker + if oldmarker != "" && marker != oldmarker { + return 0, "" + } + + // TODO(shurcooL): It's probably a good idea to simplify the 2 code paths here + // into one, always get the syntax, and discard it if the caller doesn't care. + if syntax != nil { + syn := 0 + i = skipChar(data, i, ' ') + + if i >= n { + if i == n { + return i, marker + } + return 0, "" + } + + syntaxStart := i + + if data[i] == '{' { + i++ + syntaxStart++ + + for i < n && data[i] != '}' && data[i] != '\n' { + syn++ + i++ + } + + if i >= n || data[i] != '}' { + return 0, "" + } + + // strip all whitespace at the beginning and the end + // of the {} block + for syn > 0 && isSpace(data[syntaxStart]) { + syntaxStart++ + syn-- + } + + for syn > 0 && isSpace(data[syntaxStart+syn-1]) { + syn-- + } + + i++ + } else { + for i < n && !isSpace(data[i]) { + syn++ + i++ + } + } + + *syntax = string(data[syntaxStart : syntaxStart+syn]) + } + + i = skipChar(data, i, ' ') + if i >= n || data[i] != '\n' { + if i == n { + return i, marker + } + return 0, "" + } + return i + 1, marker // Take newline into account. +} + +// fencedCodeBlock returns the end index if data contains a fenced code block at the beginning, +// or 0 otherwise. It writes to out if doRender is true, otherwise it has no side effects. +// If doRender is true, a final newline is mandatory to recognize the fenced code block. +func (p *Parser) fencedCodeBlock(data []byte, doRender bool) int { + var syntax string + beg, marker := isFenceLine(data, &syntax, "") + if beg == 0 || beg >= len(data) { + return 0 + } + + var work bytes.Buffer + work.WriteString(syntax) + work.WriteByte('\n') + + for { + // safe to assume beg < len(data) + + // check for the end of the code block + fenceEnd, _ := isFenceLine(data[beg:], nil, marker) + if fenceEnd != 0 { + beg += fenceEnd + break + } + + // copy the current line + end := skipUntilChar(data, beg, '\n') + 1 + + // did we reach the end of the buffer without a closing marker? + if end >= len(data) { + return 0 + } + + // verbatim copy to the working buffer + if doRender { + work.Write(data[beg:end]) + } + beg = end + } + + if doRender { + codeBlock := &ast.CodeBlock{ + IsFenced: true, + } + codeBlock.Content = work.Bytes() // TODO: get rid of temp buffer + + if p.extensions&Mmark == 0 { + p.addBlock(codeBlock) + finalizeCodeBlock(codeBlock) + return beg + } + + // Check for caption and if found make it a figure. + if captionContent, id, consumed := p.caption(data[beg:], []byte("Figure: ")); consumed > 0 { + figure := &ast.CaptionFigure{} + caption := &ast.Caption{} + figure.HeadingID = id + p.Inline(caption, captionContent) + + p.addBlock(figure) + codeBlock.AsLeaf().Attribute = figure.AsContainer().Attribute + p.addChild(codeBlock) + finalizeCodeBlock(codeBlock) + p.addChild(caption) + p.finalize(figure) + + beg += consumed + + return beg + } + + // Still here, normal block + p.addBlock(codeBlock) + finalizeCodeBlock(codeBlock) + } + + return beg +} + +func unescapeChar(str []byte) []byte { + if str[0] == '\\' { + return []byte{str[1]} + } + return []byte(html.UnescapeString(string(str))) +} + +func unescapeString(str []byte) []byte { + if reBackslashOrAmp.Match(str) { + return reEntityOrEscapedChar.ReplaceAllFunc(str, unescapeChar) + } + return str +} + +func finalizeCodeBlock(code *ast.CodeBlock) { + c := code.Content + if code.IsFenced { + newlinePos := bytes.IndexByte(c, '\n') + firstLine := c[:newlinePos] + rest := c[newlinePos+1:] + code.Info = unescapeString(bytes.Trim(firstLine, "\n")) + code.Literal = rest + } else { + code.Literal = c + } + code.Content = nil +} + +func (p *Parser) table(data []byte) int { + i, columns, table := p.tableHeader(data) + if i == 0 { + return 0 + } + + p.addBlock(&ast.TableBody{}) + + for i < len(data) { + pipes, rowStart := 0, i + for ; i < len(data) && data[i] != '\n'; i++ { + if data[i] == '|' { + pipes++ + } + } + + if pipes == 0 { + i = rowStart + break + } + + // include the newline in data sent to tableRow + i = skipCharN(data, i, '\n', 1) + + if p.tableFooter(data[rowStart:i]) { + continue + } + + p.tableRow(data[rowStart:i], columns, false) + } + if captionContent, id, consumed := p.caption(data[i:], []byte("Table: ")); consumed > 0 { + caption := &ast.Caption{} + p.Inline(caption, captionContent) + + // Some switcheroo to re-insert the parsed table as a child of the captionfigure. + figure := &ast.CaptionFigure{} + figure.HeadingID = id + table2 := &ast.Table{} + // Retain any block level attributes. + table2.AsContainer().Attribute = table.AsContainer().Attribute + children := table.GetChildren() + ast.RemoveFromTree(table) + + table2.SetChildren(children) + ast.AppendChild(figure, table2) + ast.AppendChild(figure, caption) + + p.addChild(figure) + p.finalize(figure) + + i += consumed + } + + return i +} + +// check if the specified position is preceded by an odd number of backslashes +func isBackslashEscaped(data []byte, i int) bool { + backslashes := 0 + for i-backslashes-1 >= 0 && data[i-backslashes-1] == '\\' { + backslashes++ + } + return backslashes&1 == 1 +} + +// tableHeaders parses the header. If recognized it will also add a table. +func (p *Parser) tableHeader(data []byte) (size int, columns []ast.CellAlignFlags, table ast.Node) { + i := 0 + colCount := 1 + for i = 0; i < len(data) && data[i] != '\n'; i++ { + if data[i] == '|' && !isBackslashEscaped(data, i) { + colCount++ + } + } + + // doesn't look like a table header + if colCount == 1 { + return + } + + // include the newline in the data sent to tableRow + j := skipCharN(data, i, '\n', 1) + header := data[:j] + + // column count ignores pipes at beginning or end of line + if data[0] == '|' { + colCount-- + } + if i > 2 && data[i-1] == '|' && !isBackslashEscaped(data, i-1) { + colCount-- + } + + columns = make([]ast.CellAlignFlags, colCount) + + // move on to the header underline + i++ + if i >= len(data) { + return + } + + if data[i] == '|' && !isBackslashEscaped(data, i) { + i++ + } + i = skipChar(data, i, ' ') + + // each column header is of form: / *:?-+:? *|/ with # dashes + # colons >= 3 + // and trailing | optional on last column + col := 0 + n := len(data) + for i < n && data[i] != '\n' { + dashes := 0 + + if data[i] == ':' { + i++ + columns[col] |= ast.TableAlignmentLeft + dashes++ + } + for i < n && data[i] == '-' { + i++ + dashes++ + } + if i < n && data[i] == ':' { + i++ + columns[col] |= ast.TableAlignmentRight + dashes++ + } + for i < n && data[i] == ' ' { + i++ + } + if i == n { + return + } + // end of column test is messy + switch { + case dashes < 3: + // not a valid column + return + + case data[i] == '|' && !isBackslashEscaped(data, i): + // marker found, now skip past trailing whitespace + col++ + i++ + for i < n && data[i] == ' ' { + i++ + } + + // trailing junk found after last column + if col >= colCount && i < len(data) && data[i] != '\n' { + return + } + + case (data[i] != '|' || isBackslashEscaped(data, i)) && col+1 < colCount: + // something else found where marker was required + return + + case data[i] == '\n': + // marker is optional for the last column + col++ + + default: + // trailing junk found after last column + return + } + } + if col != colCount { + return + } + + table = &ast.Table{} + p.addBlock(table) + p.addBlock(&ast.TableHeader{}) + p.tableRow(header, columns, true) + size = skipCharN(data, i, '\n', 1) + return +} + +func (p *Parser) tableRow(data []byte, columns []ast.CellAlignFlags, header bool) { + p.addBlock(&ast.TableRow{}) + i, col := 0, 0 + + if data[i] == '|' && !isBackslashEscaped(data, i) { + i++ + } + + n := len(data) + for col = 0; col < len(columns) && i < n; col++ { + for i < n && data[i] == ' ' { + i++ + } + + cellStart := i + + for i < n && (data[i] != '|' || isBackslashEscaped(data, i)) && data[i] != '\n' { + i++ + } + + cellEnd := i + + // skip the end-of-cell marker, possibly taking us past end of buffer + i++ + + for cellEnd > cellStart && cellEnd-1 < n && data[cellEnd-1] == ' ' { + cellEnd-- + } + + block := &ast.TableCell{ + IsHeader: header, + Align: columns[col], + } + block.Content = data[cellStart:cellEnd] + p.addBlock(block) + } + + // pad it out with empty columns to get the right number + for ; col < len(columns); col++ { + block := &ast.TableCell{ + IsHeader: header, + Align: columns[col], + } + p.addBlock(block) + } + + // silently ignore rows with too many cells +} + +// tableFooter parses the (optional) table footer. +func (p *Parser) tableFooter(data []byte) bool { + colCount := 1 + for i := 0; i < len(data) && data[i] != '\n'; i++ { + if data[i] == '|' && !isBackslashEscaped(data, i) { + colCount++ + continue + } + // remaining data must be the = character + if data[i] != '=' { + return false + } + } + + // doesn't look like a table footer + if colCount == 1 { + return false + } + + p.addBlock(&ast.TableFooter{}) + + return true +} + +// returns blockquote prefix length +func (p *Parser) quotePrefix(data []byte) int { + i := 0 + n := len(data) + for i < 3 && i < n && data[i] == ' ' { + i++ + } + if i < n && data[i] == '>' { + if i+1 < n && data[i+1] == ' ' { + return i + 2 + } + return i + 1 + } + return 0 +} + +// blockquote ends with at least one blank line +// followed by something without a blockquote prefix +func (p *Parser) terminateBlockquote(data []byte, beg, end int) bool { + if p.isEmpty(data[beg:]) <= 0 { + return false + } + if end >= len(data) { + return true + } + return p.quotePrefix(data[end:]) == 0 && p.isEmpty(data[end:]) == 0 +} + +// parse a blockquote fragment +func (p *Parser) quote(data []byte) int { + var raw bytes.Buffer + beg, end := 0, 0 + for beg < len(data) { + end = beg + // Step over whole lines, collecting them. While doing that, check for + // fenced code and if one's found, incorporate it altogether, + // irregardless of any contents inside it + for end < len(data) && data[end] != '\n' { + if p.extensions&FencedCode != 0 { + if i := p.fencedCodeBlock(data[end:], false); i > 0 { + // -1 to compensate for the extra end++ after the loop: + end += i - 1 + break + } + } + end++ + } + end = skipCharN(data, end, '\n', 1) + if pre := p.quotePrefix(data[beg:]); pre > 0 { + // skip the prefix + beg += pre + } else if p.terminateBlockquote(data, beg, end) { + break + } + // this line is part of the blockquote + raw.Write(data[beg:end]) + beg = end + } + + if p.extensions&Mmark == 0 { + block := p.addBlock(&ast.BlockQuote{}) + p.block(raw.Bytes()) + p.finalize(block) + return end + } + + if captionContent, id, consumed := p.caption(data[end:], []byte("Quote: ")); consumed > 0 { + figure := &ast.CaptionFigure{} + caption := &ast.Caption{} + figure.HeadingID = id + p.Inline(caption, captionContent) + + p.addBlock(figure) // this discard any attributes + block := &ast.BlockQuote{} + block.AsContainer().Attribute = figure.AsContainer().Attribute + p.addChild(block) + p.block(raw.Bytes()) + p.finalize(block) + + p.addChild(caption) + p.finalize(figure) + + end += consumed + + return end + } + + block := p.addBlock(&ast.BlockQuote{}) + p.block(raw.Bytes()) + p.finalize(block) + + return end +} + +// returns prefix length for block code +func (p *Parser) codePrefix(data []byte) int { + n := len(data) + if n >= 1 && data[0] == '\t' { + return 1 + } + if n >= 4 && data[3] == ' ' && data[2] == ' ' && data[1] == ' ' && data[0] == ' ' { + return 4 + } + return 0 +} + +func (p *Parser) code(data []byte) int { + var work bytes.Buffer + + i := 0 + for i < len(data) { + beg := i + + i = skipUntilChar(data, i, '\n') + i = skipCharN(data, i, '\n', 1) + + blankline := p.isEmpty(data[beg:i]) > 0 + if pre := p.codePrefix(data[beg:i]); pre > 0 { + beg += pre + } else if !blankline { + // non-empty, non-prefixed line breaks the pre + i = beg + break + } + + // verbatim copy to the working buffer + if blankline { + work.WriteByte('\n') + } else { + work.Write(data[beg:i]) + } + } + + // trim all the \n off the end of work + workbytes := work.Bytes() + + eol := backChar(workbytes, len(workbytes), '\n') + + if eol != len(workbytes) { + work.Truncate(eol) + } + + work.WriteByte('\n') + + codeBlock := &ast.CodeBlock{ + IsFenced: false, + } + // TODO: get rid of temp buffer + codeBlock.Content = work.Bytes() + p.addBlock(codeBlock) + finalizeCodeBlock(codeBlock) + + return i +} + +// returns unordered list item prefix +func (p *Parser) uliPrefix(data []byte) int { + // start with up to 3 spaces + i := skipCharN(data, 0, ' ', 3) + + if i >= len(data)-1 { + return 0 + } + // need one of {'*', '+', '-'} followed by a space or a tab + if (data[i] != '*' && data[i] != '+' && data[i] != '-') || + (data[i+1] != ' ' && data[i+1] != '\t') { + return 0 + } + return i + 2 +} + +// returns ordered list item prefix +func (p *Parser) oliPrefix(data []byte) int { + // start with up to 3 spaces + i := skipCharN(data, 0, ' ', 3) + + // count the digits + start := i + for i < len(data) && data[i] >= '0' && data[i] <= '9' { + i++ + } + if start == i || i >= len(data)-1 { + return 0 + } + + // we need >= 1 digits followed by a dot and a space or a tab + if data[i] != '.' || !(data[i+1] == ' ' || data[i+1] == '\t') { + return 0 + } + return i + 2 +} + +// returns definition list item prefix +func (p *Parser) dliPrefix(data []byte) int { + if len(data) < 2 { + return 0 + } + // need a ':' followed by a space or a tab + if data[0] != ':' || !(data[1] == ' ' || data[1] == '\t') { + return 0 + } + i := skipChar(data, 0, ' ') + return i + 2 +} + +// parse ordered or unordered list block +func (p *Parser) list(data []byte, flags ast.ListType, start int) int { + i := 0 + flags |= ast.ListItemBeginningOfList + list := &ast.List{ + ListFlags: flags, + Tight: true, + Start: start, + } + block := p.addBlock(list) + + for i < len(data) { + skip := p.listItem(data[i:], &flags) + if flags&ast.ListItemContainsBlock != 0 { + list.Tight = false + } + i += skip + if skip == 0 || flags&ast.ListItemEndOfList != 0 { + break + } + flags &= ^ast.ListItemBeginningOfList + } + + above := block.GetParent() + finalizeList(list) + p.tip = above + return i +} + +// Returns true if the list item is not the same type as its parent list +func (p *Parser) listTypeChanged(data []byte, flags *ast.ListType) bool { + if p.dliPrefix(data) > 0 && *flags&ast.ListTypeDefinition == 0 { + return true + } else if p.oliPrefix(data) > 0 && *flags&ast.ListTypeOrdered == 0 { + return true + } else if p.uliPrefix(data) > 0 && (*flags&ast.ListTypeOrdered != 0 || *flags&ast.ListTypeDefinition != 0) { + return true + } + return false +} + +// Returns true if block ends with a blank line, descending if needed +// into lists and sublists. +func endsWithBlankLine(block ast.Node) bool { + // TODO: figure this out. Always false now. + for block != nil { + //if block.lastLineBlank { + //return true + //} + switch block.(type) { + case *ast.List, *ast.ListItem: + block = ast.GetLastChild(block) + default: + return false + } + } + return false +} + +func finalizeList(list *ast.List) { + items := list.Parent.GetChildren() + lastItemIdx := len(items) - 1 + for i, item := range items { + isLastItem := i == lastItemIdx + // check for non-final list item ending with blank line: + if !isLastItem && endsWithBlankLine(item) { + list.Tight = false + break + } + // recurse into children of list item, to see if there are spaces + // between any of them: + subItems := item.GetParent().GetChildren() + lastSubItemIdx := len(subItems) - 1 + for j, subItem := range subItems { + isLastSubItem := j == lastSubItemIdx + if (!isLastItem || !isLastSubItem) && endsWithBlankLine(subItem) { + list.Tight = false + break + } + } + } +} + +// Parse a single list item. +// Assumes initial prefix is already removed if this is a sublist. +func (p *Parser) listItem(data []byte, flags *ast.ListType) int { + // keep track of the indentation of the first line + itemIndent := 0 + if data[0] == '\t' { + itemIndent += 4 + } else { + for itemIndent < 3 && data[itemIndent] == ' ' { + itemIndent++ + } + } + + var bulletChar byte = '*' + i := p.uliPrefix(data) + if i == 0 { + i = p.oliPrefix(data) + } else { + bulletChar = data[i-2] + } + if i == 0 { + i = p.dliPrefix(data) + // reset definition term flag + if i > 0 { + *flags &= ^ast.ListTypeTerm + } + } + if i == 0 { + // if in definition list, set term flag and continue + if *flags&ast.ListTypeDefinition != 0 { + *flags |= ast.ListTypeTerm + } else { + return 0 + } + } + + // skip leading whitespace on first line + i = skipChar(data, i, ' ') + + // find the end of the line + line := i + for i > 0 && i < len(data) && data[i-1] != '\n' { + i++ + } + + // get working buffer + var raw bytes.Buffer + + // put the first line into the working buffer + raw.Write(data[line:i]) + line = i + + // process the following lines + containsBlankLine := false + sublist := 0 + +gatherlines: + for line < len(data) { + i++ + + // find the end of this line + for i < len(data) && data[i-1] != '\n' { + i++ + } + + // if it is an empty line, guess that it is part of this item + // and move on to the next line + if p.isEmpty(data[line:i]) > 0 { + containsBlankLine = true + line = i + continue + } + + // calculate the indentation + indent := 0 + indentIndex := 0 + if data[line] == '\t' { + indentIndex++ + indent += 4 + } else { + for indent < 4 && line+indent < i && data[line+indent] == ' ' { + indent++ + indentIndex++ + } + } + + chunk := data[line+indentIndex : i] + + // evaluate how this line fits in + switch { + // is this a nested list item? + case (p.uliPrefix(chunk) > 0 && !p.isHRule(chunk)) || p.oliPrefix(chunk) > 0 || p.dliPrefix(chunk) > 0: + + // to be a nested list, it must be indented more + // if not, it is either a different kind of list + // or the next item in the same list + if indent <= itemIndent { + if p.listTypeChanged(chunk, flags) { + *flags |= ast.ListItemEndOfList + } else if containsBlankLine { + *flags |= ast.ListItemContainsBlock + } + + break gatherlines + } + + if containsBlankLine { + *flags |= ast.ListItemContainsBlock + } + + // is this the first item in the nested list? + if sublist == 0 { + sublist = raw.Len() + // in the case of dliPrefix we are too late and need to search back for the definition item, which + // should be on the previous line, we then adjust sublist to start there. + if p.dliPrefix(chunk) > 0 { + sublist = backUntilChar(raw.Bytes(), raw.Len()-1, '\n') + } + } + + // is this a nested prefix heading? + case p.isPrefixHeading(chunk), p.isPrefixSpecialHeading(chunk): + // if the heading is not indented, it is not nested in the list + // and thus ends the list + if containsBlankLine && indent < 4 { + *flags |= ast.ListItemEndOfList + break gatherlines + } + *flags |= ast.ListItemContainsBlock + + // anything following an empty line is only part + // of this item if it is indented 4 spaces + // (regardless of the indentation of the beginning of the item) + case containsBlankLine && indent < 4: + if *flags&ast.ListTypeDefinition != 0 && i < len(data)-1 { + // is the next item still a part of this list? + next := i + for next < len(data) && data[next] != '\n' { + next++ + } + for next < len(data)-1 && data[next] == '\n' { + next++ + } + if i < len(data)-1 && data[i] != ':' && next < len(data)-1 && data[next] != ':' { + *flags |= ast.ListItemEndOfList + } + } else { + *flags |= ast.ListItemEndOfList + } + break gatherlines + + // a blank line means this should be parsed as a block + case containsBlankLine: + raw.WriteByte('\n') + *flags |= ast.ListItemContainsBlock + } + + // if this line was preceded by one or more blanks, + // re-introduce the blank into the buffer + if containsBlankLine { + containsBlankLine = false + raw.WriteByte('\n') + } + + // add the line into the working buffer without prefix + raw.Write(data[line+indentIndex : i]) + + line = i + } + + rawBytes := raw.Bytes() + + listItem := &ast.ListItem{ + ListFlags: *flags, + Tight: false, + BulletChar: bulletChar, + Delimiter: '.', // Only '.' is possible in Markdown, but ')' will also be possible in CommonMark + } + p.addBlock(listItem) + + // render the contents of the list item + if *flags&ast.ListItemContainsBlock != 0 && *flags&ast.ListTypeTerm == 0 { + // intermediate render of block item, except for definition term + if sublist > 0 { + p.block(rawBytes[:sublist]) + p.block(rawBytes[sublist:]) + } else { + p.block(rawBytes) + } + } else { + // intermediate render of inline item + para := &ast.Paragraph{} + if sublist > 0 { + para.Content = rawBytes[:sublist] + } else { + para.Content = rawBytes + } + p.addChild(para) + if sublist > 0 { + p.block(rawBytes[sublist:]) + } + } + return line +} + +// render a single paragraph that has already been parsed out +func (p *Parser) renderParagraph(data []byte) { + if len(data) == 0 { + return + } + + // trim leading spaces + beg := skipChar(data, 0, ' ') + + end := len(data) + // trim trailing newline + if data[len(data)-1] == '\n' { + end-- + } + + // trim trailing spaces + for end > beg && data[end-1] == ' ' { + end-- + } + para := &ast.Paragraph{} + para.Content = data[beg:end] + p.addBlock(para) +} + +// blockMath handle block surround with $$ +func (p *Parser) blockMath(data []byte) int { + if len(data) <= 4 || data[0] != '$' || data[1] != '$' || data[2] == '$' { + return 0 + } + + // find next $$ + var end int + for end = 2; end+1 < len(data) && (data[end] != '$' || data[end+1] != '$'); end++ { + } + + // $$ not match + if end+1 == len(data) { + return 0 + } + + // render the display math + mathBlock := &ast.MathBlock{} + mathBlock.Literal = data[2:end] + p.addBlock(mathBlock) + + return end + 2 +} + +func (p *Parser) paragraph(data []byte) int { + // prev: index of 1st char of previous line + // line: index of 1st char of current line + // i: index of cursor/end of current line + var prev, line, i int + tabSize := tabSizeDefault + if p.extensions&TabSizeEight != 0 { + tabSize = tabSizeDouble + } + // keep going until we find something to mark the end of the paragraph + for i < len(data) { + // mark the beginning of the current line + prev = line + current := data[i:] + line = i + + // did we find a reference or a footnote? If so, end a paragraph + // preceding it and report that we have consumed up to the end of that + // reference: + if refEnd := isReference(p, current, tabSize); refEnd > 0 { + p.renderParagraph(data[:i]) + return i + refEnd + } + + // did we find a blank line marking the end of the paragraph? + if n := p.isEmpty(current); n > 0 { + // did this blank line followed by a definition list item? + if p.extensions&DefinitionLists != 0 { + if i < len(data)-1 && data[i+1] == ':' { + listLen := p.list(data[prev:], ast.ListTypeDefinition, 0) + return prev + listLen + } + } + + p.renderParagraph(data[:i]) + return i + n + } + + // an underline under some text marks a heading, so our paragraph ended on prev line + if i > 0 { + if level := p.isUnderlinedHeading(current); level > 0 { + // render the paragraph + p.renderParagraph(data[:prev]) + + // ignore leading and trailing whitespace + eol := i - 1 + for prev < eol && data[prev] == ' ' { + prev++ + } + for eol > prev && data[eol-1] == ' ' { + eol-- + } + + id := "" + if p.extensions&AutoHeadingIDs != 0 { + id = sanitizeAnchorName(string(data[prev:eol])) + } + + block := &ast.Heading{ + Level: level, + HeadingID: id, + } + block.Content = data[prev:eol] + p.addBlock(block) + + // find the end of the underline + return skipUntilChar(data, i, '\n') + } + } + + // if the next line starts a block of HTML, then the paragraph ends here + if p.extensions&LaxHTMLBlocks != 0 { + if data[i] == '<' && p.html(current, false) > 0 { + // rewind to before the HTML block + p.renderParagraph(data[:i]) + return i + } + } + + // if there's a prefixed heading or a horizontal rule after this, paragraph is over + if p.isPrefixHeading(current) || p.isPrefixSpecialHeading(current) || p.isHRule(current) { + p.renderParagraph(data[:i]) + return i + } + + // if there's a fenced code block, paragraph is over + if p.extensions&FencedCode != 0 { + if p.fencedCodeBlock(current, false) > 0 { + p.renderParagraph(data[:i]) + return i + } + } + + // if there's a figure block, paragraph is over + if p.extensions&Mmark != 0 { + if p.figureBlock(current, false) > 0 { + p.renderParagraph(data[:i]) + return i + } + } + + // if there's a definition list item, prev line is a definition term + if p.extensions&DefinitionLists != 0 { + if p.dliPrefix(current) != 0 { + ret := p.list(data[prev:], ast.ListTypeDefinition, 0) + return ret + prev + } + } + + // if there's a list after this, paragraph is over + if p.extensions&NoEmptyLineBeforeBlock != 0 { + if p.uliPrefix(current) != 0 || + p.oliPrefix(current) != 0 || + p.quotePrefix(current) != 0 || + p.codePrefix(current) != 0 { + p.renderParagraph(data[:i]) + return i + } + } + + // otherwise, scan to the beginning of the next line + nl := bytes.IndexByte(data[i:], '\n') + if nl >= 0 { + i += nl + 1 + } else { + i += len(data[i:]) + } + } + + p.renderParagraph(data[:i]) + return i +} + +// skipChar advances i as long as data[i] == c +func skipChar(data []byte, i int, c byte) int { + n := len(data) + for i < n && data[i] == c { + i++ + } + return i +} + +// like skipChar but only skips up to max characters +func skipCharN(data []byte, i int, c byte, max int) int { + n := len(data) + for i < n && max > 0 && data[i] == c { + i++ + max-- + } + return i +} + +// skipUntilChar advances i as long as data[i] != c +func skipUntilChar(data []byte, i int, c byte) int { + n := len(data) + for i < n && data[i] != c { + i++ + } + return i +} + +func skipAlnum(data []byte, i int) int { + n := len(data) + for i < n && isAlnum(data[i]) { + i++ + } + return i +} + +func skipSpace(data []byte, i int) int { + n := len(data) + for i < n && isSpace(data[i]) { + i++ + } + return i +} + +func backChar(data []byte, i int, c byte) int { + for i > 0 && data[i-1] == c { + i-- + } + return i +} + +func backUntilChar(data []byte, i int, c byte) int { + for i > 0 && data[i-1] != c { + i-- + } + return i +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/callout.go b/vendor/github.com/gomarkdown/markdown/parser/callout.go new file mode 100644 index 00000000..15858aa9 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/callout.go @@ -0,0 +1,29 @@ +package parser + +import ( + "bytes" + "strconv" +) + +// IsCallout detects a callout in the following format: <<N>> Where N is a integer > 0. +func IsCallout(data []byte) (id []byte, consumed int) { + if !bytes.HasPrefix(data, []byte("<<")) { + return nil, 0 + } + start := 2 + end := bytes.Index(data[start:], []byte(">>")) + if end < 0 { + return nil, 0 + } + + b := data[start : start+end] + b = bytes.TrimSpace(b) + i, err := strconv.Atoi(string(b)) + if err != nil { + return nil, 0 + } + if i <= 0 { + return nil, 0 + } + return b, start + end + 2 // 2 for >> +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/caption.go b/vendor/github.com/gomarkdown/markdown/parser/caption.go new file mode 100644 index 00000000..54d3f741 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/caption.go @@ -0,0 +1,70 @@ +package parser + +import ( + "bytes" +) + +// caption checks for a caption, it returns the caption data and a potential "headingID". +func (p *Parser) caption(data, caption []byte) ([]byte, string, int) { + if !bytes.HasPrefix(data, caption) { + return nil, "", 0 + } + j := len(caption) + data = data[j:] + end := p.linesUntilEmpty(data) + + data = data[:end] + + id, start := captionID(data) + if id != "" { + return data[:start], id, end + j + } + + return data, "", end + j +} + +// linesUntilEmpty scans lines up to the first empty line. +func (p *Parser) linesUntilEmpty(data []byte) int { + line, i := 0, 0 + + for line < len(data) { + i++ + + // find the end of this line + for i < len(data) && data[i-1] != '\n' { + i++ + } + + if p.isEmpty(data[line:i]) == 0 { + line = i + continue + } + + break + } + return i +} + +// captionID checks if the caption *ends* in {#....}. If so the text after {# is taken to be +// the ID/anchor of the entire figure block. +func captionID(data []byte) (string, int) { + end := len(data) + + j, k := 0, 0 + // find start/end of heading id + for j = 0; j < end-1 && (data[j] != '{' || data[j+1] != '#'); j++ { + } + for k = j + 1; k < end && data[k] != '}'; k++ { + } + // remains must be whitespace. + for l := k + 1; l < end; l++ { + if !isSpace(data[l]) { + return "", 0 + } + } + + if j > 0 && k > 0 && j+2 < k { + return string(data[j+2 : k]), j + } + return "", 0 +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/citation.go b/vendor/github.com/gomarkdown/markdown/parser/citation.go new file mode 100644 index 00000000..8ea1fbee --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/citation.go @@ -0,0 +1,86 @@ +package parser + +import ( + "bytes" + + "github.com/gomarkdown/markdown/ast" +) + +// citation parses a citation. In its most simple form [@ref], we allow multiple +// being separated by semicolons and a sub reference inside ala pandoc: [@ref, p. 23]. +// Each citation can have a modifier: !, ? or - wich mean: +// +// ! - normative +// ? - formative +// - - suppressed +// +// The suffix starts after a comma, we strip any whitespace before and after. If the output +// allows for it, this can be rendered. +func citation(p *Parser, data []byte, offset int) (int, ast.Node) { + // look for the matching closing bracket + i := offset + 1 + for level := 1; level > 0 && i < len(data); i++ { + switch { + case data[i] == '\n': + // no newlines allowed. + return 0, nil + + case data[i-1] == '\\': + continue + + case data[i] == '[': + level++ + + case data[i] == ']': + level-- + if level <= 0 { + i-- // compensate for extra i++ in for loop + } + } + } + + if i >= len(data) { + return 0, nil + } + + node := &ast.Citation{} + + citations := bytes.Split(data[1:i], []byte(";")) + for _, citation := range citations { + var suffix []byte + citation = bytes.TrimSpace(citation) + j := 0 + if citation[j] != '@' { + // not a citation, drop out entirely. + return 0, nil + } + if c := bytes.Index(citation, []byte(",")); c > 0 { + part := citation[:c] + suff := citation[c+1:] + part = bytes.TrimSpace(part) + suff = bytes.TrimSpace(suff) + + citation = part + suffix = suff + } + + citeType := ast.CitationTypeInformative + j = 1 + switch citation[j] { + case '!': + citeType = ast.CitationTypeNormative + j++ + case '?': + citeType = ast.CitationTypeInformative + j++ + case '-': + citeType = ast.CitationTypeSuppressed + j++ + } + node.Destination = append(node.Destination, citation[j:]) + node.Type = append(node.Type, citeType) + node.Suffix = append(node.Suffix, suffix) + } + + return i + 1, node +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/esc.go b/vendor/github.com/gomarkdown/markdown/parser/esc.go new file mode 100644 index 00000000..0a79aa35 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/esc.go @@ -0,0 +1,20 @@ +package parser + +// isEscape returns true if byte i is prefixed by an odd number of backslahses. +func isEscape(data []byte, i int) bool { + if i == 0 { + return false + } + if i == 1 { + return data[0] == '\\' + } + j := i - 1 + for ; j >= 0; j-- { + if data[j] != '\\' { + break + } + } + j++ + // odd number of backslahes means escape + return (i-j)%2 != 0 +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/figures.go b/vendor/github.com/gomarkdown/markdown/parser/figures.go new file mode 100644 index 00000000..6615449c --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/figures.go @@ -0,0 +1,119 @@ +package parser + +import ( + "bytes" + + "github.com/gomarkdown/markdown/ast" +) + +// sFigureLine checks if there's a figure line (e.g., !--- ) at the beginning of data, +// and returns the end index if so, or 0 otherwise. +func sFigureLine(data []byte, oldmarker string) (end int, marker string) { + i, size := 0, 0 + + n := len(data) + // skip up to three spaces + for i < n && i < 3 && data[i] == ' ' { + i++ + } + + // check for the marker characters: ! + if i+1 >= n { + return 0, "" + } + if data[i] != '!' || data[i+1] != '-' { + return 0, "" + } + i++ + + c := data[i] // i.e. the - + + // the whole line must be the same char or whitespace + for i < n && data[i] == c { + size++ + i++ + } + + // the marker char must occur at least 3 times + if size < 3 { + return 0, "" + } + marker = string(data[i-size : i]) + + // if this is the end marker, it must match the beginning marker + if oldmarker != "" && marker != oldmarker { + return 0, "" + } + + // there is no syntax modifier although it might be an idea to re-use this space for something? + + i = skipChar(data, i, ' ') + if i >= n || data[i] != '\n' { + if i == n { + return i, marker + } + return 0, "" + } + return i + 1, marker // Take newline into account. +} + +// figureBlock returns the end index if data contains a figure block at the beginning, +// or 0 otherwise. It writes to out if doRender is true, otherwise it has no side effects. +// If doRender is true, a final newline is mandatory to recognize the figure block. +func (p *Parser) figureBlock(data []byte, doRender bool) int { + beg, marker := sFigureLine(data, "") + if beg == 0 || beg >= len(data) { + return 0 + } + + var raw bytes.Buffer + + for { + // safe to assume beg < len(data) + + // check for the end of the code block + figEnd, _ := sFigureLine(data[beg:], marker) + if figEnd != 0 { + beg += figEnd + break + } + + // copy the current line + end := skipUntilChar(data, beg, '\n') + 1 + + // did we reach the end of the buffer without a closing marker? + if end >= len(data) { + return 0 + } + + // verbatim copy to the working buffer + if doRender { + raw.Write(data[beg:end]) + } + beg = end + } + + if !doRender { + return beg + } + + figure := &ast.CaptionFigure{} + p.addBlock(figure) + p.block(raw.Bytes()) + + defer p.finalize(figure) + + if captionContent, id, consumed := p.caption(data[beg:], []byte("Figure: ")); consumed > 0 { + caption := &ast.Caption{} + p.Inline(caption, captionContent) + + figure.HeadingID = id + + p.addChild(caption) + + beg += consumed + } + + p.finalize(figure) + return beg +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/include.go b/vendor/github.com/gomarkdown/markdown/parser/include.go new file mode 100644 index 00000000..2448a685 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/include.go @@ -0,0 +1,129 @@ +package parser + +import ( + "bytes" + "path" + "path/filepath" +) + +// isInclude parses {{...}}[...], that contains a path between the {{, the [...] syntax contains +// an address to select which lines to include. It is treated as an opaque string and just given +// to readInclude. +func (p *Parser) isInclude(data []byte) (filename string, address []byte, consumed int) { + i := skipCharN(data, 0, ' ', 3) // start with up to 3 spaces + if len(data[i:]) < 3 { + return "", nil, 0 + } + if data[i] != '{' || data[i+1] != '{' { + return "", nil, 0 + } + start := i + 2 + + // find the end delimiter + i = skipUntilChar(data, i, '}') + if i+1 >= len(data) { + return "", nil, 0 + } + end := i + i++ + if data[i] != '}' { + return "", nil, 0 + } + filename = string(data[start:end]) + + if i+1 < len(data) && data[i+1] == '[' { // potential address specification + start := i + 2 + + end = skipUntilChar(data, start, ']') + if end >= len(data) { + return "", nil, 0 + } + address = data[start:end] + return filename, address, end + 1 + } + + return filename, address, i + 1 +} + +func (p *Parser) readInclude(from, file string, address []byte) []byte { + if p.Opts.ReadIncludeFn != nil { + return p.Opts.ReadIncludeFn(from, file, address) + } + + return nil +} + +// isCodeInclude parses <{{...}} which is similar to isInclude the returned bytes are, however wrapped in a code block. +func (p *Parser) isCodeInclude(data []byte) (filename string, address []byte, consumed int) { + i := skipCharN(data, 0, ' ', 3) // start with up to 3 spaces + if len(data[i:]) < 3 { + return "", nil, 0 + } + if data[i] != '<' { + return "", nil, 0 + } + start := i + + filename, address, consumed = p.isInclude(data[i+1:]) + if consumed == 0 { + return "", nil, 0 + } + return filename, address, start + consumed + 1 +} + +// readCodeInclude acts like include except the returned bytes are wrapped in a fenced code block. +func (p *Parser) readCodeInclude(from, file string, address []byte) []byte { + data := p.readInclude(from, file, address) + if data == nil { + return nil + } + ext := path.Ext(file) + buf := &bytes.Buffer{} + buf.Write([]byte("```")) + if ext != "" { // starts with a dot + buf.WriteString(" " + ext[1:] + "\n") + } else { + buf.WriteByte('\n') + } + buf.Write(data) + buf.WriteString("```\n") + return buf.Bytes() +} + +// incStack hold the current stack of chained includes. Each value is the containing +// path of the file being parsed. +type incStack struct { + stack []string +} + +func newIncStack() *incStack { + return &incStack{stack: []string{}} +} + +// Push updates i with new. +func (i *incStack) Push(new string) { + if path.IsAbs(new) { + i.stack = append(i.stack, path.Dir(new)) + return + } + last := "" + if len(i.stack) > 0 { + last = i.stack[len(i.stack)-1] + } + i.stack = append(i.stack, path.Dir(filepath.Join(last, new))) +} + +// Pop pops the last value. +func (i *incStack) Pop() { + if len(i.stack) == 0 { + return + } + i.stack = i.stack[:len(i.stack)-1] +} + +func (i *incStack) Last() string { + if len(i.stack) == 0 { + return "" + } + return i.stack[len(i.stack)-1] +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/inline.go b/vendor/github.com/gomarkdown/markdown/parser/inline.go new file mode 100644 index 00000000..81766b85 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/inline.go @@ -0,0 +1,1284 @@ +package parser + +import ( + "bytes" + "regexp" + "strconv" + + "github.com/gomarkdown/markdown/ast" +) + +// Parsing of inline elements + +var ( + urlRe = `((https?|ftp):\/\/|\/)[-A-Za-z0-9+&@#\/%?=~_|!:,.;\(\)]+` + anchorRe = regexp.MustCompile(`^(<a\shref="` + urlRe + `"(\stitle="[^"<>]+")?\s?>` + urlRe + `<\/a>)`) + + // TODO: improve this regexp to catch all possible entities: + htmlEntityRe = regexp.MustCompile(`&[a-z]{2,5};`) +) + +// Inline parses text within a block. +// Each function returns the number of consumed chars. +func (p *Parser) Inline(currBlock ast.Node, data []byte) { + // handlers might call us recursively: enforce a maximum depth + if p.nesting >= p.maxNesting || len(data) == 0 { + return + } + p.nesting++ + beg, end := 0, 0 + + n := len(data) + for end < n { + handler := p.inlineCallback[data[end]] + if handler == nil { + end++ + continue + } + consumed, node := handler(p, data, end) + if consumed == 0 { + // no action from the callback + end++ + continue + } + // copy inactive chars into the output + ast.AppendChild(currBlock, newTextNode(data[beg:end])) + if node != nil { + ast.AppendChild(currBlock, node) + } + beg = end + consumed + end = beg + } + + if beg < n { + if data[end-1] == '\n' { + end-- + } + ast.AppendChild(currBlock, newTextNode(data[beg:end])) + } + p.nesting-- +} + +// single and double emphasis parsing +func emphasis(p *Parser, data []byte, offset int) (int, ast.Node) { + data = data[offset:] + c := data[0] + + n := len(data) + if n > 2 && data[1] != c { + // whitespace cannot follow an opening emphasis; + // strikethrough only takes two characters '~~' + if isSpace(data[1]) { + return 0, nil + } + if p.extensions&SuperSubscript != 0 && c == '~' { + // potential subscript, no spaces, except when escaped, helperEmphasis does + // not check that for us, so walk the bytes and check. + ret := skipUntilChar(data[1:], 0, c) + if ret == 0 { + return 0, nil + } + ret++ // we started with data[1:] above. + for i := 1; i < ret; i++ { + if isSpace(data[i]) && !isEscape(data, i) { + return 0, nil + } + } + sub := &ast.Subscript{} + sub.Literal = data[1:ret] + return ret + 1, sub + } + ret, node := helperEmphasis(p, data[1:], c) + if ret == 0 { + return 0, nil + } + + return ret + 1, node + } + + if n > 3 && data[1] == c && data[2] != c { + if isSpace(data[2]) { + return 0, nil + } + ret, node := helperDoubleEmphasis(p, data[2:], c) + if ret == 0 { + return 0, nil + } + + return ret + 2, node + } + + if n > 4 && data[1] == c && data[2] == c && data[3] != c { + if c == '~' || isSpace(data[3]) { + return 0, nil + } + ret, node := helperTripleEmphasis(p, data, 3, c) + if ret == 0 { + return 0, nil + } + + return ret + 3, node + } + + return 0, nil +} + +func codeSpan(p *Parser, data []byte, offset int) (int, ast.Node) { + data = data[offset:] + + // count the number of backticks in the delimiter + nb := skipChar(data, 0, '`') + + // find the next delimiter + i, end := 0, 0 + for end = nb; end < len(data) && i < nb; end++ { + if data[end] == '`' { + i++ + } else { + i = 0 + } + } + + // no matching delimiter? + if i < nb && end >= len(data) { + return 0, nil + } + + // trim outside whitespace + fBegin := nb + for fBegin < end && data[fBegin] == ' ' { + fBegin++ + } + + fEnd := end - nb + for fEnd > fBegin && data[fEnd-1] == ' ' { + fEnd-- + } + + // render the code span + if fBegin != fEnd { + code := &ast.Code{} + code.Literal = data[fBegin:fEnd] + return end, code + } + + return end, nil +} + +// newline preceded by two spaces becomes <br> +func maybeLineBreak(p *Parser, data []byte, offset int) (int, ast.Node) { + origOffset := offset + offset = skipChar(data, offset, ' ') + + if offset < len(data) && data[offset] == '\n' { + if offset-origOffset >= 2 { + return offset - origOffset + 1, &ast.Hardbreak{} + } + return offset - origOffset, nil + } + return 0, nil +} + +// newline without two spaces works when HardLineBreak is enabled +func lineBreak(p *Parser, data []byte, offset int) (int, ast.Node) { + if p.extensions&HardLineBreak != 0 { + return 1, &ast.Hardbreak{} + } + return 0, nil +} + +type linkType int + +const ( + linkNormal linkType = iota + linkImg + linkDeferredFootnote + linkInlineFootnote + linkCitation +) + +func isReferenceStyleLink(data []byte, pos int, t linkType) bool { + if t == linkDeferredFootnote { + return false + } + return pos < len(data)-1 && data[pos] == '[' && data[pos+1] != '^' +} + +func maybeImage(p *Parser, data []byte, offset int) (int, ast.Node) { + if offset < len(data)-1 && data[offset+1] == '[' { + return link(p, data, offset) + } + return 0, nil +} + +func maybeInlineFootnoteOrSuper(p *Parser, data []byte, offset int) (int, ast.Node) { + if offset < len(data)-1 && data[offset+1] == '[' { + return link(p, data, offset) + } + + if p.extensions&SuperSubscript != 0 { + ret := skipUntilChar(data[offset:], 1, '^') + if ret == 0 { + return 0, nil + } + for i := offset; i < offset+ret; i++ { + if isSpace(data[i]) && !isEscape(data, i) { + return 0, nil + } + } + sup := &ast.Superscript{} + sup.Literal = data[offset+1 : offset+ret] + return offset + ret, sup + } + + return 0, nil +} + +// '[': parse a link or an image or a footnote or a citation +func link(p *Parser, data []byte, offset int) (int, ast.Node) { + // no links allowed inside regular links, footnote, and deferred footnotes + if p.insideLink && (offset > 0 && data[offset-1] == '[' || len(data)-1 > offset && data[offset+1] == '^') { + return 0, nil + } + + var t linkType + switch { + // special case: ![^text] == deferred footnote (that follows something with + // an exclamation point) + case p.extensions&Footnotes != 0 && len(data)-1 > offset && data[offset+1] == '^': + t = linkDeferredFootnote + // ![alt] == image + case offset >= 0 && data[offset] == '!': + t = linkImg + offset++ + // [@citation], [@-citation], [@?citation], [@!citation] + case p.extensions&Mmark != 0 && len(data)-1 > offset && data[offset+1] == '@': + t = linkCitation + // [text] == regular link + // ^[text] == inline footnote + // [^refId] == deferred footnote + case p.extensions&Footnotes != 0: + if offset >= 0 && data[offset] == '^' { + t = linkInlineFootnote + offset++ + } else if len(data)-1 > offset && data[offset+1] == '^' { + t = linkDeferredFootnote + } + default: + t = linkNormal + } + + data = data[offset:] + + if t == linkCitation { + return citation(p, data, 0) + } + + var ( + i = 1 + noteID int + title, link, linkID, altContent []byte + textHasNl = false + ) + + if t == linkDeferredFootnote { + i++ + } + + // look for the matching closing bracket + for level := 1; level > 0 && i < len(data); i++ { + switch { + case data[i] == '\n': + textHasNl = true + + case data[i-1] == '\\': + continue + + case data[i] == '[': + level++ + + case data[i] == ']': + level-- + if level <= 0 { + i-- // compensate for extra i++ in for loop + } + } + } + + if i >= len(data) { + return 0, nil + } + + txtE := i + i++ + var footnoteNode ast.Node + + // skip any amount of whitespace or newline + // (this is much more lax than original markdown syntax) + i = skipSpace(data, i) + + // inline style link + switch { + case i < len(data) && data[i] == '(': + // skip initial whitespace + i++ + + i = skipSpace(data, i) + + linkB := i + + // look for link end: ' " ) + findlinkend: + for i < len(data) { + switch { + case data[i] == '\\': + i += 2 + + case data[i] == ')' || data[i] == '\'' || data[i] == '"': + break findlinkend + + default: + i++ + } + } + + if i >= len(data) { + return 0, nil + } + linkE := i + + // look for title end if present + titleB, titleE := 0, 0 + if data[i] == '\'' || data[i] == '"' { + i++ + titleB = i + + findtitleend: + for i < len(data) { + switch { + case data[i] == '\\': + i += 2 + + case data[i] == ')': + break findtitleend + + default: + i++ + } + } + + if i >= len(data) { + return 0, nil + } + + // skip whitespace after title + titleE = i - 1 + for titleE > titleB && isSpace(data[titleE]) { + titleE-- + } + + // check for closing quote presence + if data[titleE] != '\'' && data[titleE] != '"' { + titleB, titleE = 0, 0 + linkE = i + } + } + + // remove whitespace at the end of the link + for linkE > linkB && isSpace(data[linkE-1]) { + linkE-- + } + + // remove optional angle brackets around the link + if data[linkB] == '<' { + linkB++ + } + if data[linkE-1] == '>' { + linkE-- + } + + // build escaped link and title + if linkE > linkB { + link = data[linkB:linkE] + } + + if titleE > titleB { + title = data[titleB:titleE] + } + + i++ + + // reference style link + case isReferenceStyleLink(data, i, t): + var id []byte + altContentConsidered := false + + // look for the id + i++ + linkB := i + i = skipUntilChar(data, i, ']') + + if i >= len(data) { + return 0, nil + } + linkE := i + + // find the reference + if linkB == linkE { + if textHasNl { + var b bytes.Buffer + + for j := 1; j < txtE; j++ { + switch { + case data[j] != '\n': + b.WriteByte(data[j]) + case data[j-1] != ' ': + b.WriteByte(' ') + } + } + + id = b.Bytes() + } else { + id = data[1:txtE] + altContentConsidered = true + } + } else { + id = data[linkB:linkE] + } + + // find the reference with matching id + lr, ok := p.getRef(string(id)) + if !ok { + return 0, nil + } + + // keep link and title from reference + linkID = id + link = lr.link + title = lr.title + if altContentConsidered { + altContent = lr.text + } + i++ + + // shortcut reference style link or reference or inline footnote + default: + var id []byte + + // craft the id + if textHasNl { + var b bytes.Buffer + + for j := 1; j < txtE; j++ { + switch { + case data[j] != '\n': + b.WriteByte(data[j]) + case data[j-1] != ' ': + b.WriteByte(' ') + } + } + + id = b.Bytes() + } else { + if t == linkDeferredFootnote { + id = data[2:txtE] // get rid of the ^ + } else { + id = data[1:txtE] + } + } + + footnoteNode = &ast.ListItem{} + if t == linkInlineFootnote { + // create a new reference + noteID = len(p.notes) + 1 + + var fragment []byte + if len(id) > 0 { + if len(id) < 16 { + fragment = make([]byte, len(id)) + } else { + fragment = make([]byte, 16) + } + copy(fragment, slugify(id)) + } else { + fragment = append([]byte("footnote-"), []byte(strconv.Itoa(noteID))...) + } + + ref := &reference{ + noteID: noteID, + hasBlock: false, + link: fragment, + title: id, + footnote: footnoteNode, + } + + p.notes = append(p.notes, ref) + p.refsRecord[string(ref.link)] = struct{}{} + + link = ref.link + title = ref.title + } else { + // find the reference with matching id + lr, ok := p.getRef(string(id)) + if !ok { + return 0, nil + } + + if t == linkDeferredFootnote && !p.isFootnote(lr) { + lr.noteID = len(p.notes) + 1 + lr.footnote = footnoteNode + p.notes = append(p.notes, lr) + p.refsRecord[string(lr.link)] = struct{}{} + } + + // keep link and title from reference + link = lr.link + // if inline footnote, title == footnote contents + title = lr.title + noteID = lr.noteID + } + + // rewind the whitespace + i = txtE + 1 + } + + var uLink []byte + if t == linkNormal || t == linkImg { + if len(link) > 0 { + var uLinkBuf bytes.Buffer + unescapeText(&uLinkBuf, link) + uLink = uLinkBuf.Bytes() + } + + // links need something to click on and somewhere to go + if len(uLink) == 0 || (t == linkNormal && txtE <= 1) { + return 0, nil + } + } + + // call the relevant rendering function + switch t { + case linkNormal: + link := &ast.Link{ + Destination: normalizeURI(uLink), + Title: title, + DeferredID: linkID, + } + if len(altContent) > 0 { + ast.AppendChild(link, newTextNode(altContent)) + } else { + // links cannot contain other links, so turn off link parsing + // temporarily and recurse + insideLink := p.insideLink + p.insideLink = true + p.Inline(link, data[1:txtE]) + p.insideLink = insideLink + } + return i, link + + case linkImg: + image := &ast.Image{ + Destination: uLink, + Title: title, + } + ast.AppendChild(image, newTextNode(data[1:txtE])) + return i + 1, image + + case linkInlineFootnote, linkDeferredFootnote: + link := &ast.Link{ + Destination: link, + Title: title, + NoteID: noteID, + Footnote: footnoteNode, + } + if t == linkDeferredFootnote { + link.DeferredID = data[2:txtE] + } + if t == linkInlineFootnote { + i++ + } + return i, link + + default: + return 0, nil + } +} + +func (p *Parser) inlineHTMLComment(data []byte) int { + if len(data) < 5 { + return 0 + } + if data[0] != '<' || data[1] != '!' || data[2] != '-' || data[3] != '-' { + return 0 + } + i := 5 + // scan for an end-of-comment marker, across lines if necessary + for i < len(data) && !(data[i-2] == '-' && data[i-1] == '-' && data[i] == '>') { + i++ + } + // no end-of-comment marker + if i >= len(data) { + return 0 + } + return i + 1 +} + +func stripMailto(link []byte) []byte { + if bytes.HasPrefix(link, []byte("mailto://")) { + return link[9:] + } else if bytes.HasPrefix(link, []byte("mailto:")) { + return link[7:] + } else { + return link + } +} + +// autolinkType specifies a kind of autolink that gets detected. +type autolinkType int + +// These are the possible flag values for the autolink renderer. +const ( + notAutolink autolinkType = iota + normalAutolink + emailAutolink +) + +// '<' when tags or autolinks are allowed +func leftAngle(p *Parser, data []byte, offset int) (int, ast.Node) { + data = data[offset:] + + if p.extensions&Mmark != 0 { + id, consumed := IsCallout(data) + if consumed > 0 { + node := &ast.Callout{} + node.ID = id + return consumed, node + } + } + + altype, end := tagLength(data) + if size := p.inlineHTMLComment(data); size > 0 { + end = size + } + if end <= 2 { + return end, nil + } + if altype == notAutolink { + htmlTag := &ast.HTMLSpan{} + htmlTag.Literal = data[:end] + return end, htmlTag + } + + var uLink bytes.Buffer + unescapeText(&uLink, data[1:end+1-2]) + if uLink.Len() <= 0 { + return end, nil + } + link := uLink.Bytes() + node := &ast.Link{ + Destination: link, + } + if altype == emailAutolink { + node.Destination = append([]byte("mailto:"), link...) + } + ast.AppendChild(node, newTextNode(stripMailto(link))) + return end, node +} + +// '\\' backslash escape +var escapeChars = []byte("\\`*_{}[]()#+-.!:|&<>~") + +func escape(p *Parser, data []byte, offset int) (int, ast.Node) { + data = data[offset:] + + if len(data) <= 1 { + return 2, nil + } + + if p.extensions&NonBlockingSpace != 0 && data[1] == ' ' { + return 2, &ast.NonBlockingSpace{} + } + + if p.extensions&BackslashLineBreak != 0 && data[1] == '\n' { + return 2, &ast.Hardbreak{} + } + + if bytes.IndexByte(escapeChars, data[1]) < 0 { + return 0, nil + } + + return 2, newTextNode(data[1:2]) +} + +func unescapeText(ob *bytes.Buffer, src []byte) { + i := 0 + for i < len(src) { + org := i + for i < len(src) && src[i] != '\\' { + i++ + } + + if i > org { + ob.Write(src[org:i]) + } + + if i+1 >= len(src) { + break + } + + ob.WriteByte(src[i+1]) + i += 2 + } +} + +// '&' escaped when it doesn't belong to an entity +// valid entities are assumed to be anything matching &#?[A-Za-z0-9]+; +func entity(p *Parser, data []byte, offset int) (int, ast.Node) { + data = data[offset:] + + end := skipCharN(data, 1, '#', 1) + end = skipAlnum(data, end) + + if end < len(data) && data[end] == ';' { + end++ // real entity + } else { + return 0, nil // lone '&' + } + + ent := data[:end] + // undo & escaping or it will be converted to &amp; by another + // escaper in the renderer + if bytes.Equal(ent, []byte("&")) { + ent = []byte{'&'} + } + + return end, newTextNode(ent) +} + +func linkEndsWithEntity(data []byte, linkEnd int) bool { + entityRanges := htmlEntityRe.FindAllIndex(data[:linkEnd], -1) + return entityRanges != nil && entityRanges[len(entityRanges)-1][1] == linkEnd +} + +// hasPrefixCaseInsensitive is a custom implementation of +// strings.HasPrefix(strings.ToLower(s), prefix) +// we rolled our own because ToLower pulls in a huge machinery of lowercasing +// anything from Unicode and that's very slow. Since this func will only be +// used on ASCII protocol prefixes, we can take shortcuts. +func hasPrefixCaseInsensitive(s, prefix []byte) bool { + if len(s) < len(prefix) { + return false + } + delta := byte('a' - 'A') + for i, b := range prefix { + if b != s[i] && b != s[i]+delta { + return false + } + } + return true +} + +var protocolPrefixes = [][]byte{ + []byte("http://"), + []byte("https://"), + []byte("ftp://"), + []byte("file://"), + []byte("mailto:"), +} + +const shortestPrefix = 6 // len("ftp://"), the shortest of the above + +func maybeAutoLink(p *Parser, data []byte, offset int) (int, ast.Node) { + // quick check to rule out most false hits + if p.insideLink || len(data) < offset+shortestPrefix { + return 0, nil + } + for _, prefix := range protocolPrefixes { + endOfHead := offset + 8 // 8 is the len() of the longest prefix + if endOfHead > len(data) { + endOfHead = len(data) + } + if hasPrefixCaseInsensitive(data[offset:endOfHead], prefix) { + return autoLink(p, data, offset) + } + } + return 0, nil +} + +func autoLink(p *Parser, data []byte, offset int) (int, ast.Node) { + // Now a more expensive check to see if we're not inside an anchor element + anchorStart := offset + offsetFromAnchor := 0 + for anchorStart > 0 && data[anchorStart] != '<' { + anchorStart-- + offsetFromAnchor++ + } + + anchorStr := anchorRe.Find(data[anchorStart:]) + if anchorStr != nil { + anchorClose := &ast.HTMLSpan{} + anchorClose.Literal = anchorStr[offsetFromAnchor:] + return len(anchorStr) - offsetFromAnchor, anchorClose + } + + // scan backward for a word boundary + rewind := 0 + for offset-rewind > 0 && rewind <= 7 && isLetter(data[offset-rewind-1]) { + rewind++ + } + if rewind > 6 { // longest supported protocol is "mailto" which has 6 letters + return 0, nil + } + + origData := data + data = data[offset-rewind:] + + if !isSafeLink(data) { + return 0, nil + } + + linkEnd := 0 + for linkEnd < len(data) && !isEndOfLink(data[linkEnd]) { + linkEnd++ + } + + // Skip punctuation at the end of the link + if (data[linkEnd-1] == '.' || data[linkEnd-1] == ',') && data[linkEnd-2] != '\\' { + linkEnd-- + } + + // But don't skip semicolon if it's a part of escaped entity: + if data[linkEnd-1] == ';' && data[linkEnd-2] != '\\' && !linkEndsWithEntity(data, linkEnd) { + linkEnd-- + } + + // See if the link finishes with a punctuation sign that can be closed. + var copen byte + switch data[linkEnd-1] { + case '"': + copen = '"' + case '\'': + copen = '\'' + case ')': + copen = '(' + case ']': + copen = '[' + case '}': + copen = '{' + default: + copen = 0 + } + + if copen != 0 { + bufEnd := offset - rewind + linkEnd - 2 + + openDelim := 1 + + /* Try to close the final punctuation sign in this same line; + * if we managed to close it outside of the URL, that means that it's + * not part of the URL. If it closes inside the URL, that means it + * is part of the URL. + * + * Examples: + * + * foo http://www.pokemon.com/Pikachu_(Electric) bar + * => http://www.pokemon.com/Pikachu_(Electric) + * + * foo (http://www.pokemon.com/Pikachu_(Electric)) bar + * => http://www.pokemon.com/Pikachu_(Electric) + * + * foo http://www.pokemon.com/Pikachu_(Electric)) bar + * => http://www.pokemon.com/Pikachu_(Electric)) + * + * (foo http://www.pokemon.com/Pikachu_(Electric)) bar + * => foo http://www.pokemon.com/Pikachu_(Electric) + */ + + for bufEnd >= 0 && origData[bufEnd] != '\n' && openDelim != 0 { + if origData[bufEnd] == data[linkEnd-1] { + openDelim++ + } + + if origData[bufEnd] == copen { + openDelim-- + } + + bufEnd-- + } + + if openDelim == 0 { + linkEnd-- + } + } + + var uLink bytes.Buffer + unescapeText(&uLink, data[:linkEnd]) + + if uLink.Len() > 0 { + node := &ast.Link{ + Destination: uLink.Bytes(), + } + ast.AppendChild(node, newTextNode(uLink.Bytes())) + return linkEnd, node + } + + return linkEnd, nil +} + +func isEndOfLink(char byte) bool { + return isSpace(char) || char == '<' +} + +var validUris = [][]byte{[]byte("http://"), []byte("https://"), []byte("ftp://"), []byte("mailto://")} +var validPaths = [][]byte{[]byte("/"), []byte("./"), []byte("../")} + +func isSafeLink(link []byte) bool { + nLink := len(link) + for _, path := range validPaths { + nPath := len(path) + linkPrefix := link[:nPath] + if nLink >= nPath && bytes.Equal(linkPrefix, path) { + if nLink == nPath { + return true + } else if isAlnum(link[nPath]) { + return true + } + } + } + + for _, prefix := range validUris { + // TODO: handle unicode here + // case-insensitive prefix test + nPrefix := len(prefix) + if nLink > nPrefix { + linkPrefix := bytes.ToLower(link[:nPrefix]) + if bytes.Equal(linkPrefix, prefix) && isAlnum(link[nPrefix]) { + return true + } + } + } + + return false +} + +// return the length of the given tag, or 0 is it's not valid +func tagLength(data []byte) (autolink autolinkType, end int) { + var i, j int + + // a valid tag can't be shorter than 3 chars + if len(data) < 3 { + return notAutolink, 0 + } + + // begins with a '<' optionally followed by '/', followed by letter or number + if data[0] != '<' { + return notAutolink, 0 + } + if data[1] == '/' { + i = 2 + } else { + i = 1 + } + + if !isAlnum(data[i]) { + return notAutolink, 0 + } + + // scheme test + autolink = notAutolink + + // try to find the beginning of an URI + for i < len(data) && (isAlnum(data[i]) || data[i] == '.' || data[i] == '+' || data[i] == '-') { + i++ + } + + if i > 1 && i < len(data) && data[i] == '@' { + if j = isMailtoAutoLink(data[i:]); j != 0 { + return emailAutolink, i + j + } + } + + if i > 2 && i < len(data) && data[i] == ':' { + autolink = normalAutolink + i++ + } + + // complete autolink test: no whitespace or ' or " + switch { + case i >= len(data): + autolink = notAutolink + case autolink != notAutolink: + j = i + + for i < len(data) { + if data[i] == '\\' { + i += 2 + } else if data[i] == '>' || data[i] == '\'' || data[i] == '"' || isSpace(data[i]) { + break + } else { + i++ + } + + } + + if i >= len(data) { + return autolink, 0 + } + if i > j && data[i] == '>' { + return autolink, i + 1 + } + + // one of the forbidden chars has been found + autolink = notAutolink + } + i += bytes.IndexByte(data[i:], '>') + if i < 0 { + return autolink, 0 + } + return autolink, i + 1 +} + +// look for the address part of a mail autolink and '>' +// this is less strict than the original markdown e-mail address matching +func isMailtoAutoLink(data []byte) int { + nb := 0 + + // address is assumed to be: [-@._a-zA-Z0-9]+ with exactly one '@' + for i, c := range data { + if isAlnum(c) { + continue + } + + switch c { + case '@': + nb++ + + case '-', '.', '_': + break + + case '>': + if nb == 1 { + return i + 1 + } + return 0 + default: + return 0 + } + } + + return 0 +} + +// look for the next emph char, skipping other constructs +func helperFindEmphChar(data []byte, c byte) int { + i := 0 + + for i < len(data) { + for i < len(data) && data[i] != c && data[i] != '`' && data[i] != '[' { + i++ + } + if i >= len(data) { + return 0 + } + // do not count escaped chars + if i != 0 && data[i-1] == '\\' { + i++ + continue + } + if data[i] == c { + return i + } + + if data[i] == '`' { + // skip a code span + tmpI := 0 + i++ + for i < len(data) && data[i] != '`' { + if tmpI == 0 && data[i] == c { + tmpI = i + } + i++ + } + if i >= len(data) { + return tmpI + } + i++ + } else if data[i] == '[' { + // skip a link + tmpI := 0 + i++ + for i < len(data) && data[i] != ']' { + if tmpI == 0 && data[i] == c { + tmpI = i + } + i++ + } + i++ + for i < len(data) && (data[i] == ' ' || data[i] == '\n') { + i++ + } + if i >= len(data) { + return tmpI + } + if data[i] != '[' && data[i] != '(' { // not a link + if tmpI > 0 { + return tmpI + } + continue + } + cc := data[i] + i++ + for i < len(data) && data[i] != cc { + if tmpI == 0 && data[i] == c { + return i + } + i++ + } + if i >= len(data) { + return tmpI + } + i++ + } + } + return 0 +} + +func helperEmphasis(p *Parser, data []byte, c byte) (int, ast.Node) { + i := 0 + + // skip one symbol if coming from emph3 + if len(data) > 1 && data[0] == c && data[1] == c { + i = 1 + } + + for i < len(data) { + length := helperFindEmphChar(data[i:], c) + if length == 0 { + return 0, nil + } + i += length + if i >= len(data) { + return 0, nil + } + + if i+1 < len(data) && data[i+1] == c { + i++ + continue + } + + if data[i] == c && !isSpace(data[i-1]) { + + if p.extensions&NoIntraEmphasis != 0 { + if !(i+1 == len(data) || isSpace(data[i+1]) || isPunctuation(data[i+1])) { + continue + } + } + + emph := &ast.Emph{} + p.Inline(emph, data[:i]) + return i + 1, emph + } + } + + return 0, nil +} + +func helperDoubleEmphasis(p *Parser, data []byte, c byte) (int, ast.Node) { + i := 0 + + for i < len(data) { + length := helperFindEmphChar(data[i:], c) + if length == 0 { + return 0, nil + } + i += length + + if i+1 < len(data) && data[i] == c && data[i+1] == c && i > 0 && !isSpace(data[i-1]) { + var node ast.Node = &ast.Strong{} + if c == '~' { + node = &ast.Del{} + } + p.Inline(node, data[:i]) + return i + 2, node + } + i++ + } + return 0, nil +} + +func helperTripleEmphasis(p *Parser, data []byte, offset int, c byte) (int, ast.Node) { + i := 0 + origData := data + data = data[offset:] + + for i < len(data) { + length := helperFindEmphChar(data[i:], c) + if length == 0 { + return 0, nil + } + i += length + + // skip whitespace preceded symbols + if data[i] != c || isSpace(data[i-1]) { + continue + } + + switch { + case i+2 < len(data) && data[i+1] == c && data[i+2] == c: + // triple symbol found + strong := &ast.Strong{} + em := &ast.Emph{} + ast.AppendChild(strong, em) + p.Inline(em, data[:i]) + return i + 3, strong + case i+1 < len(data) && data[i+1] == c: + // double symbol found, hand over to emph1 + length, node := helperEmphasis(p, origData[offset-2:], c) + if length == 0 { + return 0, nil + } + return length - 2, node + default: + // single symbol found, hand over to emph2 + length, node := helperDoubleEmphasis(p, origData[offset-1:], c) + if length == 0 { + return 0, nil + } + return length - 1, node + } + } + return 0, nil +} + +// math handle inline math wrapped with '$' +func math(p *Parser, data []byte, offset int) (int, ast.Node) { + data = data[offset:] + + // too short, or block math + if len(data) <= 2 || data[1] == '$' { + return 0, nil + } + + // find next '$' + var end int + for end = 1; end < len(data) && data[end] != '$'; end++ { + } + + // $ not match + if end == len(data) { + return 0, nil + } + + // create inline math node + math := &ast.Math{} + math.Literal = data[1:end] + return end + 1, math +} + +func newTextNode(d []byte) *ast.Text { + return &ast.Text{ast.Leaf{Literal: d}} +} + +func normalizeURI(s []byte) []byte { + return s // TODO: implement +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/matter.go b/vendor/github.com/gomarkdown/markdown/parser/matter.go new file mode 100644 index 00000000..92686357 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/matter.go @@ -0,0 +1,36 @@ +package parser + +import ( + "bytes" + + "github.com/gomarkdown/markdown/ast" +) + +func (p *Parser) documentMatter(data []byte) int { + if data[0] != '{' { + return 0 + } + + consumed := 0 + matter := ast.DocumentMatterNone + if bytes.HasPrefix(data, []byte("{frontmatter}")) { + consumed = len("{frontmatter}") + matter = ast.DocumentMatterFront + } + if bytes.HasPrefix(data, []byte("{mainmatter}")) { + consumed = len("{mainmatter}") + matter = ast.DocumentMatterMain + } + if bytes.HasPrefix(data, []byte("{backmatter}")) { + consumed = len("{backmatter}") + matter = ast.DocumentMatterBack + } + if consumed == 0 { + return 0 + } + node := &ast.DocumentMatter{Matter: matter} + p.addBlock(node) + p.finalize(node) + + return consumed +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/options.go b/vendor/github.com/gomarkdown/markdown/parser/options.go new file mode 100644 index 00000000..d3d0c088 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/options.go @@ -0,0 +1,32 @@ +package parser + +import ( + "github.com/gomarkdown/markdown/ast" +) + +// Flags control optional behavior of parser. +type Flags int + +// Options is a collection of supplementary parameters tweaking the behavior of various parts of the parser. +type Options struct { + ParserHook BlockFunc + ReadIncludeFn ReadIncludeFunc + + Flags Flags // Flags allow customizing parser's behavior +} + +// Parser renderer configuration options. +const ( + FlagsNone Flags = 0 + SkipFootnoteList Flags = 1 << iota // Skip adding the footnote list (regardless if they are parsed) +) + +// BlockFunc allows to registration of a parser function. If successful it +// returns an ast.Node, a buffer that should be parsed as a block and the the number of bytes consumed. +type BlockFunc func(data []byte) (ast.Node, []byte, int) + +// ReadIncludeFunc should read the file under path and returns the read bytes, +// from will be set to the name of the current file being parsed. Initially +// this will be empty. address is the optional address specifier of which lines +// of the file to return. If this function is not set no data will be read. +type ReadIncludeFunc func(from, path string, address []byte) []byte diff --git a/vendor/github.com/gomarkdown/markdown/parser/parser.go b/vendor/github.com/gomarkdown/markdown/parser/parser.go new file mode 100644 index 00000000..c7302dfd --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/parser.go @@ -0,0 +1,812 @@ +/* +Package parser implements parser for markdown text that generates AST (abstract syntax tree). +*/ +package parser + +import ( + "bytes" + "fmt" + "strings" + "unicode/utf8" + + "github.com/gomarkdown/markdown/ast" +) + +// Extensions is a bitmask of enabled parser extensions. +type Extensions int + +// Bit flags representing markdown parsing extensions. +// Use | (or) to specify multiple extensions. +const ( + NoExtensions Extensions = 0 + NoIntraEmphasis Extensions = 1 << iota // Ignore emphasis markers inside words + Tables // Parse tables + FencedCode // Parse fenced code blocks + Autolink // Detect embedded URLs that are not explicitly marked + Strikethrough // Strikethrough text using ~~test~~ + LaxHTMLBlocks // Loosen up HTML block parsing rules + SpaceHeadings // Be strict about prefix heading rules + HardLineBreak // Translate newlines into line breaks + NonBlockingSpace // Translate backspace spaces into line non-blocking spaces + TabSizeEight // Expand tabs to eight spaces instead of four + Footnotes // Pandoc-style footnotes + NoEmptyLineBeforeBlock // No need to insert an empty line to start a (code, quote, ordered list, unordered list) block + HeadingIDs // specify heading IDs with {#id} + Titleblock // Titleblock ala pandoc + AutoHeadingIDs // Create the heading ID from the text + BackslashLineBreak // Translate trailing backslashes into line breaks + DefinitionLists // Parse definition lists + MathJax // Parse MathJax + OrderedListStart // Keep track of the first number used when starting an ordered list. + Attributes // Block Attributes + SuperSubscript // Super- and subscript support: 2^10^, H~2~O. + EmptyLinesBreakList // 2 empty lines break out of list + Includes // Support including other files. + Mmark // Support Mmark syntax, see https://mmark.nl/syntax + + CommonExtensions Extensions = NoIntraEmphasis | Tables | FencedCode | + Autolink | Strikethrough | SpaceHeadings | HeadingIDs | + BackslashLineBreak | DefinitionLists | MathJax +) + +// The size of a tab stop. +const ( + tabSizeDefault = 4 + tabSizeDouble = 8 +) + +// for each character that triggers a response when parsing inline data. +type inlineParser func(p *Parser, data []byte, offset int) (int, ast.Node) + +// ReferenceOverrideFunc is expected to be called with a reference string and +// return either a valid Reference type that the reference string maps to or +// nil. If overridden is false, the default reference logic will be executed. +// See the documentation in Options for more details on use-case. +type ReferenceOverrideFunc func(reference string) (ref *Reference, overridden bool) + +// Parser is a type that holds extensions and the runtime state used by +// Parse, and the renderer. You can not use it directly, construct it with New. +type Parser struct { + + // ReferenceOverride is an optional function callback that is called every + // time a reference is resolved. It can be set before starting parsing. + // + // In Markdown, the link reference syntax can be made to resolve a link to + // a reference instead of an inline URL, in one of the following ways: + // + // * [link text][refid] + // * [refid][] + // + // Usually, the refid is defined at the bottom of the Markdown document. If + // this override function is provided, the refid is passed to the override + // function first, before consulting the defined refids at the bottom. If + // the override function indicates an override did not occur, the refids at + // the bottom will be used to fill in the link details. + ReferenceOverride ReferenceOverrideFunc + + Opts Options + + // after parsing, this is AST root of parsed markdown text + Doc ast.Node + + extensions Extensions + + refs map[string]*reference + refsRecord map[string]struct{} + inlineCallback [256]inlineParser + nesting int + maxNesting int + insideLink bool + indexCnt int // incremented after every index + + // Footnotes need to be ordered as well as available to quickly check for + // presence. If a ref is also a footnote, it's stored both in refs and here + // in notes. Slice is nil if footnotes not enabled. + notes []*reference + + tip ast.Node // = doc + oldTip ast.Node + lastMatchedContainer ast.Node // = doc + allClosed bool + + // Attributes are attached to block level elements. + attr *ast.Attribute + + includeStack *incStack +} + +// New creates a markdown parser with CommonExtensions. +// +// You can then call `doc := p.Parse(markdown)` to parse markdown document +// and `markdown.Render(doc, renderer)` to convert it to another format with +// a renderer. +func New() *Parser { + return NewWithExtensions(CommonExtensions) +} + +// NewWithExtensions creates a markdown parser with given extensions. +func NewWithExtensions(extension Extensions) *Parser { + p := Parser{ + refs: make(map[string]*reference), + refsRecord: make(map[string]struct{}), + maxNesting: 16, + insideLink: false, + Doc: &ast.Document{}, + extensions: extension, + allClosed: true, + includeStack: newIncStack(), + } + p.tip = p.Doc + p.oldTip = p.Doc + p.lastMatchedContainer = p.Doc + + p.inlineCallback[' '] = maybeLineBreak + p.inlineCallback['*'] = emphasis + p.inlineCallback['_'] = emphasis + if p.extensions&Strikethrough != 0 { + p.inlineCallback['~'] = emphasis + } + p.inlineCallback['`'] = codeSpan + p.inlineCallback['\n'] = lineBreak + p.inlineCallback['['] = link + p.inlineCallback['<'] = leftAngle + p.inlineCallback['\\'] = escape + p.inlineCallback['&'] = entity + p.inlineCallback['!'] = maybeImage + if p.extensions&Mmark != 0 { + p.inlineCallback['('] = maybeShortRefOrIndex + } + p.inlineCallback['^'] = maybeInlineFootnoteOrSuper + if p.extensions&Autolink != 0 { + p.inlineCallback['h'] = maybeAutoLink + p.inlineCallback['m'] = maybeAutoLink + p.inlineCallback['f'] = maybeAutoLink + p.inlineCallback['H'] = maybeAutoLink + p.inlineCallback['M'] = maybeAutoLink + p.inlineCallback['F'] = maybeAutoLink + } + if p.extensions&MathJax != 0 { + p.inlineCallback['$'] = math + } + + return &p +} + +func (p *Parser) getRef(refid string) (ref *reference, found bool) { + if p.ReferenceOverride != nil { + r, overridden := p.ReferenceOverride(refid) + if overridden { + if r == nil { + return nil, false + } + return &reference{ + link: []byte(r.Link), + title: []byte(r.Title), + noteID: 0, + hasBlock: false, + text: []byte(r.Text)}, true + } + } + // refs are case insensitive + ref, found = p.refs[strings.ToLower(refid)] + return ref, found +} + +func (p *Parser) isFootnote(ref *reference) bool { + _, ok := p.refsRecord[string(ref.link)] + return ok +} + +func (p *Parser) finalize(block ast.Node) { + p.tip = block.GetParent() +} + +func (p *Parser) addChild(node ast.Node) ast.Node { + for !canNodeContain(p.tip, node) { + p.finalize(p.tip) + } + ast.AppendChild(p.tip, node) + p.tip = node + return node +} + +func canNodeContain(n ast.Node, v ast.Node) bool { + switch n.(type) { + case *ast.List: + return isListItem(v) + case *ast.Document, *ast.BlockQuote, *ast.Aside, *ast.ListItem, *ast.CaptionFigure: + return !isListItem(v) + case *ast.Table: + switch v.(type) { + case *ast.TableHeader, *ast.TableBody, *ast.TableFooter: + return true + default: + return false + } + case *ast.TableHeader, *ast.TableBody, *ast.TableFooter: + _, ok := v.(*ast.TableRow) + return ok + case *ast.TableRow: + _, ok := v.(*ast.TableCell) + return ok + } + return false +} + +func (p *Parser) closeUnmatchedBlocks() { + if p.allClosed { + return + } + for p.oldTip != p.lastMatchedContainer { + parent := p.oldTip.GetParent() + p.finalize(p.oldTip) + p.oldTip = parent + } + p.allClosed = true +} + +// Reference represents the details of a link. +// See the documentation in Options for more details on use-case. +type Reference struct { + // Link is usually the URL the reference points to. + Link string + // Title is the alternate text describing the link in more detail. + Title string + // Text is the optional text to override the ref with if the syntax used was + // [refid][] + Text string +} + +// Parse generates AST (abstract syntax tree) representing markdown document. +// +// The result is a root of the tree whose underlying type is *ast.Document +// +// You can then convert AST to html using html.Renderer, to some other format +// using a custom renderer or transform the tree. +func (p *Parser) Parse(input []byte) ast.Node { + p.block(input) + // Walk the tree and finish up some of unfinished blocks + for p.tip != nil { + p.finalize(p.tip) + } + // Walk the tree again and process inline markdown in each block + ast.WalkFunc(p.Doc, func(node ast.Node, entering bool) ast.WalkStatus { + switch node.(type) { + case *ast.Paragraph, *ast.Heading, *ast.TableCell: + p.Inline(node, node.AsContainer().Content) + node.AsContainer().Content = nil + } + return ast.GoToNext + }) + + if p.Opts.Flags&SkipFootnoteList == 0 { + p.parseRefsToAST() + } + return p.Doc +} + +func (p *Parser) parseRefsToAST() { + if p.extensions&Footnotes == 0 || len(p.notes) == 0 { + return + } + p.tip = p.Doc + list := &ast.List{ + IsFootnotesList: true, + ListFlags: ast.ListTypeOrdered, + } + p.addBlock(&ast.Footnotes{}) + block := p.addBlock(list) + flags := ast.ListItemBeginningOfList + // Note: this loop is intentionally explicit, not range-form. This is + // because the body of the loop will append nested footnotes to p.notes and + // we need to process those late additions. Range form would only walk over + // the fixed initial set. + for i := 0; i < len(p.notes); i++ { + ref := p.notes[i] + p.addChild(ref.footnote) + block := ref.footnote + listItem := block.(*ast.ListItem) + listItem.ListFlags = flags | ast.ListTypeOrdered + listItem.RefLink = ref.link + if ref.hasBlock { + flags |= ast.ListItemContainsBlock + p.block(ref.title) + } else { + p.Inline(block, ref.title) + } + flags &^= ast.ListItemBeginningOfList | ast.ListItemContainsBlock + } + above := list.Parent + finalizeList(list) + p.tip = above + + ast.WalkFunc(block, func(node ast.Node, entering bool) ast.WalkStatus { + switch node.(type) { + case *ast.Paragraph, *ast.Heading: + p.Inline(node, node.AsContainer().Content) + node.AsContainer().Content = nil + } + return ast.GoToNext + }) +} + +// +// Link references +// +// This section implements support for references that (usually) appear +// as footnotes in a document, and can be referenced anywhere in the document. +// The basic format is: +// +// [1]: http://www.google.com/ "Google" +// [2]: http://www.github.com/ "Github" +// +// Anywhere in the document, the reference can be linked by referring to its +// label, i.e., 1 and 2 in this example, as in: +// +// This library is hosted on [Github][2], a git hosting site. +// +// Actual footnotes as specified in Pandoc and supported by some other Markdown +// libraries such as php-markdown are also taken care of. They look like this: +// +// This sentence needs a bit of further explanation.[^note] +// +// [^note]: This is the explanation. +// +// Footnotes should be placed at the end of the document in an ordered list. +// Inline footnotes such as: +// +// Inline footnotes^[Not supported.] also exist. +// +// are not yet supported. + +// reference holds all information necessary for a reference-style links or +// footnotes. +// +// Consider this markdown with reference-style links: +// +// [link][ref] +// +// [ref]: /url/ "tooltip title" +// +// It will be ultimately converted to this HTML: +// +// <p><a href=\"/url/\" title=\"title\">link</a></p> +// +// And a reference structure will be populated as follows: +// +// p.refs["ref"] = &reference{ +// link: "/url/", +// title: "tooltip title", +// } +// +// Alternatively, reference can contain information about a footnote. Consider +// this markdown: +// +// Text needing a footnote.[^a] +// +// [^a]: This is the note +// +// A reference structure will be populated as follows: +// +// p.refs["a"] = &reference{ +// link: "a", +// title: "This is the note", +// noteID: <some positive int>, +// } +// +// TODO: As you can see, it begs for splitting into two dedicated structures +// for refs and for footnotes. +type reference struct { + link []byte + title []byte + noteID int // 0 if not a footnote ref + hasBlock bool + footnote ast.Node // a link to the Item node within a list of footnotes + + text []byte // only gets populated by refOverride feature with Reference.Text +} + +func (r *reference) String() string { + return fmt.Sprintf("{link: %q, title: %q, text: %q, noteID: %d, hasBlock: %v}", + r.link, r.title, r.text, r.noteID, r.hasBlock) +} + +// Check whether or not data starts with a reference link. +// If so, it is parsed and stored in the list of references +// (in the render struct). +// Returns the number of bytes to skip to move past it, +// or zero if the first line is not a reference. +func isReference(p *Parser, data []byte, tabSize int) int { + // up to 3 optional leading spaces + if len(data) < 4 { + return 0 + } + i := 0 + for i < 3 && data[i] == ' ' { + i++ + } + + noteID := 0 + + // id part: anything but a newline between brackets + if data[i] != '[' { + return 0 + } + i++ + if p.extensions&Footnotes != 0 { + if i < len(data) && data[i] == '^' { + // we can set it to anything here because the proper noteIds will + // be assigned later during the second pass. It just has to be != 0 + noteID = 1 + i++ + } + } + idOffset := i + for i < len(data) && data[i] != '\n' && data[i] != '\r' && data[i] != ']' { + i++ + } + if i >= len(data) || data[i] != ']' { + return 0 + } + idEnd := i + // footnotes can have empty ID, like this: [^], but a reference can not be + // empty like this: []. Break early if it's not a footnote and there's no ID + if noteID == 0 && idOffset == idEnd { + return 0 + } + // spacer: colon (space | tab)* newline? (space | tab)* + i++ + if i >= len(data) || data[i] != ':' { + return 0 + } + i++ + for i < len(data) && (data[i] == ' ' || data[i] == '\t') { + i++ + } + if i < len(data) && (data[i] == '\n' || data[i] == '\r') { + i++ + if i < len(data) && data[i] == '\n' && data[i-1] == '\r' { + i++ + } + } + for i < len(data) && (data[i] == ' ' || data[i] == '\t') { + i++ + } + if i >= len(data) { + return 0 + } + + var ( + linkOffset, linkEnd int + titleOffset, titleEnd int + lineEnd int + raw []byte + hasBlock bool + ) + + if p.extensions&Footnotes != 0 && noteID != 0 { + linkOffset, linkEnd, raw, hasBlock = scanFootnote(p, data, i, tabSize) + lineEnd = linkEnd + } else { + linkOffset, linkEnd, titleOffset, titleEnd, lineEnd = scanLinkRef(p, data, i) + } + if lineEnd == 0 { + return 0 + } + + // a valid ref has been found + + ref := &reference{ + noteID: noteID, + hasBlock: hasBlock, + } + + if noteID > 0 { + // reusing the link field for the id since footnotes don't have links + ref.link = data[idOffset:idEnd] + // if footnote, it's not really a title, it's the contained text + ref.title = raw + } else { + ref.link = data[linkOffset:linkEnd] + ref.title = data[titleOffset:titleEnd] + } + + // id matches are case-insensitive + id := string(bytes.ToLower(data[idOffset:idEnd])) + + p.refs[id] = ref + + return lineEnd +} + +func scanLinkRef(p *Parser, data []byte, i int) (linkOffset, linkEnd, titleOffset, titleEnd, lineEnd int) { + // link: whitespace-free sequence, optionally between angle brackets + if data[i] == '<' { + i++ + } + linkOffset = i + for i < len(data) && data[i] != ' ' && data[i] != '\t' && data[i] != '\n' && data[i] != '\r' { + i++ + } + linkEnd = i + if linkEnd < len(data) && data[linkOffset] == '<' && data[linkEnd-1] == '>' { + linkOffset++ + linkEnd-- + } + + // optional spacer: (space | tab)* (newline | '\'' | '"' | '(' ) + for i < len(data) && (data[i] == ' ' || data[i] == '\t') { + i++ + } + if i < len(data) && data[i] != '\n' && data[i] != '\r' && data[i] != '\'' && data[i] != '"' && data[i] != '(' { + return + } + + // compute end-of-line + if i >= len(data) || data[i] == '\r' || data[i] == '\n' { + lineEnd = i + } + if i+1 < len(data) && data[i] == '\r' && data[i+1] == '\n' { + lineEnd++ + } + + // optional (space|tab)* spacer after a newline + if lineEnd > 0 { + i = lineEnd + 1 + for i < len(data) && (data[i] == ' ' || data[i] == '\t') { + i++ + } + } + + // optional title: any non-newline sequence enclosed in '"() alone on its line + if i+1 < len(data) && (data[i] == '\'' || data[i] == '"' || data[i] == '(') { + i++ + titleOffset = i + + // look for EOL + for i < len(data) && data[i] != '\n' && data[i] != '\r' { + i++ + } + if i+1 < len(data) && data[i] == '\n' && data[i+1] == '\r' { + titleEnd = i + 1 + } else { + titleEnd = i + } + + // step back + i-- + for i > titleOffset && (data[i] == ' ' || data[i] == '\t') { + i-- + } + if i > titleOffset && (data[i] == '\'' || data[i] == '"' || data[i] == ')') { + lineEnd = titleEnd + titleEnd = i + } + } + + return +} + +// The first bit of this logic is the same as Parser.listItem, but the rest +// is much simpler. This function simply finds the entire block and shifts it +// over by one tab if it is indeed a block (just returns the line if it's not). +// blockEnd is the end of the section in the input buffer, and contents is the +// extracted text that was shifted over one tab. It will need to be rendered at +// the end of the document. +func scanFootnote(p *Parser, data []byte, i, indentSize int) (blockStart, blockEnd int, contents []byte, hasBlock bool) { + if i == 0 || len(data) == 0 { + return + } + + // skip leading whitespace on first line + for i < len(data) && data[i] == ' ' { + i++ + } + + blockStart = i + + // find the end of the line + blockEnd = i + for i < len(data) && data[i-1] != '\n' { + i++ + } + + // get working buffer + var raw bytes.Buffer + + // put the first line into the working buffer + raw.Write(data[blockEnd:i]) + blockEnd = i + + // process the following lines + containsBlankLine := false + +gatherLines: + for blockEnd < len(data) { + i++ + + // find the end of this line + for i < len(data) && data[i-1] != '\n' { + i++ + } + + // if it is an empty line, guess that it is part of this item + // and move on to the next line + if p.isEmpty(data[blockEnd:i]) > 0 { + containsBlankLine = true + blockEnd = i + continue + } + + n := 0 + if n = isIndented(data[blockEnd:i], indentSize); n == 0 { + // this is the end of the block. + // we don't want to include this last line in the index. + break gatherLines + } + + // if there were blank lines before this one, insert a new one now + if containsBlankLine { + raw.WriteByte('\n') + containsBlankLine = false + } + + // get rid of that first tab, write to buffer + raw.Write(data[blockEnd+n : i]) + hasBlock = true + + blockEnd = i + } + + if data[blockEnd-1] != '\n' { + raw.WriteByte('\n') + } + + contents = raw.Bytes() + + return +} + +// isPunctuation returns true if c is a punctuation symbol. +func isPunctuation(c byte) bool { + for _, r := range []byte("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~") { + if c == r { + return true + } + } + return false +} + +// isSpace returns true if c is a white-space charactr +func isSpace(c byte) bool { + return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v' +} + +// isLetter returns true if c is ascii letter +func isLetter(c byte) bool { + return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') +} + +// isAlnum returns true if c is a digit or letter +// TODO: check when this is looking for ASCII alnum and when it should use unicode +func isAlnum(c byte) bool { + return (c >= '0' && c <= '9') || isLetter(c) +} + +// TODO: this is not used +// Replace tab characters with spaces, aligning to the next TAB_SIZE column. +// always ends output with a newline +func expandTabs(out *bytes.Buffer, line []byte, tabSize int) { + // first, check for common cases: no tabs, or only tabs at beginning of line + i, prefix := 0, 0 + slowcase := false + for i = 0; i < len(line); i++ { + if line[i] == '\t' { + if prefix == i { + prefix++ + } else { + slowcase = true + break + } + } + } + + // no need to decode runes if all tabs are at the beginning of the line + if !slowcase { + for i = 0; i < prefix*tabSize; i++ { + out.WriteByte(' ') + } + out.Write(line[prefix:]) + return + } + + // the slow case: we need to count runes to figure out how + // many spaces to insert for each tab + column := 0 + i = 0 + for i < len(line) { + start := i + for i < len(line) && line[i] != '\t' { + _, size := utf8.DecodeRune(line[i:]) + i += size + column++ + } + + if i > start { + out.Write(line[start:i]) + } + + if i >= len(line) { + break + } + + for { + out.WriteByte(' ') + column++ + if column%tabSize == 0 { + break + } + } + + i++ + } +} + +// Find if a line counts as indented or not. +// Returns number of characters the indent is (0 = not indented). +func isIndented(data []byte, indentSize int) int { + if len(data) == 0 { + return 0 + } + if data[0] == '\t' { + return 1 + } + if len(data) < indentSize { + return 0 + } + for i := 0; i < indentSize; i++ { + if data[i] != ' ' { + return 0 + } + } + return indentSize +} + +// Create a url-safe slug for fragments +func slugify(in []byte) []byte { + if len(in) == 0 { + return in + } + out := make([]byte, 0, len(in)) + sym := false + + for _, ch := range in { + if isAlnum(ch) { + sym = false + out = append(out, ch) + } else if sym { + continue + } else { + out = append(out, '-') + sym = true + } + } + var a, b int + var ch byte + for a, ch = range out { + if ch != '-' { + break + } + } + for b = len(out) - 1; b > 0; b-- { + if out[b] != '-' { + break + } + } + return out[a : b+1] +} + +func isListItem(d ast.Node) bool { + _, ok := d.(*ast.ListItem) + return ok +} diff --git a/vendor/github.com/gomarkdown/markdown/parser/ref.go b/vendor/github.com/gomarkdown/markdown/parser/ref.go new file mode 100644 index 00000000..0b59a196 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/parser/ref.go @@ -0,0 +1,89 @@ +package parser + +import ( + "bytes" + "fmt" + + "github.com/gomarkdown/markdown/ast" +) + +// parse '(#r)', where r does not contain spaces. Or. +// (!item) (!item, subitem), for an index, (!!item) signals primary. +func maybeShortRefOrIndex(p *Parser, data []byte, offset int) (int, ast.Node) { + if len(data[offset:]) < 4 { + return 0, nil + } + // short ref first + data = data[offset:] + i := 1 + switch data[i] { + case '#': // cross ref + i++ + Loop: + for i < len(data) { + c := data[i] + switch { + case c == ')': + break Loop + case !isAlnum(c): + if c == '_' || c == '-' || c == ':' { + i++ + continue + } + i = 0 + break Loop + } + i++ + } + if i >= len(data) { + return 0, nil + } + if data[i] != ')' { + return 0, nil + } + + id := data[2:i] + node := &ast.CrossReference{} + node.Destination = id + + return i + 1, node + + case '!': // index + i++ + start := i + i = skipUntilChar(data, start, ')') + + // did we reach the end of the buffer without a closing marker? + if i >= len(data) { + return 0, nil + } + + if len(data[start:i]) < 1 { + return 0, nil + } + + idx := &ast.Index{} + + idx.ID = fmt.Sprintf("idxref:%d", p.indexCnt) + p.indexCnt++ + + idx.Primary = data[start] == '!' + buf := data[start:i] + + if idx.Primary { + buf = buf[1:] + } + items := bytes.Split(buf, []byte(",")) + switch len(items) { + case 1: + idx.Item = bytes.TrimSpace(items[0]) + return i + 1, idx + case 2: + idx.Item = bytes.TrimSpace(items[0]) + idx.Subitem = bytes.TrimSpace(items[1]) + return i + 1, idx + } + } + + return 0, nil +} diff --git a/vendor/github.com/gomarkdown/markdown/todo.md b/vendor/github.com/gomarkdown/markdown/todo.md new file mode 100644 index 00000000..be8bb55c --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/todo.md @@ -0,0 +1,7 @@ +# Things to do + +[ ] docs: add examples like https://godoc.org/github.com/dgrijalva/jwt-go (put in foo_example_test.go). Or see https://github.com/garyburd/redigo/blob/master/redis/zpop_example_test.go#L5 / https://godoc.org/github.com/garyburd/redigo/redis or https://godoc.org/github.com/go-redis/redis + +[ ] figure out expandTabs and parser.TabSizeEight. Are those used? + +[ ] SoftbreakData is not used diff --git a/vendor/github.com/gomarkdown/markdown/tracking-perf.md b/vendor/github.com/gomarkdown/markdown/tracking-perf.md new file mode 100644 index 00000000..40b95183 --- /dev/null +++ b/vendor/github.com/gomarkdown/markdown/tracking-perf.md @@ -0,0 +1,189 @@ +## Tracking perf changes + +Initial performance: +``` +goos: darwin +goarch: amd64 +pkg: github.com/gomarkdown/markdown +BenchmarkEscapeHTML-8 2000000 823 ns/op 0 B/op 0 allocs/op +BenchmarkSmartDoubleQuotes-8 300000 5033 ns/op 9872 B/op 56 allocs/op +BenchmarkReferenceAmps-8 100000 19538 ns/op 26776 B/op 150 allocs/op +BenchmarkReferenceAutoLinks-8 100000 17574 ns/op 24544 B/op 132 allocs/op +BenchmarkReferenceBackslashEscapes-8 30000 50977 ns/op 76752 B/op 243 allocs/op +BenchmarkReferenceBlockquotesWithCodeBlocks-8 200000 8546 ns/op 12864 B/op 65 allocs/op +BenchmarkReferenceCodeBlocks-8 200000 9000 ns/op 14912 B/op 70 allocs/op +BenchmarkReferenceCodeSpans-8 200000 8856 ns/op 14992 B/op 69 allocs/op +BenchmarkReferenceHardWrappedPara-8 200000 6599 ns/op 11312 B/op 57 allocs/op +BenchmarkReferenceHorizontalRules-8 100000 15483 ns/op 23536 B/op 98 allocs/op +BenchmarkReferenceInlineHTMLAdvances-8 200000 6839 ns/op 12150 B/op 62 allocs/op +BenchmarkReferenceInlineHTMLSimple-8 100000 19940 ns/op 28488 B/op 117 allocs/op +BenchmarkReferenceInlineHTMLComments-8 200000 7455 ns/op 13440 B/op 64 allocs/op +BenchmarkReferenceLinksInline-8 100000 16425 ns/op 23664 B/op 147 allocs/op +BenchmarkReferenceLinksReference-8 30000 54895 ns/op 66464 B/op 416 allocs/op +BenchmarkReferenceLinksShortcut-8 100000 17647 ns/op 23776 B/op 158 allocs/op +BenchmarkReferenceLiterQuotesInTitles-8 200000 9367 ns/op 14832 B/op 95 allocs/op +BenchmarkReferenceMarkdownBasics-8 10000 129772 ns/op 130848 B/op 378 allocs/op +BenchmarkReferenceMarkdownSyntax-8 3000 502365 ns/op 461411 B/op 1411 allocs/op +BenchmarkReferenceNestedBlockquotes-8 200000 7028 ns/op 12688 B/op 64 allocs/op +BenchmarkReferenceOrderedAndUnorderedLists-8 20000 79686 ns/op 107520 B/op 374 allocs/op +BenchmarkReferenceStrongAndEm-8 200000 10020 ns/op 17792 B/op 78 allocs/op +BenchmarkReferenceTabs-8 200000 12025 ns/op 18224 B/op 81 allocs/op +BenchmarkReferenceTidyness-8 200000 8985 ns/op 14432 B/op 71 allocs/op +PASS +ok github.com/gomarkdown/markdown 45.375s +``` + +After switching to using interface{} for Node.Data: +``` +BenchmarkEscapeHTML-8 2000000 929 ns/op 0 B/op 0 allocs/op +BenchmarkSmartDoubleQuotes-8 300000 5126 ns/op 9248 B/op 56 allocs/op +BenchmarkReferenceAmps-8 100000 19927 ns/op 17880 B/op 154 allocs/op +BenchmarkReferenceAutoLinks-8 100000 20732 ns/op 17360 B/op 141 allocs/op +BenchmarkReferenceBackslashEscapes-8 30000 50267 ns/op 38128 B/op 244 allocs/op +BenchmarkReferenceBlockquotesWithCodeBlocks-8 200000 8988 ns/op 10912 B/op 67 allocs/op +BenchmarkReferenceCodeBlocks-8 200000 8611 ns/op 12256 B/op 74 allocs/op +BenchmarkReferenceCodeSpans-8 200000 8256 ns/op 11248 B/op 69 allocs/op +BenchmarkReferenceHardWrappedPara-8 200000 6739 ns/op 9856 B/op 57 allocs/op +BenchmarkReferenceHorizontalRules-8 100000 15503 ns/op 15600 B/op 104 allocs/op +BenchmarkReferenceInlineHTMLAdvances-8 200000 6874 ns/op 10278 B/op 62 allocs/op +BenchmarkReferenceInlineHTMLSimple-8 100000 22271 ns/op 18552 B/op 121 allocs/op +BenchmarkReferenceInlineHTMLComments-8 200000 8315 ns/op 10736 B/op 64 allocs/op +BenchmarkReferenceLinksInline-8 100000 16155 ns/op 16912 B/op 152 allocs/op +BenchmarkReferenceLinksReference-8 30000 52387 ns/op 38192 B/op 445 allocs/op +BenchmarkReferenceLinksShortcut-8 100000 17111 ns/op 16592 B/op 167 allocs/op +BenchmarkReferenceLiterQuotesInTitles-8 200000 9164 ns/op 12048 B/op 97 allocs/op +BenchmarkReferenceMarkdownBasics-8 10000 129262 ns/op 87264 B/op 416 allocs/op +BenchmarkReferenceMarkdownSyntax-8 3000 496873 ns/op 293906 B/op 1559 allocs/op +BenchmarkReferenceNestedBlockquotes-8 200000 6854 ns/op 10192 B/op 64 allocs/op +BenchmarkReferenceOrderedAndUnorderedLists-8 20000 79633 ns/op 55024 B/op 447 allocs/op +BenchmarkReferenceStrongAndEm-8 200000 9637 ns/op 12176 B/op 78 allocs/op +BenchmarkReferenceTabs-8 100000 12164 ns/op 13776 B/op 87 allocs/op +BenchmarkReferenceTidyness-8 200000 8677 ns/op 11296 B/op 75 allocs/op +``` + +Not necessarily faster, but uses less bytes per op (but sometimes more allocs). + +After tweaking the API: +``` +$ ./s/run-bench.sh + +go test -bench=. -test.benchmem +goos: darwin +goarch: amd64 +pkg: github.com/gomarkdown/markdown +BenchmarkEscapeHTML-8 2000000 834 ns/op 0 B/op 0 allocs/op +BenchmarkSmartDoubleQuotes-8 300000 3486 ns/op 6160 B/op 27 allocs/op +BenchmarkReferenceAmps-8 100000 18158 ns/op 14792 B/op 125 allocs/op +BenchmarkReferenceAutoLinks-8 100000 16824 ns/op 14272 B/op 112 allocs/op +BenchmarkReferenceBackslashEscapes-8 30000 44066 ns/op 35040 B/op 215 allocs/op +BenchmarkReferenceBlockquotesWithCodeBlocks-8 200000 6868 ns/op 7824 B/op 38 allocs/op +BenchmarkReferenceCodeBlocks-8 200000 7157 ns/op 9168 B/op 45 allocs/op +BenchmarkReferenceCodeSpans-8 200000 6663 ns/op 8160 B/op 40 allocs/op +BenchmarkReferenceHardWrappedPara-8 300000 4821 ns/op 6768 B/op 28 allocs/op +BenchmarkReferenceHorizontalRules-8 100000 13033 ns/op 12512 B/op 75 allocs/op +BenchmarkReferenceInlineHTMLAdvances-8 300000 4998 ns/op 7190 B/op 33 allocs/op +BenchmarkReferenceInlineHTMLSimple-8 100000 17696 ns/op 15464 B/op 92 allocs/op +BenchmarkReferenceInlineHTMLComments-8 300000 5506 ns/op 7648 B/op 35 allocs/op +BenchmarkReferenceLinksInline-8 100000 14450 ns/op 13824 B/op 123 allocs/op +BenchmarkReferenceLinksReference-8 30000 52561 ns/op 35104 B/op 416 allocs/op +BenchmarkReferenceLinksShortcut-8 100000 15616 ns/op 13504 B/op 138 allocs/op +BenchmarkReferenceLiterQuotesInTitles-8 200000 7772 ns/op 8960 B/op 68 allocs/op +BenchmarkReferenceMarkdownBasics-8 10000 121436 ns/op 84176 B/op 387 allocs/op +BenchmarkReferenceMarkdownSyntax-8 3000 487404 ns/op 290818 B/op 1530 allocs/op +BenchmarkReferenceNestedBlockquotes-8 300000 5098 ns/op 7104 B/op 35 allocs/op +BenchmarkReferenceOrderedAndUnorderedLists-8 20000 74422 ns/op 51936 B/op 418 allocs/op +BenchmarkReferenceStrongAndEm-8 200000 7888 ns/op 9088 B/op 49 allocs/op +BenchmarkReferenceTabs-8 200000 10061 ns/op 10688 B/op 58 allocs/op +BenchmarkReferenceTidyness-8 200000 7152 ns/op 8208 B/op 46 allocs/op +ok github.com/gomarkdown/markdown 40.809s +``` + +After refactoring Renderer: +``` +BenchmarkEscapeHTML-8 2000000 883 ns/op 0 B/op 0 allocs/op +BenchmarkSmartDoubleQuotes-8 300000 3717 ns/op 6208 B/op 29 allocs/op +BenchmarkReferenceAmps-8 100000 19135 ns/op 14680 B/op 123 allocs/op +BenchmarkReferenceAutoLinks-8 100000 17142 ns/op 14176 B/op 110 allocs/op +BenchmarkReferenceBackslashEscapes-8 30000 54616 ns/op 35088 B/op 217 allocs/op +BenchmarkReferenceBlockquotesWithCodeBlocks-8 200000 7993 ns/op 7872 B/op 40 allocs/op +BenchmarkReferenceCodeBlocks-8 200000 8285 ns/op 9216 B/op 47 allocs/op +BenchmarkReferenceCodeSpans-8 200000 7684 ns/op 8208 B/op 42 allocs/op +BenchmarkReferenceHardWrappedPara-8 200000 5595 ns/op 6816 B/op 30 allocs/op +BenchmarkReferenceHorizontalRules-8 100000 16444 ns/op 12560 B/op 77 allocs/op +BenchmarkReferenceInlineHTMLAdvances-8 200000 5415 ns/op 7238 B/op 35 allocs/op +BenchmarkReferenceInlineHTMLSimple-8 100000 19867 ns/op 15512 B/op 94 allocs/op +BenchmarkReferenceInlineHTMLComments-8 200000 6026 ns/op 7696 B/op 37 allocs/op +BenchmarkReferenceLinksInline-8 100000 14864 ns/op 13664 B/op 120 allocs/op +BenchmarkReferenceLinksReference-8 30000 52479 ns/op 34816 B/op 401 allocs/op +BenchmarkReferenceLinksShortcut-8 100000 15812 ns/op 13472 B/op 135 allocs/op +BenchmarkReferenceLiterQuotesInTitles-8 200000 7767 ns/op 8880 B/op 68 allocs/op +BenchmarkReferenceMarkdownBasics-8 10000 131065 ns/op 84048 B/op 386 allocs/op +BenchmarkReferenceMarkdownSyntax-8 2000 515604 ns/op 289953 B/op 1501 allocs/op +BenchmarkReferenceNestedBlockquotes-8 200000 5655 ns/op 7152 B/op 37 allocs/op +BenchmarkReferenceOrderedAndUnorderedLists-8 20000 84188 ns/op 51984 B/op 420 allocs/op +BenchmarkReferenceStrongAndEm-8 200000 8664 ns/op 9136 B/op 51 allocs/op +BenchmarkReferenceTabs-8 100000 11110 ns/op 10736 B/op 60 allocs/op +BenchmarkReferenceTidyness-8 200000 7628 ns/op 8256 B/op 48 allocs/op +ok github.com/gomarkdown/markdown 40.841s +``` + +After Node refactor to have Children array: +``` +BenchmarkEscapeHTML-8 2000000 901 ns/op 0 B/op 0 allocs/op +BenchmarkSmartDoubleQuotes-8 300000 3905 ns/op 6224 B/op 31 allocs/op +BenchmarkReferenceAmps-8 100000 22216 ns/op 15560 B/op 157 allocs/op +BenchmarkReferenceAutoLinks-8 100000 20335 ns/op 14824 B/op 146 allocs/op +BenchmarkReferenceBackslashEscapes-8 20000 69174 ns/op 37392 B/op 316 allocs/op +BenchmarkReferenceBlockquotesWithCodeBlocks-8 200000 8443 ns/op 7968 B/op 48 allocs/op +BenchmarkReferenceCodeBlocks-8 200000 9250 ns/op 9392 B/op 58 allocs/op +BenchmarkReferenceCodeSpans-8 200000 8515 ns/op 8432 B/op 54 allocs/op +BenchmarkReferenceHardWrappedPara-8 200000 5738 ns/op 6856 B/op 34 allocs/op +BenchmarkReferenceHorizontalRules-8 100000 20864 ns/op 13648 B/op 93 allocs/op +BenchmarkReferenceInlineHTMLAdvances-8 200000 6187 ns/op 7310 B/op 40 allocs/op +BenchmarkReferenceInlineHTMLSimple-8 50000 23793 ns/op 16128 B/op 114 allocs/op +BenchmarkReferenceInlineHTMLComments-8 200000 7060 ns/op 7840 B/op 44 allocs/op +BenchmarkReferenceLinksInline-8 100000 18432 ns/op 14496 B/op 153 allocs/op +BenchmarkReferenceLinksReference-8 20000 67666 ns/op 37136 B/op 502 allocs/op +BenchmarkReferenceLinksShortcut-8 100000 19324 ns/op 13984 B/op 162 allocs/op +BenchmarkReferenceLiterQuotesInTitles-8 200000 8998 ns/op 9320 B/op 83 allocs/op +BenchmarkReferenceMarkdownBasics-8 10000 160908 ns/op 88152 B/op 518 allocs/op +BenchmarkReferenceMarkdownSyntax-8 2000 707160 ns/op 303801 B/op 2044 allocs/op +BenchmarkReferenceNestedBlockquotes-8 200000 6740 ns/op 7248 B/op 45 allocs/op +BenchmarkReferenceOrderedAndUnorderedLists-8 10000 115808 ns/op 55052 B/op 626 allocs/op +BenchmarkReferenceStrongAndEm-8 100000 10540 ns/op 9416 B/op 72 allocs/op +BenchmarkReferenceTabs-8 100000 13171 ns/op 10968 B/op 77 allocs/op +BenchmarkReferenceTidyness-8 200000 8903 ns/op 8404 B/op 62 allocs/op +PASS +ok github.com/gomarkdown/markdown 43.477s +``` +It's slower (but opens up possibilities for further improvements). + +After refactoring to make ast.Node a top-level thing. +``` +BenchmarkEscapeHTML-8 2000000 829 ns/op 0 B/op 0 allocs/op +BenchmarkSmartDoubleQuotes-8 300000 3998 ns/op 6192 B/op 31 allocs/op +BenchmarkReferenceAmps-8 50000 27389 ns/op 15480 B/op 153 allocs/op +BenchmarkReferenceAutoLinks-8 50000 23106 ns/op 14656 B/op 137 allocs/op +BenchmarkReferenceBackslashEscapes-8 10000 112435 ns/op 36696 B/op 315 allocs/op +BenchmarkReferenceBlockquotesWithCodeBlocks-8 200000 9227 ns/op 7856 B/op 46 allocs/op +BenchmarkReferenceCodeBlocks-8 200000 10469 ns/op 9248 B/op 54 allocs/op +BenchmarkReferenceCodeSpans-8 200000 10522 ns/op 8368 B/op 54 allocs/op +BenchmarkReferenceHardWrappedPara-8 200000 6354 ns/op 6784 B/op 34 allocs/op +BenchmarkReferenceHorizontalRules-8 50000 32393 ns/op 13952 B/op 87 allocs/op +BenchmarkReferenceInlineHTMLAdvances-8 200000 6894 ns/op 7238 B/op 40 allocs/op +BenchmarkReferenceInlineHTMLSimple-8 50000 32942 ns/op 15864 B/op 110 allocs/op +BenchmarkReferenceInlineHTMLComments-8 200000 8181 ns/op 7776 B/op 44 allocs/op +BenchmarkReferenceLinksInline-8 100000 21679 ns/op 14400 B/op 148 allocs/op +BenchmarkReferenceLinksReference-8 20000 83928 ns/op 36688 B/op 473 allocs/op +BenchmarkReferenceLinksShortcut-8 100000 22053 ns/op 13872 B/op 153 allocs/op +BenchmarkReferenceLiterQuotesInTitles-8 100000 10784 ns/op 9296 B/op 81 allocs/op +BenchmarkReferenceMarkdownBasics-8 5000 237097 ns/op 87760 B/op 480 allocs/op +BenchmarkReferenceMarkdownSyntax-8 1000 1465402 ns/op 300769 B/op 1896 allocs/op +BenchmarkReferenceNestedBlockquotes-8 200000 7461 ns/op 7152 B/op 45 allocs/op +BenchmarkReferenceOrderedAndUnorderedLists-8 5000 212256 ns/op 53724 B/op 553 allocs/op +BenchmarkReferenceStrongAndEm-8 100000 13018 ns/op 9264 B/op 72 allocs/op +BenchmarkReferenceTabs-8 100000 15005 ns/op 10752 B/op 71 allocs/op +BenchmarkReferenceTidyness-8 200000 10308 ns/op 8292 B/op 58 allocs/op +PASS +ok github.com/gomarkdown/markdown 42.176s +``` |