DOM Parsing and Serialization

Living Standard — Last Updated 4 August 2014

This Version:
http://domparsing.spec.whatwg.org/
Participate:
Send feedback to whatwg@whatwg.org (archives) or file a bug (open bugs)
IRC: #whatwg on Freenode
Version History:
https://github.com/whatwg/domparsing/commits
Editor:
Ms2ger <ms2ger@gmail.com>

Abstract

This specification aims to describe DOM APIs related to parsing markup into DOM trees and serializing DOM trees into markup, with a strong focus on compatibility with existing content.

Table of contents

  1. Issues
  2. 1 Conformance
    1. 1.1 Dependencies
    2. 1.2 Extensibility
  3. 2 Terminology
  4. 3 Parsing and serializing Nodes
    1. 3.1 Parsing
    2. 3.2 Serializing
  5. 4 The DOMParser interface
  6. 5 The XMLSerializer interface
  7. 6 Extensions to the Element interface
    1. 6.1 innerHTML
    2. 6.2 outerHTML
    3. 6.3 insertAdjacentHTML()
  8. 7 Extensions to the Text interface
  9. 8 Extensions to the Range interface
  10. References
  11. Acknowledgements

Issues

Various issues are listed in the rest of the document.

This specification currently requires using the XML Parser for some APIs, when in an XML document. It is unclear whether consensus can be found for this approach.

1 Conformance

All diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked non-normative. Everything else in this specification is normative.

The key words "MUST", "MUST NOT", "REQUIRED", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in the normative parts of this document are to be interpreted as described in RFC2119. For readability, these words do not appear in all uppercase letters in this specification. [RFC2119]

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and terminate these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent. (In particular, the algorithms defined in this specification are intended to be easy to follow, and not intended to be performant.)

User agents may impose implementation-specific limits on otherwise unconstrained inputs, e.g. to prevent denial of service attacks, to guard against running out of memory, or to work around platform-specific limitations.

When a method or an attribute is said to call another method or attribute, the user agent must invoke its internal API for that attribute or method so that e.g. the author can't change the behavior by overriding attributes or methods with custom properties or functions in ECMAScript.

Unless otherwise stated, string comparisons are done in a case-sensitive manner.

If an algorithm calls into another algorithm, any exception that is thrown by the latter (unless it is explicitly caught), must cause the former to terminate, and the exception to be propagated up to its caller.

1.1 Dependencies

The IDL fragments in this specification must be interpreted as required for conforming IDL fragments, as described in the Web IDL specification. [WEBIDL]

Some of the terms used in this specification are defined in DOM4 and HTML. [DOM] [HTML]

1.2 Extensibility

Vendor-specific proprietary extensions to this specification are strongly discouraged. Authors must not use such extensions, as doing so reduces interoperability and fragments the user base, allowing only users of specific user agents to access the content in question.

If vendor-specific extensions are needed, the members should be prefixed by vendor-specific strings to prevent clashes with future versions of this specification. Extensions must be defined so that the use of extensions neither contradicts nor causes the non-conformance of functionality defined in the specification.

When vendor-neutral extensions to this specification are needed, either this specification can be updated accordingly, or an extension specification can be written that overrides the requirements in this specification. When someone applying this specification to their activities decides that they will recognise the requirements of such an extension specification, it becomes an applicable specification for the purposes of conformance requirements in this specification.

2 Terminology

The term context object means the object on which the method or attribute being discussed was called.

3 Parsing and serializing Nodes

3.1 Parsing

The following steps form the fragment parsing algorithm, whose arguments are a markup string and a context element.

  1. If the context element's node document is an HTML document: let algorithm be the HTML fragment parsing algorithm.

    If the context element's node document is an XML document: let algorithm be the XML fragment parsing algorithm.

  2. Invoke algorithm with markup as the input, and context element as the context element.

  3. Let new children be the nodes returned.

  4. Let fragment be a new DocumentFragment whose node document is context element's node document.

  5. Append each node in new children to fragment (in order).

    This ensures the node document for the new nodes is correct.

  6. Return fragment.

3.2 Serializing

To serialize a Node node, the user agent must run the following steps:

  1. Let document be node's node document.
  2. If document is an HTML document, return an HTML serialization of node.
  3. Otherwise, document is an XML document. Return an XML serialization of node.

The serialization of a DocumentType node is the string returned by the following steps:

  1. Let name be node's name.
  2. Let public ID be node's public ID.
  3. Let system ID be node's system ID.
  4. Let ids be as follows:
    if both public ID and system ID are the empty string

    the empty string

    if system ID is the empty string

    the concatenation of

    1. "PUBLIC";
    2. " "" (U+0020 SPACE, U+0022 QUOTATION MARK);
    3. public ID;
    4. """ (U+0022 QUOTATION MARK).
    if public ID is the empty string

    the concatenation of

    1. "SYSTEM";
    2. " "" (U+0020 SPACE, U+0022 QUOTATION MARK);
    3. system ID;
    4. """ (U+0022 QUOTATION MARK).
    otherwise

    the concatenation of

    1. "PUBLIC";
    2. " "" (U+0020 SPACE, U+0022 QUOTATION MARK);
    3. public ID;
    4. "" "" (U+0022 QUOTATION MARK, U+0020 SPACE, U+0022 QUOTATION MARK);
    5. system ID;
    6. """ (U+0022 QUOTATION MARK).
  5. Return the concatenation of

    1. "<!DOCTYPE";
    2. " " (U+0020 SPACE);
    3. name;
    4. " " (U+0020 SPACE);
    5. ids;
    6. ">" (U+003E GREATER-THAN SIGN).

No attempt is made to ensure that the result of this algorithm is sensible if the public or system ID contains a """ character.


To produce an HTML serialization of a Node node, the user agent must run the appropriate steps, depending on node's interface:

Element
Document
DocumentFragment

Run the HTML fragment serialization algorithm on node. Return the returned string.

Comment
Text
You tell me.
DocumentType

Return the serialization of node.

ProcessingInstruction
You tell me.

To produce an XML serialization of a Node node, the user agent must run the appropriate steps, depending on node's interface:

Element

Return the concatenation of the following strings:

  1. "<" (U+003C LESS-THAN SIGN);
  2. the value of node's tagName attribute;

    escaping / throwing

  3. the XML serialization of node's attributes;
  4. ">" (U+003E GREATER-THAN SIGN);
  5. the serialization of node's children, in order;
  6. "</" (U+003C LESS-THAN SIGN, U+002F SOLIDUS);
  7. the value of node's tagName attribute;
  8. ">" (U+003E GREATER-THAN SIGN).
Document

Run the XML fragment serialization algorithm on node. Return the string this produced.

Comment

Return the concatenation of

  1. "<!--" (U+003C LESS-THAN SIGN, U+0021 EXCLAMATION MARK, U+002D HYPHEN-MINUS, U+002D HYPHEN-MINUS);
  2. node's data;
  3. "-->" (U+002D HYPHEN-MINUS, U+002D HYPHEN-MINUS, U+003E GREATER-THAN SIGN).
Text
  1. Let data be node's data.
  2. If node has its serialize as CDATA flag set, run the following steps:

    1. If data doesn't match the CData production, throw an InvalidStateError exception and terminate the entire algorithm.
    2. Let markup be the concatenation of "<![CDATA[", data, and "]]>".
  3. Otherwise, run the following steps:

    1. Let markup be data.
    2. Replace any occurrences of "&" in markup by "&amp;".
    3. Replace any occurrences of "<" in markup by "&lt;".
    4. Replace any occurrences of ">" in markup by "&gt;".
  4. Return markup.
DocumentFragment
  1. Let markup be the empty string.
  2. For each child of node, in order, produce an XML serialization of the child and concatenate the result to markup.
  3. Return markup.
DocumentType

Return the serialization of node.

ProcessingInstruction

Return the concatenation of

  1. "<?" (U+003C LESS-THAN SIGN, U+003F QUESTION MARK);
  2. node's target;
  3. " " (U+0020 SPACE);
  4. node's data;
  5. "?>" (U+003F QUESTION MARK, U+003E GREATER-THAN SIGN).

The XML serialization of the attributes of an element element is the result of the following algorithm:

  1. Let result be the empty string.
  2. For each attribute attr in element attributes, in order, append the following strings to result:

    1. " " (U+0020 SPACE);
    2. attr's name;

      escaping / throwing

    3. "="" (U+003D EQUALS SIGN, U+0022 QUOTATION MARK);
    4. attr's value;

      escaping / throwing

    5. """ (U+0022 QUOTATION MARK).
  3. Return result.

4 The DOMParser interface

enum SupportedType {
  "text/html",
  "text/xml",
  "application/xml",
  "application/xhtml+xml",
  "image/svg+xml"
};

[Constructor]
interface DOMParser {
  Document parseFromString(DOMString str, SupportedType type);
};

The DOMParser() constructor must return a new DOMParser object.

The parseFromString(str, type) method must run these steps, depending on type:

"text/html"

Parse str with an HTML parser, and return the newly created document.

The scripting flag must be set to "disabled".

meta elements are not taken into account for the encoding used, as a Unicode stream is passed into the parser.

script elements get marked unexecutable and the contents of noscript get parsed as markup.

"text/xml"
"application/xml"
"application/xhtml+xml"
"image/svg+xml"
  1. Parse str with a namespace-enabled XML parser.

  2. If the previous step didn't return an error, return the newly created document and terminate these steps.

  3. Let document be a newly-created Document.

    The intention is that this document does not support document.load(). It is not clear if it needs to be a XMLDocument for other reasons (such as stringification).

  4. Let root be a new Element, with its local name set to "parsererror" and its namespace set to "http://www.mozilla.org/newlayout/xml/parsererror.xml".

    At this point user agents may append nodes to root, for example to describe the nature of the error.

  5. Append root to document.

  6. Return document.

In any case, the returned document's content type must be the type argument.

It is currently unclear what the URL of the returned document should be.

Results for a test case:

GeckoOperaChrome
document.location null
document.URL unsupported unsupported ""
document.documentURI Page URL null null

Anne van Kesteren suggests using the default, about:blank.

The returned document's encoding is the default, UTF-8.

5 The XMLSerializer interface

[Constructor]
interface XMLSerializer {
  DOMString serializeToString(Node root);
};

The XMLSerializer() constructor must return a new XMLSerializer object.

The serializeToString(root) method must produce an XML serialization of root and return the result.

6 Extensions to the Element interface

partial interface Element {
  [TreatNullAs=EmptyString] attribute DOMString innerHTML;
  [TreatNullAs=EmptyString] attribute DOMString outerHTML;
  void insertAdjacentHTML(DOMString position, DOMString text);
};

6.1 innerHTML

The innerHTML IDL attribute represents the markup of the Element's contents.

element . innerHTML [ = value ]

Returns a fragment of HTML or XML that represents the element's contents.

Can be set, to replace the contents of the element with nodes parsed from the given string.

In the case of an XML document, will throw an InvalidStateError if the Element cannot be serialized to XML, and a SyntaxError if the given string is not well-formed.

On getting, if the context object's node document is an HTML document, then the attribute must return the result of running the HTML fragment serialization algorithm on the context object; otherwise, the context object's node document is an XML document, and the attribute must return the result of running the XML fragment serialization algorithm on the context object instead (this might throw an exception instead of returning a string).

On setting, these steps must be run:

  1. Let fragment be the result of invoking the fragment parsing algorithm with the new value as markup, and the context object as the context element.

  2. Replace all with fragment within the context object.

6.2 outerHTML

The outerHTML IDL attribute represents the markup of the Element and its contents.

element . outerHTML [ = value ]

Returns a fragment of HTML or XML that represents the element and its contents.

Can be set, to replace the element with nodes parsed from the given string.

In the case of an XML document, will throw an InvalidStateError if the element cannot be serialized to XML, and a SyntaxError if the given string is not well-formed.

Throws a NoModificationAllowedError exception if the parent of the element is the Document node.

On getting, if the context object's node document is an HTML document, then the attribute must return the result of running the HTML fragment serialization algorithm on a fictional node whose only child is context object; otherwise, the context object's node document is an XML document, and the attribute must return the result of running the XML fragment serialization algorithm on that fictional node instead (this might throw an exception instead of returning a string).

On setting, the following steps must be run:

  1. Let parent be the context object's parent.

  2. If parent is null, terminate these steps. There would be no way to obtain a reference to the nodes created even if the remaining steps were run.

  3. If parent is a Document, throw a NoModificationAllowedError exception and terminate these steps.

  4. If parent is a DocumentFragment, let parent be a new Element with

  5. Let fragment be the result of invoking the fragment parsing algorithm with the new value as markup, and parent as the context element.

  6. Replace the context object with fragment within the context object's parent.

6.3 insertAdjacentHTML()

element . insertAdjacentHTML(position, text)

Parses the given string text as HTML or XML and inserts the resulting nodes into the tree in the position given by the position argument, as follows:

"beforebegin"
Before the element itself.
"afterbegin"
Just inside the element, before its first child.
"beforeend"
Just inside the element, after its last child.
"afterend"
After the element itself.

Throws a SyntaxError exception if the arguments have invalid values (e.g., in the case of an XML document, if the given string is not well-formed).

Throws a NoModificationAllowedError exception if the given position isn't possible (e.g. inserting elements after the root element of a Document).

The insertAdjacentHTML(position, text) method must run these steps:

  1. Use the first matching item from this list:

    If position is an ASCII case-insensitive match for the string "beforebegin"
    If position is an ASCII case-insensitive match for the string "afterend"

    Let context be the context object's parent.

    If context is null or a document, throw a NoModificationAllowedError and terminate these steps.

    If position is an ASCII case-insensitive match for the string "afterbegin"
    If position is an ASCII case-insensitive match for the string "beforeend"

    Let context be the context object.

    Otherwise

    Throw a SyntaxError exception.

  2. If context is not an Element or the following are all true:

    let context be a new Element with

  3. Let fragment be the result of invoking the fragment parsing algorithm with text as markup, and parent as the context element.

  4. Use the first matching item from this list:

    If position is an ASCII case-insensitive match for the string "beforebegin"

    Insert fragment into the context object's parent before the context object.

    If position is an ASCII case-insensitive match for the string "afterbegin"

    Insert fragment into the context object before its first child.

    If position is an ASCII case-insensitive match for the string "beforeend"

    Append fragment to the context object.

    If position is an ASCII case-insensitive match for the string "afterend"

    Insert fragment into the context object's parent before the context object's next sibling.

7 Extensions to the Text interface

partial interface Text {
  attribute boolean serializeAsCDATA;
};
text . serializeAsCDATA [ = value ]
Controls whether, in XML, this node is serialized as a CDATA section.

Text nodes have an additional associated flag, the serialize as CDATA flag.

The serializeAsCDATA attribute must return true if the context object has its serialize as CDATA flag set, or false otherwise.

Setting the serializeAsCDATA attribute must, if the new value is true, set the context object's serialize as CDATA flag, or unset it otherwise.

8 Extensions to the Range interface

partial interface Range {
  DocumentFragment createContextualFragment(DOMString fragment);
};
fragment = range . createContextualFragment(fragment)

Returns a DocumentFragment, created from the markup string given.

The createContextualFragment(fragment) method must run these steps:

  1. Let node be the context object's start node.

    Let element be as follows, depending on node's interface:

    Document
    DocumentFragment
    null
    Element
    node
    Text
    Comment
    node's parent element
    DocumentType
    ProcessingInstruction
    DOM4 prevents this case. [DOM]
  2. If either element is null or the following are all true:

    let element be a new element with

  3. Let fragment node be the result of invoking the fragment parsing algorithm with fragment as markup, and element as the context element.

  4. For each script in fragment node, unset the "parser-inserted" and "already started" flags.

    This step is intended to be equivalent to not setting those flags in the first place, and to ensure that scripts are run when fragment node is inserted into a document.

  5. Return fragment node.

References

All references are normative unless marked "Non-normative".

[DOM]
DOM, Anne van Kesteren, Aryeh Gregor and Ms2ger. WHATWG.
[HTML]
HTML, Ian Hickson. WHATWG.
[RFC2119]
Key words for use in RFCs to Indicate Requirement Levels, Scott Bradner. IETF.
[WEBIDL]
Web IDL, Cameron McCormack. W3C.

Acknowledgements

Thanks to Anne van Kesteren, Aryeh Gregor, Boris Zbarsky, David Håsäther, Henri Sivonen, Ryosuke Niwa, Simon Pieters, timeless and Travis Leithead for their useful comments.

Special thanks to Ian Hickson for defining the innerHTML and outerHTML attributes, and the insertAdjacentHTML() method in HTML and his useful comments. [HTML]