Hacking a HTML renderer in plain browser-side JS

Just because

Christoph Burgmer

@cburgmer, cburgmer@thoughtworks.com

JSConf 2014

What are we up to?

Misuse new browser features
Use our JS fu to manipulate SVG, HTML, CSS, DOM and CSSOM
Hopefully have some fun on the way

Our task

Trigger HTML rendering capabilities from JavaScript
- Get a screenshot from a web page
- Render HTML on to a canvas

Provide the missing (i.e. a "prollyfill")

var ctx = canvas.getContext("2d");
ctx.drawHTML(htmlString, 0, 0);

Sounds easy?

Quick solution

In Firefox

var canvas = document.getElementById('mycanvas');
var ctx = canvas.getContext("2d");
ctx.drawWindow(content, 0, 0, w, h, "rgb(0,0,0)");

That was easy

But :(

Only works with "Chrome privileges" and only in Firefox
Not executable in a web page
- Security concerns
- Would allow any script to read any content
So much for that

SVG to the rescue

<foreignObject> allows to embed foreign content, including (X)HTML

<svg xmlns="http://www.w3.org/2000/svg">
    <foreignObject width="100%" height="100%">
        <html>
            <strong>
                This is an image containing HTML
            </strong>
        </html>
    </foreignObject>
</svg>

Can we build on that?

XHTML

SVG is XML and only allows XML in <foreignObject>

Firefox

var doc = document.implementation.createHTMLDocument("");
doc.documentElement.innerHTML = ourHTML;
doc.documentElement.setAttribute("xmlns",
    doc.documentElement.namespaceURI);
var xhtml = (new XMLSerializer())
    .serializeToString(doc.documentElement);

Need "shim" for Chrome & Safari :(

Inline data

The future

var blob = new Blob(["<svg xmlns=...>"],
        {"type": "image/svg+xml"});
return URL.createObjectURL(blob);

The present: data URIs

<img src="data:image/svg+xml;charset=utf-8,<svg xmlns='http://www..."/>

Bringing this together

Now we can render this

var image = new Image();
image.src = "data:image/svg+xml;charset=utf-8," +
    encodeURIComponent("<svg xmlns='http://www...'");
canvas.getContext("2d").drawImage(image, 0, 0);

Congratulations, we are done!

We embeded HTML as XHTML using <foreignObject> into an SVG
... which we converted into a data URI to yield an <img>
... which we drew with the canvas' drawImage().

All good?

SVG + external resources

SVGs are not allowed to include external references.

HTML allows

<img> and <input type="image">
<link rel="stylesheet" href="...">
div { background-img: "url(...)"}
@import url(...)
@font face { src: url(...)}

We could try embed them.

Finding external elements (1)

<img> and <input type="image"> are easy

var doc = document.implementation.createHTMLDocument("");
doc.documentElement.innerHTML = "<html><img src="..."/></html>";
var images = doc.getElementsByTagName("img");

images.forEach(function (image) {
    loadAndEmbedImageSrc(image.src);
});

Embedding <img>

<img src="externalImage.png"/>

Replace image src with data URI

<img src="data:image/png;base64,iVBORw0KGgo...">

Get the image content via AJAX

var xhr = new XMLHttpRequest();
xhr.addEventListener("load", function () {
    image.src = "data:image/png;base64,"
        + btoa(xhr.response); // base64
});
xhr.open('GET', "externalImage.png");
xhr.send();

binary AJAX

AJAX was designed for XML, i. e. text
Not designed for binary
XMLHttpRequest2 knows about binary blobs, missing in Safari

Workaround:

xhr.addEventListener("load", function () {
    var content = xhr.response,
        binaryContent = "";

    for (var i = 0; i < content.length; i++) {
        binaryContent += String.fromCharCode(
            content.charCodeAt(i) & 0xFF);
    }
});

Finding external elements (2)

How do we embed background-image?

Working on CSS resources needs parsing CSS

var rules = parseCss("span {" +
    "background-image: url('externalImage.png');" +
"}");

Our minimal CSS parser

The browser implements a full-blown CSS parser

We can access it through the CSSOM

function parseCss(content) {
    var styleElement = document.createElement("style");

    styleElement.textContent = content;
    // the style will only be parsed once
    //   it is added to a document
    document.implementation.createHTMLDocument("")
        .body.appendChild(styleElement);
    return styleElement.sheet.cssRules;
}

Putting the CSS stuff together

We now have our rules

var rules = parseCss("span {" +
    "background-image: url('externalImage.png');" +
"}");

Then iterate over rules

rules.forEach(function (rule) {
    var backgroundImage = rule.style.getPropertyValue(
        'background-image');
    if (backgroundImage) {
        doInlineImageUrl(backgroundImage);
    }
});

Finally same stuff as for our <img> before

Finding external elements (3)

Next up is embedding <link rel="stylesheet" href="sheetUrl">
Simple at first
- Load the sheetUrl via AJAX
- Then replace <link> with <style>PUT_CONTENT_HERE</style>
However: relative links inside the stylesheet will break

relative URLs

Relative resource URLs are relative to the sheet's URL
Example ../images/sprite.png in assets/style/default.css becomes assets/images/sprite.png after embedding

Using the URL parser from Node.js

fontFaceRules.forEach(function (rule) {
    var url = getFontFaceSrc(rule),
        adaptedUrl = url.resolve(sheetUrl, url);

    updateBackgroundImageSrc(rule, adaptedUrl);
});

There you go

We rendered HTML through SVG's <foreignObject>
and inlined all external resources
1. loading binary content through AJAX,
2. embedding images and fonts through data URIs,
3. parsing CSS for style resources, and
4. adapting relative paths

Demo time - click me

Animation from http://craftymind.com/factory/html5video/CanvasVideo.html

Issues

same-origin policy (work around with CORS)
Re-reading from a canvas not allowed (Safari & Chrome)
Waiting for <foreignObject> to make it into IE (in development)
Form inputs don't render
Performance (Y u no DOM in web worker?!)

Incomplete list of workarounds

Draw HTML through SVG
Convert HTML to XHTML (corner cases exist)
Spider & inline externals
Execute JS in temporary iframe
XHR proxy object to fix up urls
XHR proxy object to notify on finished requests
Blob fix for old Webkits
First style rule ignored in Safari
Background images not always showing

Binary AJAX
Gather viewport size from iframe
DOMParser not parsing HTML
Detect parse errors for XHR with Document
Detect MIME type
Escape closing script tags (buggy)
Hashing through JSON.stringify
background-image url() serialization
CSSFontFaceRule.setCssText()

Outstanding browser issues

Make canvas drawWindow Web-accessible :) (https://bugzilla.mozilla.org/show_bug.cgi?id=329693)
xhr2 responseType='document' fails on local pages (https://bugzilla.mozilla.org/show_bug.cgi?id=942138)
History API throws nasty exception when used within an iframe (with no src) (https://bugzilla.mozilla.org/show_bug.cgi?id=777526)
Canvas is tainted after drawing SVG including foreignObject (https://code.google.com/p/chromium/issues/detail?id=294129 and https://bugs.webkit.org/show_bug.cgi?id=17352)
CSSOM backgroundImage incorrectly returns empty url() when created on-the-fly (http://code.google.com/p/chromium/issues/detail?id=161644)
Support mutation of CSSFontFaceRule DOM objects from JavaScript (https://bugzilla.mozilla.org/show_bug.cgi?id=443978)
context.drawImage() on canvas.toDataURL("image/png") should be idempotent (https://bugzilla.mozilla.org/show_bug.cgi?id=790468)
Webkit fails to render SVGs in HTML inside foreignObject (https://github.com/cburgmer/rasterizeHTML.js/issues/39)
foreignObject not supported (http://status.modern.ie/svgforeignobjectelement)

What is this all about?

All this went into rasterizeHTML.js (and inlineresources and xmlserializer and css-font-face-src)
248 ppl on Github think it's useful, but for what??!
My own use case: underlying technique for CSS testing tool: CSS Critic

Thanks for listening

Oh, and go make something cool with it.