Write Capture 2

[Noah Sloan]

Good News Everyone!

I'm happy to announce the beta release of Write Capture 2. It's a
complete rewrite of Write Capture that simulates document.write rather
than capturing and post processing the output. While it's impossible
to replicate 100% of document.write's quirks, I think it should
eventually be more compatible with scripts than the current Write
Capture.

Please check it out here and report any bugs you find:

https://github.com/iamnoah/writeCapture/tree/writeCapture2

You'll also need to include element.write.js, which can be found in
the lib directory.

Advantages:

1. DOM elements are generally available immediately. That means no more hacks!
2. No jQuery or nolib support required!

Differences (may be good or bad):

1. The API is very different:
elementWrite.toElement(document.getElementById('id')).write(html).close();
// very important that you call close
2. All script tags now load and execute asynchronously. That will
generally mean a more responsive UI.

Known Edge Case:

1. document.written script tags will execute asynchronously, so they
wont be immediately available to the writing script. This sort of
dependency is rather perverse and should be rare (I'm not even sure it
would work cross browser). Unfortunately, it's impossible to block a
script being executed while we load and execute another script, so if
you hit this limitation, you'll have to work with your ad server or
use an iframe.

Cheers,
Noah

[Eric Hochberger]

Hey Noah,

writeCapture2.js seems to fix some huge bugs for me with crummy broken ads!  Is there anyway to use jQuery selectors with it?  I hate the idea of using document.getElementById...

Thank you for this amazing library!

[noah]

Sure, just unwrap the jQuery object:

elementWrite.toElement($('.whatever')[0]).write(html).close();

[Eric Hochberger]

Hmm, well, yeah, I guess there's that :) Any shot you'll be releasing it as a plugin so I can get callback functionality? As in if I need one script to run first followed by another, how could I do that in current system?

Also, would you mind if I sent you an ad tag that's been giving me hell?

<script type="text/javascript" language="javascript" src="http://www35.glam.com/gad/glamadapt_mobile.act?;sz=320x50;tt=m;afid=367631667;ord='+Math.random()*10000000000000000+';tile=1;_glto='+ new Date().getTimezoneOffset() + ';_g_cv=3;"></script>

Some times it writes out some crazy tracking code that seems to escape writeCapture2.js


Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

[noah]

The 2nd arg is a callback:

writeCapture($('.whatever')[0],function() {
// done now, so you can load another script
}).write(html).close(); // don't forget to close

If you can make a repeatable jsfiddle/jsbin, I'll take a look. You'll
need to hardcode a random value that causes the problem.

[noah]

Oh, and I just noticed that in the previous examples, I wrote
elementWrite.toElement instead of writeCapture. That's not going to
capture document.writes, so I hope no one was confused.

[Eric Hochberger]

Does this fiddle work for you?

http://jsbin.com/agekom/3

It runs fine for my non-mobile ad tags. It's just this one that's an issue.

Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

[noah]

Your fiddle has this in it: Math.random()*10000000000000000; so it's
not giving me consistent results. I reloaded several times and didn't
see an issue. Please hard code a value that will consistently
reproduce the issue.

[Vincent Voyer]

Hello, great this project is still alive.

How would you link this to controljs? I saw you had a controljs plugin, is this still possible?

thanks

[Vincent Voyer]

One more question : would it be possible to capture document.write globally and have it automatically use writeCapture ?

[noah]

You can use it with control JS via the plugin, but since control JS is
not actively developed, I'd use a different script loader.

[noah]

For the original write capture, autoInject does what you're describing.

[Vincent Voyer]

I wanna use writeCapture2 of course so Ill try to include the autoInject from v1, what do you think ?

About controlJS and another script loader, do you have one in mind or are you using one? Labjs could be a good candidate right?

Im planning on using writeCapture as part of my work at http://fasterize.com, my main concern is lazyloading and I did a module for images https://github.com/fasterize/lazyload

It will be a generic async script loading hosted on https://github.com/vvo/lazyjs 

So I will help in debugging and developing (if my code fits your needs) for sure.

[Eric Hochberger]

Hey Noah,

That's just a cache buster and does not actually determine which script runs. If I do enough refreshes eventually I get one.

The other issue is I don't think it's ever loading an ad. When I view the source it looks like it should be running a least a placeholder ad.

I've cleaned up the code a bit here, including removing the cache buster, but still no ad:

http://jsbin.com/olifut/2

If I run the javascript directly:

http://jsbin.com/olifut/3

Ad.

Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

[noah]

On first, look, I'm guessing the issue is the comment in the resulting
script <!--//. I'll need some time to look into it further.

[noah]

I prefer AMD loaders like requireJS myself. It's generally better to
build all your JS into one file for production than load a lot of
files in parallel, and an AMD loader lets you keep dependencies
separate and manages the order for you.

[noah]

OK, there was definitely an issue with comments in scripts that is now
fixed. I see ads some of the time. Other times your ad server doesn't
appear to be returning anything (I made requests for the same script
just to be sure).

[Eric Hochberger]

Holy crap! That fixed it. Working perfectly!

Once I get it fully running on all my sites I'll let you know if I run into any more issues. I'd don't even think I can express how much I appreciate this library and that fix.

Your script is a godsend with the archaic slow ad servers out there. Thank you!

Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

[Derek Brans]

Hey Naoh,
I just came across your project and was impressed with the elegance of writeCapture1.
I've been writing an parser+DOM writer over the past few months. There are a lot of complexities to this approach. To name a couple:

Parsing
ins elements can be both inline or block. However, you don't want them to close when they contain a block element, and you don’t want them to close other inline elements. http://jsfiddle.net/dbrans/fw98N/5/

DOM manipulation
Setting attributes is not easy. ie doesn’t let you set the name attribute dynamically and there are exceptions like style.cssText. jQuery’s attributes.js has some other examples.

InnerHTML skirts these issues and others by having the browser do the parsing and node creation for you.

Did you move to a parser+DOM approach to fix the partial-tag problem you described in the README? What other reasons made you abandon innerHTML?

Cheers,
Derek

[noah]

Hi Derek,

While it's true that innerHTML is more reliable in terms of parsing,
the natural behavior of document.write is to have the elements in the
DOM immediately. While you're right that some unusual cases will parse
differently using all the rules the browser has, for the vast majority
of the document.write use cases (mostly ads) you're not going to
encounter an ins tag or something unusual like that, but you will
encounter scripts that write out a div or iframe or object or
something and expect it to be immediately available. So writeCapture2
may be less reliable in terms of nesting (so may mess up rendering
slightly), but is less likely to fail completely.

That said, I'd gladly take a pull request for more robust parsing
(with proper tests of course).

[Derek Brans]

Hi Noah, 

Do these examples get to the heart of innerHTML's weaknesses?

<script>

document.write('<div id="a" >');

var el = document.getElementById('a');

// do something with el.

document.write('<p>hello</p></div>');

</script>

or worse (with the '<p' cutt off):

<script>

document.write('<div id="a" ><p');

var el = document.getElementById('a');

// do something with el.

document.write('>hello</p></div>');

</script>

[noah]

That's the basic idea, but scripts do all sorts of weird things.
They'll set attributes, they'll create an iframe and write that, try
to get the parent element, etc.

[Eric Hochberger]

Hey Noah,

Not sure if you have time to look again, but I want to launch my site with writeCapture2 this week but have been running into a problem. When I make a bunch of simultaneous calls to writeCapture, my ads seem to pop up in random places.

Example:

http://jsbin.com/agekom/4/

Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

On May 29, 2012, at 9:45 PM, noah wrote:

[noah]

Yea, managing ordering in WC2 is up to you. The 2nd argument is a
callback, so you can use that to chain them together. Or you can write
them all as one call if it's a single block.

[noah]

Sorry that wasn't clear. Multiple calls at the same time will conflict
with each other. There can only be one document.write at a time, and
the browser is free to intersperse the scripts in whatever order it
pleases.

[Eric Hochberger]

Perfectly clear now. I will take advantage of callback to fix. Thanks!

Sent from my iPhone. Forgive all typos.

[Eric Hochberger]

And fixed! If anyone is curious I ended up using a single function to take care of all writeCaptures w/ a ghetto mutex variable. I even threw in a little setTimeout of 100 ms during callback to keep things safe.

Thanks for the wonderful library and amazing tech support!

Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

[Vincent Voyer]

Hello, could you share your code?

On https://gist.github.com/ for example or directly by email.

thank you.

--
Vincent Voyer

06 13 92 69 96

[noah]

If you use jQuery, here's a simple plugin:

https://gist.github.com/3176323

(I haven't tested this yet, but that's the general idea)

[Micky Hulse]

Hello,

On Wed, Jul 25, 2012 at 6:51 AM, Vincent Voyer <Vincen...@gmail.com> wrote:
> Hello, could you share your code?

I would also be interested in seeing your code.

The jQuery version that noah posted is perfect, but I'd like to see a
non-jQuery version.

I could take the time to reverse engineer the jQuery example, but if
you feel like sharing, I'd love to not reinvent a wheel on this one.
:D

Thanks!
M

[Eric Hochberger]

My code is quite ghetto, untested (but running live), and somewhat jQuery dependent. It's of the format:

https://gist.github.com/3189712

Obviously it can be very DRY'd up and no clue how much the Timeouts are needed. But I very very much needed to make sure my tags loaded in the proper order based on some shit my ad network does.

Hopefully this helps someone, since this code rescued my project. Have I said thank you a nauseating number of times yet, Noah?

Sincerely,

Eric Hochberger
er...@mediavine.com
@EricTVF

Office: (954) 519-2181 x 191

[Micky Hulse]

On Fri, Jul 27, 2012 at 11:44 AM, Eric Hochberger <er...@mediavine.com> wrote:
> https://gist.github.com/3189712

Awesome! It's cool to see your code, thank you for sharing!

Also, "mutex" is a new term to me... I'll have to look it up. Learning
something new every day. :D

> Hopefully this helps someone, since this code rescued my project. Have I said thank you a nauseating number of times yet, Noah?

2000% agree! Noah, you are a ROCK STAR!

[OT]

Btw, my latest tests show that IEs and Firefox 3.6 are working!!!!!
Not sure what you did, but I am ecstatic! :)

I'm going to post my findings in the other thread I started.

[/OT]

Cheers,
Micky

[Debugger]

Hello,

I am using wc2 , its working fine everywhere except IE(7,8).

http://jsfiddle.net/ruHNf/2/ , in its callback I am moving my content inside other div but in IE its not working.Need a solution.

[Noah Sloan]

This line:

document.getElementById('test1').innerHTML =
document.getElementById('test').innerHTML;

Is not a good idea. You're copying script tags that were written out
and may end up being executed again. You should write the ad out where
you want it. If you need it several places, you need to write it
multiple times.

Make sure you clear IE's browser cache. There was a bug until a week
ago that broke the callback in IE8-.

[Ulhas Shringare]

If instead of  "document.getElementById('test1').innerHTML = document.getElementById('test').innerHTML;" , i directly write ad in same div wc2 appends it with previous ad.

Is there any option to remove previous ad content and then overwrite it with new ad content?

--
Regards,
Ulhas Shringare

Contact - 9821553636
Email : ul...@gigfrix.com / shringar...@gmail.com

[noah]

Nope. You'll have to remove the old content yourself.

[Ulhas Shringare]

Now I saw real problem that in IE(7/8) callback for wc2 is not working.

http://jsfiddle.net/ruHNf/3/ , this is my fiddle.

[noah]

That's interesting. By which I mean really really stupid. Your ad
script is writing out noscript tags... which is completely pointless.
Apparently IE doesn't like it when you try to do things with noscript
tags in JavaScript. I can't say I blame it.

Anyhow, I added some checks in WC2 for noscripts so clear your browser
cache and try it again.

On Fri, Aug 3, 2012 at 2:40 AM, Ulhas Shringare

[agdm]

=================
$('test').html('');
=================

[Dan Silk]

Hi Noah, 

Your writeCapture API looks promising.  I worked on a similar project where we tried to capture ad code before it could change the DOM.  I was able to capture the document.write statement sure enough, but subsequent SCRIPT tags were loaded by the ad code that used document.createElement to place the ad markup on the page.  I was unable to get around this issue so I'm wondering if writeCapture can handle this as well.

Thanks,

-Dan