sending a http request and evaluating the response as browser does in a chrome extension

[jack]

hi,

i want to send a http request in a chrome extension using fetch api http://stackoverflow.com/questions/9028234/is-there-a-way-to-not-send-cookies-when-making-an-xmlhttprequest-on-the-same-ori/30050285#30050285 and then, get the responseText.

For example, if I fetch the url "http://www.nytimes.com", i would get the response from nytimes as responseText. after receiving the responseText, browser starts loading this page. To load the page, browser loads all css, image etc. files and also, evaluates the javascript code and  makes xmlhttprequests as well if asked by the scripts and so on. after all this is done, the browser goes to "complete" state from "loading". now the text of the page is different than the what is originally downloaded as responseText.

I want an api that can do this programmatically for me in chrome extension, i.e., i pass the responseText and it gives me back the content of the page after doing what browser does to a response received for a main_frame request. Or if not possible programmatically, what other ways exist to do this task in a chrome extension.

I hope my question is clear. thanks a lot in advance.

[PhistucK]

You can create an iFrame within your background page (and inject content script into it), but it will also send the current cookies and everything.

I do not think there are other ways within a Chrome extension (unlike Chrome applications).

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "Chromium-extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To post to this group, send email to chromium-...@chromium.org.
Visit this group at http://groups.google.com/a/chromium.org/group/chromium-extensions/.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/1645e299-6732-4b2e-b7fb-d3c1138aa795%40chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

[jagdish achara]

Thanks PhistucK!

What about opting for opening a new Incognito window behind the main window and a new tab in that window. and then, load the page there? 

Yes, doing it in an iFrame in the background page would mean that it is not isolated or in private mode. This is the main reason to use fetch api in the first place...so you are right. that solution is not good for this purpose.

[PhistucK]

The user may spot that something happens (another taskbar button, a glitch, a windows that shows up for a fraction of a second). And the user may close it when they spot it (or ask themselves why). Also, if the user happened to have browsed (in incognito mode) the website you are trying to load already, the cookies of that visit will not be deleted (or if they visited another website, that share third party resources with your website, and those resources set third party cookies, you will get them as well) until all of the incognito tabs are closed.

I am not sure there is a good advice here.

However, why do you need to run scripts and such? I see in StackOverflow that you want the RSS feed, which does not include scripts, styles or anything else.

☆PhistucK

[jagdish achara]

This is basically to parse the page when the loading is complete and try to find out the difference in the contents of the page (for example, Ads received) when the user is tracked through cookies and when he is not, i.e., without any cookies.

[PhistucK]

By the way, did you check the recently announced GMail API?

https://developers.google.com/gmail/api/

☆PhistucK

[jagdish achara]

I did not know about that. Thanks for letting me know.

[jagdish achara]

I was thinking of what PhistucK proposed, i.e., creating iFrame within the background page. But as PhistucK pointed out, it would also send the current cookies and everything.  Therefore, I came up with one possible solution to avoid sending cookies.

What we can do is to capture the requests in onBeforeRequestHeaders() and delete the cookie of the requests with tabId = -1. This would delete the cookies of all the requests sent by the browser which are not related to a tab. However, it comes with a cost: http requests sent by other extension and the browser itself would also be captured here and we would not be able to distinguish only the requests that came from my extension. So I would probably end up deleting cookies for http requests from other extensions and browser http requests like when opening a new tab or browser update or something.

What do you guys think of this solution? I think it's better than opening a new Incognito window behind current window and trying to hid that. 

On Wed, Aug 26, 2015 at 6:29 PM, PhistucK <phis...@gmail.com> wrote:

[PhistucK]

I understand the GMail API is not suitable for you, then, right?

☆PhistucK

[jagdish achara]

Oups...now I understand this is why you sent me the link to the Gmail api. 

As far as what I understood form it, it is to fetch/search emails. no?

[PhistucK]

I believe so. What else do you need?

☆PhistucK

[jagdish achara]

I thought my question is clear.

I re-iterate it: In my chrome extension, I want to make a http request to a webpage "xyz.com" and then, load the http response thus received as the browser does. This should be done without any cookies. Something like when we open a new tab in Incognito mode and then, visit "xyz.com". So the request to xyz.com is sent without any cookie and then, all the subsequent requests (resulting from the loading of the page like downloading image, css file and xmlhttprequests made from the execution of the included javascript code) are also made without any cookies. 

[PhistucK]

This question is clear, but in the StackOverflow, you mentioned the GMail feed as you goal. Then you mentioned (within the thread) that you also want to get rid of the ads. So I suggested the GMail API and you still seem to want something else.

Perhaps I am a bit overwhelmed since you have multiple directions and goals and I happen to be combining them even though they are completely separate and unrelated.

☆PhistucK

[jagdish achara]

OK. BTW, what do u think about the solution I proposed based on your and rob's suggestion of using iframes for this in the extension.

[PhistucK]

​It sounds like a pretty big hammer.

☆PhistucK

[jagdish achara]

In what sense? Performance? Usability?

[PhistucK]

Both of them. Performance, yes, but usability as well - you can ruin the request and the user will get degraded experiences in some aspect.

☆PhistucK

[jagdish achara]

Yeah, right. But I don't see a way to distinguish the requests coming from my extension. Do you think there is a way out of it?

[Antony Sargent]

FYI, one API we do have which comes at least somewhat close to what you're requesting is the <webview> tag for packaged apps. A packaged app is a little different than an extension and can't use all the same APIs as extensions can, but this might serve your needs or could be a companion to your extension (packaged apps and extensions can communicate with each other using messaging). 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAKnO5RsesEj%2BmHtLH36fBzfNUixVzVX-VvKtL4nR6Vc4p7HL7w%40mail.gmail.com.

[jagdish achara]

That's a great suggestion, Antony!

Is there a way I can put the app as a pre-requisite for installation of my extension, i.e., the user would automatically be prompted to install the app with my extension. 

Also, the app has to be started/opened when the extension wants to communicate with it or it is enough to have it installed.

Thanks again.

[PhistucK]

I see that Chrome applications can have background pages as well, so it looks like it does not have to be opened by the user first, only installed.

☆PhistucK

[Antony Sargent]

On Sat, Aug 29, 2015 at 9:52 AM, jagdish achara <jagdish...@gmail.com> wrote:

That's a great suggestion, Antony!

Is there a way I can put the app as a pre-requisite for installation of my extension, i.e., the user would automatically be prompted to install the app with my extension. 

There isn't any way to express one being a prerequisite for the other ; instead you can use messaging from one to try and contact the other and if that fails then put some UI explaining things to the user with a link to the entry in the webstore so they can choose to install it if they wish. If you do this make sure you understand the program policies and single purpose policy though as outlined here:

https://developer.chrome.com/webstore/program_policies

https://developer.chrome.com/extensions/single_purpose

(Probably the most important thing to keep in mind is that if your extension has useful functionality apart from the app, then you should not make installation of the app required, but rather optional). 

 

Also, the app has to be started/opened when the extension wants to communicate with it or it is enough to have it installed.

It's enough to have it installed, and the app should make sure to register a listener for the runtime.onConnectExternal event so its event page can be woken up when appropriate (for security purposes, in your listener when you get a connection make sure you check the sender.id field of the port to make sure you're talking to the right extension and not some other unrelated one trying to trick your app).