How to use chrome extension to communicate with local application?

[Zhan Benjamin]

Hello, I have a quest that i need to crawl some of information from a website.

The problem is I need to sign up to access the pages with information i need and the account is very precious. So I can not use pyspider or other ways because it is easy be banned.

Thankfully, there are not much pages need be crwaled. So I think how about I download the html files (not page source code but codes get from chrome inspect elements function) by chrome extensions and send it to the local application which can analysis the html file to get the information I needed. 

So I try write a chrome extension simply with code:

chrome.tabs.executeScript({

    code: 'document.document.body.innerHTML'

  }


and I dont know how to send the string in this ver to my local application. I totally new to javascript or web development. So can some one tell me how to find a easy way to use chrome extension communicate with local application? Or can someone think a better way to reach my goal?

[PhistucK]

There are a few options here.
1. Create a local web server that will receive that string via HTTP requests, like a POST (or whatever, but not GET, since the content may be huge) request. From your extension, use fetch or XMLHttpRequest to send a request to your local server. The server will communicate with (or be part of) the local application.

2. Use native messaging. That also requires a separate application that gets messages as input and does whatever.

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "Chromium-Extensions-Announce" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsubscribe...@chromium.org.
To post to this group, send email to chromium-extensions@chromium.org.
Visit this group at https://groups.google.com/a/chromium.org/group/chromium-extensions/.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/0938a786-21ec-45ff-834c-e723b55712b9%40chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

[Scott Mathews]

[Zhan Benjamin]

PhistucK,

thanks for your advice, communication via http is a good idea.

So I used Python to build a local web sever and use XMLHttpRequest in the chrome extension to send data. It works well in http page, but in the https page, I get a error log:

Mixed Content: The page at 'https://www.baidu.com/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://localhost:8000/'. This request has been blocked; the content must be served over HTTPS.

Obviously, https page dont allow me transfer data to my http web sever. How can I fix it?

在 2017年3月30日星期四 UTC+8下午3:54:09，PhistucK写道：

There are a few options here.
1. Create a local web server that will receive that string via HTTP requests, like a POST (or whatever, but not GET, since the content may be huge) request. From your extension, use fetch or XMLHttpRequest to send a request to your local server. The server will communicate with (or be part of) the local application.

2. Use native messaging. That also requires a separate application that gets messages as input and does whatever.

☆PhistucK

On Thu, Mar 30, 2017 at 10:19 AM, Zhan Benjamin <gnbenja...@gmail.com> wrote:

Hello, I have a quest that i need to crawl some of information from a website.

The problem is I need to sign up to access the pages with information i need and the account is very precious. So I can not use pyspider or other ways because it is easy be banned.

Thankfully, there are not much pages need be crwaled. So I think how about I download the html files (not page source code but codes get from chrome inspect elements function) by chrome extensions and send it to the local application which can analysis the html file to get the information I needed. 

So I try write a chrome extension simply with code:

chrome.tabs.executeScript({

    code: 'document.document.body.innerHTML'

  }


and I dont know how to send the string in this ver to my local application. I totally new to javascript or web development. So can some one tell me how to find a easy way to use chrome extension communicate with local application? Or can someone think a better way to reach my goal?

--
You received this message because you are subscribed to the Google Groups "Chromium-Extensions-Announce" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.
To post to this group, send email to chromium-...@chromium.org.

[PhistucK]

Did you add "http://localhost" to the "permissions" key in the manifest of your extension?

☆PhistucK

On Thu, Mar 30, 2017 at 1:15 PM, Zhan Benjamin <gnbenja...@gmail.com> wrote:

PhistucK,

thanks for your advice, communication via http is a good idea.

So I used Python to build a local web sever and use XMLHttpRequest in the chrome extension to send data. It works well in http page, but in the https page, I get a error log:

Mixed Content: The page at 'https://www.baidu.com/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://localhost:8000/'. This request has been blocked; the content must be served over HTTPS.

Obviously, https page dont allow me transfer data to my http web sever. How can I fix it?

在 2017年3月30日星期四 UTC+8下午3:54:09，PhistucK写道：

There are a few options here.
1. Create a local web server that will receive that string via HTTP requests, like a POST (or whatever, but not GET, since the content may be huge) request. From your extension, use fetch or XMLHttpRequest to send a request to your local server. The server will communicate with (or be part of) the local application.

2. Use native messaging. That also requires a separate application that gets messages as input and does whatever.

☆PhistucK

On Thu, Mar 30, 2017 at 10:19 AM, Zhan Benjamin <gnbenja...@gmail.com> wrote:

Hello, I have a quest that i need to crawl some of information from a website.

The problem is I need to sign up to access the pages with information i need and the account is very precious. So I can not use pyspider or other ways because it is easy be banned.

Thankfully, there are not much pages need be crwaled. So I think how about I download the html files (not page source code but codes get from chrome inspect elements function) by chrome extensions and send it to the local application which can analysis the html file to get the information I needed. 

So I try write a chrome extension simply with code:

chrome.tabs.executeScript({

    code: 'document.document.body.innerHTML'

  }


and I dont know how to send the string in this ver to my local application. I totally new to javascript or web development. So can some one tell me how to find a easy way to use chrome extension communicate with local application? Or can someone think a better way to reach my goal?

--
You received this message because you are subscribed to the Google Groups "Chromium-Extensions-Announce" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsubscribe...@chromium.org.

To post to this group, send email to chromium-...@chromium.org.
Visit this group at https://groups.google.com/a/chromium.org/group/chromium-extensions/.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/0938a786-21ec-45ff-834c-e723b55712b9%40chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

--
You received this message because you are subscribed to the Google Groups "Chromium-Extensions-Announce" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To post to this group, send email to chromium-extensions@chromium.org.

Visit this group at https://groups.google.com/a/chromium.org/group/chromium-extensions/.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/0419eda7-13cb-45ea-87b8-0a4f1b90565a%40chromium.org.

[Zhan Benjamin]

yes，It works well in http page。I think https page dont allow javascript communication with http sever. 

在 2017年3月30日星期四 UTC+8下午7:29:27，PhistucK写道：

Did you add "http://localhost" to the "permissions" key in the manifest of your extension?

☆PhistucK

On Thu, Mar 30, 2017 at 1:15 PM, Zhan Benjamin <gnbenja...@gmail.com> wrote:

PhistucK,

thanks for your advice, communication via http is a good idea.

So I used Python to build a local web sever and use XMLHttpRequest in the chrome extension to send data. It works well in http page, but in the https page, I get a error log:

Mixed Content: The page at 'https://www.baidu.com/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://localhost:8000/'. This request has been blocked; the content must be served over HTTPS.

Obviously, https page dont allow me transfer data to my http web sever. How can I fix it?

在 2017年3月30日星期四 UTC+8下午3:54:09，PhistucK写道：

There are a few options here.
1. Create a local web server that will receive that string via HTTP requests, like a POST (or whatever, but not GET, since the content may be huge) request. From your extension, use fetch or XMLHttpRequest to send a request to your local server. The server will communicate with (or be part of) the local application.

2. Use native messaging. That also requires a separate application that gets messages as input and does whatever.

☆PhistucK

On Thu, Mar 30, 2017 at 10:19 AM, Zhan Benjamin <gnbenja...@gmail.com> wrote:

Hello, I have a quest that i need to crawl some of information from a website.

The problem is I need to sign up to access the pages with information i need and the account is very precious. So I can not use pyspider or other ways because it is easy be banned.

Thankfully, there are not much pages need be crwaled. So I think how about I download the html files (not page source code but codes get from chrome inspect elements function) by chrome extensions and send it to the local application which can analysis the html file to get the information I needed. 

So I try write a chrome extension simply with code:

chrome.tabs.executeScript({

    code: 'document.document.body.innerHTML'

  }


and I dont know how to send the string in this ver to my local application. I totally new to javascript or web development. So can some one tell me how to find a easy way to use chrome extension communicate with local application? Or can someone think a better way to reach my goal?

--
You received this message because you are subscribed to the Google Groups "Chromium-Extensions-Announce" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To post to this group, send email to chromium-...@chromium.org.
Visit this group at https://groups.google.com/a/chromium.org/group/chromium-extensions/.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/0938a786-21ec-45ff-834c-e723b55712b9%40chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

--
You received this message because you are subscribed to the Google Groups "Chromium-Extensions-Announce" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To post to this group, send email to chromium-...@chromium.org.
Visit this group at https://groups.google.com/a/chromium.org/group/chromium-extensions/.

[Zhan Benjamin]

I solved this problem by read http://stackoverflow.com/questions/7699615/cross-domain-xmlhttprequest-using-background-pages/7699773#7699773 .

The solution is dont use XMLHttpRequest directly in content page, but use it in background.js.

在 2017年3月31日星期五 UTC+8上午9:34:33，Zhan Benjamin写道：