want to reuse the same socket in Chrome

[all smooth]

Greetings all,

I like to reuse the socket connection to send/receive tcp request/response once I done https request/response. 

 

Here is the scenario,

 The customized Chrome sends https requests, and receives response ( I can monitor this flow in net module within customized Chrome). Since http uses persistent connection, so after all expected responses done I assume the ssl connection is still alive. Now the question is how do I reuse *THIS* connection for my own tcp send/recv and return the recv (it's web page)  to ask Chrome to display. The subsequent requests based on this page will all through the same connection.  

In fact, there are three tasks here:

1. my own tcp request to reuse the previously connected ssl socket.

2. How to get my own tcp responses (web page) to render into Chrome.

3. the subsequent Chrome http requests based on 2's webpage should use the same 1's ssl connection.

It's like I need to ask customized Chrome to use the specified socket.  

I would like someone give suggestions/hints and possibilities. Where should I start to exploring. I guess net/socket is some entry point. I'd like to have more details.

cheers,

all smooth

[David Michael]

Are you able to step back and give some information about what you are trying to accomplish from the user's perspective? The approach you're suggesting sounds difficult and fraught with peril, and I don't see what advantage(s) it gives you.

On Wed, Jun 5, 2013 at 1:41 PM, all smooth <guang...@gmail.com> wrote:

Greetings all,

I like to reuse the socket connection to send/receive tcp request/response once I done https request/response. 

 

Here is the scenario,

 The customized Chrome sends https requests, and receives response ( I can monitor this flow in net module within customized Chrome). Since http uses persistent connection, so after all expected responses done I assume the ssl connection is still alive. Now the question is how do I reuse *THIS* connection for my own tcp send/recv and return the recv (it's web page)  to ask Chrome to display. The subsequent requests based on this page will all through the same connection.  

So you want to use the socket for stuff beyond HTTP? You intend to also have a custom server to make this work, I assume? Have you thought about just using all the stuff that's already in the Web stack to accomplish your goal?

In fact, there are three tasks here:

1. my own tcp request to reuse the previously connected ssl socket.

2. How to get my own tcp responses (web page) to render into Chrome.

This is what HTTPS already does for you. What do you need that it doesn't have?

 

3. the subsequent Chrome http requests based on 2's webpage should use the same 1's ssl connection.

I'm not sure what you mean here exactly, but it sounds a bit like a proxy (http://en.wikipedia.org/wiki/Proxy_server). If you can get what you need with existing technology (e.g., a proxy), you'll save yourself a lot of work and pain.

 

It's like I need to ask customized Chrome to use the specified socket.  

I would like someone give suggestions/hints and possibilities. Where should I start to exploring. I guess net/socket is some entry point. I'd like to have more details.

cheers,

all smooth

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
 
 
 

[all smooth]

Thanks much for response.

Here is the scenario,

 The customized Chrome sends https requests, and receives response ( I can monitor this flow in net module within customized Chrome). Since http uses persistent connection, so after all expected responses done I assume the ssl connection is still alive. Now the question is how do I reuse *THIS* connection for my own tcp send/recv and return the recv (it's web page)  to ask Chrome to display. The subsequent requests based on this page will all through the same connection.  

So you want to use the socket for stuff beyond HTTP? You intend to also have a custom server to make this work, I assume? Have you thought about just using all the stuff that's already in the Web stack to accomplish your goal?

yes, this is target for specific server.  

In fact, there are three tasks here:

1. my own tcp request to reuse the previously connected ssl socket.

2. How to get my own tcp responses (web page) to render into Chrome.

This is what HTTPS already does for you. What do you need that it doesn't have?

Different.  After Chrome use the underlying socket for https request, I want to use this socket(server know it's special) again for my own tcp request. The requirement cannot get everything done in web stack.  

 

 

3. the subsequent Chrome http requests based on 2's webpage should use the same 1's ssl connection.

I'm not sure what you mean here exactly, but it sounds a bit like a proxy (http://en.wikipedia.org/wiki/Proxy_server). If you can get what you need with existing technology (e.g., a proxy), you'll save yourself a lot of work and pain.

 

 It's similar to proxy server. but we want to build this feature inside Chrome. 

[David Michael]

On Wed, Jun 5, 2013 at 3:19 PM, all smooth <guang...@gmail.com> wrote:

Thanks much for response.

Here is the scenario,

 The customized Chrome sends https requests, and receives response ( I can monitor this flow in net module within customized Chrome). Since http uses persistent connection, so after all expected responses done I assume the ssl connection is still alive. Now the question is how do I reuse *THIS* connection for my own tcp send/recv and return the recv (it's web page)  to ask Chrome to display. The subsequent requests based on this page will all through the same connection.  

So you want to use the socket for stuff beyond HTTP? You intend to also have a custom server to make this work, I assume? Have you thought about just using all the stuff that's already in the Web stack to accomplish your goal?

yes, this is target for specific server.  

In fact, there are three tasks here:

1. my own tcp request to reuse the previously connected ssl socket.

2. How to get my own tcp responses (web page) to render into Chrome.

This is what HTTPS already does for you. What do you need that it doesn't have?

Different.  After Chrome use the underlying socket for https request, I want to use this socket(server know it's special) again for my own tcp request. The requirement cannot get everything done in web stack.  

But why does it have to be TCP? Let the browser manage that, and use some of the features available to you at the web platform level. If you own the page, use XMLHTTPRequest or WebSockets to communicate with the server. If you are using arbitrary content, then maybe you can use an extension instead of modifying Chromium.

 

 

 

3. the subsequent Chrome http requests based on 2's webpage should use the same 1's ssl connection.

I'm not sure what you mean here exactly, but it sounds a bit like a proxy (http://en.wikipedia.org/wiki/Proxy_server). If you can get what you need with existing technology (e.g., a proxy), you'll save yourself a lot of work and pain.

 

 It's similar to proxy server. but we want to build this feature inside Chrome. 

If you're customizing Chromium anyway, maybe you can ship a proxy bundled with your custom Chromium fork, and make your custom Chromium fork always talk to that proxy?

The more you tinker too much with Chromium's internals, the more difficulty you are going to have keeping up with Chromium changes (even if you get it to work initially). And the more you try to subvert standard technologies like HTTPS, the more trouble you are going to have getting your stuff to actually work properly over real web proxies and routers.

Just my opinion. But I think you should think carefully about your requirements and try to find another approach that works with the existing architecture of Chromium and existing standards rather than against them.

[Ryan Hamilton]

On Wed, Jun 5, 2013 at 12:41 PM, all smooth <guang...@gmail.com> wrote:

Greetings all,

I like to reuse the socket connection to send/receive tcp request/response once I done https request/response. 

 

Here is the scenario,

 The customized Chrome sends https requests, and receives response ( I can monitor this flow in net module within customized Chrome). Since http uses persistent connection, so after all expected responses done I assume the ssl connection is still alive. Now the question is how do I reuse *THIS* connection for my own tcp send/recv and return the recv (it's web page)  to ask Chrome to display. The subsequent requests based on this page will all through the same connection.  

In fact, there are three tasks here:

1. my own tcp request to reuse the previously connected ssl socket.

2. How to get my own tcp responses (web page) to render into Chrome.

3. the subsequent Chrome http requests based on 2's webpage should use the same 1's ssl connection.

It's like I need to ask customized Chrome to use the specified socket.  

​This is going to be virtually impossible without rewriting vast portions of the Chrome network stack, I suspect.  The sockets are hidden from requests inside of the various SocketPools which Chrome access via the HttpStreamFactory.  To do what you propose requires penetrating numerous different layers of abstraction and I think is totally impractical.

Cheers,

Ryan

 

I would like someone give suggestions/hints and possibilities. Where should I start to exploring. I guess net/socket is some entry point. I'd like to have more details.

cheers,

all smooth

[all smooth]

On Wednesday, June 5, 2013 2:46:50 PM UTC-7, David Michael wrote:

On Wed, Jun 5, 2013 at 3:19 PM, all smooth <guang...@gmail.com> wrote:

Thanks much for response.

Here is the scenario,

 The customized Chrome sends https requests, and receives response ( I can monitor this flow in net module within customized Chrome). Since http uses persistent connection, so after all expected responses done I assume the ssl connection is still alive. Now the question is how do I reuse *THIS* connection for my own tcp send/recv and return the recv (it's web page)  to ask Chrome to display. The subsequent requests based on this page will all through the same connection.  

So you want to use the socket for stuff beyond HTTP? You intend to also have a custom server to make this work, I assume? Have you thought about just using all the stuff that's already in the Web stack to accomplish your goal?

yes, this is target for specific server.  

In fact, there are three tasks here:

1. my own tcp request to reuse the previously connected ssl socket.

2. How to get my own tcp responses (web page) to render into Chrome.

This is what HTTPS already does for you. What do you need that it doesn't have?

Different.  After Chrome use the underlying socket for https request, I want to use this socket(server know it's special) again for my own tcp request. The requirement cannot get everything done in web stack.  

But why does it have to be TCP? Let the browser manage that, and use some of the features available to you at the web platform level. If you own the page, use XMLHTTPRequest or WebSockets to communicate with the server. If you are using arbitrary content, then maybe you can use an extension instead of modifying Chromium.

 

Server at this stage knows this TCP request specifically for this connection. If use high level API, such as the two you listed, then don't have knowledge for this socket, thus failed on server side. we don't own the content. 

 

 

3. the subsequent Chrome http requests based on 2's webpage should use the same 1's ssl connection.

I'm not sure what you mean here exactly, but it sounds a bit like a proxy (http://en.wikipedia.org/wiki/Proxy_server). If you can get what you need with existing technology (e.g., a proxy), you'll save yourself a lot of work and pain.

 

 It's similar to proxy server. but we want to build this feature inside Chrome. 

If you're customizing Chromium anyway, maybe you can ship a proxy bundled with your custom Chromium fork, and make your custom Chromium fork always talk to that proxy?

yes, this is possible. but might not neat. besides work on a proxy is not easy job too. especially the target platform might be Android. too much uncertainty. 

The more you tinker too much with Chromium's internals, the more difficulty you are going to have keeping up with Chromium changes (even if you get it to work initially). And the more you try to subvert standard technologies like HTTPS, the more trouble you are going to have getting your stuff to actually work properly over real web proxies and routers.

we understand this potential problem. so i will isolate the customization part as much as can. 

Just my opinion. But I think you should think carefully about your requirements and try to find another approach that works with the existing architecture of Chromium and existing standards rather than against them.

 

Thanks for your opinion, we are in the stage prototyping and exploring the solution.  

But now, can you tell me if it's possible to identify the specific socket and designate it for the subsequent request?

[all smooth]

Hi Ryan,

thanks for response.

​This is going to be virtually impossible without rewriting vast portions of the Chrome network stack, I suspect.  The sockets are hidden from requests inside of the various SocketPools which Chrome access via the HttpStreamFactory.  To do what you propose requires penetrating numerous different layers of abstraction and I think is totally impractical.

What if disable socket pool, or reduce the pool size to only 1? performance will be bad. Another approach after initial https req/rep, I update all the socket in pool inside to contains those information. Pl note the customized chrome is only intended to connect to customer's specific server. For this special purpose, even I disable the pool size to 1, then why care. the wiki document (http://en.wikipedia.org/wiki/HTTP_persistent_connection) for http persistent connection also mention the per-client at most have two socket per website.(According to RFC 2616 (page 46), a single-user client should not maintain more than 2 connections with any server or proxy.)

Any thoughts? 

btw, for Chrome, if each tab process will maintain its own socket pool?

cheers,

all smooth

[Ryan Hamilton]

I think these approaches are completely impractical.  The scope of work required to implement them is very large and would require penetrating all kinds of layers of abstraction which would leave you with hideous spaghetti code.  I can't discourage you strongly enough from this course of action.  This path is doomed to failure, I fear.

[Ryan Sleevi]

I think Ryan H's answer pretty much contains the information at hand:

It's an intentional design goal of Chromium's network stack NOT to
expose the underlying socket through these layers. Any patches to do
so would, with a 99.9% certainty, be something that would be "dead on
arrival" because of this.

Because of this, I'm not really sure it's applicable for the
chromium-dev community, or something that would be valuable to
furthering Chromium's four S's of Security, Stability, Speed, and
Simplicity.

If you follow the code, you'll find your answer - the relationship
between the StreamSocket interface, the ClientSocketHandle, and the
ClientSocketPool. As Ryan said, you're looking at forking Chromium to
accomplish those goals - and at that point, any assistance we can
realistically provide is gone, because now you're asking questions
about your fork, and not Chromium.

On Wed, Jun 5, 2013 at 3:33 PM, all smooth <guang...@gmail.com> wrote:
> Hi Ryan,
> thanks for response.
>
>
>> This is going to be virtually impossible without rewriting vast portions
>> of the Chrome network stack, I suspect. The sockets are hidden from
>> requests inside of the various SocketPools which Chrome access via the
>> HttpStreamFactory. To do what you propose requires penetrating numerous
>> different layers of abstraction and I think is totally impractical.
>>
> What if disable socket pool, or reduce the pool size to only 1? performance
> will be bad. Another approach after initial https req/rep, I update all the
> socket in pool inside to contains those information. Pl note the customized
> chrome is only intended to connect to customer's specific server. For this
> special purpose, even I disable the pool size to 1, then why care. the wiki
> document (http://en.wikipedia.org/wiki/HTTP_persistent_connection) for http
> persistent connection also mention the per-client at most have two socket
> per website.(According to RFC 2616 (page 46), a single-user client should
> not maintain more than 2 connections with any server or proxy.)
>
> Any thoughts?
>
> btw, for Chrome, if each tab process will maintain its own socket pool?

No. The point of the connection pool is to pool connections. This
includes between tabs.

[all smooth]

Thanks Ryan S for taking time to response.

On Wednesday, June 5, 2013 3:49:44 PM UTC-7, Ryan Sleevi wrote:

I think Ryan H's answer pretty much contains the information at hand:

It's an intentional design goal of Chromium's network stack NOT to
expose the underlying socket through these layers. Any patches to do
so would, with a 99.9% certainty, be something that would be "dead on
arrival" because of this.

I cannot say this is not doable before the real attempt/investigate. If not doable, I must provide very concrete details to convince others, not just say no. This is part of purpose of prototyping and exploring.  btw, in my previously exploring Chrome, I get traffic intercept done, and work nicely. in fact those things are do able. the effort is fully under control. 

Because of this, I'm not really sure it's applicable for the
chromium-dev community, or something that would be valuable to
furthering Chromium's four S's of Security, Stability, Speed, and
Simplicity.

If you follow the code, you'll find your answer - the relationship
between the StreamSocket interface, the ClientSocketHandle, and the
ClientSocketPool. As Ryan said, you're looking at forking Chromium to
accomplish those goals - and at that point, any assistance we can
realistically provide is gone, because now you're asking questions
about your fork, and not Chromium.  

No. The point of the connection pool is to pool connections. This
includes between tabs.

[Ryan Sleevi]

On Wed, Jun 5, 2013 at 4:21 PM, all smooth <guang...@gmail.com> wrote:
> Thanks Ryan S for taking time to response.
>
>
> On Wednesday, June 5, 2013 3:49:44 PM UTC-7, Ryan Sleevi wrote:
>>
>> I think Ryan H's answer pretty much contains the information at hand:
>>
>> It's an intentional design goal of Chromium's network stack NOT to
>> expose the underlying socket through these layers. Any patches to do
>> so would, with a 99.9% certainty, be something that would be "dead on
>> arrival" because of this.
>
> I cannot say this is not doable before the real attempt/investigate. If not
> doable, I must provide very concrete details to convince others, not just
> say no. This is part of purpose of prototyping and exploring. btw, in my
> previously exploring Chrome, I get traffic intercept done, and work nicely.
> in fact those things are do able. the effort is fully under control.

It's software. Anything is doable.

At this point, if I can perhaps over-simplify things, the questions of
interest to Chromium-dev are:
1) Does Chromium support this:
Answer) No. By design.
2) Is Chromium interested in supporting this:
Answer) No. By design.
3) What's the best way to do this?
Answer) You shouldn't.

I generally hate to say "No, we don't want X", because priorities may
change at any time, but the design, layering, security goals, and
overall composition are such that there's no desire to add the
additional complexity, fragility, and risk associated with a change,
particularly when it's not in the interest of or needed for the goal
of making the best possible browser we can for Chromium users.

I'm not trying to be too dismissive here, as I'm absolutely thrilled
you're able to find Chromium useful as a starting point for whatever
project you're trying to do, but I think that when our goals and yours
are this opposed, it's not something we can reasonably help you on.
Because of this, this may not be the best venue for those discussions.

I guess what I'm trying to say is that we're more than happy to answer
questions of "Did you consider X" or "Is X possible", but if the
answer is "Yes, and we decided not to" or "No, it's not possible",
then follow-up questions like "Well, how would I do it if I wanted to"
are things you're going to have to answer on your own.

Ryan

[Albert J. Wong (王重傑)]

Let me say it explicitly.

Your questions indicate an attempt to use chromium APIs far outside their intended design.  Because of this, answering them is costing an inordinate amount of energy from the various area experts. The consensus seems to be that you are breaking abstractions in a way that we will not support.

If you have to convince others, perhaps you can point them to this thread so they can see the (strong) feedback that this isn't the right approach.

Alternatively, if the only method for disproof you will accept is a complete prototype attempt, then feel free to do it.  However, I would expect less and less support from this list.

-Albert

 

Ryan

[all smooth]

On Wednesday, June 5, 2013 3:46:25 PM UTC-7, Ryan Hamilton wrote:

I think these approaches are completely impractical.  The scope of work required to implement them is very large and would require penetrating all kinds of layers of abstraction which would leave you with hideous spaghetti code.  I can't discourage you strongly enough from this course of action.  This path is doomed to failure, I fear.

Possible it's going to fail. Bu before I claim it's not practical I need to scope out the effort and find the hard to overcome difficulties. 

[all smooth]

I guess what I'm trying to say is that we're more than happy to answer
questions of "Did you consider X" or "Is X possible", but if the
answer is "Yes, and we decided not to" or "No, it's not possible",
then follow-up questions like "Well, how would I do it if I wanted to"
are things you're going to have to answer on your own.

 

Hi Ryan, is this your opinion or this group's policy?  I think all my question does not violate any policy. I am looking for Chrome developers' inside thoughts. But if they don't want  to share due to resource limitation, like someone said, cost a lot of time and energy, and money, then it's fine for me. If someone happen to know it and want to share without much effort, I feel very happy to learn the point. 

 

Ryan

[Ryan Sleevi]

You've heard from multiple OWNERS from src/net that what you're trying
to do is inconsistent/contrary to the design goals of the network
module, and to strongly discourage you from doing so. Attempts to
upstream such changes would, as I said, almost certainly be rejected
because of this.

You've also heard several alternatives proposed that are consistent
with the design goals, but rejected them. At that point, there's not
much we can really do to help you further in your quest to solve your
problem. You'll have to write this code yourself, and I'm afraid we
cannot help you.

I'm not trying to shut down any questions - as I said, we're happy to
try to answer questions where we can. After all, there is always room
for improvement.

Unfortunately, questions such as "How do I do X", when "doing X" is
clearly something we don't intend to support, are likely to simply go
unanswered. This is part of what it means to "not support" doing "X" -
at that point, you're going to have to figure how how to "do X" on
your own, with the code as your reference.

To be perfectly clear, getting the underlying socket is not something
we intend to support.

Again, this is software - anything is possible - but you're going to
have to work out how to do that on your own, because your solution
will ultimately be tailored to the problem you're trying to solve,
which is not a problem we're trying to solve.

[all smooth]

On Wednesday, June 5, 2013 5:16:32 PM UTC-7, Ryan Sleevi wrote:

 Attempts to
upstream such changes would, as I said, almost certainly be rejected
because of this.

I think this is completely irrelevant here. I never said I will upstream the changes. I think you know well this change is only for private usage. 

 

I'm not trying to shut down any questions - as I said, we're happy to
try to answer questions where we can. After all, there is always room
for improvement.

I did not request any improvement on this part. I only wish it's supported.  Hopefully I made myself clear.

 

Unfortunately, questions such as "How do I do X", when "doing X" is
clearly something we don't intend to support, are likely to simply go
unanswered. This is part of what it means to "not support" doing "X" -
at that point, you're going to have to figure how how to "do X" on
your own, with the code as your reference.

To be perfectly clear, getting the underlying socket is not something
we intend to support.

But I still can ask question to get someone share their knowledge on this part, right? It's open and free here, and people ask question, if someone happen to know and willing, then answer, otherwise just ignore. 

Again, this is software - anything is possible - but you're going to
have to work out how to do that on your own, because your solution
will ultimately be tailored to the problem you're trying to solve,
which is not a problem we're trying to solve.

I never intent to ask Chromium change source code to comply my requirement --that would be ridiculous. What I did here is ask some questions and looking for other developers sharing their experience on solving those problems. Those problems could be generic, even asked in Chromium forum. Again, if some of them think it's too much effort to reply, just ignore and it's perfect for me to accept. But if someone are willing to share his/her experience, I really appreciated. I learned a lot in this forum from various informative replies. 

I hope the reply is establishing, constructing, not just negative without any useful information.

Best,

all smooth

[Ryan Sleevi]

On Wed, Jun 5, 2013 at 6:42 PM, all smooth <guang...@gmail.com> wrote:
>
>
> On Wednesday, June 5, 2013 5:16:32 PM UTC-7, Ryan Sleevi wrote:
>>
>> Attempts to
>> upstream such changes would, as I said, almost certainly be rejected
>> because of this.
>>
> I think this is completely irrelevant here. I never said I will upstream the
> changes. I think you know well this change is only for private usage.

This discussion group is primarily focused on development of the
Chromium browser - eg: "upstream".

>
>>
>>
>> I'm not trying to shut down any questions - as I said, we're happy to
>> try to answer questions where we can. After all, there is always room
>> for improvement.
>
> I did not request any improvement on this part. I only wish it's supported.
> Hopefully I made myself clear.

I would consider a request for it to be supported a request for improvement.

>
>>
>> Unfortunately, questions such as "How do I do X", when "doing X" is
>> clearly something we don't intend to support, are likely to simply go
>> unanswered. This is part of what it means to "not support" doing "X" -
>> at that point, you're going to have to figure how how to "do X" on
>> your own, with the code as your reference.
>>
>> To be perfectly clear, getting the underlying socket is not something
>> we intend to support.
>
> But I still can ask question to get someone share their knowledge on this
> part, right? It's open and free here, and people ask question, if someone
> happen to know and willing, then answer, otherwise just ignore.

Generally, a response of "This is not something that is supported nor
intended to be supported" is meant to be a polite indication that
further questions in that line of inquiry are off-topic, and may be
better suited for another forum, such as one devoted to your private
fork, or by individual exploration of the code.

That is what it means to be "not supported" - that this forum is not
really an appropriate place for continued discussion. Even if it's
technically possible, it's not something encouraged, and thus you
won't really find people offering it up. You'll have to see the code
yourself. It's not meant to cut off discussion, but in the event that
what's trying to be accomplished is so wildly inconsistent with the
design goals, it's very reasonable to see it as that.

We're more than willing to help understand your problem, as well as
potentially even propose solutions that are consistent with the design
goals for the Chromium browser, but if you find those design goals
incompatible with your goals, you're not really going to find support
here. Continuing the line of inquiry after such polite responses will
typically end up costing a sizable bit of good will and further
willingness to answer questions.