Node.js standard and urlfetch

[Aleksander Efremov]

All outbound requests in the "appengine" of "python" and "go" issue via "urlfetch" service. But for "node.js" it isn't available. There is improvements on using the "urlfetch" service instead of a native libraries in "python" and "go"? And why it isn't implemented for "node.js"?

[Attila-Mihaly Balazs]

I believe (but I'm not 100%) that the new runtimes (node.js, Java 8, Python 3.7 in beta, etc) are moving AppEngine standard more towards Flex. Ie. they will be full containers (thought managed and updated by Google) where you can install arbitrary binaries. This means that they don't/won't need services like "urlfetch" because you can connect to arbitrary TCP/UDP endpoints (though I'm sure there are/will be some restrictions like not being able to send emails).

All the best,

Attila

[Aleksander Efremov]

I more want to know there are restrictions on fetching arbitrary URLs via "urlfetch" or "native libraries"? I rewrote the Go (was used the "urlfetch" service) implementation of the proxy service to Node.js (uses native http/https libraries) and I want to be sure a performance didn't change.

[Yasser Karout (Cloud Platform Support)]

Currently, there is no implementation of urlfetch for Node.js so you would have to use the standard HTTP library or other libraries to issue HTTP requests.

I created a feature request on the issue tracker website which will get forwarded to engineering. You can click on the star next to the request title on the top of the page to mark that you are interested in the feature and receive email updates.

However there are no ETAs or guarantees of implementation for feature requests.

Regarding performance, all HTTP requests, whether urlfetch is used or not, are issued using App Engine’s URL Fetch service for efficiency and scaling purposes. Meaning performance will not differ while using native libraries or urlfetch. 

One advantage that urlfetch has is that it provides you with an interface to make asynchronous requests.

[Aleksander Efremov]

Thanks, then I will star of course that feature request. And it's very good there isn't difference.

[Jeff Schnitzer]

There's one big disadvantage of the URLFetch service, which is that it's limited to 32MB inbound and 10MB outbound. Those are pretty small numbers in this day and age.

In Java8-land I've used both Apache HttpClient and OkHttp and both work normally.

Jeff

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/39c3df33-ceec-4af6-8204-c878b3ab9630%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aleksander Efremov]

oh, yes, it's important note, thanks.

[Steren Giannini]

Hi,

As commented on the bug: It is not clear what problem a Node.js implementation of urlfetch would solve.

Today, developers can use idiomatic methods to send asynchronous requests to HTTP endpoints, for example by using the native `http` module. Did you try it?

Steren

Product Manager

App Engine

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/10f3f503-8b21-453f-80d4-acbe2d8e7e3b%40googlegroups.com.

[Attila-Mihaly Balazs]

^^^ This.

However, Steren, please correct me if I'm wrong but I believe that to use the "idiomatic methods to send asynchronous requests to HTTP endpoints" one needs to enable "sockets": https://cloud.google.com/appengine/docs/standard/python/sockets/ (and also needs to be a paying application).

Regards,

Attila

[Yasser Karout (Cloud Platform Support)]

As a workaround to this, you can use the Blobstore API which allows users to upload large files through an HTML form. The browser uploads and stores the file directly to the Blobstore then rewrites the user's request. The data is sent using blob keys to the URL path specified.

The documentation provides sample code.

On Friday, July 6, 2018 at 10:58:38 AM UTC-4, Jeff Schnitzer wrote:

There's one big disadvantage of the URLFetch service, which is that it's limited to 32MB inbound and 10MB outbound. Those are pretty small numbers in this day and age.

In Java8-land I've used both Apache HttpClient and OkHttp and both work normally.

Jeff

On Thu, Jul 5, 2018 at 10:06 PM Aleksander Efremov <mr.e...@gmail.com> wrote:

Thanks, then I will star of course that feature request. And it's very good there isn't difference.

On Thursday, July 5, 2018 at 11:58:27 PM UTC+4, Yasser Karout (Cloud Platform Support) wrote:

Currently, there is no implementation of urlfetch for Node.js so you would have to use the standard HTTP library or other libraries to issue HTTP requests.

I created a feature request on the issue tracker website which will get forwarded to engineering. You can click on the star next to the request title on the top of the page to mark that you are interested in the feature and receive email updates.

However there are no ETAs or guarantees of implementation for feature requests.

Regarding performance, all HTTP requests, whether urlfetch is used or not, are issued using App Engine’s URL Fetch service for efficiency and scaling purposes. Meaning performance will not differ while using native libraries or urlfetch. 

One advantage that urlfetch has is that it provides you with an interface to make asynchronous requests.

On Tuesday, July 3, 2018 at 2:01:26 AM UTC-4, Aleksander Efremov wrote:

All outbound requests in the "appengine" of "python" and "go" issue via "urlfetch" service. But for "node.js" it isn't available. There is improvements on using the "urlfetch" service instead of a native libraries in "python" and "go"? And why it isn't implemented for "node.js"?

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

[Jason Collins]

For nodejs8, you do not need the Sockets API. In fact, it's not even possible. Just connect to other servers like you would in a VM environment. 

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/HWSKQIiAmeg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f24ddd92-d4ba-4157-bfe2-3de6e3a31949%40googlegroups.com.

[Aleksander Efremov]

Yes, I use `http`/`https` native libraries and I use streaming of course when it's possible. But as I understood from the AppEngine documentation, streaming doesn't work and all responses are buffered in any case. Do you know why streaming isn't ready yet and when it will possible?

[Jason Collins]

Streaming doesn't work for inbound requests (i.e., requests from a client to your App Engine app), but should work fine for outbound requests originating from your App Engine app to another server.

You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/HWSKQIiAmeg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/adedb59c-3c3b-41ba-ba25-7fbc289e75cc%40googlegroups.com.

[Aleksander Efremov]

https://cloud.google.com/appengine/docs/standard/nodejs/how-requests-are-handled

Streaming Responses

App Engine does not support streaming responses where data is sent in incremental chunks to the client while a request is being processed. All data from your code is collected as described above and sent as a single HTTP response.

вт, 10 июл. 2018 г., 19:10 Jason Collins <jason.a...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAL_dBeaRCKQgezGdXjWy-fFh1HvedQ7yTFaV6ozyfbWBYS%2BBYQ%40mail.gmail.com.

[Aleksander Efremov]

But yes, you're correct. Only for me is more important a streaming between the client and the my app.

вт, 10 июл. 2018 г., 23:40 Aleksander Efremov <mr.e...@gmail.com>: