How do I use pubsub emulator for local testing?

[Kush Goyal]

I am following these 2 tutorials:

1. pubsub python queues: https://cloud.google.com/python/getting-started/using-pub-sub

2. pubsub python client api: https://cloud.google.com/appengine/docs/flexible/python/writing-and-responding-to-pub-sub-messages

I found out there is a emulator available for pubsub: https://cloud.google.com/pubsub/docs/emulator

But I did find any documentation to use the emulator. Can anyone please post sample code.

My code does not go past client = pubsub.Client()

I get credentials error.

[Kir Titievsky]

Kush,  The Pub/Sub emulator is detected using environment variables.  The page you refer to gives: export PUBSUB_EMULATOR_HOST=localhost:8590 as an example.  You will find the actual emulator host and port when you start the emulator. 

It looks like you might need to set PUBSUB_EMULATOR variable for python (rather than PUBSUB_EMULATOR_HOST).  Would you try and report your findings please?

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/6b5df3e5-88d5-440f-848c-9e00fefa9336%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kir Titievsky | Product Manager | Google Cloud Pub/Sub 

[Kush Goyal]

Hi Kir,

I am using django for writing my app.

I run the emulator using command: 

gcloud beta emulators pubsub start

Then I set the PUBSUB_EMULATOR and PUBSUB_EMULATOR_HOST environment variables and then run my testing using python manage.py runserver

I set the variable values using the command 

gcloud beta emulators pubsub env-init

I am using this code to publish a message

def publish(request):
 ps = pubsub.Client()
 topic = ps.topic(PUBSUB_TOPIC)
 topic.publish('Example payload'.encode('utf-8'))
 print("sent message")
 return Response({'detail': 'ok'})

On running this code I do not get any credential error like before but there is no response from the pubsub emulator. There is no response from the subscriber either.

Also, how do I register topics and subscribers on the emulator?

Thanks,

Kush

[Jordan (Cloud Platform Support)]

In the same tutorial you are following you created a subscription with an endpoint URL. Likewise, when you configure the environment variable and specified the port that you want the emulator to run on (e.g '8590'), you need to then setup a subscriber to have the endpoint of the emulator (e.g 'localhost:8590'). 

You do this by setting the endpoint of a subscriber using 'subscription = topic.subscription(subscription_name, push_endpoint='http://localhost:8590')' in your code, as shown in the documentation. 

[Kir Titievsky]

Same goes for creating a topic in the emulator.  The normal python client library API for creating topics should create one in the emulator once the environment variables point the client library at the emulator rather than production service.

On Fri, Apr 28, 2017 at 3:48 PM, 'Jordan (Cloud Platform Support)' via Google Cloud Pub/Sub Discussions <cloud-pubs...@googlegroups.com> wrote:

In the same tutorial you are following you created a subscription with an endpoint URL. Likewise, when you configure the environment variable and specified the port that you want the emulator to run on (e.g '8590'), you need to then setup a subscriber to have the endpoint of the emulator (e.g 'localhost:8590'). 

You do this by setting the endpoint of a subscriber using 'subscription = topic.subscription(subscription_name, push_endpoint='http://localhost:8590')' in your code, as shown in the documentation. 

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/39761936-c881-4206-bfa8-76b12c8b15ac%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Kush Goyal]

@Kir: thanks for the help. I was not setting PUBSUB_EMULATOR_HOST. Once I set it properly the emulator started working. I didn't knew that the command gcloud beta emulators pubsub env-init will not set the variable, it prints the command only.

@Jordan: push_endpoint is url of the method you want to call when a message is published. It should be something like http://<your-local-service-url>/subscriber/message-handler/

[Jordan (Cloud Platform Support)]

You are correct Kush, using "http://<your-local-service-url>/..." with the emulator port running on localhost (as shown in my previous post) is the correct endpoint for development. As pointed out by Kir, "<your-local-service-url>" should be used for both Topic and Subscriber. 

If you use the endpoint from the tutorial "https://YOUR_APP_ID.appspot.com/..." this will direct your application to use the production endpoint and will not interact with the local Pub/Sub emulator.  

[Robert Christian]

Not clear on how to setup push endpoint with simulator.  

When I change the target push URL to anything but app engine it complains with "Invalid push endpoint given".

My steps:

Set up topic:

gcloud beta pubsub topics create my-topic

Start emulator:

[pubsub] INFO: [ManagedChannelImpl@783e6358] Created with target localhost:8085

Set env variable:

export PUBSUB_EMULATOR_HOST=localhost:8085

Start service:

$ python main.py

... which listens for push queue messages:

@APP.route('/pubsub/push', methods=['POST'])
def pubsub_push():

...

Create subscription:

$ gcloud beta pubsub subscriptions create my-sub \

>      --topic my-topic \

>      --push-endpoint \

>          http://localhost:8081/pubsub/push?token=123.987 \

>      --ack-deadline 30

And fails:

ERROR: Failed to create subscription [projects/my-project/subscriptions/my-sub]: Invalid push endpoint given (endpoint=http://localhost:8081/pubsub/push?token=123.987). Refer to https://cloud.google.com/pubsub/subscriber#create for more information.

[Kush Goyal]

your step 1 should be to start the pubsub emulator.

Then export the env variable.

And then create your topics and subscriptions.

This is the way I do it and there are no errors.

[idan haviv]

Hi Robert,

Have you found a solution to emulate push subscription?

בתאריך יום חמישי, 10 באוגוסט 2017 בשעה 21:47:18 UTC+3, מאת Robert Christian:

[muath aljafari]

how can I configure Real-time Developer Notifications https://developer.android.com/google/play/billing/realtime_developer_notifications.html to push to a local topic in the emulator, do could you suggest any alternative please.

[Kir Titievsky]

Muath, 

I don't believe you can, but the question is best addressed to the android dev notifications team and community.  Pub/Sub is just the platform; the developer notification feature and its detailed capabilities are not defined by Pub/Sub.

I'd be very curious to know why you'd want your notifications in an emulator rather than the Pub/Sub service.  Could you tell us more?

--

You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/ab758cdf-f407-42ad-a14c-185947bd7c07%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[muath aljafari]

I am sorry if my question was unclear or a bit weird, what I actually wanted to do is just let the Pub/sub service subscriber to push to localhost. what I did is I configured the developer notification feature to push to my Pub/Sub topic and I configured a push subscriber (https://cloud.google.com/pubsub/docs/push) to push to a published site and every thing worked successfully, but I just want the subscriber to push to localhost for development and testing purposes but I couldn't or don't know how to do that.
my thanks to you.

On Monday, February 5, 2018 at 6:17:53 PM UTC+2, Kir Titievsky wrote:

Muath, 

I don't believe you can, but the question is best addressed to the android dev notifications team and community.  Pub/Sub is just the platform; the developer notification feature and its detailed capabilities are not defined by Pub/Sub.

I'd be very curious to know why you'd want your notifications in an emulator rather than the Pub/Sub service.  Could you tell us more?

On Mon, Feb 5, 2018 at 10:19 AM muath aljafari <muath....@gmail.com> wrote:

how can I configure Real-time Developer Notifications https://developer.android.com/google/play/billing/realtime_developer_notifications.html to push to a local topic in the emulator, do could you suggest any alternative please.

On Friday, April 28, 2017 at 10:48:32 PM UTC+3, Jordan (Cloud Platform Support) wrote:

In the same tutorial you are following you created a subscription with an endpoint URL. Likewise, when you configure the environment variable and specified the port that you want the emulator to run on (e.g '8590'), you need to then setup a subscriber to have the endpoint of the emulator (e.g 'localhost:8590'). 

You do this by setting the endpoint of a subscriber using 'subscription = topic.subscription(subscription_name, push_endpoint='http://localhost:8590')' in your code, as shown in the documentation. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud Pub/Sub Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-pubsub-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-pubsub-discuss/ab758cdf-f407-42ad-a14c-185947bd7c07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.