How do I find out what I am being billed for?

[Peter Camilleri]

I started playing with GAE a couple weeks ago. I've only played around with the tutorials so far. I successfully deployed the bookshelf and visit apps.

I checked my billing today and found that I have burnt through about 1/5th of my free 300$ credit. This seems pretty extreme.

The billing page doesnt really tell me what I've been billed for. My biggest expense is

"Credit FreeTrial:2017-10-22T00:00:00.000-07:002af76aae-dd42-421e-XXXX-XXXXXXXXX" which is 50$

Should I be deleting my app engine instance when I'm not using it? 

Even so, these prices seem pretty extreme, it looks like App engine will cost me around $75 a week to host my project. The GAE calculator gives me a significantly smaller estimate.

Where am I losing money?

[Ani Hatzis]

Hi Peter,

$75 per week sounds a lot for a non-used tutorial app, that normally should stay below the free tier. Maybe there is more than one app running since you played with the tutorials. One other explanation would be that you have chosen an expensive instance class (e.g. F4, F4G1), or that there are too many idle instances running, or forgotten cron-jobs, etc. Best approach is to study all the expenses for the billing account where the costs were charged and identify the big hitters.

In order to see your detailed expenses in Google Cloud Platform:

1. Go to Google Cloud Console > Billing
2. Click on each billing account (probably you only have one listed for all your projects)
3. On the left navigation under Billing, click on Transactions
4. You will see a list of transactions. Make sure that above the table, you have selected:
1. Detailed transaction view
2. All transactions
3. All time
5. Each costs item will show you which project has consumed a certain service or resource, the quantity (in hours, counts etc.), and the cost in USD
6. Some lines will show payments (I assume, the free 300$ credit should appear as the last item at the bottom of the page)

More information here: View Your Cost and Payment History.

Most tutorials in the documentation of Google Cloud Platform also include how to delete the training assets after the tutorial is completed, so no more costs will occur.

However, if you do not want to delete a Google App Engine app, you have two options to save costs (at least for not running the app instances):

• Disable the app until you want to work again with the app. The app will stop serving requests, until re-enabled. Data and state of the app will be kept, but may result in costs (e.g. size of datastore).
  
1. In Cloud Console for the project that has the GAE app, go to App Engine > Settings
2. Click on Disable application button
3. Enter the appp ID and confirm the action
• Apply a daily billing limit for the app. Once the daily spending limit is reached, the app will stop working, any further requests will be responded with server errors.
1. In Cloud Console for the project that has the GAE app, go to App Engine > Settings
   
2. Click on the Edit button above
3. Enter a USD value (e.g. 1) into the input field Daily spending
4. Click on Save

Both options relate to GAE costs (typically instance hours, some traffic, datastore ops), but other costs may incur, let's say, you have created the app with a Cloud SQL instance, significant Datastore or Storage size. Just check the Transactions page to see which costs were actually charged. The documentation provide lots of information on billing, as well as pricing, quotas, and free tiers for each product.

Optionally, you can also setup budgets and alerts, so you will receive an email at certain budget milestones (reset each month). I strongly recommend to setup at least one alert for each billing account, so you won't be surprised at the end of a billing period in case you forgot to delete or disable something.

Ani

[Peter Camilleri]

On Sunday, 5 November 2017 05:11:43 UTC+11, Ani Hatzis wrote:

Hi Peter,

$75 per week sounds a lot for a non-used tutorial app, that normally should stay below the free tier. Maybe there is more than one app running since you played with the tutorials. One other explanation would be that you have chosen an expensive instance class (e.g. F4, F4G1),

I didnt choose an instance class. How do I do this? Isn't that more something you do in the compute engine? Not the app engine?

I have recently tried to specify resources, e.g. only 1 instance, minimal ram, etc. but I dont did this after I found out my costs were unusually high

 

or that there are too many idle instances running, or forgotten cron-jobs, etc. Best approach is to study all the expenses for the billing account where the costs were charged and identify the big hitters.

How do I find idle instances? I found some older instances of apps were running so I deleted them. I think that is what you are referring to. IS this normal?

 

In order to see your detailed expenses in Google Cloud Platform:

1. Go to Google Cloud Console > Billing
2. Click on each billing account (probably you only have one listed for all your projects)
3. On the left navigation under Billing, click on Transactions
4. You will see a list of transactions. Make sure that above the table, you have selected:
1. Detailed transaction view
2. All transactions
3. All time
5. Each costs item will show you which project has consumed a certain service or resource, the quantity (in hours, counts etc.), and the cost in USD
6. Some lines will show payments (I assume, the free 300$ credit should appear as the last item at the bottom of the page)

Yea this is where I found out I was paying 50$ for resources. Although I dont find the descriptions to be very helpful, which is why I came here looking for help.

 

More information here: View Your Cost and Payment History.

Most tutorials in the documentation of Google Cloud Platform also include how to delete the training assets after the tutorial is completed, so no more costs will occur.

However, if you do not want to delete a Google App Engine app, you have two options to save costs (at least for not running the app instances):

• Disable the app until you want to work again with the app. The app will stop serving requests, until re-enabled. Data and state of the app will be kept, but may result in costs (e.g. size of datastore).
  
1. In Cloud Console for the project that has the GAE app, go to App Engine > Settings
2. Click on Disable application button
3. Enter the appp ID and confirm the action
• Apply a daily billing limit for the app. Once the daily spending limit is reached, the app will stop working, any further requests will be responded with server errors.
1. In Cloud Console for the project that has the GAE app, go to App Engine > Settings
   
2. Click on the Edit button above
3. Enter a USD value (e.g. 1) into the input field Daily spending
4. Click on Save

Both options relate to GAE costs (typically instance hours, some traffic, datastore ops), but other costs may incur, let's say, you have created the app with a Cloud SQL instance, significant Datastore or Storage size. Just check the Transactions page to see which costs were actually charged. The documentation provide lots of information on billing, as well as pricing, quotas, and free tiers for each product.

Yes, I also have a cloud SQL instance. But it is literally just 1 database with 3 tables, each with about 4-5 fields (these are from tutorials). So it's realistically a tiny database. but i still got charged 15$ for using that. What I would like is a breakdown of where my money went, because all I know is that I lost 15$ to CloudSQL but have no idea why.

[Ani Hatzis]

On Sunday, 5 November 2017 17:06:37 UTC+1, Peter Camilleri wrote:

I didnt choose an instance class. How do I do this? Isn't that more something you do in the compute engine? Not the app engine?

I have recently tried to specify resources, e.g. only 1 instance, minimal ram, etc. but I dont did this after I found out my costs were unusually high

App Engine standard environment also supports different instances classes with different pricing. You define the instance classes (instance_class) and the type of scaling in the app.yaml files of your GAE app (see An Overview of App Engine, How Instances are Managed, Managing App Resources, and the app.yaml Reference). If I remember correctly, the defaults will be the cheapest instance class (F1) and auto-scaling. There can be many reasons why an instance would be automatically started (one of the big selling points of App Engine actually), so shutting instances down manually might have no persistent effect at all. Also note that for apps in standard environment, the charges will always be calculated on base of the F1 price, where the higher instance classes are priced as a multiple of the F1 price, i.e. if you have used F4, their instance hours will be 4 times of what they actually have been, but with the price of F1.

NB: Since each app comes with a free tier of 28 instance hours per day (F1 or B1), an un-used app should remain for free, unless there is something wrong in your configuration as explained in my earlier post. In order to learn more about this, it makes sense to check the logging of the app to see what requests where handled by the billed instances. Maybe it is unexpected traffic from outside (a crawler?), a cron-job that runs every now and then, or there is a bug in a task in one of your task-queues and the task is retried indefinitely.

On the other hand, App Engine flexible environment are deployed to virtual machine types as specified, and the billing is different from standard environment, see Pricing - Flexible environment instances.

It would help if you could tell us which version of App Engine you have in use (programming language, and whether standard or flexible environment). Above I have linked to the Python version.

 

 

or that there are too many idle instances running, or forgotten cron-jobs, etc. Best approach is to study all the expenses for the billing account where the costs were charged and identify the big hitters.

How do I find idle instances? I found some older instances of apps were running so I deleted them. I think that is what you are referring to. IS this normal?

 

You can find active and idle instances on your App Engine > Dashboard or App Engine > Instances. Toggle the version and chart type ("Instances") above the charts, and the period between 1 hour and 30 days. It not only shows the instances, but you can also access the requests logs for each instance to figure out what is causing the instances to not be shut down (as mentioned earlier).

 

Yea this is where I found out I was paying 50$ for resources. Although I dont find the descriptions to be very helpful, which is why I came here looking for help.

If you want, you could copy-paste the lines here. But remove all project IDs first and anything else you consider sensitive information.

 

Yes, I also have a cloud SQL instance. But it is literally just 1 database with 3 tables, each with about 4-5 fields (these are from tutorials). So it's realistically a tiny database. but i still got charged 15$ for using that. What I would like is a breakdown of where my money went, because all I know is that I lost 15$ to CloudSQL but have no idea why.

Different from Cloud Datastore, Cloud SQL is also charging for instances (as their is a managed VM being the database), and maybe also CPU and memory if you have dedicated-core instances. See Cloud SQL Pricing. Unfortunatley, I'm not very familiar with Cloud SQL, so I can't tell how you could save money for unused Cloud SQL resources, but probably disabling its instance might at least save the instance costs until you actually need it.

[Peter Camilleri]

Last night at 12:00 I recorded this from the billing page

App Engine Flex Instance Core Hours 8,640.47 Minute $9.92

And this morning I recorded this

App Engine Flex Instance Core Hours 191.79 Hour $13.22

8600 minutes is about 144 hours

191 - 144 = 47 hours of time used in 9 hours. So it looks like I've got 5 apps running simultaneously which is eating up my credit.

My versions page looks like this. Could this be where my credit is going?

I thought that these instances do not consume resources if they are stopped? The lower 6 instances are also created when I execute "gcloud app deploy app.yaml" with no extra arguments. So should I instead be specifying a specific version each time I deploy (e.g. dev or prod) so that I am overwriting the current instance and not creating a new instance each time?

However these are all inside of my "default" service, so I thought they would naturally be overwriting the last deployed version naturally.

On Monday, 6 November 2017 05:11:04 UTC+11, Ani Hatzis wrote:

On Sunday, 5 November 2017 17:06:37 UTC+1, Peter Camilleri wrote:

I didnt choose an instance class. How do I do this? Isn't that more something you do in the compute engine? Not the app engine?

I have recently tried to specify resources, e.g. only 1 instance, minimal ram, etc. but I dont did this after I found out my costs were unusually high

App Engine standard environment also supports different instances classes with different pricing. You define the instance classes (instance_class) and the type of scaling in the app.yaml files of your GAE app (see An Overview of App Engine, How Instances are Managed, Managing App Resources, and the app.yaml Reference). If I remember correctly, the defaults will be the cheapest instance class (F1) and auto-scaling. There can be many reasons why an instance would be automatically started (one of the big selling points of App Engine actually), so shutting instances down manually might have no persistent effect at all. Also note that for apps in standard environment, the charges will always be calculated on base of the F1 price, where the higher instance classes are priced as a multiple of the F1 price, i.e. if you have used F4, their instance hours will be 4 times of what they actually have been, but with the price of F1.

NB: Since each app comes with a free tier of 28 instance hours per day (F1 or B1), an un-used app should remain for free, unless there is something wrong in your configuration as explained in my earlier post. In order to learn more about this, it makes sense to check the logging of the app to see what requests where handled by the billed instances. Maybe it is unexpected traffic from outside (a crawler?), a cron-job that runs every now and then, or there is a bug in a task in one of your task-queues and the task is retried indefinitely. 

On the other hand, App Engine flexible environment are deployed to virtual machine types as specified, and the billing is different from standard environment, see Pricing - Flexible environment instances.

It would help if you could tell us which version of App Engine you have in use (programming language, and whether standard or flexible environment). Above I have linked to the Python version.

Python flexible environment (although this is only because the standard environment tutorials using CloudSQL wouldnt work out of the box.)

 

 

 

or that there are too many idle instances running, or forgotten cron-jobs, etc. Best approach is to study all the expenses for the billing account where the costs were charged and identify the big hitters.

How do I find idle instances? I found some older instances of apps were running so I deleted them. I think that is what you are referring to. IS this normal?

 

You can find active and idle instances on your App Engine > Dashboard or App Engine > Instances. Toggle the version and chart type ("Instances") above the charts, and the period between 1 hour and 30 days. It not only shows the instances, but you can also access the requests logs for each instance to figure out what is causing the instances to not be shut down (as mentioned earlier).

 

Yea this is where I found out I was paying 50$ for resources. Although I dont find the descriptions to be very helpful, which is why I came here looking for help.

If you want, you could copy-paste the lines here. But remove all project IDs first and anything else you consider sensitive information.

From the instances page, only one instance

Instances 

	ID	Debug mode	QPS last minute	Start Time	VM IP
xxx-default-dev-xxxx	Disabled	0	4 Nov 2017, 22:16:16	35.xxx.xxx.xxx	SSH

 

 

Yes, I also have a cloud SQL instance. But it is literally just 1 database with 3 tables, each with about 4-5 fields (these are from tutorials). So it's realistically a tiny database. but i still got charged 15$ for using that. What I would like is a breakdown of where my money went, because all I know is that I lost 15$ to CloudSQL but have no idea why.

Different from Cloud Datastore, Cloud SQL is also charging for instances (as their is a managed VM being the database), and maybe also CPU and memory if you have dedicated-core instances. See Cloud SQL Pricing. Unfortunatley, I'm not very familiar with Cloud SQL, so I can't tell how you could save money for unused Cloud SQL resources, but probably disabling its instance might at least save the instance costs until you actually need it.

No worries. I quickly checked and found my instance was not using the cheapest instance type. So that cut my SQL costs by about 80% right there.

 

[Ani Hatzis]

On Sunday, 5 November 2017 23:59:42 UTC+1, Peter Camilleri wrote:

Last night at 12:00 I recorded this from the billing page

App Engine Flex Instance Core Hours 8,640.47 Minute $9.92

And this morning I recorded this

App Engine Flex Instance Core Hours 191.79 Hour $13.22

8600 minutes is about 144 hours

191 - 144 = 47 hours of time used in 9 hours. So it looks like I've got 5 apps running simultaneously which is eating up my credit.

OK. In case that there is really just one instance running all the time (as your Versions page suggested), it might be that you have the instance configure with multiple vCPUs (cores). Flexible environment is charged per core hour (see doc here). So, an instance with four cores running for one (real) hour will be charged with 4 core hours. In the 9 hours you had ~ $3.3 costs for core hours, ~ $0.37/core hour, depending on your region, we talk about an instance with maybe 8 cores. Check your app.yaml file for a "resources" configuration (see doc here). What does the "cpu" value say? If you want to can copy-paste the section here.

 

My versions page looks like this. Could this be where my credit is going?

I thought that these instances do not consume resources if they are stopped? The lower 6 instances are also created when I execute "gcloud app deploy app.yaml" with no extra arguments. So should I instead be specifying a specific version each time I deploy (e.g. dev or prod) so that I am overwriting the current instance and not creating a new instance each time? 

However these are all inside of my "default" service, so I thought they would naturally be overwriting the last deployed version naturally.

You are right, stopped instances don't consume resources (at least not for the VM). What you are seeing in this page are all versions of the app that have been deployed until now. When you deploy a version, by default, instance of an older version is stopped. A new instance spins up and gets all the traffic. So your current version is in the first line, handling 100% of the traffic, running 1 instance. Below are the older versions (no traffic and no instances).

Older versions are kept, so you have the opportunity to roll-back to an older version in case your new version shows elevated errors. You can delete versions that you don't need anymore though, but in that case you can't just roll-back to them.

Python flexible environment (although this is only because the standard environment tutorials using CloudSQL wouldnt work out of the box.)

Have you followed these Cloud SQL instructions of standard environment here? If you had no other reason for flexible, I would recommend the standard environment. Albeit there are many good reasons for choosing flexible environment over standard, flexible needs more administrative overhead than standard. In standard it's easier to focus on your own code.

 

No worries. I quickly checked and found my instance was not using the cheapest instance type. So that cut my SQL costs by about 80% right there.

Great! And add budgets & alerts to your billing account. It's really helpful to stay alerted about potential config errors wreaking havocs to your credits during your first steps.

[Peter Camilleri]

On Monday, 6 November 2017 10:54:14 UTC+11, Ani Hatzis wrote:

On Sunday, 5 November 2017 23:59:42 UTC+1, Peter Camilleri wrote:

Last night at 12:00 I recorded this from the billing page

App Engine Flex Instance Core Hours 8,640.47 Minute $9.92

And this morning I recorded this

App Engine Flex Instance Core Hours 191.79 Hour $13.22

8600 minutes is about 144 hours

191 - 144 = 47 hours of time used in 9 hours. So it looks like I've got 5 apps running simultaneously which is eating up my credit.

OK. In case that there is really just one instance running all the time (as your Versions page suggested), it might be that you have the instance configure with multiple vCPUs (cores). Flexible environment is charged per core hour (see doc here). So, an instance with four cores running for one (real) hour will be charged with 4 core hours. In the 9 hours you had ~ $3.3 costs for core hours, ~ $0.37/core hour, depending on your region, we talk about an instance with maybe 8 cores. Check your app.yaml file for a "resources" configuration (see doc here). What does the "cpu" value say? If you want to can copy-paste the section here.

Nothing of the sort. I was just following the cloudsql tutorial (https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/appengine/flexible/cloudsql)

runtime: python

env: flex

entrypoint: gunicorn -b :$PORT main:app

#[START env]

env_variables:

    # Replace user, password, database, and instance connection name with the values obtained

    # when configuring your Cloud SQL instance.

    SQLALCHEMY_DATABASE_URI: >-

      mysql+pymysql://sensitive-data-removed

#[END env]

#[START cloudsql_settings]

# Replace project and instance with the values obtained  when configuring your

# Cloud SQL instance.

beta_settings:

    cloud_sql_instances: sensitive-data-removed

#[END cloudsql_settings]

But I will try specifying the resources. Does this look ok?

manual_scaling:

  instances: 1

resources:

  cpu: 2

  memory_gb: 1.4

  disk_size_gb: 10

 

 

My versions page looks like this. Could this be where my credit is going?

I thought that these instances do not consume resources if they are stopped? The lower 6 instances are also created when I execute "gcloud app deploy app.yaml" with no extra arguments. So should I instead be specifying a specific version each time I deploy (e.g. dev or prod) so that I am overwriting the current instance and not creating a new instance each time? 

However these are all inside of my "default" service, so I thought they would naturally be overwriting the last deployed version naturally.

You are right, stopped instances don't consume resources (at least not for the VM). What you are seeing in this page are all versions of the app that have been deployed until now. When you deploy a version, by default, instance of an older version is stopped. A new instance spins up and gets all the traffic. So your current version is in the first line, handling 100% of the traffic, running 1 instance. Below are the older versions (no traffic and no instances).

Older versions are kept, so you have the opportunity to roll-back to an older version in case your new version shows elevated errors. You can delete versions that you don't need anymore though, but in that case you can't just roll-back to them.

Python flexible environment (although this is only because the standard environment tutorials using CloudSQL wouldnt work out of the box.)

Have you followed these Cloud SQL instructions of standard environment here? If you had no other reason for flexible, I would recommend the standard environment. Albeit there are many good reasons for choosing flexible environment over standard, flexible needs more administrative overhead than standard. In standard it's easier to focus on your own code.

Yea I followed that tutorial and my database couldnt connect to cloudsql. I tried the flexible equivalent tutorial and it worked fine, so I continued with flexible (I thought the pricing difference would be negligible for the moment, and I just wanted to get my app development going. I'm not fussed over a several dollars a month, but I dont want to be haemorrhaging 60$).

I'll give standard another go.

[Ani Hatzis]

I'm surprised your app.yaml has no "resources" section defined, because the default would be "cpu: 1", which wouldn't add up with the instance core hours you have reported from your billing. But the default is also "automatic_scaling", so maybe between the moments you have checked, your app scaled automatically to many more than the one instance. The chart could tell you that. However, just to be safe, it might be worth to ask the Billing Support for help understanding the reason why instance core hours of the current version ID "dev" were or are burning through so fast in your app.

But I will try specifying the resources. Does this look ok?

manual_scaling:

  instances: 1

resources:

  cpu: 2

  memory_gb: 1.4

  disk_size_gb: 10

 

Yes, this looks correct. When you deploy, explicitly state a new version ID, so Billing Support has a chance to look into the current version's yaml files (or whatever they might need).

Good luck and have fun!