Local setup to Azure giving error

[Sutapa Karmakar]

Hi

I am trying to setup CCF in local to connect to Azure and getting below error. Can someone help with any info?

[0] Issues checking in progress...
[0] Proxy error: Could not proxy request /api/regions/emissions-factors from 127.0.0.1:3000 to http://localhost:4000/.
[0] See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
[0]
[0] Proxy error: Could not proxy request /api/footprint?start=2021-08-16&end=2022-08-16&limit=90&skip=0 from 127.0.0.1:3000 to http://localhost:4000/.
[0] See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
[0]
[0] No issues found.
[1] 2022-08-16T10:54:52.825Z [server] info: Cloud Carbon Footprint Server listening at http://localhost:4000
[1] 2022-08-16T10:55:11.852Z [api] info: Regions emissions factors API request started
[1] 2022-08-16T10:55:11.877Z [api] info: Footprint API request started.
[1] 2022-08-16T10:55:11.879Z [api] warn: GroupBy parameter not specified. This will be required in the future.
[1] 2022-08-16T10:55:11.891Z [Cache] info: Using local cache file...
[1]
[1] <--- Last few GCs --->
[1]
[1] [23840:0000025B939C35B0]   328248 ms: Scavenge 8065.7 (8218.1) -> 8057.5 (8222.9) MB, 39.3 / 0.0 ms  (average mu = 0.323, current mu = 0.286) allocation failure
[1] [23840:0000025B939C35B0]   376511 ms: Mark-sweep 8071.8 (8224.4) -> 8061.5 (8230.9) MB, 48234.4 / 1.5 ms  (average mu = 0.191, current mu = 0.035) allocation failure scavenge might not succeed
[1]
[1]
[1] <--- JS stacktrace --->
[1]
[1] FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
[1]  1: 00007FF762757C4F v8::internal::CodeObjectRegistry::~CodeObjectRegistry+114207
[1]  2: 00007FF7626E5EC6 DSA_meth_get_flags+65542
[1]  3: 00007FF7626E6D7D node::OnFatalError+301
[1]  4: 00007FF76301B6CE v8::Isolate::ReportExternalAllocationLimitReached+94
[1]  5: 00007FF763005CAD v8::SharedArrayBuffer::Externalize+781
[1]  6: 00007FF762EA907C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1468
[1]  7: 00007FF762EB5D29 v8::internal::Heap::PublishPendingAllocations+1129
[1]  8: 00007FF762EB2CFA v8::internal::Heap::PageFlagsAreConsistent+2842
[1]  9: 00007FF762EA5959 v8::internal::Heap::CollectGarbage+2137
[1] 10: 00007FF762EA3B10 v8::internal::Heap::AllocateExternalBackingStore+2000
[1] 11: 00007FF762EC1FC7 v8::internal::Factory::AllocateRawWithAllocationSite+231
[1] 12: 00007FF762ECB552 v8::internal::Factory::NewJSObjectFromMap+50
[1] 13: 00007FF762D3137D v8::internal::JSObject::New+141
[1] 14: 00007FF762D311F4 v8::internal::JSObject::MigrateToMap+388
[1] 15: 00007FF762FC2543 v8::internal::Builtins::code_handle+92051
16: 00007FF762FC17D0 v8::internal::Builtins::code_handle+88608
[1] 17: 00007FF7630A94C1 v8::internal::SetupIsolateDelegate::SetupHeap+494673
[1] 18: 00007FF763038F56 v8::internal::SetupIsolateDelegate::SetupHeap+34534
[1] 19: 0000025B957A1FD1
[0] Proxy error: Could not proxy request /api/footprint?start=2021-08-16&end=2022-08-16&limit=90&skip=0 from localhost:3000 to http://localhost:4000/.
[0] See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNRESET).
[0]

[Tobias Gerbothe]

Hi,

I would expect that the memory of your local machine is too low.  FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Try adding additional memory to your system, use another system or adapt the configuration in order to process less accounts etc.

Best,
Tobias

[Sutapa Karmakar]

Thanks. Yes, setting the memory worked with less accounts

[ashutosh tripathi]

I am facing the same issue.
I have 10GB of RAM available still it is throwing this error.
Tried it on multiple systems.

Any help would be appreciated.

[C. G.]

As mentionned in a previous post in July 13, 2022 : https://groups.google.com/g/cloud-carbon-footprint/c/_wxI5LbKHSg 

A feature is in development to query less Azure account at the same time because it's won't work with very big quantity of account. I have more than 200 account and got the same error. 

Best,

Clement

[Sutapa Karmakar]

Hi Clement

Thanks a lot for the reply. Will try this and will post in the thread.

I was able to sort out the javascript heap out of memory error which I posted earlier by setting the below in terminal in Azure VM where I was running the code.

export NODE_OPTIONS=--max_old_space_size=4096

I was able to get data with the api -http://xx.xxx.xx.xx:4000/api/footprint?start=2022-08-19&end=2022-08-19&groupBy=day. I had limited data so maybe this worked for me.I could see the api being called in the console.

However when I am hiiting the client at http:// /xx.xxx.xx.xx:3000, I am getting the proxy error .

[0] Issues checking in progress...
[0] Proxy error: Could not proxy request /api/regions/emissions-factors from 127.0.0.1:3000 to http://localhost:4000/.
[0] See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
[0]
[0] Proxy error: Could not proxy request /api/footprint?start=2021-08-16&end=2022-08-16&limit=90&skip=0 from 127.0.0.1:3000 to http://localhost:4000/.
[0] See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
[0]

I have tried the solutions provided to append "/" and use 127.0.0.1 or use the private IP to route the call from external IP of client to API however no luck.

Regards

Sutapa

[Tobias Gerbothe]

Hey Sutapa,

The feature mentioned above is polling the azure account with a limit which is per default set to 90. The browser runs out of memory in my case and I figured out that the memory footprint in the browser scales with the request size, almost independently from the limit set.

You can try to set the limit to higher values than 90. For local dev: Go to your client\.env file and add the following line: 

REACT_APP_PAGE_LIMIT=<your page limit here>

I tried several values and I request our azure acounts for almost the whole year 2022 and end up with billions of entries. It not even worked with a limit of 6000000000000000, I just gave up after 300 requests. It's unclear for me what a suitable value is. I guess you need to try out what works best for you. 

You can also deactivate the feature, for me it worked without any issue, by just adding the following line to the client\.env

REACT_APP_DISABLE_CACHE=false

Please note: the variables 'REACT_APP_DISABLE_CACHE' and 'REACT_APP_PAGE_LIMIT' are currently not documented. Furthermore, in case you use the ci.yml to deploy your app to GCP, you need to add the environment variables by your own since it is not prepared.

Best,

Tobias

[Sutapa Karmakar]

Thanks for the reply and explanation. Will try this and post if things work.

[Cloud Carbon Footprint]

Hi all,

Just adding in one note - the REACT_APP_PAGE_LIMIT and REACT_APP_DISABLE_CACHE are for the mongo implementation only (documentation to come very soon!), so it will not be helpful for this issue.

Best,
The Cloud Carbon Footprint Team at Thoughtworks