Local Cloud Build copying node_modules

[Jack Woodward]

Hi,

The local cloud build emulator is great for testing however I am noticing it takes a while to initialise it. I was doing a test in another folder where it had only 1 file and it worked alot quicker. In my main project its a node app so a big node_modules folder which it seems to be copying. I have used the .gcloudignore file which includes the node_modules folder so it should ignore it id expect.

cloud-build-local \

--config=cloudbuild.yaml \

--dryrun=false \

--write-workspace=../cloudbuild-workspace

.

Thanks

Jack

[George (Cloud Platform Support)]

Hello Jack, 

Local Cloud Build emulator runs on one machine, it is slower than proper Cloud Build, which is scalable, and works so as to minimize run time. 

You might be right about the copy operation you noticed, but .gcloudignore is not expected to help here. For one, this feature is meant to reduce traffic from your development machine to Cloud, not to prevent local writes from one directory to another. Also, commands are different: cloud-build-local vs. gcloud builds submit. In what concerns the local Cloud Build emulator, the command pattern is: 

cloud-build-local --config=[BUILD_CONFIG] --dryrun=false --push [SOURCE_CODE]

You may find related details on the "Building and debugging locally" page. 

The .gcloudignore feature works with the gcloud builds submit command. If the source is a local directory, this command skips the files specified in the .gcloudignore file. If a .gitignore file is present in the local source directory, gcloud will use a Git-compatible .gcloudignore file that respects your .gitignored files. The global .gitignore is not respected. This feature is provided with the reduction of Internet traffic in mind, and to avoid unnecessary costs. 

[Jack Woodward]

I understand that it will he slowly and that's fine as it's just for testing but I think it would be useful for the local build emulator to respect the gcloudignore or gitignore file as technically it could be copying files into the local build workspace that it actually shouldn't and therefore when it actually runs on Google cloud build it wouldn't actually work. This would then naturally speed up builds for projects with large dependencies and increase the testability of those build scripts before deploying them.

--
You received this message because you are subscribed to a topic in the Google Groups "Google Cloud Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-dev/PncgZecL1UA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-cloud-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-dev/9b4ffb95-69f9-4a65-a644-5620f3aaad4e%40googlegroups.com.

[Tiago (Google Cloud Platform Support)]

Hello Jack,

This has been acknowledged by the team responsible for the `cloud-build-local` repository [1] as an open issue. I'd suggest to add any relevant information and findings you might have gathered from your specific use cases in that thread.

[1] https://github.com/GoogleCloudPlatform/cloud-build-local/issues/84

On Monday, May 13, 2019 at 2:12:56 PM UTC-4, Jack Woodward wrote:

I understand that it will he slowly and that's fine as it's just for testing but I think it would be useful for the local build emulator to respect the gcloudignore or gitignore file as technically it could be copying files into the local build workspace that it actually shouldn't and therefore when it actually runs on Google cloud build it wouldn't actually work. This would then naturally speed up builds for projects with large dependencies and increase the testability of those build scripts before deploying them.

On Sun, 12 May 2019, 20:03 'George (Cloud Platform Support)' via Google Cloud Developers, <google-cloud-dev@googlegroups.com> wrote:

Hello Jack, 

Local Cloud Build emulator runs on one machine, it is slower than proper Cloud Build, which is scalable, and works so as to minimize run time. 

You might be right about the copy operation you noticed, but .gcloudignore is not expected to help here. For one, this feature is meant to reduce traffic from your development machine to Cloud, not to prevent local writes from one directory to another. Also, commands are different: cloud-build-local vs. gcloud builds submit. In what concerns the local Cloud Build emulator, the command pattern is: 

cloud-build-local --config=[BUILD_CONFIG] --dryrun=false --push [SOURCE_CODE]

You may find related details on the "Building and debugging locally" page. 

The .gcloudignore feature works with the gcloud builds submit command. If the source is a local directory, this command skips the files specified in the .gcloudignore file. If a .gitignore file is present in the local source directory, gcloud will use a Git-compatible .gcloudignore file that respects your .gitignored files. The global .gitignore is not respected. This feature is provided with the reduction of Internet traffic in mind, and to avoid unnecessary costs. 

--
You received this message because you are subscribed to a topic in the Google Groups "Google Cloud Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-dev/PncgZecL1UA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to google-cloud-dev+unsubscribe@googlegroups.com.