VM cloning

[Dail Software]

Hello.

I'm used to KVM, but my enterprise decided to use GCP because they say it is safer... Hah.

At the moment they're reconfiguring every instance from scratch: they create an ampty vm, add disks, mount them in place, install software inside newly mounted disks, configure software and start operations. However, I want to make a template (I'm not sure if it matches with Google's template definition) of a Debian with n disks, one virtual disk per service and an extra for backups. The system should pop-up with all the applications installed and configured for defaults. Those applicationds are usually part of the repositories, so an apt-clone would suffice, however, others are not, so I'd need custom scripts to make deployments from scratch. I'll make one template per machine type, so the number of disks will be the same for all the instances made from a single template. This is straightforward in KVM, and I'm kinda mad about Google not offering this after the crazy prices they're charging us.

What I want is a way to "clone" my VM, tell Google "make a copy" and obtain a cloned machine with the UUIDs, hostname and so on altered to match Google configs to be able to get ssh keys and st*pid stuff like that automatically (my boss seems to like to have the ability to put a ssh key from the web interface -.-). also, I like to leave Rnapshot and the backup stuff configured BEFORE deployment to avoid forgetting it, so what I really want is a clone of the system which get "sysprep'd" (or linux equivalent on the first run).

They say there's a clone button "create similar", but thats not cloning, thats just copying the d*** machine definition file to a new location and create a new disk... Its just insulting my intelligence xD.

PD: Sorry for my mood, but I'm so angry that such an expensive service does, after all, perform worse than doing the same on local used servers >.<

I want to be able to make it across projects, of course.

[Justin Reiners]

I do this all the time, https://cloud.google.com/sdk/gcloud/reference/compute/disks/snapshot

you can do it in the gui by going to compute engine, disks, open the disk, and click create snapshot.

then go to compute engine snapshots, and create a new VM from the snapshot. It's identical to the last machine, just a different IP and hostname. I also have this scripted to run every night, once a week, and once a month so I have daily, weekly, and monthly backups of everything. If I need to manually scale, I just create machines from last nights snapshot.

Take a look at the documentation, It's easily scriptable as well.

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/648e5abe-cd08-43de-a377-8ffc0cdc86e1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Larbi (Google Cloud Support)]

Hi,

You can create images from the VM and you can share this image by just granting role "compute.imageUser" to another user from another project to access, and not just that, you can start Sharing Images, Snapshots, and Disks across project and organization.

The first Snapshot is full copy, it a back up data from your persistent disks.there are different from public images and custom images, which are used primarily to create instances or configure instance templates. Snapshots are incremental and automatically compressed.

You can use instance templates to create VM instances and managed instance groups. Instance templates define the machine type, boot disk image or container image, zone, labels, and other instance properties. This will help you quickly create VM instances based off of a preexisting configuration.

[Paul Nash]

In addition to the recommendations posted here about using Snapshots or Instance Templates, if you need to do special per-instance configuration to change settings or something, you could use a startup script to run commands the first time the new instance is booted, or if there are only a few things, you could set them as VM instance metadata and have the VM read from that (or combine these).

--

© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/ef280c72-99c9-4182-a3b6-5ed416fd8c92%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Paul R. Nash |

Group Product Manager, Compute Engine |

paul...@google.com |

206-876-1620

[Dail Software]

I managed to achieve something, thanks to your advice.

To prepare a VM with 4 disks for deployment across projects you need:

1.- Authorized user. Mine is "owner", so I haven't tested myself, but they say below that the required permission is "compute.imageUser".

2.- One "disk image" (Google Image Console) per disk you want to create in your machine. Keep in mind Google I/O polytics when designing your layout, however, using one disk per pertition apparently allows google to automatically expand partitions on reboot when more disk space is assigned. It may be worth giving it a try if you're not very confident with CLI partition tools. Disks images are available across your projects.

3.- One instance template (per project?), configured with all the disks and images on their correct possition. I've not tested fstab behaviour yet, but it will probably crash and require manual remounting the first time: gotta look at it.

4.- Patience: The web interface doesnt allow you to add additional disks from other projects, however, CGP itself allows it, so you need to use beta gcloud commands:

gcloud beta compute --project=%proyecto% instance-templates create %nametem% --machine-type=n1-standard-1 --network=projects/%proyecto%/global/networks/default --network-tier=PREMIUM --maintenance-policy=MIGRATE --service-account=%servacc% --scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append --tags=http-server,https-server --image=debian9-webservices-os-1 --image-project=%proyectoImgs% --boot-disk-size=20GB --no-boot-disk-auto-delete --boot-disk-type=pd-standard --boot-disk-device-name=nombre-template --create-disk=mode=rw,size=10,image=projects/%proyectoImgs%/global/images/debian9-webservices-backup-1 --create-disk=mode=rw,size=10,image=projects/%proyectoImgs%/global/images/debian9-webservices-apache-1 --create-disk=mode=rw,size=10,image=projects/%proyectoImgs%/global/images/debian9-webservices-mysql-1 --create-disk=mode=rw,size=10,image=projects/%proyectoImgs%/global/images/debian9-webservices-tomcat-1

I made it with windows variables for clarity:

%proyecto% is the proyect where you want to create the template

%nametem% is the template's name

%proyectoImgs% is the project where disk images are stored. Obviously you'll need to change "debian9-webservices-tomcat-1" and other names to your image names.

%servacc% is the google services account.

This is just an example command. If you want to build a complet template via GUI, I recommend you to select all the template options in the same project that the images reside, then, before submitting, use "equivalent console command" button to get your full command and customice it to point to the other project.