Issue in Scalar server instance is in "resuming" status

[Rajeev Gangur]

Hello 

the Scalar server instance is in "resuming" status continuously ..for almost half day to a day.. it does not enable pause or resume option..Can somebody help how it can be made running status again?

Regards

Rajeev

[Marc O'Brien]

Hi Rajeev,

More details are needed to begin looking in to this issue.  What version of Scalr are you using? What Cloud Provider, and what steps are taken leading up to this behavior?  Are instances showing as "Running" in the Cloud Provider dashboard when Scalr shows them as "Resuming" or are they showing the same status there as well?  Scalarizr agent logs will be useful as well if you can upload them here.

Many thanks,
Wm. Marc O'Brien
Scalr Technical Support

[Matthieu Serrepuy]

Hi !

I am having the exact same behavior.
We are using Azure as provider and some times, some instances switch to "Resuming" state.
I couldn't find anything obvious in the scalarizr logs on my own.

Looking at tcpdump, I couldn't find traffic between our scalr instance and the problematic one.

I was not paying so much attention to this, but it seems it prevents the auto-scaling to work properly.

Can I have assistance on how to troubleshoot this issue ? Next time it happens I'll be glad to provide logs or traces to fix this issue for good.

Thanks,

Matt

[ryann...@pingidentity.com]

We have this problem as well. frequently it's just an issue with the scalrizer agent not being able to start. either from uncaught python exceptions to the update client not being able to resolve the url to the repo etc. for most of our issues bypassing scalr and getting credentials from AWS/Azure and manually updating the scalrizer agent fixes the issue, In most of these instances the scalrizer logs are unavailable via scalr since the update agent can't start the scalrizer agent and so there is no logs passed to the admin node.

[Matthieu Serrepuy]

Hi !

I am still looking for a way to solve/mitigate the issue when it is happening.
I Used to terminate the instance to solve the issue (as it blocks the scaling) but it seems a reboot solve it. So it is probably a mater of restarting a good service but I can't find the good one or the good thing to trigger to re-schedule a connection :(

Anybody else is having this issue and is able to solve it without downtime ?

[Matthieu Serrepuy]

Hi !

As it seems to be caused by the agent upgrade, what is the best way to disable agent auto-update (I'd rather upgrade them one by one and choose when I'll upgrade them) ?
I supposed I should set scalr.scalarizr_update.mode, but not sure which value I should choose between solo and client.

Anybody has a clear explanation about the difference between these two modes ?

Thanks,

Regards,

Matt


[matthieu...@gmail.com]

One of my instances stucked in 'resuming' state was using scalarizr 5.2.11.
I upgraded it to 5.4 manually (with sudo apt-get install scalarizr).

The instance is still in resuming state but when I go to "Extented instance information" > "Scalr Internal Properties", I can see that scalr.version property has been updated to 5.4 so I guess it means the agent and my scalr server are able to communicate, so I would exclude a network cause.

I am still not able to go back to a normal state without a reboot, which is pretty annoying.

Matt

[Igor Savchenko]

Hi,

Can you please send me full scalarizr logs: /var/log/scalarizr_*.log files.

Thanks,

Igor

--
You received this message because you are subscribed to the Google Groups "scalr-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalr-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matthieu Serrepuy]

$ 1 08:56:33 mserrepuy@xxxxxxxxx:~$ dpkg -l|grep scalar
ii  scalarizr                          5.2.2-1                           amd64        The full stack of scalarizr
ii  scalarizr-azure                    5.2.2-1                           amd64        no description given
$ 0 08:57:44 mserrepuy@xxxxxxxx:~$ sudo apt-get install scalarizr
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following additional packages will be installed:
  scalarizr-azure
The following packages will be upgraded:
  scalarizr scalarizr-azure
2 upgraded, 0 newly installed, 0 to remove and 151 not upgraded.
Need to get 64.5 MB of archives.
After this operation, 1170 kB disk space will be freed.
Do you want to continue? [Y/n] Y
Get:1 http://repo.scalr.net/apt-plain stable/ scalarizr-azure 5.4.0-1 [1022 B]
Get:2 http://repo.scalr.net/apt-plain stable/ scalarizr 5.4.0-1 [64.5 MB]
Fetched 64.5 MB in 6s (10.3 MB/s)                                                                                                     
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LC_TIME = "en_US.UTF-8",
        LC_MONETARY = "fr_FR.UTF-8",
        LC_CTYPE = "en_US.UTF-8",
        LC_ADDRESS = "fr_FR.UTF-8",
        LC_TELEPHONE = "fr_FR.UTF-8",
        LC_NAME = "fr_FR.UTF-8",
        LC_MEASUREMENT = "fr_FR.UTF-8",
        LC_IDENTIFICATION = "fr_FR.UTF-8",
        LC_NUMERIC = "fr_FR.UTF-8",
        LC_PAPER = "fr_FR.UTF-8",
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
locale: Cannot set LC_ALL to default locale: No such file or directory
(Reading database ... 108015 files and directories currently installed.)
Preparing to unpack .../scalarizr-azure_5.4.0-1_amd64.deb ...
Unpacking scalarizr-azure (5.4.0-1) over (5.2.2-1) ...
Preparing to unpack .../scalarizr_5.4.0-1_amd64.deb ...
You're about to install scalarizr!
Unpacking scalarizr (5.4.0-1) over (5.2.2-1) ...
scalarizr has been upgraded!
Setting up scalarizr (5.4.0-1) ...
Thank you for installing scalarizr!
Setting up scalarizr-azure (5.4.0-1) ...
$ 0 08:59:31 mserrepuy@xxxxxxxx:~$ dpkg -l|grep scalar
ii  scalarizr                          5.4.0-1                           amd64        The full stack of scalarizr
ii  scalarizr-azure                    5.4.0-1                           amd64        no description given

Attached the logs before and after the upgrade.

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to scalr-discus...@googlegroups.com.

[Matthieu Serrepuy]

Hi !

News from my issue (that still occurs after a lot of tries).
I finally found this in the scalr php-cloud_poller.log :

service/php-cloud_poller.log:Jun 21 14:05:07 +00:00 - FarmLog@59840 - WARN - [FarmID: 5] Server '88a29a62-0afb-4363-a1f1-fce01eaebff9' (Platform: azure) is not running (Status in cloud: succeeded/stopping, Status in Scalr: Running).

I am looking at the Azure consule and I can see an event called "Write VirtualMachines" in status "Succeed" so I guess this is the one creating my issue.

I am contacting Azure to see what is this job but in the meantime, is there any change in my setup/config to ignore these for now ?

Thanks,

Matt