docker ps showing only one docker is up and running. [sonic 202012]

[balaji]

Hi,

We are trying multiple combinations in config_db.json.

During this time dockers get killed.

In /var/log/ directory checked "syslog" file, could not get valid reason.

Observed logs that all dockers get killed by calling pid.

On facing this issue, show version, reboot, config reload -y or any other won't work.

Log Snippet:

=========== 

observed similar logs for all dockers

 ERR monit[620]: 'bgp|bgpmon' status failed (9) -- nn

o output

Mar 28 22:42:02.945103 sonic INFO dhclient[1651]: XMT: Solicit on eth0, intervall

 128680ms.

Mar 28 22:43:23.348441 sonic ERR monit[620]: 'telemetry|telemetry' program timedd

 out after 5.017 m. Killing program with pid 25693

Mar 28 22:43:23.349676 sonic ERR monit[620]: 'telemetry|telemetry' status failedd

 (9) -- no output

Mar 28 22:43:23.351129 sonic ERR monit[620]: 'telemetry|dialout_client' program

timed out after 5.017 m. Killing program with pid 25694

Mar 28 22:43:23.354139 sonic ERR monit[620]: 'telemetry|dialout_client' status ff

ailed (9) -- no output

&

Mar 27 03:22:56 sonic systemd-sysctl[402]: Couldn't write '1' to 'kernel/panic_oo

n_stackoverflow', ignoring: No such file or directory

Mar 27 03:22:56 sonic systemd-sysctl[402]: Couldn't write '1' to 'kernel/panic_oo

n_stackoverflow', ignoring: No such file or directory

root@sonic:/# docker ps

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS               NAMES

4c9b23bf0d78        docker-database:latest   "/usr/local/bin/dock…"   43 hours ago        Up 43 hours                             database

root@sonic:/#

Regards,

Balaji

[balaji]

SONiC image details

root@sonic:~# show version

SONiC Software Version: SONiC.202012.0-c65a8a22

Distribution: Debian 10.9

Kernel: 4.19.0-12-2-amd64

Build commit: c65a8a22

Build date: Sun May  2 14:30:55 UTC 2021

Built by: balaji@sfstypc01

Platform: x86_64-accton_as7716_32x-r0

HwSKU: Accton-AS7716-32X

ASIC: broadcom

ASIC Count: 1

Serial Number: 771632X1716006

Uptime: 02:11:38 up  1:10,  1 user,  load average: 0.09, 0.40, 0.59

Regards,

Balaji

[zhenghui cai]

You might need to share the log files and check if there is any crash.

--
You received this message because you are subscribed to the Google Groups "sonicproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonicproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonicproject/be69b1a3-9c50-4e21-a0b6-c96a98d48aean%40googlegroups.com.

[balaji]

Attaching logs.

We are not able to see exact reason for crash in syslog file.

Apr  5 22:33:39.976101 sonic INFO bgp#supervisord 2021-04-05 22:33:39,975 INFO stopped: fpmsyncd (terminated by SIGTERM)

Apr  5 22:33:39.978274 sonic INFO bgp#supervisord 2021-04-05 22:33:39,977 INFO stopped: bgpmon (terminated by SIGTERM