Cannot read proc file system since 31/1

Anita Wong

unread,

Feb 4, 2018, 2:23:29 AM2/4/18

to gce-discussion

Hi, starting from 31/1, I started seeing below logs in 4 of my VMs (out of total 40).

Jan 31 14:08:07 web-dev-public dhclient[1185]: DHCPREQUEST on eth0 to 169.254.169.254 port 67 (xid=0x466eef7d)

Jan 31 14:08:07 web-dev-public dhclient[1185]: DHCPACK from 169.254.169.254 (xid=0x466eef7d)

Jan 31 14:08:09 web-dev-public kernel: Kernel logging (proc) stopped.

Jan 31 14:08:09 web-dev-public rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1260" x-info="http://www.rsyslog.com"] exiting on signal 15.

Jan 31 14:08:10 web-dev-public kernel: imklog 5.8.10, log source = /proc/kmsg started.

Jan 31 14:08:10 web-dev-public rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="26361" x-info="http://www.rsyslog.com"] start

Jan 31 14:08:10 web-dev-public kernel: Cannot read proc file system: 9 - Bad file descriptor.

........

The log "kernel: Cannot read proc file system: 9 - Bad file descriptor." will then appear continuously with a frequency that the disk space will be used up in few days

I've tried to reboot the VMs but then this will appear again after a day or so.

Those 4 VMs are serving different applications and from the first line of the log, it should be receiving some commands from metadata.google.internal (169.254.169.254) to start with.

All these happens at 31/1 around 12pm(+8).

The VMs are all "CentOS release 6.9 (Final)".

Please help as some VMs are hosting services for our client and I cannot always reboot them.

Thanks a lot.

Carlos (Cloud Platform Support)

unread,

Feb 4, 2018, 1:32:56 PM2/4/18

to gce-discussion

Hi Anita,

We have received some reports of VMs running CentOS 6 having such behavior. Our backend team is currently investigating. In the meantime the only workaround I have seen worked is restarting affected services like "rsyslogd".

I will check for updates during the next days.

Evgeny Inberg

unread,

Feb 4, 2018, 2:19:23 PM2/4/18

to gce-discussion

We have the same issue during the last week or so.

lawrence pan

unread,

Feb 5, 2018, 9:00:59 AM2/5/18

to gce-discussion

Hi Carlos ,

Have any update ?

Carlos (Cloud Platform Support)於 2018年2月5日星期一 UTC+8上午2時32分56秒寫道：

Carlos (Cloud Platform Support)

unread,

Feb 5, 2018, 1:07:02 PM2/5/18

to gce-discussion

Thank you all for your reports.

Our engineering team is working with high priority this issue. You can follow up for updates in this public report.

Carlos (Cloud Platform Support)

unread,

Feb 6, 2018, 4:07:39 PM2/6/18

to gce-discussion

Thank you for your patience on this issue. An update package was released yesterday, please check the public issue for details.

Patrik Majer

unread,

Feb 13, 2018, 10:35:11 AM2/13/18

to gce-discussion

Hi,

we have same issues on several GCP servers..
Can you check this server? 35.187.36.203

Many thanks,
Patrik

Strauss Bornman

unread,

Feb 26, 2018, 10:20:22 AM2/26/18

to gce-discussion

We seem to be having the same issue a bit later, is this issue been resolved/

On Sunday, February 4, 2018 at 9:23:29 AM UTC+2, Anita Wong wrote:

Justin Reiners

unread,

Feb 28, 2018, 6:22:40 PM2/28/18

to gce-discussion

I am also seeing this error for the last few days, it's filled my drive a couple times now, anything we can do? I'm in US-CENTRAL1-A for most of my instances.

Reply all

Reply to author

Forward