svc:/network/rpc/bind:default in maintenance mode - Solaris 10
underh20
of services are not starting up. The zone is up and running but we
can't login remotely via SSH. Any idea how to resolve it.
Thx, Bill
# svcs -vx
svc:/network/rpc/bind:default (RPC bindings)
State: maintenance since Tue Sep 06 23:36:25 2011
Reason: Start method failed repeatedly, last exited with status 1.
See: http://sun.com/msg/SMF-8000-KS
See: man -M /usr/share/man -s 1M rpcbind
See: /var/svc/log/network-rpc-bind:default.log
Impact: 15 dependent services are not running:
svc:/network/rpc/gss:default
svc:/network/nfs/client:default
svc:/system/filesystem/autofs:default
svc:/network/rpc/rstat:default
svc:/network/nfs/nlockmgr:default
svc:/network/nfs/status:default
svc:/network/nfs/cbd:default
svc:/network/nfs/mapid:default
svc:/application/management/dmi:default
svc:/network/rpc/cde-ttdbserver:tcp
svc:/network/rpc/smserver:default
svc:/system/filesystem/volfs:default
svc:/network/rpc/rusers:default
svc:/network/nfs/rquota:default
svc:/network/rpc/cde-calendar-manager:default
svc:/application/print/server:default (LP print server)
State: disabled since Tue Sep 06 23:36:14 2011
Reason: Disabled by an administrator.
See: http://sun.com/msg/SMF-8000-05
See: man -M /usr/share/man -s 1M lpsched
Impact: 2 dependent services are not running:
svc:/application/print/rfc1179:default
svc:/application/print/ipp-listener:default
svc:/application/management/snmpdx:default (Sun Solstice Enterprise
Master Agent)
State: disabled since Tue Sep 06 23:36:14 2011
Reason: Disabled by an administrator.
See: http://sun.com/msg/SMF-8000-05
See: man -M /usr/share/man/ -s 1M snmpdx
Impact: 1 dependent service is not running:
svc:/application/management/dmi:default
svc:/application/management/sma:default (net-snmp SNMP daemon)
State: maintenance since Tue Sep 06 23:36:27 2011
Reason: Start method failed repeatedly, last exited with status 1.
See: http://sun.com/msg/SMF-8000-KS
See: man -M /usr/share/man/ -s 1M snmpd
See: /var/svc/log/application-management-sma:default.log
Impact: This service is not running.
svc:/network/security/ktkt_warn:default (Kerberos V5 warning messages
daemon)
State: maintenance since Tue Sep 06 23:36:33 2011
Reason: Restarter svc:/network/inetd:default gave no explanation.
See: http://sun.com/msg/SMF-8000-9C
See: man -M /usr/share/man -s 1M ktkt_warnd
Impact: This service is not running.
svc:/network/rpc-100235_1/rpc_ticotsord:default (100235)
State: maintenance since Tue Sep 06 23:36:33 2011
Reason: Restarter svc:/network/inetd:default gave no explanation.
See: http://sun.com/msg/SMF-8000-9C
Impact: This service is not running.
svc:/system/webconsole:console (java web console)
State: offline since Wed Sep 07 00:07:48 2011
Reason: Start method is running.
See: http://sun.com/msg/SMF-8000-C4
See: man -M /usr/share/man -s 1M smcwebserver
See: /var/svc/log/system-webconsole:console.log
Impact: This service is not running.
svc:/application/stosreg:default (Service Tag OS Registry Inserter)
State: maintenance since Tue Sep 06 23:36:28 2011
Reason: Start method failed repeatedly, last exited with status 1.
See: http://sun.com/msg/SMF-8000-KS
See: man -M /usr/share/man -s 1M stclient
See: /var/svc/log/application-stosreg:default.log
Impact: This service is not running.
svc:/application/font/stfsloader:default (Standard Type Services
Framework (STSF) Font Server loader)
State: maintenance since Tue Sep 06 23:36:32 2011
Reason: Restarter svc:/network/inetd:default gave no explanation.
See: http://sun.com/msg/SMF-8000-9C
Impact: This service is not running.
Ian Collins
> Our solaris 10 panic and rebooted. One of the zones came up but a lot
> of services are not starting up. The zone is up and running but we
> can't login remotely via SSH. Any idea how to resolve it.
> Thx, Bill
>
> # svcs -vx
> svc:/network/rpc/bind:default (RPC bindings)
> State: maintenance since Tue Sep 06 23:36:25 2011
> Reason: Start method failed repeatedly, last exited with status 1.
> See: http://sun.com/msg/SMF-8000-KS
> See: man -M /usr/share/man -s 1M rpcbind
> See: /var/svc/log/network-rpc-bind:default.log
What does the log say?
--
Ian Collins
underh20
# cat /var/svc/log/network-rpc-bind:default.log
[ Jan 14 06:53:07 Enabled. ]
[ Jan 14 06:53:48 Rereading configuration. ]
[ Jan 14 06:56:27 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 14 06:56:27 Method "start" exited with status 0 ]
[ Jan 14 10:00:14 Enabled. ]
[ Jan 14 10:00:26 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 14 10:00:26 Method "start" exited with status 0 ]
[ Jan 14 10:38:46 Enabled. ]
[ Jan 14 10:38:57 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 14 10:38:57 Method "start" exited with status 0 ]
[ Jan 14 10:43:50 Stopping because service disabled. ]
[ Jan 14 10:43:50 Executing stop method ("/lib/svc/method/rpc-bind stop 170629") ]
[ Jan 14 10:43:55 Method "stop" exited with status 0 ]
[ Jan 14 10:51:43 Enabled. ]
[ Jan 14 10:51:55 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 14 10:51:55 Method "start" exited with status 0 ]
[ Jan 14 10:57:11 Stopping because service disabled. ]
[ Jan 14 10:57:11 Executing stop method ("/lib/svc/method/rpc-bind stop 1996") ]
[ Jan 14 10:57:16 Method "stop" exited with status 0 ]
[ Jan 14 10:57:45 Enabled. ]
[ Jan 14 10:57:57 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 14 10:57:57 Method "start" exited with status 0 ]
[ Jan 14 11:00:20 Stopping because service disabled. ]
[ Jan 14 11:00:20 Executing stop method ("/lib/svc/method/rpc-bind stop 171149") ]
[ Jan 14 11:00:25 Method "stop" exited with status 0 ]
[ Jan 14 11:00:58 Enabled. ]
[ Jan 14 11:01:10 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 14 11:01:10 Method "start" exited with status 0 ]
[ Jan 23 07:31:48 Stopping because service disabled. ]
[ Jan 23 07:31:48 Executing stop method ("/lib/svc/method/rpc-bind stop 2424") ]
[ Jan 23 07:31:53 Method "stop" exited with status 0 ]
[ Jan 23 11:30:18 Enabled. ]
[ Jan 23 11:30:30 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 23 11:30:30 Method "start" exited with status 0 ]
[ Jan 23 12:00:11 Enabled. ]
[ Jan 23 12:00:24 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 23 12:00:24 Method "start" exited with status 0 ]
[ Jan 23 12:10:17 Enabled. ]
[ Jan 23 12:10:29 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 23 12:10:29 Method "start" exited with status 0 ]
[ Jan 23 12:32:55 Stopping because service disabled. ]
[ Jan 23 12:32:56 Executing stop method ("/lib/svc/method/rpc-bind stop 1684") ]
[ Jan 23 12:33:01 Method "stop" exited with status 0 ]
[ Jan 23 12:33:34 Enabled. ]
[ Jan 23 12:33:45 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 23 12:33:45 Method "start" exited with status 0 ]
[ Jan 23 12:42:26 Stopping because service disabled. ]
[ Jan 23 12:42:27 Executing stop method ("/lib/svc/method/rpc-bind stop 1944") ]
[ Jan 23 12:42:32 Method "stop" exited with status 0 ]
[ Jan 23 12:43:04 Enabled. ]
[ Jan 23 12:43:17 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jan 23 12:43:17 Method "start" exited with status 0 ]
[ Feb 2 14:25:10 Stopping because service disabled. ]
[ Feb 2 14:25:10 Executing stop method ("/lib/svc/method/rpc-bind stop 2554") ]
[ Feb 2 14:25:15 Method "stop" exited with status 0 ]
[ Feb 3 12:22:09 Enabled. ]
[ Feb 3 12:22:21 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 12:22:22 Method "start" exited with status 0 ]
[ Feb 3 12:26:50 Enabled. ]
[ Feb 3 12:27:02 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 12:27:02 Method "start" exited with status 0 ]
[ Feb 3 12:28:49 Enabled. ]
[ Feb 3 12:29:00 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 12:29:00 Method "start" exited with status 0 ]
[ Feb 3 12:33:00 Stopping because service disabled. ]
[ Feb 3 12:33:00 Executing stop method ("/lib/svc/method/rpc-bind stop 85339") ]
[ Feb 3 12:33:05 Method "stop" exited with status 0 ]
[ Feb 3 12:33:39 Enabled. ]
[ Feb 3 12:33:51 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 12:33:51 Method "start" exited with status 0 ]
[ Feb 3 14:21:38 Stopping because service disabled. ]
[ Feb 3 14:21:38 Executing stop method ("/lib/svc/method/rpc-bind stop 55209") ]
[ Feb 3 14:21:44 Method "stop" exited with status 0 ]
[ Feb 3 14:22:15 Enabled. ]
[ Feb 3 14:22:27 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 14:22:27 Method "start" exited with status 0 ]
[ Feb 3 14:32:32 Stopping because service disabled. ]
[ Feb 3 14:32:32 Executing stop method ("/lib/svc/method/rpc-bind stop 86360") ]
[ Feb 3 14:32:37 Method "stop" exited with status 0 ]
[ Feb 3 14:33:08 Enabled. ]
[ Feb 3 14:33:20 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 14:33:21 Method "start" exited with status 0 ]
[ Feb 3 15:27:39 Stopping because service disabled. ]
[ Feb 3 15:27:40 Executing stop method ("/lib/svc/method/rpc-bind stop 56216") ]
[ Feb 3 15:27:45 Method "stop" exited with status 0 ]
[ Feb 3 15:28:18 Enabled. ]
[ Feb 3 15:28:29 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 15:28:29 Method "start" exited with status 0 ]
[ Feb 3 16:04:13 Stopping because service disabled. ]
[ Feb 3 16:04:13 Executing stop method ("/lib/svc/method/rpc-bind stop 86901") ]
[ Feb 3 16:04:18 Method "stop" exited with status 0 ]
[ Feb 3 16:04:50 Enabled. ]
[ Feb 3 16:05:03 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Feb 3 16:05:03 Method "start" exited with status 0 ]
[ May 9 20:38:57 Stopping because service disabled. ]
[ May 9 20:38:57 Executing stop method ("/lib/svc/method/rpc-bind stop 56873") ]
[ May 9 20:39:02 Method "stop" exited with status 0 ]
[ May 9 20:39:44 Enabled. ]
[ May 9 20:40:00 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ May 9 20:40:00 Method "start" exited with status 0 ]
[ May 15 23:56:49 Stopping because service disabled. ]
[ May 15 23:56:49 Executing stop method ("/lib/svc/method/rpc-bind stop 796051") ]
[ May 15 23:56:54 Method "stop" exited with status 0 ]
[ May 15 23:57:33 Enabled. ]
[ May 15 23:57:46 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ May 15 23:57:46 Method "start" exited with status 0 ]
[ May 28 03:01:27 Stopping because service disabled. ]
[ May 28 03:01:28 Executing stop method ("/lib/svc/method/rpc-bind stop 624421") ]
[ May 28 03:01:33 Method "stop" exited with status 0 ]
[ May 29 22:25:33 Enabled. ]
[ May 29 22:25:46 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ May 29 22:25:46 Method "start" exited with status 0 ]
[ May 29 22:49:51 Stopping because service disabled. ]
[ May 29 22:49:51 Executing stop method ("/lib/svc/method/rpc-bind stop 2248") ]
[ May 29 22:49:56 Method "stop" exited with status 0 ]
[ May 29 23:52:49 Enabled. ]
[ May 29 23:53:02 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ May 29 23:53:02 Method "start" exited with status 0 ]
[ May 30 00:06:58 Stopping because service disabled. ]
[ May 30 00:06:58 Executing stop method ("/lib/svc/method/rpc-bind stop 2781") ]
[ May 30 00:07:04 Method "stop" exited with status 0 ]
[ May 30 00:07:35 Enabled. ]
[ May 30 00:07:48 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ May 30 00:07:48 Method "start" exited with status 0 ]
[ Jun 3 15:36:58 Stopping because service disabled. ]
[ Jun 3 15:36:58 Executing stop method ("/lib/svc/method/rpc-bind stop 1351") ]
[ Jun 3 15:37:03 Method "stop" exited with status 0 ]
[ Jun 3 15:37:44 Enabled. ]
[ Jun 3 15:37:55 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 3 15:37:55 Method "start" exited with status 0 ]
[ Jun 6 19:36:38 Stopping because service disabled. ]
[ Jun 6 19:36:38 Executing stop method ("/lib/svc/method/rpc-bind stop 18963") ]
[ Jun 6 19:36:43 Method "stop" exited with status 0 ]
[ Jun 6 19:37:22 Enabled. ]
[ Jun 6 19:37:38 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 6 19:37:39 Method "start" exited with status 0 ]
[ Jun 6 19:46:15 Stopping because service disabled. ]
[ Jun 6 19:46:15 Executing stop method ("/lib/svc/method/rpc-bind stop 72796") ]
[ Jun 6 19:46:20 Method "stop" exited with status 0 ]
[ Jun 6 19:47:02 Enabled. ]
[ Jun 6 19:47:13 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 6 19:47:13 Method "start" exited with status 0 ]
[ Jun 7 09:53:07 Stopping because service disabled. ]
[ Jun 7 09:53:07 Executing stop method ("/lib/svc/method/rpc-bind stop 38286") ]
[ Jun 7 09:53:12 Method "stop" exited with status 0 ]
[ Jun 7 09:53:40 Enabled. ]
[ Jun 7 09:53:52 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 7 09:53:52 Method "start" exited with status 0 ]
[ Jun 7 09:56:39 Enabled. ]
[ Jun 7 09:56:51 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 7 09:56:51 Method "start" exited with status 0 ]
[ Jun 7 13:42:13 Enabled. ]
[ Jun 7 13:42:25 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 7 13:42:25 Method "start" exited with status 0 ]
[ Jun 7 13:44:37 Stopping because service disabled. ]
[ Jun 7 13:44:37 Executing stop method ("/lib/svc/method/rpc-bind stop 44490") ]
[ Jun 7 13:44:42 Method "stop" exited with status 0 ]
[ Jun 7 13:45:11 Enabled. ]
[ Jun 7 13:45:22 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 7 13:45:22 Method "start" exited with status 0 ]
[ Jun 19 07:36:23 Enabled. ]
[ Jun 19 07:36:36 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 19 07:36:36 Method "start" exited with status 0 ]
[ Jun 19 07:39:22 Stopping because service disabled. ]
[ Jun 19 07:39:22 Executing stop method ("/lib/svc/method/rpc-bind stop 114069") ]
[ Jun 19 07:39:27 Method "stop" exited with status 0 ]
[ Jun 19 07:40:01 Enabled. ]
[ Jun 19 07:40:14 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 19 07:40:14 Method "start" exited with status 0 ]
[ Jun 20 15:07:38 Enabled. ]
[ Jun 20 15:07:58 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 15:07:58 Method "start" exited with status 0 ]
[ Jun 20 15:15:46 Enabled. ]
[ Jun 20 15:16:00 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 15:16:00 Method "start" exited with status 0 ]
[ Jun 20 17:57:22 Enabled. ]
[ Jun 20 17:57:37 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 17:57:37 Method "start" exited with status 0 ]
[ Jun 20 18:01:06 Enabled. ]
[ Jun 20 18:01:19 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 18:01:19 Method "start" exited with status 0 ]
[ Jun 20 18:09:17 Enabled. ]
[ Jun 20 18:09:28 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 18:09:28 Method "start" exited with status 0 ]
[ Jun 20 18:17:32 Enabled. ]
[ Jun 20 18:17:52 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 18:17:52 Method "start" exited with status 0 ]
[ Jun 20 18:25:34 Enabled. ]
[ Jun 20 18:25:45 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 18:25:45 Method "start" exited with status 0 ]
[ Jun 20 19:14:17 Enabled. ]
[ Jun 20 19:14:29 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 19:14:29 Method "start" exited with status 0 ]
[ Jun 20 19:22:23 Enabled. ]
[ Jun 20 19:22:36 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 19:22:36 Method "start" exited with status 0 ]
[ Jun 20 19:30:35 Enabled. ]
[ Jun 20 19:30:47 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 19:30:48 Method "start" exited with status 0 ]
[ Jun 20 19:38:43 Enabled. ]
[ Jun 20 19:38:55 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 19:38:55 Method "start" exited with status 0 ]
[ Jun 20 20:21:21 Enabled. ]
[ Jun 20 20:21:33 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 20:21:33 Method "start" exited with status 0 ]
[ Jun 20 20:26:43 Stopping because service disabled. ]
[ Jun 20 20:26:43 Executing stop method ("/lib/svc/method/rpc-bind stop 281") ]
[ Jun 20 20:26:48 Method "stop" exited with status 0 ]
[ Jun 20 20:27:22 Enabled. ]
[ Jun 20 20:27:35 Executing start method ("/lib/svc/method/rpc-bind start") ]
[ Jun 20 20:27:35 Method "start" exited with status 0 ]
[ Sep 6 23:33:27 Enabled. ]
[ Sep 6 23:34:43 Enabled. ]
[ Sep 6 23:36:13 Enabled. ]
[ Sep 6 23:36:24 Executing start method ("/lib/svc/method/rpc-bind start") ]
rpcbind failed with 1.
[ Sep 6 23:36:24 Method "start" exited with status 1 ]
[ Sep 6 23:36:24 Executing start method ("/lib/svc/method/rpc-bind start") ]
rpcbind failed with 1.
[ Sep 6 23:36:24 Method "start" exited with status 1 ]
[ Sep 6 23:36:25 Executing start method ("/lib/svc/method/rpc-bind start") ]
rpcbind failed with 1.
[ Sep 6 23:36:25 Method "start" exited with status 1 ]
[ Sep 6 23:46:04 Rereading configuration. ]
#
Ian Collins
[Context is good!]
> here's what the log shows. Any idea how we could resolve this. Thx
>
> [ Sep 6 23:36:24 Method "start" exited with status 1 ]
> [ Sep 6 23:36:25 Executing start method ("/lib/svc/method/rpc-bind start") ]
> rpcbind failed with 1.
> [ Sep 6 23:36:25 Method "start" exited with status 1 ]
> [ Sep 6 23:46:04 Rereading configuration. ]
> #
1 is EPERM, so it looks like you have a file permission problem.
Try running rpcbind by hand (under truss if it helps) and see what happens.
--
Ian Collins
underh20
# /usr/sbin/rpcbind
#
# ps -ef | grep rpcbind
root 8675 26300 0 04:59:50 pts/5 0:00 grep rpcbind
How do I run it with "truss" ?
I also notice some messages in /var/adm/messages after issuing the command :
Sep 7 04:58:31 serverX java[18998]: [ID 360237 user.error] libpkcs11: /usr/lib/security/pkcs11_softtoken.so unexpected failure in ELF signature verification. See cryptoadm(1M). Skipping this plug-in.
Sep 7 04:58:31 serverXjava[18998]: [ID 360237 user.error] libpkcs11: /usr/lib/security/pkcs11_so
Thx,
Bill
underh20
Any idea what the issue might be? Thanks, Bill
# truss -d /usr/sbin/rpcbind
Base time stamp: 1315386396.4115 [ Wed Sep 7 05:06:36 EDT 2011 ]
0.0000 execve("/usr/sbin/rpcbind", 0xFFBFFE64, 0xFFBFFE6C) argc = 1
0.0063 resolvepath("/usr/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12
0.0068 resolvepath("/usr/sbin/rpcbind", "/usr/sbin/rpcbind", 1023) = 17
0.0072 stat64("/usr/sbin/rpcbind", 0xFFBFFB20) = 0
0.0075 open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT
0.0078 stat64("/usr/sfw/lib/libsocket.so.1", 0xFFBFF2B0) Err#2 ENOENT
0.0082 stat64("/lib/libsocket.so.1", 0xFFBFF2B0) = 0
0.0084 resolvepath("/lib/libsocket.so.1", "/lib/libsocket.so.1", 1023) = 19
0.0089 open("/lib/libsocket.so.1", O_RDONLY) = 3
0.0091 mmap(0x00010000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ALIGN, 3, 0) = 0xFF3A0000
0.0094 mmap(0x00010000, 122880, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF380000
0.0096 mmap(0xFF380000, 44594, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF380000
0.0098 mmap(0xFF39C000, 4277, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 49152) = 0xFF39C000
0.0101 munmap(0xFF38C000, 65536) = 0
0.0103 memcntl(0xFF380000, 14320, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0105 close(3) = 0
0.0107 stat64("/usr/sfw/lib/libnsl.so.1", 0xFFBFF2B0) Err#2 ENOENT
0.0110 stat64("/lib/libnsl.so.1", 0xFFBFF2B0) = 0
0.0112 resolvepath("/lib/libnsl.so.1", "/lib/libnsl.so.1", 1023) = 16
0.0116 open("/lib/libnsl.so.1", O_RDONLY) = 3
0.0119 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0123 mmap(0x00010000, 729088, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF280000
0.0126 mmap(0xFF280000, 596478, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF280000
0.0128 mmap(0xFF322000, 33825, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 598016) = 0xFF322000
0.0130 mmap(0xFF32C000, 23312, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANON, -1, 0) = 0xFF32C000
0.0132 munmap(0xFF312000, 65536) = 0
0.0136 memcntl(0xFF280000, 89972, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0137 close(3) = 0
0.0139 mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFF3F0000
0.0142 stat64("/usr/sfw/lib/libwrap.so.1", 0xFFBFF2B0) = 0
0.0144 resolvepath("/usr/sfw/lib/libwrap.so.1", "/usr/sfw/lib/libwrap.so.1.0", 1023) = 27
0.0148 open("/usr/sfw/lib/libwrap.so.1", O_RDONLY) = 3
0.0151 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0154 mmap(0x00010000, 106496, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF360000
0.0156 mmap(0xFF360000, 25325, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF360000
0.0158 mmap(0xFF378000, 5294, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 32768) = 0xFF378000
0.0160 munmap(0xFF368000, 65536) = 0
0.0163 memcntl(0xFF360000, 10060, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0164 close(3) = 0
0.0166 stat64("/usr/sfw/lib/libscf.so.1", 0xFFBFF2B0) Err#2 ENOENT
0.0169 stat64("/lib/libscf.so.1", 0xFFBFF2B0) = 0
0.0172 resolvepath("/lib/libscf.so.1", "/lib/libscf.so.1", 1023) = 16
0.0176 open("/lib/libscf.so.1", O_RDONLY) = 3
0.0178 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0181 mmap(0x00010000, 172032, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF250000
0.0183 mmap(0xFF250000, 97551, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF250000
0.0185 mmap(0xFF278000, 4942, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 98304) = 0xFF278000
0.0188 munmap(0xFF268000, 65536) = 0
0.0190 memcntl(0xFF250000, 20500, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0192 close(3) = 0
0.0194 stat64("/usr/sfw/lib/libc.so.1", 0xFFBFF2B0) Err#2 ENOENT
0.0196 stat64("/lib/libc.so.1", 0xFFBFF2B0) = 0
0.0199 resolvepath("/lib/libc.so.1", "/lib/libc.so.1", 1023) = 14
0.0203 open("/lib/libc.so.1", O_RDONLY) = 3
0.0205 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0208 mmap(0x00010000, 1294336, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF100000
0.0209 mmap(0xFF100000, 1241517, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF100000
0.0211 mmap(0xFF230000, 35901, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 1245184) = 0xFF230000
0.0214 mmap(0xFF23A000, 1552, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANON, -1, 0) = 0xFF23A000
0.0217 memcntl(0xFF100000, 145604, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0219 close(3) = 0
0.0223 stat64("/lib/libdoor.so.1", 0xFFBFF2B0) = 0
0.0226 resolvepath("/lib/libdoor.so.1", "/lib/libdoor.so.1", 1023) = 17
0.0230 open("/lib/libdoor.so.1", O_RDONLY) = 3
0.0233 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0235 mmap(0x00010000, 81920, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF340000
0.0237 mmap(0xFF340000, 6727, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF340000
0.0239 mmap(0xFF352000, 944, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 8192) = 0xFF352000
0.0241 munmap(0xFF342000, 65536) = 0
0.0243 memcntl(0xFF340000, 3560, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0245 close(3) = 0
0.0247 stat64("/lib/libuutil.so.1", 0xFFBFF2B0) = 0
0.0249 resolvepath("/lib/libuutil.so.1", "/lib/libuutil.so.1", 1023) = 18
0.0253 open("/lib/libuutil.so.1", O_RDONLY) = 3
0.0256 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0258 mmap(0x00010000, 106496, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF0E0000
0.0260 mmap(0xFF0E0000, 26032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF0E0000
0.0262 mmap(0xFF0F8000, 1882, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 32768) = 0xFF0F8000
0.0265 munmap(0xFF0E8000, 65536) = 0
0.0267 memcntl(0xFF0E0000, 6652, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0269 close(3) = 0
0.0271 stat64("/lib/libgen.so.1", 0xFFBFF2B0) = 0
0.0274 resolvepath("/lib/libgen.so.1", "/lib/libgen.so.1", 1023) = 16
0.0277 open("/lib/libgen.so.1", O_RDONLY) = 3
0.0280 mmap(0xFF3A0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFF3A0000
0.0282 mmap(0x00010000, 98304, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF0C0000
0.0284 mmap(0xFF0C0000, 22089, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFF0C0000
0.0285 mmap(0xFF0D6000, 2303, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 24576) = 0xFF0D6000
0.0288 munmap(0xFF0C6000, 65536) = 0
0.0290 memcntl(0xFF0C0000, 5656, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
0.0291 close(3) = 0
0.0315 mmap(0x00010000, 24576, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF390000
0.0318 munmap(0xFF3A0000, 32768) = 0
0.0320 getcontext(0xFFBFF990)
0.0321 getrlimit(RLIMIT_STACK, 0xFFBFF970) = 0
0.0323 getpid() = 26623 [26583]
0.0324 setustack(0xFF392A88)
0.0327 mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFF3A0000
0.0330 sigfillset(0xFF23937C) = 0
0.0333 stat64("/platform/SUNW,SPARC-Enterprise-T5220/lib/libc_psr.so.1", 0xFFBFEF40) = 0
0.0337 resolvepath("/platform/SUNW,SPARC-Enterprise-T5220/lib/libc_psr.so.1", "/platform/sun4v/lib/libc_psr.so.1", 1023) = 33
0.0342 open("/platform/SUNW,SPARC-Enterprise-T5220/lib/libc_psr.so.1", O_RDONLY) = 3
0.0345 mmap(0x00010000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ALIGN, 3, 0) = 0xFF240000
0.0348 munmap(0xFF242000, 24576) = 0
0.0350 close(3) = 0
0.0354 getrlimit(RLIMIT_NOFILE, 0xFFBFFDF4) = 0
0.0356 setrlimit(RLIMIT_NOFILE, 0xFFBFFDF4) = 0
0.0358 fstat(-1, 0xFFBFFD08) Err#9 EBADF
0.0360 mkdir("/var/run/daemon", 0755) Err#17 EEXIST
0.0363 chmod("/var/run/daemon", 0755) = 0
0.0366 chown("/var/run/daemon", 1, 12) = 0
0.0368 getprivimplinfo(0xFFBFF4A8, 2076) = 0
0.0370 mmap(0x00010000, 65536, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFF0A0000
0.0374 sysconfig(_CONFIG_NGROUPS) = 16
0.0377 zone_lookup("") = 11
0.0379 zone_getattr(11, ZONE_ATTR_PRIVSET, 0xFF390248, 12) = 12
0.0381 getppriv(PRIV_EFFECTIVE, {c3f71fdbeee1280008000000}) = 0
0.0389 setgroups(0, 0x00000000) = 0
0.0391 setgid(12) = 0
0.0393 getppriv(PRIV_PERMITTED, {c3f71fdbeee1280008000000}) = 0
0.0396 setppriv(PRIV_SET, PRIV_EFFECTIVE, {c3f71fdbeee1280008000000}) = 0
0.0398 setreuid(1, 1) = 0
0.0400 setppriv(PRIV_SET, PRIV_LIMIT, {000000000000000000000000}) = 0
0.0403 setppriv(PRIV_SET, PRIV_PERMITTED, {000402438800200008000000}) = 0
0.0405 setppriv(PRIV_SET, PRIV_INHERITABLE, {000000000000000000000000}) = 0
0.0408 getpid() = 26623 [26583]
0.0410 corectl(CC_GET_PROCESS_PATH, "core", 1024, 26623) = 0
0.0412 getpid() = 26623 [26583]
0.0414 corectl(CC_SET_PROCESS_PATH, "/var/tmp/core.%f.%t", 20, 26623) = 0
0.0417 setpflags(8, 0) = 0
0.0420 open("/etc/netconfig", O_RDONLY|O_LARGEFILE) Err#13 EACCES [file_dac_search]
0.0423 fstat(-1, 0xFFBFF0D8) Err#9 EBADF
0.0425 open("/dev/conslog", O_WRONLY) Err#13 EACCES [file_dac_search]
0.0428 fcntl(-1, F_SETFD, 0x00000001) Err#9 EBADF
0.0429 fstat(-1, 0xFFBFF0D8) Err#9 EBADF
0.0431 fstat(-1, 0xFFBFFB38) Err#9 EBADF
0.0434 _exit(1)
#
Casper H.S. Dik
>ok. I got the following output after running "truss" on rpcbind.
>Any idea what the issue might be? Thanks, Bill
># truss -d /usr/sbin/rpcbind
> 0.0420 open("/etc/netconfig", O_RDONLY|O_LARGEFILE) Err#13 EACCES [file_dac_search]
> 0.0425 open("/dev/conslog", O_WRONLY) Err#13 EACCES [file_dac_search]
That sounds like an error.
What is the permission of /, /etc and /etc/netconfig and the device files?
%
ls -Lld / /etc /etc/netconfig /dev /devices /devices/pseudo /dev/conslog
drwxr-xr-x 52 root root 67 Jul 31 14:28 /
drwxr-xr-x 382 root sys 382 Aug 23 15:15 /dev
crw-rw-rw- 1 root sys 21, 0 Nov 1 2008 /dev/conslog
drwxr-xr-x 4 root sys 20 Oct 20 2010 /devices
drwxr-xr-x 4 root sys 410 Oct 20 2010 /devices/pseudo
drwxr-xr-x 102 root sys 278 Aug 27 14:54 /etc
-rw-r--r-- 1 root sys 2137 Aug 14 2010 /etc/netconfig
I think there might be an issue with the permissions of /.
Casper
--
underh20
# ls -Lld / /etc /etc/netconfig /dev /devices /devices/pseudo /dev/conslog
/devices: No such file or directory
/devices/pseudo: No such file or directory
drwxr-x--- 22 root root 29 Sep 7 02:03 /
drwxr-x--- 12 root root 51 Sep 6 23:36 /dev
crw-r----- 1 root root 21, 0 Sep 6 21:34 /dev/conslog
drwxr-xr-x 81 root sys 232 Sep 7 00:01 /etc
-rw-r--r-- 1 root sys 1239 Aug 6 2010 /etc/netconfig
Ian Collins
Please keep context!
> Here's the output of permissions I got. This is the zone, so I don't have the /devices or /devices/psudo directories. Thx, Bill
>
> # ls -Lld / /etc /etc/netconfig /dev /devices /devices/pseudo /dev/conslog
> /devices: No such file or directory
> /devices/pseudo: No such file or directory
> drwxr-x--- 22 root root 29 Sep 7 02:03 /
This is wrong, see Casper's list (that you snipped!).
You could try changing the permissions to the expected value.
> drwxr-x--- 12 root root 51 Sep 6 23:36 /dev
Same here.
--
Ian Collins
underh20
Any idea how I can change it ?
# ls -ltd /
drwxr-xr-x 22 root root 29 Sep 7 02:03 /
#
# ls -ltd /dev
drwxr-x--- 12 root root 51 Sep 6 23:36 /dev
#
# chmod 755 /dev
chmod: WARNING: can't change /dev
#
Casper H.S. Dik
Ah, ok it's a zone. So you won't have /devices and al the devices
under /dev are actual device nodes.
Some of the directories can be changed as they are under the control of
the zone; it seems like someone run chmod -R g-w,o-rw on your zone; that
breaks it.
pkgchk should be able part of the broken modes and some need to be
done from the global zone, specifically /dev.
Casper
--
Sami Ketola
> At this zone, I could change the permission of root but not /dev
> Any idea how I can change it ?
Please verify if your system has been hacked. Poorly made rootkits
fsck up permissions on vital system files.
Sami
Paroksha
u should check all ur File Systems are available or not.
go into each mount point and check data is available or not.
check /etc/vfstab entries with df -h output.
i think its problem with mounting the FS at boot time.
so comment out(#) any unwanted FS in vfstab and then reboot.
it should come up with all process running fine
let me know
gho...@gmail.com
I've had the same problem and the issue was the permissions of /.
Had to correct it from the global zone with chown root:root /zone/root
and chmod 755 /zone/root
Best regards,
Guillermo
Casper H.S. Dik
>On Wednesday, September 7, 2011 1:16:06 AM UTC-3, Bill Mackler wrote:
>> Our solaris 10 panic and rebooted. One of the zones came up but a lot
>> of services are not starting up. The zone is up and running but we
>> can't login remotely via SSH. Any idea how to resolve it.
>> Thx, Bill
>>
>> # svcs -vx
>> svc:/network/rpc/bind:default (RPC bindings)
>Had to correct it from the global zone with chown root:root /zone/root
>and chmod 755 /zone/root
But yes, modifying the permissions of / will cause failures of
the first non-root services started and rpcbind might well be
the first.
Casper