Sync time with AD
5,150 views
Skip to first unread message
scott
Oct 24, 2012, 6:39:57 PM10/24/12
to isilon-u...@googlegroups.com
I have an Isilon cluster joined to Active Directory. It was my understanding this automatically becomes my cluster's time source as well, but this afternoon I noticed my cluster time had drifted a few minutes from AD time.
mbisilon02-1# isi auth ads time
Wed Oct 24 15:40:51 2012
mbisilon02-1# date
Wed Oct 24 15:38:53 PDT 2012
Is there something extra I need to do to keep the clocks synced?
-Scott Hunter
Blake Golliher
Oct 24, 2012, 7:24:11 PM10/24/12
to isilon-u...@googlegroups.com
You could add an ntp server with isi_ntp_config
-Blake
-Blake
Greg Jones
Oct 24, 2012, 8:46:28 PM10/24/12
to isilon-u...@googlegroups.com
Scott,
I'm working through this exact issue with Isilon. They recommend that we don't use both NTP and AD for time services and suggested that we only use AD.
After making the change, we noticed our time was off by 15 seconds within a week. We contacted support and asked them why the time was off. This is what we were told: If you use AD for time services, Isilon nodes will not adjust their time unless they're a minute or more off. This sound unreasonable to me, but we are following up to see if we can adjust the one minute mark or go back to using NTP.
I'll keep the group posted on how we resolve this.
Greg
Richard Kunert
Oct 24, 2012, 8:53:14 PM10/24/12
to isilon-u...@googlegroups.com
We use NTP but the NTP servers are our domain controllers. Haven't had any problems, time is right on.
Richard
Jerry Uanino
Oct 24, 2012, 9:50:05 PM10/24/12
to isilon-u...@googlegroups.com
We use NTP, but our AD servers are time synced to our NTP servers as well.
David Bell
Oct 25, 2012, 3:43:32 AM10/25/12
to isilon-u...@googlegroups.com
We had the same dilema when setting up our clusters. Isilon support insists you should not run AD and NTP together, but when we started looking at the "smbtime" scripts on the Isilon that are responsible for time sync against AD we decided we had to use NTP. If I remember correctly it only rarely (ever N hours or so) checked to make sure time was accurate, and then only did a change if it was a minute or so out, and then used a time-sensitive protocol to sync the time.
As such we've been running with NTP ever since (and our AD uses the same NTP servers). So far no problems - and time has remained in sync!
Cheers,
David
Donald Coleman
Oct 25, 2012, 9:05:46 AM10/25/12
to isilon-u...@googlegroups.com, isilon-u...@googlegroups.com
Scott,
If you plan to stay with smbtime it is a cron job which checks time against your ad server every 6 hours. There are two way you can go with this 1. Depending on your version of OneFS you can adjust the cron to run every 2 hours or even every hour for more accuracy. The Job sits in the /etc/crontab if I remember correct. 2) you can create your own cron job to run the command;
# isi_for_array -s "isi auth ads time --sync --force"
To force a sync at regular intervals. The other option is as everyone has pointed out to use ntp which depending on various factors of your environment can be more accurate. Again use one or the other never both.
Donald
Sent from my iPhone
If you plan to stay with smbtime it is a cron job which checks time against your ad server every 6 hours. There are two way you can go with this 1. Depending on your version of OneFS you can adjust the cron to run every 2 hours or even every hour for more accuracy. The Job sits in the /etc/crontab if I remember correct. 2) you can create your own cron job to run the command;
# isi_for_array -s "isi auth ads time --sync --force"
To force a sync at regular intervals. The other option is as everyone has pointed out to use ntp which depending on various factors of your environment can be more accurate. Again use one or the other never both.
Donald
Sent from my iPhone
gtjones
Oct 29, 2012, 12:37:27 PM10/29/12
to isilon-u...@googlegroups.com
We could never get a clear answer out of Isilon why the time was so far off when using smbtime or why you should use smbtime instead of NTP. My colleague and I are speculating that earlier versions of Isilon had problems (we're running 6.5.5.7) running NTP and smbtime together. We decided to go back to NTP time because of the inaccuracy of smbtime. We noticed that when you configure the cluster for NTP using the GUI, the crontab entry for smbtime is commented out.
On Wednesday, October 24, 2012 8:46:29 PM UTC-4, gtjones wrote:
To see the system cron jobs: cat /etc/crontab
The smbtime script is located at /etc/mcp/override/crontab.smbtime which calls /etc/mcp/scripts/smbtime_cron.py
Greg
On Wednesday, October 24, 2012 8:46:29 PM UTC-4, gtjones wrote:
Scott,I'm working through this exact issue with Isilon. They recommend that we don't use both NTP and AD for time services and suggested that we only use AD.After making the change, we noticed our time was off by 15 seconds within a week. We contacted support and asked them why the time was off. This is what we were told: If you use AD for time services, Isilon nodes will not adjust their time unless they're a minute or more off. This sound unreasonable to me, but we are following up to see if we can adjust the one minute mark or go back to using NTP.I'll keep the group posted on how we resolve this.
Greg
Andrew Stack
Feb 8, 2013, 2:20:02 PM2/8/13
to isilon-u...@googlegroups.com
I have gone down this rabbit hole. The bottom line is that using AD for keeping time on your cluster is a horrible solution. Even if you ratchet down the time diff on querying AD you still will encounter drift. Why? Simply put - The clocks on the Isilon nodes are utter crap. Only NTP keeps your cluster in sync with Atomic Time at an acceptable level for time sensitive apps. Why? Because NTP queries the time source every 64 seconds and propagates this value down to it's other nodes over IB. This affectively negates the drift issue.
There are some tricks to configuring NTP. I have written a guide which you're welcome to use. I also recommend you read up on Isilon's Support Site by querying NTP.
Real World Configuration - Step by Step
Step 1 In GUI go to Cluster -> Cluster Settings - > NTP and enter in the NTP servers that Genentech uses:  Step 2 NTP service uses 3 'chimmer' nodes on each cluster to synchronize time with NTP servers. These in turn do a comparison (self check) among each other and determine the time to push to the rest of the cluster. The 3 nodes must be connected to the network and none should be node 1. Step 3 Open up a shell to the cluster - In this case nas-vv-is3. Determine which nodes you want to use. In this case we will be using 9-12. Run the following command to Exclude other nodes from pulling time from NTP servers. nas-vv-is3-9# isi_ntp_config add exclude 1 2 3 4 5 6 7 8 9 Nodes excluded from contacting external NTP servers: 1 2 3 4 5 6 7 8 9 nas-vv-is3-9# Step 3 Verify the overall settings: nas-vv-is3-9# isi_ntp_config list NTP Server Setting: Nodes excluded from contacting external NTP servers: 1 2 3 4 5 6 7 8 9 NTP Auth Keyfile: No Keyfile is configured. nas-vv-is3-9# Stop and start the ntpd service which will refresh the time and sync time with NTP source. nas-vv-is3-9# isi_for_array -s "killall ntpd && ntpdate ntp1.gene.com" nas-vv-is3-1: 13 Dec 15:41:08 ntpdate[41359]: step time server 128.137.1.1 offset 1034.675395 sec nas-vv-is3-2: 13 Dec 15:41:08 ntpdate[49625]: step time server 128.137.1.1 offset 1034.644325 sec nas-vv-is3-3: 13 Dec 15:41:09 ntpdate[54635]: step time server 128.137.1.1 offset 1034.642789 sec nas-vv-is3-4: 13 Dec 15:41:09 ntpdate[47364]: step time server 128.137.1.1 offset 1034.577387 sec nas-vv-is3-5: 13 Dec 15:41:10 ntpdate[51548]: step time server 128.137.1.1 offset 1034.670115 sec nas-vv-is3-6: 13 Dec 15:41:10 ntpdate[59781]: step time server 128.137.1.1 offset 1034.647192 sec nas-vv-is3-7: 13 Dec 15:41:10 ntpdate[48438]: step time server 128.137.1.1 offset 1034.648501 sec nas-vv-is3-8: 13 Dec 15:41:11 ntpdate[48500]: step time server 128.137.1.1 offset 1034.660797 sec nas-vv-is3-9: 13 Dec 15:41:11 ntpdate[49939]: step time server 128.137.1.1 offset 1034.673386 sec nas-vv-is3-10: 13 Dec 15:41:12 ntpdate[63791]: step time server 128.137.1.1 offset 1033.560341 sec nas-vv-is3-11: 13 Dec 15:41:12 ntpdate[57463]: step time server 128.137.1.1 offset 1034.668393 sec nas-vv-is3-12: 13 Dec 15:41:13 ntpdate[55012]: step time server 128.137.1.1 offset 1034.581385 sec nas-vv-is3-9# Step 5 Verify that the time is now accurate by comparing it to a known good clock and then running (there should be no more than a second delay between node responses). nas-vv-is3-9# isi_for_array -s date nas-vv-is3-1: Thu Dec 13 15:47:31 PST 2012 nas-vv-is3-2: Thu Dec 13 15:47:31 PST 2012 nas-vv-is3-3: Thu Dec 13 15:47:32 PST 2012 nas-vv-is3-4: Thu Dec 13 15:47:32 PST 2012 nas-vv-is3-5: Thu Dec 13 15:47:32 PST 2012 nas-vv-is3-6: Thu Dec 13 15:47:33 PST 2012 nas-vv-is3-7: Thu Dec 13 15:47:33 PST 2012 nas-vv-is3-8: Thu Dec 13 15:47:33 PST 2012 nas-vv-is3-9: Thu Dec 13 15:47:33 PST 2012 nas-vv-is3-10: Thu Dec 13 15:47:34 PST 2012 nas-vv-is3-11: Thu Dec 13 15:47:34 PST 2012 nas-vv-is3-12: Thu Dec 13 15:47:34 PST 2012 nas-vv-is3-9# Step 6 Lastly you can dig deep by looking at the ntpq command which is explained here: ntpq output explained ntpq output explained
|
Reply all
Reply to author
Forward
0 new messages