Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Warning: Your VMS system may be attacking other systems
483 views
Skip to first unread message
Michael Moroney
Feb 1, 2014, 10:40:58 AM2/1/14
to
There is a NTP-based DDOS going on, and VMS systems will participate.
Recently, a friend wondered why the NTP process on his Alpha was racking
up hours of CPU time and zillions of I/Os. Figuring it was a bug in NTP,
he stopped and restarted NTP a couple of times, to no effect. Later he
and another friend figured it was part of a DDOS amplification attack. A
system on the internet sends a NTP query packet with the forged source of
a victim. The target responds (to the victim) with packets many times
larger than the original query. Doing this to many systems results in a
flood of data to the victim with little outgoing traffic from the bad guy.
Last night I noticed my TCPIP$NTP_1 process had racked up 2 1/2 hours of
CPU time and enough I/Os to run into the next column. Looking at NTP, I
see some 600 systems on the internet (all likely zombies) had poked at NTP
on my system. My system was participating in the DDOS. I stopped NTP
until I figure out what to do to exclude random attackers.
Anyway, if you are running a VMS system connected to the net, look at
your TCPIP$NTP_1 process, if it's racking up hours of CPU time and
zillions of I/Os, it is likely participating.
I don't know what other OS's participate, but it's probably several, since
so many widgets use NTP to set time these days.
I'll reply to this when I find a good way to handle this.
Recently, a friend wondered why the NTP process on his Alpha was racking
up hours of CPU time and zillions of I/Os. Figuring it was a bug in NTP,
he stopped and restarted NTP a couple of times, to no effect. Later he
and another friend figured it was part of a DDOS amplification attack. A
system on the internet sends a NTP query packet with the forged source of
a victim. The target responds (to the victim) with packets many times
larger than the original query. Doing this to many systems results in a
flood of data to the victim with little outgoing traffic from the bad guy.
Last night I noticed my TCPIP$NTP_1 process had racked up 2 1/2 hours of
CPU time and enough I/Os to run into the next column. Looking at NTP, I
see some 600 systems on the internet (all likely zombies) had poked at NTP
on my system. My system was participating in the DDOS. I stopped NTP
until I figure out what to do to exclude random attackers.
Anyway, if you are running a VMS system connected to the net, look at
your TCPIP$NTP_1 process, if it's racking up hours of CPU time and
zillions of I/Os, it is likely participating.
I don't know what other OS's participate, but it's probably several, since
so many widgets use NTP to set time these days.
I'll reply to this when I find a good way to handle this.
Stephen Hoffman
Feb 1, 2014, 11:26:42 AM2/1/14
to
On 2014-02-01 15:40:58 +0000, Michael Moroney said:
> There is a NTP-based DDOS going on, and VMS systems will participate.
> ...
> There is a NTP-based DDOS going on, and VMS systems will participate.
> I'll reply to this when I find a good way to handle this.
Here’s a write-up on and a workaround for the NTP DDoS, pending an
upgrade to a newer version of NTP:
https://isc.sans.edu/forums/diary/NTP+reflection+attack/17300
This is the US CERT alert:
https://www.us-cert.gov/ncas/alerts/TA14-013A
I'd expect the workaround applies to OpenVMS, as well.
--
Pure Personal Opinion | HoffmanLabs LLC
Bill Gunshannon
Feb 1, 2014, 12:33:14 PM2/1/14
to
In article <lcj4ia$1aq$2...@pcls7.std.com>,
How about blocking all ntp traffic both in and out at your firewall
except for the specific address of your ntp peers?
bill
--
Bill Gunshannon | de-moc-ra-cy (di mok' ra see) n. Three wolves
bill...@cs.scranton.edu | and a sheep voting on what's for dinner.
University of Scranton |
Scranton, Pennsylvania | #include <std.disclaimer.h>
except for the specific address of your ntp peers?
bill
--
Bill Gunshannon | de-moc-ra-cy (di mok' ra see) n. Three wolves
bill...@cs.scranton.edu | and a sheep voting on what's for dinner.
University of Scranton |
Scranton, Pennsylvania | #include <std.disclaimer.h>
Jan-Erik Soderholm
Feb 1, 2014, 12:56:48 PM2/1/14
to
an NTP server to stop it replying to the monlist command?
The workaround will not protect the target (VMS) system (?).
Jan-Erik.
Michael Moroney
Feb 1, 2014, 1:14:30 PM2/1/14
to
Stephen Hoffman <seao...@hoffmanlabs.invalid> writes:
>Here'™s a write-up on and a workaround for the NTP DDoS, pending an
>upgrade to a newer version of NTP:
>https://isc.sans.edu/forums/diary/NTP+reflection+attack/17300
>This is the US CERT alert:
>https://www.us-cert.gov/ncas/alerts/TA14-013A
>I'd expect the workaround applies to OpenVMS, as well.
Thanks, Hoff.
>Here'™s a write-up on and a workaround for the NTP DDoS, pending an
>upgrade to a newer version of NTP:
>https://isc.sans.edu/forums/diary/NTP+reflection+attack/17300
>This is the US CERT alert:
>https://www.us-cert.gov/ncas/alerts/TA14-013A
>I'd expect the workaround applies to OpenVMS, as well.
To see if your system has been abused, do the following:
$ ntpdc :== $tcpip$ntpdc
$ ntpdc
monlist
You should only see the peers you use listed. If you see a list of
600 random systems, you're being abused.
For simplicity, here's how to avoid VMS from being abused by this:
Edit the file SYS$SYSROOT:[TCPIP$NTP]TCPIP$NTP.CONF and add the following
lines at the end:
# NTP Reflection DDOS attack
restrict default kod nomodify notrap nopeer noquery
restrict 127.0.0.1
The first restrict disables abusable things, the second allows you to do
those things locally.
Stop and restart NTP.
Michael Moroney
Feb 1, 2014, 1:17:49 PM2/1/14
to
bi...@server2.cs.scranton.edu (Bill Gunshannon) writes:
>How about blocking all ntp traffic both in and out at your firewall
>except for the specific address of your ntp peers?
That will work if you are using fixed, static addresses.
>How about blocking all ntp traffic both in and out at your firewall
>except for the specific address of your ntp peers?
If you are using one of the round-robin systems such as 0.us.pool.ntp.org
(like I am), the address changes every so often, and that firewall rule
will break NTP. (0.us.pool.ntp.org may translate to 1.2.3.4 one time and
a few seconds later be 5.6.7.8)
David Froble
Feb 1, 2014, 2:10:47 PM2/1/14
to
the problem, cause my internet conncetion is through Verizon Wireless
Broadband. Not only do I have a NAT router, but, it seems that Verizon
does something similar, such that my external IP address is not
reachable from the internet since there is another scheme similar to NAT
that Verizon is using.
However, on both a VAX and an Alpha, I'm seeing about 1 I/O per second
on the NTP_1 process. Anybody know what NTP is doing to generate I/Os
so often? I figured there would be 1 or a couple I/Os each time it went
to check with a peer, which is not that often.
Stephen Hoffman
Feb 1, 2014, 3:48:25 PM2/1/14
to
On 2014-02-01 19:10:47 +0000, David Froble said:
> Ok, checked, and don't seem to have the problem. Didn't think i'd have
> the problem, cause my internet conncetion is through Verizon Wireless
> Broadband. Not only do I have a NAT router, but, it seems that Verizon
> does something similar, such that my external IP address is not
> reachable from the internet since there is another scheme similar to
> NAT that Verizon is using.
If you have IPv4 NAT, then you have a firewall against remote access,
> Ok, checked, and don't seem to have the problem. Didn't think i'd have
> the problem, cause my internet conncetion is through Verizon Wireless
> Broadband. Not only do I have a NAT router, but, it seems that Verizon
> does something similar, such that my external IP address is not
> reachable from the internet since there is another scheme similar to
> NAT that Verizon is using.
unless you've port-forwarded NAT.
JF Mezei
Feb 2, 2014, 1:01:56 AM2/2/14
to
On 14-02-01 14:10, David Froble wrote:
> However, on both a VAX and an Alpha, I'm seeing about 1 I/O per second
> on the NTP_1 process. Anybody know what NTP is doing to generate I/Os
> so often?
They are bored, so it is the equivalent of twiddling their thumbs :-)
> However, on both a VAX and an Alpha, I'm seeing about 1 I/O per second
> on the NTP_1 process. Anybody know what NTP is doing to generate I/Os
> so often?
Many ntp servers have parameters to set polling intervals. Can't
remember if VMS has that.
CISCO routers do not have such parameters, but they adjust the interval
based on variations in latency/line conditions. Minimum is 64 seconds
if I recall properly.
BTW, unless you want to be a public NTS server, there is no reason to
keep inboud 123 open. Your servers make outboud requests to
authoritative servers but don't need to accept requests from outside
your lan.
This prevents your server from being told to generate the "monlist". But
does not prevent your IP address from receiviung the result of a monlist
issued to some foreign server with your forged IP address.
Your system may dismiss those packets because they are addressed to a
post that is not being listened to, but it still gnerates traffic.
David Froble
Feb 2, 2014, 9:10:43 AM2/2/14
to
I'm not saying that I've set up good protection, I'm saying that Verizon
has things set up, probably to save on having assigned IP addresses,
that the IP address they assign to me is not accessable from the
outside. This actually is not good, for me.
It's as if NTP is doing constant (1 per second) polling of something ...
David Froble
Feb 2, 2014, 9:33:11 AM2/2/14
to
2 Feb 04:43:39 ntp[566]: offset: 0.001564 sec freq: 13.533 ppm poll:
1024 sec
error: 0.003185
2 Feb 05:43:42 ntp[566]: offset: -0.000629 sec freq: 13.337 ppm
poll: 1024 se
c error: 0.003476
2 Feb 06:43:45 ntp[566]: offset: -0.010766 sec freq: 12.807 ppm
poll: 1024 se
c error: 0.003022
2 Feb 07:43:49 ntp[566]: offset: -0.009745 sec freq: 12.748 ppm
poll: 1024 se
c error: 0.003044
2 Feb 08:43:52 ntp[566]: offset: -0.002093 sec freq: 13.131 ppm
poll: 1024 se
c error: 0.002265
Apparently it's talking to the clock every 2-4 seconds ....
JF Mezei
Feb 2, 2014, 10:58:04 AM2/2/14
to
On 14-02-02 09:33, David Froble wrote:
>
> Apparently it's talking to the clock every 2-4 seconds ....
Does it have the privileges to change the clock ? (is it OPER that is
>
> Apparently it's talking to the clock every 2-4 seconds ....
needed ?)
Try to set your clock back 30 seconds or such and see if NTP will fix it
(and how quickly its gets fixed)
I really really dislike undully long logs because it makes spotting
problems next to impossible.
From man ntp.conf :
minpoll minpoll
maxpoll maxpoll
These options specify the minimum and maximum poll intervals for NTP
messages, in seconds to the power of two. The maximum poll interval
defaults to 10 (1,024s), but can be increased by the maxpoll option to
an upper limit of 17 (36.4 h).
The minimum poll interval defaults to 6 (64 s), but can be decreased by
the minpoll option to a lower limit of 4 (16 s).
Note sure if those are supported by the VMS version of ntpd. I believe
the ntp.conf file is in TCPIP$ETC directory if I remember correctly. As
you are seeing entries every 3 seconds, this may b some other mechanism
in play here.
From what I have read, the ntp daemon can go into a "learning mode"
which lasts normally about 15 minutes wyere it builds statistics on
clock drift as well as jitter with the data connection to the remote NTP
server and this is written to a file which is used to determine optimum
polling interval.
It is possible that during that learning mode, the ntp deamon does very
frequent polling. Might explain what you see. (assuming the VMS NTP
daemon came from the standard unix ones)
Scott Dorsey
Feb 2, 2014, 11:09:53 AM2/2/14
to
>c error: 0.003044
> 2 Feb 08:43:52 ntp[566]: offset: -0.002093 sec freq: 13.131 ppm
>poll: 1024 se
>c error: 0.002265
>
>Apparently it's talking to the clock every 2-4 seconds ....
Yes, that's what it's for. And if you set it up in a network of machines,
> 2 Feb 08:43:52 ntp[566]: offset: -0.002093 sec freq: 13.131 ppm
>poll: 1024 se
>c error: 0.002265
>
>Apparently it's talking to the clock every 2-4 seconds ....
it will talk to all of them and average all the errors out between them
constantly in real time, giving you a more accurate clock than any of them
individually.
If this is not what you want, and you do not want constant synchronization,
consider using ntpdate at boot time instead of running an ntp daemon.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."
David Froble
Feb 3, 2014, 12:35:21 PM2/3/14
to
When the alert was recently posted here, I took a look at my (usually
quiet) systems and noticed that NTP was, over time, doing lots of
buffered I/O. So I got curious. That's all.
Most of the time NTP is the major task on my 2 running VMS systems.
Sort of a sad thing to admit.
Neil Rieck
Mar 19, 2014, 1:25:40 PM3/19/14
to
On Monday, February 3, 2014 12:35:21 PM UTC-5, David Froble wrote:
> Scott Dorsey wrote:
>
> > David Froble <da...@tsoft-inc.com> wrote:
>
> >> Ok, got interested, did some looking ...
>
> >> c error: 0.003044
>
> >> 2 Feb 08:43:52 ntp[566]: offset: -0.002093 sec freq: 13.131 ppm
>
> >> poll: 1024 se
>
Is posting to COV via google still broken?
> Scott Dorsey wrote:
>
> > David Froble <da...@tsoft-inc.com> wrote:
>
> >> Ok, got interested, did some looking ...
>
> >> c error: 0.003044
>
> >> 2 Feb 08:43:52 ntp[566]: offset: -0.002093 sec freq: 13.131 ppm
>
> >> poll: 1024 se
>
NSR
Neil Rieck
Mar 19, 2014, 1:28:32 PM3/19/14
to
Neil Rieck
Mar 19, 2014, 1:35:35 PM3/19/14
to
Please watch this video before enabling NTP
Tha Attack That Could Disrupt The Whole Internet - Computerphile
Neil Rieck
Waterloo, Ontario, Canada.
Tha Attack That Could Disrupt The Whole Internet - Computerphile
Neil Rieck
Waterloo, Ontario, Canada.
Neil Rieck
Mar 19, 2014, 1:36:27 PM3/19/14
to
Neil Rieck
Mar 19, 2014, 1:38:44 PM3/19/14
to
Just in case you think I've lost my mind, I have been experiencing problems posting to COV via Google since Sunday morning from home and work. Not surprisingly, posting with Chrome seems most stable.
NSR
NSR
Neil Rieck
Mar 19, 2014, 1:41:19 PM3/19/14
to
Please watch this video before enabling NTP
The Attack That Could Disrupt The Whole Internet - Computerphile
http://www.youtube.com/watch?v=BcDZS7iYNsA
Neil Rieck
Mar 19, 2014, 1:43:06 PM3/19/14
to
>
>
> Please watch this video before enabling NTP
>
> The Attack That Could Disrupt The Whole Internet - Computerphile
>
> http://www.youtube.com/watch?v=BcDZS7iYNsA
>
> Neil Rieck
>
> Waterloo, Ontario, Canada.
Okay so the solution is to keep submitting the comment until it is accepted. I wonder if anyone at Google is monitoring the error logs.
>
> Please watch this video before enabling NTP
>
> The Attack That Could Disrupt The Whole Internet - Computerphile
>
> http://www.youtube.com/watch?v=BcDZS7iYNsA
>
> Neil Rieck
>
> Waterloo, Ontario, Canada.
NSR
Stephen Hoffman
Mar 19, 2014, 6:03:14 PM3/19/14
to
On 2014-03-19 17:41:19 +0000, Neil Rieck said:
> Please watch this video before enabling NTP
You can...
> Please watch this video before enabling NTP
...block inbound access to NTP TCP port 123 at your perimeter firewall;
there's generally no need for in-bound access.
...disable the monlist command within NTP.
...upgrade to a current NTP version (than any that's provided with
OpenVMS), and you'll be fine.
While you're concerned about this stuff, also make sure you don't have
DNS zone transfers enabled (or better: block both TCP and UDP port 53
inbound), and ensure that you do disable Wordpress pingback support
should you be running that.
If you have a reasonably configured firewall and aren't opening far
more ports than you should be, then only the Wordpress stuff is an
issue — if you're running that.
Neil Rieck
Mar 20, 2014, 6:29:02 PM3/20/14
to
On Saturday, February 1, 2014 10:40:58 AM UTC-5, Michael Moroney wrote:
Neil Rieck
Kitchener / Waterloo / Cambridge,
Ontario, Canada.
http://www3.sympatico.ca/n.rieck/OpenVMS.html
Stephen Hoffman
Mar 20, 2014, 10:20:18 PM3/20/14
to
On 2014-03-20 22:29:02 +0000, Neil Rieck said:
> Our NTP daemon was under attack last month so I shut down. Next, I
> wrote a little batch job which runs every two hours performing an
> NTPDATE operation.
FWIW, your NTP daemon wasn't really being attacked so much as it was
> Our NTP daemon was under attack last month so I shut down. Next, I
> wrote a little batch job which runs every two hours performing an
> NTPDATE operation.
part of the attack — somebody else was really getting hammered.
Why is your NTP daemon externally accessible? I'd look into that, and
into what other ports are open.
Related:
<http://krebsonsecurity.com/2014/03/blogs-of-war-dont-be-cannon-fodder/>
Unrelated: <http://www.eternal-september.org>
Gary Parker
Apr 24, 2014, 4:51:49 AM4/24/14
to
On Saturday, 1 February 2014 18:14:30 UTC, Michael Moroney wrote:
> For simplicity, here's how to avoid VMS from being abused by this:
>
> Edit the file SYS$SYSROOT:[TCPIP$NTP]TCPIP$NTP.CONF and add the following
>
> lines at the end:
>
> # NTP Reflection DDOS attack
>
> restrict default kod nomodify notrap nopeer noquery
>
> restrict 127.0.0.1
Thanks for the above. I just wanted that, if you're running IPv6, you'll also need to add the following:
> For simplicity, here's how to avoid VMS from being abused by this:
>
> Edit the file SYS$SYSROOT:[TCPIP$NTP]TCPIP$NTP.CONF and add the following
>
> lines at the end:
>
> # NTP Reflection DDOS attack
>
> restrict default kod nomodify notrap nopeer noquery
>
> restrict 127.0.0.1
restrict -6 default kod nomodify notrap nopeer noquery
restrict -6 ::1
It may be obvious to some but I floundered for a while wondering why some ntpdc monlist queries from my Mac succeeded and others failed when I'd only added the first two lines.
0 new messages