exploits for network stack like CVE-2014-2523 and Qubes

[Oleg Artemiev]

Hello, list.

I've a question: do I understand right, that if attacker is able to send a crafted network packet for a network - network-connected VM (work, personal,...), then he will be able to jump through all the VMs that are connected to the net via Network VM, even indirectly?
Or Qubes has rules in firewall VM that will drop any traffic not matching allowed protocols before they reach kernel packet processing upper the protocol number processing branch?

Currently I've made this in my ordinary Linux (untested):

# banned protocols
BANNED_PROTOCOLS=(
 # remote code execution via IP stack: www.opennet.ru/opennews/art.shtml?num=39355
 dccp
 # protocols below are banned just in cause them are definitely not used (though this is not tested yet):
 igmp
 ggp
 st
 egp
 igp
 pup
 hmp
 xns-idp
 rdp
 iso-tp4
 xtp
 ddp
 idpr-cmtp
 ipv6
 ipv6-route
 ipv6-frag
 idrp
 rsvp
 gre
 esp
 ah
 skip
 ipv6-icmp
 ipv6-nonxt
 ipv6-opts
 rspf
 vmtp
 eigrp
 ospf
 ax.25
 encap
 pim
 vrrp
 l2tp
 isis
 fc
 mobility-header
 udplite
 udp-lite
 mpls-in-ip
 manet
 hip
 shim6
 wesp
 rohc
)

...
        # Ban protocols
        [ $VERBOSE ] && echo -e "\tSetting ban rules.."
        for proto in ${BANNED_PROTOCOLS[@]}; do
          $IPFW -t raw -I PREROUTING -p $proto -j DROP
        done

Is it unsafe for ordinary networking abilities (connect to the net, use ssh, openvpn, ssl,..) to remove any of protocol above?

[Alex Dubois]

AFAiK inbound connections are not accepted unless explicitly configured. 

Outbound connections are initiated by the AppVM.

Potential risk is UDP protocol as spoofable by an infected AppVM, but I think FirewallVM would drop returned packet as "originating" from the wrong interface.

So I am not convinced this adds value but I am only a Qubes user.

Alex

--
You received this message because you are subscribed to the Google Groups "qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.
Visit this group at http://groups.google.com/group/qubes-users.
For more options, visit https://groups.google.com/d/optout.

[Marek Marczykowski-Górecki]

On 23.03.2014 09:06, Alex Dubois wrote:
> AFAiK inbound connections are not accepted unless explicitly configured.

Yes, this is correct.

> Outbound connections are initiated by the AppVM.
>
> Potential risk is UDP protocol as spoofable by an infected AppVM, but I think FirewallVM would drop returned packet as "originating" from the wrong interface.

This is also correct - both firewallvm and netvm will drop packets with
spoofed IP originating from VMs.

--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[Oleg Artemiev]

On Sun, Mar 23, 2014 at 12:49 PM, Marek Marczykowski-Górecki
<marm...@invisiblethingslab.com> wrote:
> On 23.03.2014 09:06, Alex Dubois wrote:
>> AFAiK inbound connections are not accepted unless explicitly configured.
>
> Yes, this is correct.
>
>> Outbound connections are initiated by the AppVM.
>>
>> Potential risk is UDP protocol as spoofable by an infected AppVM, but I think FirewallVM would drop returned packet as "originating" from the wrong interface.
>
> This is also correct - both firewallvm and netvm will drop packets with
> spoofed IP originating from VMs.

The CVE-2014-2523 problem is in the stack and I am not sure that
connections are accepted after the stack deals with protocol type. I
didn't get deep enough with this CVE yet, I made my question based on
the
unclear details on netwrok stack internal behaviour.

So, the claims given by information summary provided by opennet.ru was
that it is enough for the specially crafted packet to enter the stack
and it could lead to remote code execution before accepting the
connection.
If you would notice the workaround - it tells treat dccp type packets
should be processed in ICOMING _raw_ . Accordingly to the 'man
iptables'

raw:
This table is used mainly for configuring exemptions
from connection tracking in combination with the NOTRACK target. It
registers at the netfilter hooks with higher priority and is thus
called before ip_conntrack, or any other IP tables.
It provides the following built-in chains: PREROUTING (for packets
arriving via any network interface) OUTPUT (for packets generated
by local processes)

this table is subject to work before any other IP connection handling.
I assume blindly that the exploit if published will work for
unprotected stacks the same way - before the connection is treated,
this may include transit packets
that are not subject to making connection on the stack dealing with
packets of that type. With some luck, it is possible to predict
(currently this believed unpredictable) the port made by NATing
machine for a NATed machine AND guess or brute other requirements
(that
the packet of some type must conform to pass from outside to the
connection endpoint on the NATed machine) and thus attack the host
currently believed impossible to attack.

By the way, current believes of one-way-initiated data flows via NAT
are based on the claim that hosts have RNGs, but after Snowden
revelations I could accept that RNGs used by current software are just
hard-but-possible-to-predict PRNGs in some cases.

So please confirm (and if possible - explain) this questionable statements:

0. The deals with protocol numbers in the stack do not allow errors
made in parsing some protocol to execute any data ever in any case.

1. The Qubes provides security from exploits made for network stack
that work before the TCP/IP connection is subject to deal with, but
work directly on receive by the buggy stack (see above related to
table 'raw').

2. This one CVE is the type that doesn't affect Qubes at all since it
doesn't work the way described in 1.

Thanks in advance.


--
Bye.Olli.
gpg --search-keys grey_olli
Key fingerprint = 9901 6808 768C 8B89 544C 9BE0 49F9 5A46 2B98 147E
Blog keys (mostly in russian): http://grey-olli.livejournal.com/tag/

[Joanna Rutkowska]

On 03/23/14 11:16, Oleg Artemiev wrote:

(...)

> So please confirm (and if possible - explain) this questionable statements:
>
> 0. The deals with protocol numbers in the stack do not allow errors
> made in parsing some protocol to execute any data ever in any case.
>

Hm, I don't understand what you're saying here.

> 1. The Qubes provides security from exploits made for network stack
> that work before the TCP/IP connection is subject to deal with, but
> work directly on receive by the buggy stack (see above related to
> table 'raw').
>

Yes, via separate network VMs and network-disconnected VMs. More:

http://theinvisiblethings.blogspot.com/2011/09/playing-with-qubes-networking-for-fun.html

http://wiki.qubes-os.org/trac/wiki/SecurityCriticalCode

> 2. This one CVE is the type that doesn't affect Qubes at all since it
> doesn't work the way described in 1.
>

I don't know the details of this CVE -- perhaps you, or somebody, would
be willing to perform analysis and send it to the list?

Generally -- if there is an exploitable bug in the VM's kernel TCP/IP
stack and the attacker is allowed to send offending packets to this VM,
then Qubes cannot stop the exploit from exploiting the VM. Qubes cannot
stop any kind of exploits against code running in VMs.

This problem of possible chained exploitation of VMs networking stacks
has been discussed several times on the list in the past, and also
mentioned under the links given above. All Qubes offers is to give you
tools for creating advanced networking topologies to prevent such attacks.

I've just had a thought that one additional layer of protection (which
would not require separation of networks) might be to introduce a
networking proxy VM which would implement a form of bridging/proxying
via qrexec (so this proxy vm would not get any Xen net
frontends/backends). It's more difficult that one might think though, as
connection multiplexing would need to be cleverly done.

joanna.

[Joanna Rutkowska]

... and would need to be implement on level high-enough to block the
stack-level attacks from propagating to the VMs behind the proxy. In
practice this would mean level 3 or 4 even (tcp/udp).

joanna.

[Oleg Artemiev]

On Sunday, March 23, 2014 4:16:35 PM UTC+4, Joanna Rutkowska wrote:

On 03/23/14 11:16, Oleg Artemiev wrote:

(...)

> So please confirm (and if possible - explain) this questionable statements:
>
> 0. The deals with protocol numbers in the stack do not allow errors
> made in parsing some protocol to execute any data ever in any case.
>

Hm, I don't understand what you're saying here.

I've made a post in qubes devel with details and wrote a post in Russian in my blog
(updated it a few seconds ago with Russian explaination of what I meant posting in English).
Sorry for my poor English that I can't give the meaning. :(

> 1. The Qubes provides security from exploits made for network stack
> that work before the TCP/IP connection is subject to deal with, but
> work directly on receive by the buggy stack (see above related to
> table 'raw').
Yes, via separate network VMs and network-disconnected VMs. More:

http://theinvisiblethings.blogspot.com/2011/09/playing-with-qubes-networking-for-fun.html

http://wiki.qubes-os.org/trac/wiki/SecurityCriticalCode

Yes, but the post was not about isolation provided by Xen for Dom0 and other network disconnected stuff.

> 2. This one CVE is the type that doesn't affect Qubes at all since it
> doesn't work the way described in 1.

I don't know the details of this CVE -- perhaps you, or somebody, would
be willing to perform analysis and send it to the list?

I definitely will dig the subject more till I can understand details and provide qualified explanation (I'm not an
experienced kernel programmer, though I've read some stuff around this and also I've a CCNA / linux admin skills),
so I hope my notes would not be foolish poking around.
Unfortunately I'm quite busy currently and not that effective as I could be. So I can't promise this will be faster to just wait for details from me.

 

Generally -- if there is an exploitable bug in the VM's kernel TCP/IP
stack and the attacker is allowed to send offending packets to this VM,
then Qubes cannot stop the exploit from exploiting the VM. Qubes cannot
stop any kind of exploits against code running in VMs.

This problem of possible chained exploitation of VMs networking stacks
has been discussed several times on the list in the past, and also
mentioned under the links given above. All Qubes offers is to give you
tools for creating advanced networking topologies to prevent such attacks.

Well, currently I see 3 type of network stack exploits (not in order I mentioned before):

The first one:
remote code execution before iptables code in table raw gets control.
For these default Qubes NetVM->FirewallVM->AppVm chain probably not bulletproof.
And I'm currently not deep enough into Qubes and into Linux ip stack design and isolation
in both of these to say something as an expert. Some chances that such a remote code
execution is impossible till someone make a prepared hole in the stack.
 
The second one:
remote code execution inside the iptables/kernel code working with the table raw.
Same comments as above.

The third one:
remote code execution inside the iptables/kernel code that is getting executed after the table raw managing code.

The 3d one is easily avoidable by making default -j DROP rule in table 'raw' for any type of protocol that is surely
no commonly in use (not popular).

That rule is not default for current Qubes Firewall VM settings as I got from the list (I've no Qubes running nearby to check), but I could be wrong.

So I vote for making such a -j DROP for a list of not-popular protocols be default for next release of Qubes. What do you think about this?
 

I've just had a thought that one additional layer of protection (which
would not require separation of networks) might be to introduce a
networking proxy VM which would implement a form of bridging/proxying
via qrexec (so this proxy vm would not get any Xen net
frontends/backends). It's more difficult that one might think though, as
connection multiplexing would need to be cleverly done.

Interesting. But currently I'm not deep enough in Qubes to fully understand what you're talking about,
but this is subject to dig in the future. Thank you for your andwer and links for reading.

If you still do not understand what I mean - sorry for my English, you could try to translate the post related to
the CVE in my blog made in Russian. BTW: all posts about Qubes are marked with 'qubes' tag.
 

joanna.

[Marek Marczykowski-Górecki]

IIUC you are saying mostly about protocol analysis code used for connection
tracking. Currently our kernel all such code have in modules (where it is
possible) and that modules aren't loaded by default. This applies for example
to dccp, sctp, sip, tfp etc. But of course this doesn't protect against
exploits for more basic network stack parts (like TCP processing).

[Oleg Artemiev]

For this also, you got my question right.

> Currently our kernel all such code have in modules (where it is
> possible) and that modules aren't loaded by default. This applies for example
> to dccp, sctp, sip, tfp etc. But of course this doesn't protect against
> exploits for more basic network stack parts (like TCP processing).

well.. Still not fully clear. Let me continue, correct me when I'm wrong:

1. What does the stack when modules for this one protocol type are
present on the file system, but not loaded
and it got the data that if this protocol type?
Does it drop any packet that has packet type of something from
/etc/protocols or does it load the module?

If I remember correctly from last time I dealt w/ lots of iptables
rules the modules had to be loaded by operator exactly
specifying them as a comand in script . And according to you - the
things has not changed since that time.

Fine. :)

2. An exploit with remote code execution is most time some data that
gets execution as code,
or a some clever set of replacements of code instructions with a
set of remotely programmed jumps,
or just even one byte replacement in correct code.

Has it any sense to make default protection for at least some of all
possible remote code manipulations?
This is questionable. Does defense from only some situations costs
time? This is also questionable.

So, for just only one example of many possibilities almost unrelated
to reality and the way the network stack does its way:

switch (expression)
{
case constant1:
codes to be executed if expression equals to constant1,
lets think this does no luck for attacker;
break;
case constant2:
codes to be executed if expression equals to constant2,
lets think this does no luck for attacker;
break;
constraint N-2:
break;
case constraintN-1:
case constraintN:
default:
codes to be executed if expression doesn't match to any cases,
lets think that this code is vulnerable that way that jumping here
gaves a remote data execution,
the data is somewhere else;
}

well, lets think that this code is will lead to remote code execution
only if "jmp far ptr address_of_code" landz between default and
constraint N-1.
When the code is made this way and the attacker has to brute adress
for success, our chances to escape landing of remote code-bullet are
probability
based.

And probability to be alive will increase a little when space between
1 and N-2 grows (each have 'break;' that stops attacker from killing
us).

My main question is:

** if we specify exact restrictions by adding -j DROP rule for any
possible protocol numbers that we know we do not use - would we raise
chance to be alive? **

That is question the answer I need. If yes - I'm right spending my own
time to add the -j DROP rules for computers of mine.

I know, though, from Joanna position that non-principal defense is not
a subject to spend time on.

But as you all alreafy know from recent posts - I disagree - we
should use as much chances as we can.

I was saved by this way of thinking when I decided to use
insecure-linux-with-non-perfect-stack-smashing-protection, not the
OpenBSD that was well known to people
to be much secure, but also OpenBSD people these days were against
implementation of non-perfect solutions as were Solar Designer kernel
patches
from their point of view.

Thank you for reading and explaining details of Qubes internals. :)

[Oleg Artemiev]

On Sunday, March 23, 2014 4:16:35 PM UTC+4, Joanna Rutkowska wrote:

On 03/23/14 11:16, Oleg Artemiev wrote:

(...)

> So please confirm (and if possible - explain) this questionable statements:
>
> 0. The deals with protocol numbers in the stack do not allow errors
> made in parsing some protocol to execute any data ever in any case.
>

Hm, I don't understand what you're saying here.

> 1. The Qubes provides security from exploits made for network stack
> that work before the TCP/IP connection is subject to deal with, but
> work directly on receive by the buggy stack (see above related to
> table 'raw').
>

Yes, via separate network VMs and network-disconnected VMs. More:

http://theinvisiblethings.blogspot.com/2011/09/playing-with-qubes-networking-for-fun.html

http://wiki.qubes-os.org/trac/wiki/SecurityCriticalCode

> 2. This one CVE is the type that doesn't affect Qubes at all since it
> doesn't work the way described in 1.
>

I don't know the details of this CVE -- perhaps you, or somebody, would
be willing to perform analysis and send it to the list?

Accordingly to the comment posted in my blog by watchcat@livejournal this vulnerability is result of this variable misuse:
-------------------------------------------------------------
struct dccp_hdr _dh, *dh;
...
skb_header_pointer(skb, dataoff, sizeof(_dh), &dh);

but it should be:
skb_header_pointer(skb, dataoff, sizeof(_dh), &_dh);
-------------------------------------------------------------

This is a classic example of ability to shoot your leg in C.

So having default restrictions added to drop the protocols not used by adding -j DROP as I suggested would be, if accepted, a doubled protection from the bugs like that.
This type of protection may help. You 're the project leader, your turn - is it OK to make such a double protection.
For my devices from now I ultimately insist  the final list of protocols blocked in firewall in table 'raw' is a must have for secure firewall. Even when you do double protection sometimes explots are just that 0-days.

Due to the code details and vulnerability information available publically I bet on worms that would exploit this automatically sooner or later: this one vuln has been present for years in the Linux kernel.
As other funny things similar to M$ DCOM exploit well known to the security world. =)

Though, after turning into distribution that keeps security patches longer than fedora (I vote for this) the Qubes users upating their OS on a regular base would be protected longer from such a stuff.
The users that have their operating systems living in the separated networks would be still vulnerable to similar bug in another protocol connection tracking code w/o the suggested rules in -j DROP
in table raw these users are potentially more vulnerable from my point of view. BTW, any serious security isolated networks usually have air gap betwen internal network w/o wireless and the any
other network connected to the net directly.  Unfortunately these networks are usually a subject to easy local compromise by insiders - OPs of such companies are often tight by rules that make them
ignore some or even all updates. I can claim this at least for one such a serious company I've seen myself. So should you make this kind of double protection for all the Qubes users not updating the OS is your judge.


BTW: I've made a ticket @work for this as QA for our OPs team (sysadmins). Willing to look how do they will argue with my opinion. :)

 

joanna.

If you need more comments on this one  CVE - let me know.

For me this CVE is the one type of three vuln types I told yout,  that Qubes is protected from by default install: network stack configuration. This is accordingly to Marek proofs related to details of current Qubes network stack configuration.

Thanks for  you and your team, and all patience you  have to reexplain the things. :)

[Oleg Artemiev]

On Sunday, March 23, 2014 11:18:37 PM UTC+4, Oleg Artemiev wrote:

For me this CVE is the one type of three vuln types I told yout,  that Qubes is protected from by default install: network stack configuration. This is accordingly to Marek proofs related to details of current Qubes network stack configuration.

Thanks for  you and your team, and all patience you  have to reexplain the things. :)

and more words defending my position:

my argue on double protection is the requirement, making users to have to allow exact list of protocols them 're willing to deal with.

This makes attack landing surface decrease, IMO, since after testing users not knowing what to disallow can use simple rule: everything already disallowed is okay to leave. Users that rule their firewalls not by default drop will always find a way to open a tunnel for attack escalation.

BTW: I'll try to argue that with our OPs for ordinary linux they manage for our company. Though I can't bet I'll win. :|

[Oleg Artemiev]

Let me write my words for our OPs team to the list, I'll be easier to
link to the post if discussion w/ admins will lead to argue. :)

The claim that restricting protocols to exact safe list with every
possible number disabled, would lead to security increase, is based on
the defense made from regression bugs in code of disabled protocols
and code that processes these disabled protocols only when them are
enabled.

This is the thing that will make any linux distro configured that way
be more secure by default.

As about current Qubes settings, my claims look this way:

once { the user has enabled NAT and connection tracking in firewall
VM} then {hole for attack from the FirewallVM to VMs around {in both
directions} is ready to use} ONLY IF {regresson has happen} AND ONLY
IF {the attacker has some other network stack execution execution vuln
working as a bridge to make {probable} regression bug in the
networking code from either AppVM or either NetVM to deliver attack
via that bridging}.

And usual NSA-speculations for fun and thinking:

Due to regular zerodays arriving that were sitting inside the kernel
for years the same way as DCOM was sitting in Windows for years (and
fixed after months) I'm not willing to bet that chain
NetVM->FirewallVM->AnyAppVM be enough secure w/o additional work for
lowering attack landing

And, Qubes specifics:

the abilities of any goverment law enforcement or security service to
buy 0days for any OS well known insisting on secrecy deal by paing a
lot I would bet on use more restrictions available on the market in
addition with isolation made by Qubes. I.e. possibly slightly weaker
grsecurity setup will anyway cover a lot of cases. Also some other
stuff. My vote to make something acceptable to all and push it to
default settings of the OS. This also means that if I will use Qubes a
lot - I'll look how to tune my setup never use default users and other
stuff that has to be tuned for the small RAT to work w/o errors. =)

BTW: The gov automation problem is that often enough the exploit
bought does not work by default and they have no direct connection to
the author who can quickly tune it. Though them may try themsleves. ;)

The poking around this is funny enough - most people on the list're
able to do similar predictions and do their own bets. :)

/me is over for this evening. =)

[Alex Dubois]

Alex

On 23 Mar 2014, at 19:35, Oleg Artemiev <grey...@gmail.com> wrote:

On Sunday, March 23, 2014 11:18:37 PM UTC+4, Oleg Artemiev wrote:

For me this CVE is the one type of three vuln types I told yout,  that Qubes is protected from by default install: network stack configuration. This is accordingly to Marek proofs related to details of current Qubes network stack configuration.

Thanks for  you and your team, and all patience you  have to reexplain the things. :)

and more words defending my position:

my argue on double protection is the requirement, making users to have to allow exact list of protocols them 're willing to deal with.

I would rather have an explicit

Iptables -j CHECKSTATE -p tcp --sport 443 --dport 1025:65565

And a drop rule bellow...

Doing white list rather than blacklist.