Porting NDIS intermediate driver to Linux
tanix
driver equivalent functionality in Linux?
How would you go about it?
It is for monitoring firewall app and driver has to be high performance.
NDIS is just fine for it.
I'd like it to sit just above the network card driver on the stack.
--
Programmer's Goldmine collections:
Tens of thousands of code examples and expert discussions on
C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript,
organized by major topics of language, tools, methods, techniques.
Burkhardt Braun
On Dec 28, 8:58 am, ta...@mongo.net (tanix) wrote:
> Anybody has any ideas what's involved in getting the NDIS intermediate
> driver equivalent functionality in Linux?
>
> How would you go about it?
>
> It is for monitoring firewall app and driver has to be high performance.
> NDIS is just fine for it.
deal with performance issues later on.
Burkhardt Braun
tanix
>Hi,
>> Anybody has any ideas what's involved in getting the NDIS intermediate
>> driver equivalent functionality in Linux?
>> How would you go about it?
>I would first try ndiswrapper and
Oh, cool. I DO like that one. That saves the day.
>> It is for monitoring firewall app and driver has to be high performance.
>> NDIS is just fine for it.
>deal with performance issues later on.
Well, not sure there are going to be any performance issues with NDIS
driver if that wrapper is not lame.
But who knows, there ARE some nasty issues with spinlocks, etc.
Did you have any experience with ndiswrapper?
What kinds of problems can I expect with it?
And I'd like to know how much performance suffers.
Nike Chen
Why do you just use netfilter mechanism in Linux kernel?
tanix
>On 12=E6=9C=8828=E6=97=A5, =E4=B8=8B=E5=8D=884=E6=99=8258=E5=88=86, ta...@m=
>ongo.net (tanix) wrote:
>> Anybody has any ideas what's involved in getting the NDIS intermediate
>> driver equivalent functionality in Linux?
>>
>> How would you go about it?
>>
>> It is for monitoring firewall app and driver has to be high performance.
>> NDIS is just fine for it.
>>
>> I'd like it to sit just above the network card driver on the stack.
>
Well, I looked at it in passing a while back.
Did not have an impression it was either low enough on the stack
or could support things I need to do.
The rules need to be uploaded to the low level driver.
At high bandwidths, some queues, that hold some pending packets
may have to stay on that queue for minutes if not longer.
If some packet arrives and there is no rule for it,
the packet is not simply dropped because the user mode, the app
may be in a prompted mode. Meaning, until user comes back from
kitchen and clicks on one of the choices for unknown packet,
that packet is held in the driver queue untill either user
responds and a new rule is created and passed to a driver,
and a packed is allowed to pass through, or it simply times out.
Furethermore, I have at least 4 distinct rulesets, such as
Stealth, Trusted, Policy and per adapter. Some of these rulesets
are big. So performance at high bandwidth is a must.
And there are all sorts of intricate mechanisms in the driver.
My impression was that netfilter is a gadget on the user level.
Not that I ever remember anything of significance of it.
It was a couple of years ago.
Also, I need to control everything on the box as far as network
traffic goes. I should be able to intercept even a strandard
firewall traffic, such as netfilter. This is important.
For example, under Windows, this particular NDIS driver is
the only properly written and fully compliant driver in any
firewall I know off and does not violate the fundamental
principles of OS and driver design. All other drivers I saw
used hacks, poking into kernel mode data structures and ugly
things like that. As a result, you could not install more than
one of those net apps because your box would blue screen,
and I have verify it first hand.
So...
Things are a bit intricate and highest level of performance
and control of the wire are a must.
For example, if you simply click a single checkbox, you can
see every single packet in the monitor window, and that is
a LOT of processing. If a single element in the chaing of
events, interfaces is not up to snuff, you'll lock up your
box within milliseconds.
Things like that.
I did not have an impression that netfilter is going to do
the trick even though I am just beginning to look into this
port issue. Right now trying to get some bits and pieces of
info from people that know what they are talking about.
tanix
Btw, does netfilter support the IOCTL mechanism?
Burkhardt Braun
On Dec 28, 10:58 am, ta...@mongo.net (tanix) wrote:
> In article <7e4f100a-310a-44ea-b40b-e56a4c868...@s31g2000yqs.googlegroups.com>, Burkhardt Braun <burkhardt.br...@gmx.net> wrote:
>
> >Hi,
> >On Dec 28, 8:58=A0am, ta...@mongo.net (tanix) wrote:
> >> Anybody has any ideas what's involved in getting the NDIS intermediate
> >> driver equivalent functionality in Linux?
> >> How would you go about it?
> >I would first try ndiswrapper and
>
> Oh, cool. I DO like that one. That saves the day.
>
> >> It is for monitoring firewall app and driver has to be high performance.
> >> NDIS is just fine for it.
> >deal with performance issues later on.
>
> Well, not sure there are going to be any performance issues with NDIS
> driver if that wrapper is not lame.
>
> But who knows, there ARE some nasty issues with spinlocks, etc.
>
> Did you have any experience with ndiswrapper?
No not know, I guess I make some expreiences mid of 2010.
> What kinds of problems can I expect with it?
I know the existence of this kind of wrapper a few years now and bugs
should be fixed nowadays.
If you figure out problems, it is a good idea to let us know about it.
Regards
Burkhardt Braun
tanix
>Hello,
>> In article <7e4f100a-310a-44ea-b40b-e56a4c868...@s31g2000yqs.googlegroups=
>> >> Anybody has any ideas what's involved in getting the NDIS intermediate
>> >> driver equivalent functionality in Linux?
>> >> How would you go about it?
>> >I would first try ndiswrapper and
>> Oh, cool. I DO like that one. That saves the day.
>that is at least presumable.
Actually, looks like a pretty comple set in terms of supporting
NDIS. Actually, I saw more stuff handled than I saw in other places.
I looked for a sec at a couple of performance critical routines
and so far, does not look like it should be a bottleneck.
>> >> It is for monitoring firewall app and driver has to be high performance.
>> >> NDIS is just fine for it.
>> >deal with performance issues later on.
>> Well, not sure there are going to be any performance issues with NDIS
>> driver if that wrapper is not lame.
>> But who knows, there ARE some nasty issues with spinlocks, etc.
>> Did you have any experience with ndiswrapper?
>No not know, I guess I make some expreiences mid of 2010.
>> What kinds of problems can I expect with it?
>I know the existence of this kind of wrapper a few years now and bugs
>should be fixed nowadays.
>If you figure out problems, it is a good idea to let us know about it.
How much do you pay?
:--}
Pavel A.
news:hh9rvh$5fe$3...@news.eternal-september.org...
> Anybody has any ideas what's involved in getting the NDIS intermediate
> driver equivalent functionality in Linux?
>
> How would you go about it?
In Linux, you don't. You just have the source of the netcard driver
and the OS kernel, and can hack it as you like.
Good luck.
--pa
tanix
That is what I was afraid of.
So, is there such a thing as stack?
Btw, there is this ndiswrapper thing if you know.
They claim they can run the NDIS driver.
I spent a few minutes on this. But still do not see how it sits in
the network stack.
>Good luck.
>--pa
Pavel A.
<snip>
> So, is there such a thing as stack?
AFAIK, no. It is a Windoze bloatware thing; we don't need this in Linux.
> Btw, there is this ndiswrapper thing if you know.
> They claim they can run the NDIS driver.
> I spent a few minutes on this. But still do not see how it sits in
> the network stack.
AFAIK, ndiswrapper just converts a miniport to a Linux network device
driver, nothing more,
It does not support IMs.
--pa
tanix
>"tanix" <ta...@mongo.net> wrote in message
>news:hhcgc5$n2d$1...@news.eternal-september.org...
>
><snip>
>> So, is there such a thing as stack?
>
>AFAIK, no. It is a Windoze bloatware thing; we don't need this in Linux.
WHAT?
So, how do I install my driver so it intercepts all the network
traffic from anything in the system?
You think the network stack is some kind of "bloatware" idea?
I do not belive what I am seeing here.
>> Btw, there is this ndiswrapper thing if you know.
>> They claim they can run the NDIS driver.
>> I spent a few minutes on this. But still do not see how it sits in
>> the network stack.
>
>AFAIK, ndiswrapper just converts a miniport to a Linux network device
>driver, nothing more,
>It does not support IMs.
What?
And what about PROTOCOL part of im driver?
Maxim S. Shatskih
> traffic from anything in the system?
I think that Linux possibly has its own packet filtering platform, which is more like WFP then like NDIS IM.
--
Maxim S. Shatskih
Windows DDK MVP
ma...@storagecraft.com
http://www.storagecraft.com
tanix
>> So, how do I install my driver so it intercepts all the network
>> traffic from anything in the system?
>
>is more like WFP then like NDIS IM.
Well, one of these days I will dig into it.
But the very idea that there is no network stack
simply blew my mind.
Actually, from looking at some linux sourse a while back,
I DID have a feeling that it is all just "hey, slug it out!".
That seems as strange to me, as it gets.
Unfortunately, I AM considering porting my firewall to linux.
But if there is no network stack, forget about it, baby.
Not interested. And without that firewall, I feel like I am
blindfolded on the net. I can not trust their funky "filters",
whatever they are. That is just not dynamic enough and not
flexible enough.
To my opinion, you have to have a MONITORING firewall,
with fully fledged GUI, where you can respond to ANY situation
in a matter of a couple of mouse clicks.
Just saw too many things to doubt that one.
And this single feauture was able to help me to recover
from the most lethal and powerful rootkit I could even dream of.
If I did not have that ability, I am not sure I would be able
to recover.
Secondly, I was able to log the entire global network,
running this rootkit operation.
The kind of things they do, will simply boggle your mind.
At least it boggled mine, and I saw it all, in and out by now.
And I published some of my findings and logs to identify
that entire network.
What someone does about it is their issue. Not mine.
I doubt anyone ever had a chance to log it SO good.
:--}
>--=20
>Maxim S. Shatskih
>Windows DDK MVP
>ma...@storagecraft.com
>http://www.storagecraft.com
--
Maxim S. Shatskih
There should be something like WFP or NDIS6 filters.
At least FreeBSD has such a thing.
> To my opinion, you have to have a MONITORING firewall,
> with fully fledged GUI, where you can respond to ANY situation
> in a matter of a couple of mouse clicks.
On a server? :-)
--
J de Boyne Pollard
PA> AFAIK, no. It is a Windoze bloatware thing; we don't need this in
Linux.
Please don't let silly knee-jerk prejudice inhibit actual
correctness. Pretty much all operating systems that support Internet
protocols have protocol stacks of some form. The clue is in the name
"TCP/IP". It stands for "TCP over IP". That's a stack of two
protocols, Transmission Control Protocol layered over Internet
Protocol. Hence similarly "UDP/IP" and indeed "ICMP/IP". Linux has a
networking stack, just like most other internetwork-capable operating
systems have. The solution domain largely models the problem domain
in this respect. Your favourite WWW search engine should be able to
quickly turn up M. Tim Jones' anatomical outline of it and Jonathan
Sevy's walkthrough of it. And that's just for starters.
Rainer Weikusat
> t> So, is there such a thing as stack?
> PA> AFAIK, no. It is a Windoze bloatware thing; we don't need this in
> Linux.
>
> Please don't let silly knee-jerk prejudice inhibit actual
> correctness. Pretty much all operating systems that support Internet
> protocols have protocol stacks of some form.
The question was related to a network interface driver. That's a
different 'stack' (if any).
alberto
We mustn't confuse protocol stacks with driver stacks. Linux doesn't
have the concept of driver stacks, it's a Windows thing.
Still, if you Google for "Linux Packet Filtering", you'll get more
information than you'll probably need!
Alberto.
On Dec 30, 10:59 am, J de Boyne Pollard <J.deBoynePoll...@Tesco.NET>
wrote:
tanix
>> But if there is no network stack, forget about it, baby.
>
>There should be something like WFP or NDIS6 filters.
>
>At least FreeBSD has such a thing.
>
>> To my opinion, you have to have a MONITORING firewall,
>> with fully fledged GUI, where you can respond to ANY situation
>> in a matter of a couple of mouse clicks.
>
>On a server? :-)
Why not?
Never thought about this one, but if your server has a firewall
and all the traffic for your LAN is going throug that server,
I'd say you MUST have a monitoring firewall. Because you can
very easily and precisely adjust things you want to see in terms
of abnormal traffic and immediately respond to any situation.
After all, your limitation is the size of your rule set.
Because that is the bottleneck on the driver level.
And the amount of rules does not differ THAT much depending
on whether you run a server or a client box.
Sure, you might have a more complicated rule set.
On the other hand, server does not need to be as specific
as client might be. So, it is really a non issue at least
from what I see right now.
>--=20
>Maxim S. Shatskih
>Windows DDK MVP
>ma...@storagecraft.com
>http://www.storagecraft.com
>
--
tanix
>t> So, is there such a thing as stack?
>
>PA> AFAIK, no. It is a Windoze bloatware thing; we don't need this in
>Linux.
>
>Please don't let silly knee-jerk prejudice inhibit actual
>correctness. Pretty much all operating systems that support Internet
>protocols have protocol stacks of some form.
I hope so.
:--}
Otherwise...
> The clue is in the name
>"TCP/IP". It stands for "TCP over IP". That's a stack of two
>protocols, Transmission Control Protocol layered over Internet
>Protocol. Hence similarly "UDP/IP" and indeed "ICMP/IP". Linux has a
>networking stack,
Cool. That is what I want to hear.
The next question is can I insert my NDIS driver equivalent
into the stack?
Yes or no.
> just like most other internetwork-capable operating
>systems have. The solution domain largely models the problem domain
>in this respect.
I like that one.
:--}
> Your favourite WWW search engine should be able to
>quickly turn up M. Tim Jones' anatomical outline of it and Jonathan
>Sevy's walkthrough of it.
Thanx for the pointers. But I am already talking here.
If you have some info, ideas, suggestions, fine.
> And that's just for starters.
--
tanix
Well, my issue is basically can I have an equivalent of the NDIS
intermediate driver (protocol and miniport ends) in Linux.
Basically, NDIS intermediate driver is just fine with me.
I does guarantees me that I can do what I want to and that is
intercept any and all traffic and to be able to drop/allow
any of it depending on what updates to rule set the app dynamically
supplies to driver.
I need to be able to intercept the traffic from any app
and I need to control which traffic is to be allowed.
That's the whole point of it.
tanix
>
>We mustn't confuse protocol stacks with driver stacks. Linux doesn't
>have the concept of driver stacks, it's a Windows thing.
>
>Still, if you Google for "Linux Packet Filtering", you'll get more
>information than you'll probably need!
Well, I was trying to get some feedback here.
Looks like I do not have as much control under Linux as I have
under windows.
I need pending queues for all unknown traffic and with these tables
there is no such a concept. If you start some operation and it turns
out it may result in some unknown traffic, I'd like you to be able
to continue if you respond fast enough from the user level app, so
the app that generates this traffic isn't even aware that something
happened in terms of you creating new rules while it is simply
waiting for some traffic.
So, I need to have control over those queues, such as setting the
timeout times or deleting some entries on the queues and things
like that.
I do not see how this could be done with these tables.
>Alberto.
>
>
>On Dec 30, 10:59=A0am, J de Boyne Pollard <J.deBoynePoll...@Tesco.NET>
>wrote:
>> t> So, is there such a thing as stack?
>>
>> PA> AFAIK, no. It is a Windoze bloatware thing; we don't need this in
>> Linux.
>>
>> Please don't let silly knee-jerk prejudice inhibit actual
>> correctness. =A0Pretty much all operating systems that support Internet
>> protocols have protocol stacks of some form. The clue is in the name
>> "TCP/IP". =A0It stands for "TCP over IP". =A0That's a stack of two
>> protocols, Transmission Control Protocol layered over Internet
>> Protocol. =A0Hence similarly "UDP/IP" and indeed "ICMP/IP". =A0Linux has =
>a
>> networking stack, just like most other internetwork-capable operating
>> systems have. =A0The solution domain largely models the problem domain
>> in this respect. =A0Your favourite WWW search engine should be able to
>> quickly turn up M. Tim Jones' anatomical outline of it and Jonathan
>> Sevy's walkthrough of it. =A0And that's just for starters.
Rainer Weikusat
> In article <69101f33-41d6-4a0f...@t12g2000vbk.googlegroups.com>, alberto <amor...@ieee.org> wrote:
>>
>>We mustn't confuse protocol stacks with driver stacks. Linux doesn't
>>have the concept of driver stacks, it's a Windows thing.
>>
>>Still, if you Google for "Linux Packet Filtering", you'll get more
>>information than you'll probably need!
>
> Well, I was trying to get some feedback here.
>
> Looks like I do not have as much control under Linux as I have
> under windows.
>
> I need pending queues for all unknown traffic and with these tables
> there is no such a concept. If you start some operation and it turns
> out it may result in some unknown traffic, I'd like you to be able
> to continue if you respond fast enough from the user level app, so
> the app that generates this traffic isn't even aware that something
> happened in terms of you creating new rules while it is simply
> waiting for some traffic.
What you basically want is to hook in the communication path between
some 'network protocol' and a device driver. Provided that IP is OK
for you, it should be possible to implement what you want to have with
a custom netfilter extension module providing a new target. This would
then need a single rule which 'redirects' some or all traffic to your
'target', which could then police it in arbitrary ways internally.
tanix
Well, we'd have to dig deeper into this thing.
But I still fail to see how can I implement the queues
for the unknown packets (traffic for which there are no rules).
Rainer Weikusat
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:
>>ta...@mongo.net (tanix) writes:
[...]
>>> I need pending queues for all unknown traffic and with these tables
>>> there is no such a concept. If you start some operation and it turns
>>> out it may result in some unknown traffic, I'd like you to be able
>>> to continue if you respond fast enough from the user level app, so
>>> the app that generates this traffic isn't even aware that something
>>> happened in terms of you creating new rules while it is simply
>>> waiting for some traffic.
>>
>>What you basically want is to hook in the communication path between
>>some 'network protocol' and a device driver. Provided that IP is OK
>>for you, it should be possible to implement what you want to have with
>>a custom netfilter extension module providing a new target. This would
>>then need a single rule which 'redirects' some or all traffic to your
>>'target', which could then police it in arbitrary ways internally.
>
> Well, we'd have to dig deeper into this thing.
> But I still fail to see how can I implement the queues
> for the unknown packets (traffic for which there are no rules).
With a catch-all rule, ie, one which just specifies a target (-j
<something>). But coming to think of it, the better idea might be to
extend the 'traffic control facility' (usually used for QoS). Some
user-level information about that is available here:
This is extensible in form of modules, too, although lesser used,
presumably, because of the extraordinarily poor documentation (there
is also a tc manpage). This would enable you to attach a so-called
'qdisc' (queueing discipline) to an interface which could basically
enforce an arbitrary 'communication policy' directly above the device
driver itself.
tanix
Thanx. I looked into that thing, and from what I saw so far, is WOW!
This is going to be a grand headache to port to from NDIS based
approach.
Yes, generality seems to be supreme and performance is probably
the best you can manage. But the amount of heeadaches you are going
to experience is something else.
>This is extensible in form of modules, too, although lesser used,
>presumably, because of the extraordinarily poor documentation (there
>is also a tc manpage). This would enable you to attach a so-called
>'qdisc' (queueing discipline) to an interface which could basically
>enforce an arbitrary 'communication policy' directly above the device
>driver itself.
Well, I just started to think what it all means in terms of porting
the intermediate NDIS driver and it almost cracked my head to even
think about this.
Basically, it can not be ported.
You'd have to rewrite the driver's end of your app and forget about
the very concept of driver, and forget about everything even
remotely related to it.
Quite dissapointing, I'd say.
That's probably at least a 3 month project, if you are lucky,
just to handle the "driver" end of it and completely remodel
your app so it generates all these insane marks, lil do I care
if they exist.
Tough.
Rainer Weikusat
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:
>>ta...@mongo.net (tanix) writes:
[tc]
>>This is extensible in form of modules, too, although lesser used,
>>presumably, because of the extraordinarily poor documentation (there
>>is also a tc manpage). This would enable you to attach a so-called
>>'qdisc' (queueing discipline) to an interface which could basically
>>enforce an arbitrary 'communication policy' directly above the device
>>driver itself.
>
> Well, I just started to think what it all means in terms of porting
> the intermediate NDIS driver and it almost cracked my head to even
> think about this.
Assuming that has a remotely sane structure, meaning, there is some
'entry point' the protocol module sitting above it uses to hand over
frames to it and some 'exit point' where frames supposed to be
transmitted again leave it, with 'some processing logic' in between,
it shouldn't be that difficult: Only the entry and exit points need
major adaptions. All of the tc stuff, that is, the qdisc, just
provides a way to hook your code into the network stack such that it
can police traffic. You don't need to care for any of the existing
queueing disciplines except insofar they might be useful as code
examples. Also, there is no reason to use the data channel for control
communication except if that happened to be convenient. A so-called
'character device', basically, a file an application can open to send
data to some kernel code and receive data from it, might be a better
choice.
Kaz Kylheku
> I need to be able to intercept the traffic from any app
> and I need to control which traffic is to be allowed.
Smells like a fine-grained security policy: SELinux.
Maxim S. Shatskih
> major adaptions.
No. The in-memory representation of a packet can also be different a lot.
For instance, in Windows/NDIS it is _discontiguous_ (this is done to allow the protocols to add headers by chaining one more buffer at front).
IIRC in Linux it is not so, the packets are allocated by the lowest level, with the size being calculated at the highest level (to include all headers). So, the packets are contiguous.
--
Rainer Weikusat
>> it shouldn't be that difficult: Only the entry and exit points need
>> major adaptions.
>
> No. The in-memory representation of a packet can also be different a
> lot.
Not really, since they necessarily contain data in the same format
somewhere.
> For instance, in Windows/NDIS it is _discontiguous_ (this is done to
> allow the protocols to add headers by chaining one more buffer at
> front).
Haunted by mbufs, I presume. But a single block of memory can be
treated as a list with only one element.
> IIRC in Linux it is not so, the packets are allocated by the lowest
> level, with the size being calculated at the highest level (to
> include all headers).
'packet' is a somewhat ill-defined term in this context. And your
statement cannot possibly ever make any sense in the way you wrote it,
since data is received 'at the highest level' (that would be the
socket interface) and then travels to the lowest (the device
driver). For this to be possible, buffer space must obviously be
allocated before 'the lowest level' is reached. But this is completely
tangential to the original question and while I am willing to try to
help someone even though this is difficult in absence of any details,
I am not willing to hold entry-level lectures on 'Linux networking' to
an audience of people likely unwilling to listen.
Maxim S. Shatskih
> treated as a list with only one element.
Correct, but not vice versa.
So, old-style Linux code, which treats each packet as contiguous memory area, can be ported to NDIS in an easy way. Backward port is harder, at least if you will not go on with the perf-losing option of flattening all packets.
Anyway: I know that FreeBSD has a framework (IIRC called "ipf") to inject packet filters to the kernel code, which is more or less similar to Vista+ WFP and NDIS6 filters. So, I also expect Linux to have one.
I would suggest the OP to study both _modern_ (not 2.2) Linux mainstream (not exotic and unfashionable in the Linux world) facilities and NDIS6 filtering. Probably it is not a hard task to write the filtering driver portable across the two.