How to bring au_to_attr(3) back to the userland?

[Mateusz Piotrowski]

Hello,

I participate in Google Summer of Code at FreeBSD this year. My project is about
converting Linux Audit logs to the BSM format (see my wiki[0]).

Recently, I've come across a problem with the libbsm(3) API. I'd like to be able
to generate an attribute token. Unfortunatelly, au_to_attr which generates those
tokens is not available in the userland (I email FreeBSD-hackers at FreeBSD
about this issue[1]).

Together with my mentor we came up with a few possible solutions to this problem
but we are not sure which one is the best. This is why I'd like to dicuss the
pros and cons.

Solutions:

1. The first idea is to add a userland version of the au_to_attr function. The
implementation would be similar to the one of the au_to_exec_* functions.

(See sys/security/audit/bsm_token.c[2].)

2. The second idea is to bring back the vattr structure. At the moment
au_to_attr has one paramter of type `struct vnode_au_info`. Historically,
au_to_attr used `struct vattr`. A possible solution is to bring vattr to the
userland and change the parameter of au_to_attr back to `struct vattr`.

At the moment `struct vattr` is included in sys/vnode.h but it lacks the
interace.

(I summed up everything I know on this wiki page[3].)

3. The last idea is to make `struct vnode_au_info` and `au_to_attr` accessible
from the userland (by simply unwrapping the prototypes from the KERNEL/_KERNEL
conditional compilation macros).

Cheers,

-Mateusz

[0]: https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools
[1]: https://lists.freebsd.org/pipermail/freebsd-hackers/2016-August/049835.html
[2]: https://github.com/freebsd/freebsd/blob/af3e10e5a78d3af8cef6088748978c6c612757f0/sys/security/audit/bsm_token.c#L1281-L1405
[3]: https://github.com/0mp/freebsd/wiki/vattr(99://github.com/0mp/freebsd/wiki/vattr(99)

_______________________________________________
freebsd...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hacke...@freebsd.org"

[Robert N. M. Watson]

Hi Mateusz:

I’m hear about your project — a way to import Linux audit records would be extremely useful to the community of BSM users and tool writers.

My memory is a bit hazy, but I believe the reason we did not stick with Solaris’s API in this instance is that, in both Mac OS X and FreeBSD, struct vattr is an in-kernel data structure that incorporates kernel-internal data types, and is subject to (moderately frequent) changes in a way that would disrupt the userspace ABI. In more recent versions of Mac OS X, struct vattr has been replaced with struct vfs_attr.

The ideal choice would be to take the existing public data structure describing file attributes (struct stat) and use that — unfortunately, not all of the fields we require (in particular the filesystem ID) are present in the stat structure.

I’m slightly unhappy with the name vnode_au_info, as although in FreeBSD, Mac OS X, and Solaris, “vnode” is the term used, that’s not portable to Linux, where the term “inode” is used. We also want to avoid being almost but not quite API compatible with the Solaris BSM API — e.g., having an au_to_attr(3) that takes a different pointer type is likely a recipe for trouble.

As a result, I feel a bit conflicted on the topic, but suspect the easiest path is to rename vnode_au_info to struct au_vattr, and add a new API au_to_vattr(3) that accepts a pointer to a new struct au_vattr pointer. We would continue to not provide an au_to_attr symbol in the OpenBSM libbsm. This would align us with the Solaris model, accepting that it is a “vnode attribute token”, but avoid confusion about function signatures / prototypes / types. I think I’d also re-craft struct au_vattr a bit to use the same types that are present in the underlying token (e.g., uint32_t, …) rather than OS-local types (e.g., dev_t), in order to force consumers of the API to cast to the BSM type, rather than having that occur transparently within OpenBSM, which could cause information loss invisible to the caller (they should do the information loss for us).

Not sure you how you feel about that strategy?

Robert

[Mateusz Piotrowski]

Hello,

>> [0]: https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools
>> [1]: https://lists.freebsd.org/pipermail/freebsd-hackers/2016-August/049835.html
>> [2]: https://github.com/freebsd/freebsd/blob/af3e10e5a78d3af8cef6088748978c6c612757f0/sys/security/audit/bsm_token.c#L1281-L1405
>> [3]: https://github.com/0mp/freebsd/wiki/vattr(99://github.com/0mp/freebsd/wiki/vattr(99)
>>

>

> I’m hear about your project — a way to import Linux audit records would be extremely useful to the community of BSM users and tool writers.

Thank you. It is really motivating to hear that after a couple of weeks of starting
implementing the conversion over and over again.

> My memory is a bit hazy, but I believe the reason we did not stick with Solaris’s API in this instance is that, in both Mac OS X and FreeBSD, struct vattr is an in-kernel data structure that incorporates kernel-internal data types, and is subject to (moderately frequent) changes in a way that would disrupt the userspace ABI. In more recent versions of Mac OS X, struct vattr has been replaced with struct vfs_attr.
>
> The ideal choice would be to take the existing public data structure describing file attributes (struct stat) and use that — unfortunately, not all of the fields we require (in particular the filesystem ID) are present in the stat structure.

This sounds like the best temporarily idea although the "au_vattr" plan seems to be much
more reasonable to me.

> I’m slightly unhappy with the name vnode_au_info, as although in FreeBSD, Mac OS X, and Solaris, “vnode” is the term used, that’s not portable to Linux, where the term “inode” is used. We also want to avoid being almost but not quite API compatible with the Solaris BSM API — e.g., having an au_to_attr(3) that takes a different pointer type is likely a recipe for trouble.
>
> As a result, I feel a bit conflicted on the topic, but suspect the easiest path is to rename vnode_au_info to struct au_vattr, and add a new API au_to_vattr(3) that accepts a pointer to a new struct au_vattr pointer. We would continue to not provide an au_to_attr symbol in the OpenBSM libbsm. This would align us with the Solaris model, accepting that it is a “vnode attribute token”, but avoid confusion about function signatures / prototypes / types. I think I’d also re-craft struct au_vattr a bit to use the same types that are present in the underlying token (e.g., uint32_t, …) rather than OS-local types (e.g., dev_t), in order to force consumers of the API to cast to the BSM type, rather than having that occur transparently within OpenBSM, which could cause information loss invisible to the caller (they should do the information loss for us).
>
> Not sure you how you feel about that strategy?

To sum it up. The idea is to:

1. Rename vnode_au_info to au_vattr.
2. Keep au_to_attr away from the userland.
3. Add au_to_vattr (the parameter of which is struct au_vattr) to the libbsm
API and make it available to the userland.
4. Re-craft au_vattr to use the same types that are present in the underlying
attribute token.

I am not sure if I understand this properly; do we want to simply rename
vnode_au_info to au_vattr and make it available in the userland after a couple
of modifications? If so then it sounds like a good idea to me as long as I don't
break something accidentally. Wouldn't renaming and modifying struct vnode_au_info
cause compatibility problems and potentially break someone's software?

Apart from that it sounds like a reasonable solution.

Thank you very much for the detailed introduction to the complexity of this problem.
Although the GSoC is coming to an end and I plan to focus on integrating my work
into auditdistd, I hope to apply the solutions we discuss here sometime later.

-Mateusz

[Robert N. M. Watson]

On 17 Aug 2016, at 00:18, Mateusz Piotrowski <0...@FreeBSD.org> wrote:

> To sum it up. The idea is to:
>
> 1. Rename vnode_au_info to au_vattr.
> 2. Keep au_to_attr away from the userland.
> 3. Add au_to_vattr (the parameter of which is struct au_vattr) to the libbsm
> API and make it available to the userland.
> 4. Re-craft au_vattr to use the same types that are present in the underlying
> attribute token.
>
> I am not sure if I understand this properly; do we want to simply rename
> vnode_au_info to au_vattr and make it available in the userland after a couple
> of modifications? If so then it sounds like a good idea to me as long as I don't
> break something accidentally. Wouldn't renaming and modifying struct vnode_au_info
> cause compatibility problems and potentially break someone's software?

I guess you have two choices:

(1) Retain existing KPIs to slightly ease merging to FreeBSD and Mac OS X; they can adopt the new in-kernel interfaces when ready.

(2) Simply remove the old KPIs and consider it a feature.

The former probably does marginally ease merging the new OpenBSM version (one fewer kernel changes for FreeBSD and Mac OS X at the point of merge), so I see no harm in retaining it. However, as it’s ifdef’d _KERNEL || KERNEL in the OpenBSM header, it has not been exposed to user applications, just the kernel.

Remember that changes in these structures don’t affect the layout and interpretation of the tokens at all — it’s really just on the producer side that a KPI changes — and the information we’re able to expose.

The existing vnode_au_info isn’t really an appropriate public interface, so do make sure not to remove the ifdefs preventing its use — instead, we should focus on a new interface that is appropriate to be public by virtue of (a) having an appropriate struct type argument that has both the fields we require and is as non-OS-specific as possible; (b) doesn’t conflict with the current interface on FreeBSD/Mac OS X; and (c) doesn’t conflict with the current interface on Solaris.

> Apart from that it sounds like a reasonable solution.
>
> Thank you very much for the detailed introduction to the complexity of this problem.
> Although the GSoC is coming to an end and I plan to focus on integrating my work
> into auditdistd, I hope to apply the solutions we discuss here sometime later.

The problems are all about compatibility, and I think we have a reasonable path here that does what we need without too much work.

Robert

[Robert N. M. Watson]

On 23 Sep 2016, at 11:09, Konrad Witaszczyk <d...@freebsd.org> wrote:

>> I guess you have two choices:
>>
>> (1) Retain existing KPIs to slightly ease merging to FreeBSD and Mac OS X; they can adopt the new in-kernel interfaces when ready.
>

> I think it won't be hard to adopt the changes in the FreeBSD kernel together
> with the changes in libbsm. Would you still consider it as an issue because of
> macOS if we fix it in FreeBSD? I don't know how important it is to their
> developers to stick with the current OpenBSM implementation.


While the kernel and userspace share code from OpenBSM in both FreeBSD and Mac OS X, it’s useful to be able to upgrade userspace without necessarily changing kernel code — e.g., if security patches are required in parsing, etc. I think it would be best to differentiate the new programming interface by giving it a new name, and keeping the existing interface, but marked to be removed at a future date. We could even discourage its use by making if #ifdef OPENBSM_DEPRECATED or such, requiring that it be explicitly enabled to be available to hint to those doing merges that it’s time to move to the new KPI.