WrMeta

[Dave Keck]

Hey list,

I'm optimizing a program that reads image metadata using
CGImageSourceCreateWithURL(). This program should strictly read from
disk, but as it iterates over thousands of photos, the kernel is
writing data to disk at >1 MB/sec. fs_usage reports many instances of
the following:

WrMeta[asyncP] D=0x... B=0x2000 /dev/disk... 0.097343 W kernel_task

This is not the kernel paging-out to disk, as Activity Monitor reports
a constant value for "Page outs" for the lifetime of the program.

What is 'WrMeta'?
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (Darwi...@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/darwin-dev/darwin-dev-garchive-73044%40googlegroups.com

This email sent to darwin-dev-g...@googlegroups.com

[Shantonu Sen]

It's a write of filesystem metadata. This is likely updates to the "atime" last accessed time for every file you enumerate, which causes writes to the HFS+ catalog (as well as journal when journaling is active).

You can try an experiment to see if this is the case with:

$ mount -t hfs | head -1
/dev/disk0s4 on / (hfs, local, journaled)
$ sudo mount -t hfs -u -o noatime /dev/disk0s4 /
$ mount -t hfs | head -1
/dev/disk0s4 on / (hfs, local, journaled, noatime)
$

Since this should be asynchronous, I'd be surprised if this is causing a significant performance degradation...

Shantonu

On Jul 6, 2012, at 2:22 PM, Dave Keck <dave...@gmail.com> wrote:

> Hey list,
>
> I'm optimizing a program that reads image metadata using
> CGImageSourceCreateWithURL(). This program should strictly read from
> disk, but as it iterates over thousands of photos, the kernel is
> writing data to disk at >1 MB/sec. fs_usage reports many instances of
> the following:
>
> WrMeta[asyncP] D=0x... B=0x2000 /dev/disk... 0.097343 W kernel_task
>
> This is not the kernel paging-out to disk, as Activity Monitor reports
> a constant value for "Page outs" for the lifetime of the program.
>
> What is 'WrMeta'?
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Darwin-dev mailing list (Darwi...@lists.apple.com)
> Help/Unsubscribe/Update your Subscription:

> https://lists.apple.com/mailman/options/darwin-dev/ssen%40apple.com
>
> This email sent to ss...@apple.com

[Dave Keck]

> Since this should be asynchronous, I'd be surprised if this is causing a significant performance degradation...

Thanks Shantonu! -- it was indeed updating the access time.

Is there a supported way to prevent my process from updating the
access time? My initial benchmarks show a 28% performance increase in
processing 4000 photos (82 seconds before, 64 seconds after).

David

[Tilghman Lesher]

On Fri, Jul 6, 2012 at 5:10 PM, Dave Keck <dave...@gmail.com> wrote:
>> Since this should be asynchronous, I'd be surprised if this is causing a significant performance degradation...
>
> Thanks Shantonu! -- it was indeed updating the access time.
>
> Is there a supported way to prevent my process from updating the
> access time? My initial benchmarks show a 28% performance increase in
> processing 4000 photos (82 seconds before, 64 seconds after).

Not your process, but you can at the filesystem mount level:
http://oroyphoto.com/2010/11/12/mac-os-x-noatime-settings-for-10-6/

-Tilghman

[Chris Suter]

Hi,

On Sun, Jul 8, 2012 at 6:14 AM, Tilghman Lesher <tilg...@meg.abyt.es> wrote:

On Fri, Jul 6, 2012 at 5:10 PM, Dave Keck <dave...@gmail.com> wrote:
>> Since this should be asynchronous, I'd be surprised if this is causing a significant performance degradation...
>
> Thanks Shantonu! -- it was indeed updating the access time.
>
> Is there a supported way to prevent my process from updating the
> access time? My initial benchmarks show a 28% performance increase in
> processing 4000 photos (82 seconds before, 64 seconds after).

Not your process, but you can at the filesystem mount level:
http://oroyphoto.com/2010/11/12/mac-os-x-noatime-settings-for-10-6/

According to the Kernel sources, there’s something called “rapid aging” that might be relevant. Documentation is sparse so I don’t know its intended use, but it looks like something you can set per-process that will prevent access times from being set.

Kind regards,

Chris

[Tilghman Lesher]

Per process or per thread, apparently. The usage is to call sysctl(3)
with KERN_RAGEVNODE and one of 4 magic values:
/* KERN_RAGEVNODE types */
#define KERN_RAGE_PROC 1
#define KERN_RAGE_THREAD 2
#define KERN_UNRAGE_PROC 3
#define KERN_UNRAGE_THREAD 4

The IOS documentation suggests that it's to be used only by background
tasks which request rapid aging, but other than that, I can find no
mention of the recommended way to ask for rapid aging. Comments in
the kernel also suggest that when an open vnode is accessed by a
"normal" process, rapid aging is turned off.

[Dave Keck]

Thanks for your help Chris & Tilghman -- with your insight I'm seeing
25-35% speedups in my app.

Here's an example code snippet to help those in the future; it
disables access-time updates for the current thread:

#include <sys/types.h>
#include <sys/sysctl.h>
int sysctlValue = KERN_RAGE_THREAD;
int sysctlResult = sysctlbyname("kern.rage_vnode", NULL, NULL,
&sysctlValue, sizeof(sysctlValue));
assert(!sysctlResult);