kernel panic reading from Go?

[Tom Lieber]

I have 735 text files in an Upspin directory, 408kB total. I can grep and cat them just fine, but this Go loop kernel panics before it can finish, even if I sleep between reads:

for _, path := range paths {

        os.ReadFile(path)

}

That makes it hard to write client programs! Does anyone else struggle with this?

It's been a problem for as long as I've used upspinfs on macOS, though I'm currently using Go 1.17.6 on macOS 12.2 on an Apple M1.

The full trace is here. The headline is "panic(cpu 4 caller 0xfffffe002120d808): lck_mtx_unlock(): Attempt to release lock not owned by thread (0xfffffe2999d8c060) @locks_arm.c:1687" and of course, "Kernel Extensions in backtrace: io.macfuse.filesystems.macfuse(2084.20)[66ADEEBF-34FC-3531-B0CB-3DC4811653DC]@0xfffffe0020694f20→0xfffffe00206a772b".

[David Presotto]

Two questions.  Does it always die after reading the same number of files?  Or is it a particular file? My initial guess is I'm holding two many inside equivalents open and hitting a bug in macfuse.

I'm currently trying to get upspinfs working on my Mac again so i can't directly test anything.

--
You received this message because you are subscribed to the Google Groups "Upspin" group.
To unsubscribe from this group and stop receiving emails from it, send an email to upspin+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/upspin/5c6e9921-c0cd-4bee-bdf7-9727eaa8b2fbn%40googlegroups.com.

[Tom Lieber]

It's a different number of files each time, even if I read them in the same order, fresh after a reboot. Sometimes it reads all of them without issue, though that's rare. I've tried with and without the cacheserver.

Re-reading my message, I should clarify that it's "grep foo *" and "cat *" that work fine. It seems to be something about the way Go opens the files that triggers the panic?

[Russ Cox]

[Again, reply all, sorry for the dup Tom]

This looks like a macfuse problem and not an upspin problem. Nothing upspin can do should cause macfuse to unlock a lock on the wrong kernel thread. 

That said, the Go loop differs from grep * and cat * in that it probably does end up issuing the system calls from a variety of different (user process) threads rather than a single one, and perhaps macfuse assumes the reads or the open and close come from the same thread. In Go they often don't. 

But again, the problem is macfuse and only macfuse can fix it.

Best,

Russ

 

[David Presotto]

On a perhaps only slightly related subject, I made a version of bazil fuse using Tom's changes that should work both on old and new macos's.  It uses the old way of mounting pre BigSur and Tom's way afterwards.  The question is what to do with it. The bazil tree has stopped supporting macos and I made these changes to the last version that did.  Not sure it makes sense to branch off the bazil git tree or just do something else.  Any suggestions?

--

You received this message because you are subscribed to the Google Groups "Upspin" group.
To unsubscribe from this group and stop receiving emails from it, send an email to upspin+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/upspin/CADSkJJW7jXq2Uq3LFTX1rmG%2BA_t13wL4a3BsB4b29d5mO-FGYw%40mail.gmail.com.

[Aram Hăvărneanu]

As an immediate measure, forking bazil makes sense. In the future it
might be worthwhile to investigate whether we can use the undocumented
9P support in macOS. Or perhaps we can export NFS/CIFS on loopback and
mount it from there.

--
Aram Hăvărneanu

[Tom Lieber]

Does anyone have access to an Intel Mac and some time to reproduce my kernel panic? The author is taking a look, but having trouble getting Upspin set up.