KfsClient crashes when mounting QFS via Fuse

[Алексей Ступников]

Hello QFS developers!

We have run into an issue with QFS mount via FUSE. A KfsClient crashes when I try to upload many small files (~ 1MB size) into a cluster. KfsClient basically uses all available RAM and crashes.

How to reproduce:

1. Mount QFS via FUSE to /mnt/qfs/
2. Create 1 MB file and upload 1000 its copies into a cluster:

#!/bin/bash
for i in seq 1 1000; do
  cp "/tmp/test.data" "/mnt/qfs/data/test.data."$i
done

I understand that it is not a regular use-case, but it looks 100% legitimate. I am wondering, is it possible to fix this issue? I believe that it possible to limit an amount of RAM used by KfsClient and hold upcoming write requests while serving the buffered ones.

[Michael Ovsiannikov]

Alexey,

While QFS fuse read write mode is still very limited / unsupported, the recent changes (the master’s branch head) might make the test case (with limited memory) that you’re describing work. For [almost] all practical purposes I’d recommend to use cptoqfs and read only (the default) fuse mount mode.

— Mike.

--
You received this message because you are subscribed to the Google Groups "QFS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qfs-devel+...@googlegroups.com.
To post to this group, send email to qfs-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qfs-devel/66b21a4c-3ba4-42ab-ace7-337e720dbf9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[astup...@mirantis.com]

On Wednesday, November 29, 2017 at 9:43:36 AM UTC+3, movsiannikov wrote:
> Alexey,
>
>
>
>
>
> While QFS fuse read write mode is still very limited / unsupported, the recent changes (the master’s branch head) might make the test case (with limited memory) that you’re describing work. For [almost] all practical purposes I’d recommend to
> use cptoqfs and read only (the default) fuse mount mode.
>
>
>
>
>
> — Mike.
>
>
>
>
>
>
>
> On Nov 20, 2017, at 7:02 AM, Алексей Ступников <aleksey....@gmail.com> wrote:
>
>
>
>
>
> Hello QFS developers!
>
>
>
> We have run into an issue with QFS mount via FUSE. A KfsClient crashes when I try to upload many small files (~ 1MB size) into a cluster. KfsClient basically uses all available RAM and crashes.
>
>
>
> How to reproduce:
>
>
>
>

> Mount QFS via FUSE to /mnt/qfs/Create 1 MB file and upload 1000 its copies into a cluster:

>
>
>
>
>
>
> #!/bin/bash
>
> for i
> in seq
> 1
> 1000;
> do
>
>   cp "/tmp/test.data"
> "/mnt/qfs/data/test.data."$i
>
> done
>
>
>
>
>
> I understand that it is not a regular use-case, but it looks 100% legitimate. I am wondering, is it possible to fix this issue? I believe that it possible to limit an amount of RAM used by KfsClient and hold upcoming write requests while serving the buffered
> ones.
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "QFS Development" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to
> qfs-devel+...@googlegroups.com.
>
> To post to this group, send email to
> qfs-...@googlegroups.com.
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qfs-devel/66b21a4c-3ba4-42ab-ace7-337e720dbf9b%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

Got it, Mike. Thank you for your reply.