BSFS bug, 'function not implemented' for gzip and chmod

[Anthony Colombo]

Hello all.

I am using the BSFS mounting system , and receiving the below error "function not implemented" when calling gzip and chmod (see output.logs attached)


The output logs show that I have the correct path to my files, and that the files are present within the specific directory.  The output prints the AppSession.json information (project_id, file name, appresults_id, and all the files within the desired directory). 
  the output printed to screen  show that the directory exists; hence there is not an error "file not found" etc.

The problem is I am not allowed to perform any functions, such as chmod, and gzip on the files in this directory (which printed out, and exist).

Please help me solve this bug.

[Lilian Janin]

Hi Anthony,

It's all normal:

BSFS is read-only.

The "function not implemented" messages are due to BSFS not implementing callbacks to "mknod" (create new file) or "mkdir" internally (as it doesn't support those anyway).

The file-system should probably have been mounted read-only to have a better "permission denied" message.

Lilian

[Lilian Janin]

PS: I wrote "mkdir" instead of "chmod", but the same explanation is valid for both.

Possible solution: output files can go to /data/output, and temporary ones to /data/scratch

[Ramsingh Laboratory]

I am using the Illumina virtual machine version 12 (the latest version)  and the bsfs logs report

2015-06-28 01:46:51.234 WARN  [140525655676672] BSFS void Illumina::FSProvider::DiskCacheManager<V>::Initialize() [with V = Illumina::FSProvider::ILruBlock].178 - The path /data/bsfs_cache/data/26642805/data/input not xfs file system, which cannot support LRU in 'SparseFile' mode, so LRU switching to 'BlockFile' mode

Do I need to have an XFS filesystem?   I have ran the app on ext3 (on linux computer) and the Illumina machine is an ext4 filesystem.    is XFS a requirement for data processing?

Thank you very much Lilian
Sincerely,
Anthony C.

[Ramsingh Laboratory]

In my local machine I see a bsfs_cache directory with read-write access

basespace@ubuntu:/$ ls /data/
26642805  bsfs_cache

Would you recommend running the processing commands on the files located in the bsfs_cache, and moving these results to the /data/output/project-Id  directory? 
Thank you very much

[Lilian Janin]

Hi Anthony,

 

Do I need to have an XFS filesystem?   I have ran the app on ext3 (on linux computer) and the Illumina machine is an ext4 filesystem.    is XFS a requirement for data processing?

This I have no idea. Hopefully a BSFS developer will be able to reply. You may need to wait a bit as it's company holiday this week.

 

Would you recommend running the processing commands on the files located in the bsfs_cache, and moving these results to the /data/output/project-Id  directory?  

Definitely not. This bsfs_cache directory is used by BSFS to store internal resources (such as downloaded file blocks).

The best place to store data before moving it to /data/output is definitely /data/scratch.

Lilian

[Mayank Tyagi]

Hey Anthony,

The error you are seeing in the BSFS log is just a warning that displays for BSFS.  By default, BSFS works on an XFS filesystem, but if its not detected then BSFS will automatically switch to 'BlockFile' mode but by default will run in 'SparseFile' mode.  In order to hide this warning so that it runs in 'BlockFile' mode by default, you can add the following to your callback in formbuilder:

{
  "CommandLine": [
    "my_command_line"
  ],
  "ContainerImageId": "my_docker_image",
  "Properties": {},
  "Options": [
    "bsfs.lrumode=BlockFile"

  ]
}

That should make it so you no longer see the errors pop up.  Please let us know if you run into any other issues or if you have any questions.

Sincerely,

The BaseSpace Platform Team