[ANNOUNCEMENT] Linux POSIX file system test suite
Szabolcs Szakacsits
Hello file system developers,
There are several POSIX file system test suites: closed source, commercial,
one which needs reading 174 pages installation guide, etc. Because of these
frustrations when Pawel Jakub Dawidek ported ZFS to FreeBSD, he also wrote
such a test suite quickly.
Last year the NTFS-3G team ported it to Linux/ext3 and Linux/NTFS-3G to
validate Jean-Pierre Andre's full file permissions and ownership support
for NTFS-3G. We sent our patches to Pawel for integration but this doesn't
seem to happen him (he didn't see problems but is busy).
Since this topic regularly appears on several lists, we are also often
asked about it and NTFS-3G does need it to be maintained, hence we decided
to release it and if nobody else would like to maintain it then we will do
so.
The test suite mostly checks POSIX compliance and works for FreeBSD,
Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list
of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename,
rmdir, symlink, truncate, unlink. There are currently 1950 regression
tests.
Availability:
http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz
and in the NTFS-3G CVS as pjd-fstest module:
http://sourceforge.net/cvs/?group_id=181143
The usage is extremely simple:
# tar czf pjd-fstest-20080402.tgz
# cd pjd-fstest-20080402
# vi tests/conf
Change 'fs' to file system type you want to test (UFS, ZFS, ext3, ntfs-3g).
# make
It will compile fstest utility which is used by regression tests.
# cd /path/to/file/system/you/want/to/test/
The test must be run as root user and requires a few basic Perl modules.
# prove -r /path/to/fstest/
It's also possible to run individual set of tests:
# /path/to/fstest/tests/chown/00.t
Or make single system call tests:
# fstest mkdir foo 0750
0
# fstest mkdir foo 0750
mkdir returned -1
EEXIST
The test suite is easy to understand, modify and extend. For instance doing
a test cases for the above examples is only
expect 0 fstest mkdir foo 0750
expect EEXIST fstest mkdir foo 0750
The default file system type is ext3 and it passes all tests.
NTFS-3G also passes all the tests if the latest PERMISSION_HANDLING_BRANCH
CVS branch is used with the below UserMapping file placed in the .NTFS-3G
control directory:
--------------------------------------------------------------->
:500:S-1-5-21-2271520284-214583110-2989893066-513
500::S-1-5-21-2271520284-214583110-2989893066-1007
# default mapping pattern
::S-1-5-21-2271520284-214583110-2989893066-10000
<--------------------------------------------------------------
Many thanks to Pawel Jakub Dawidek for writing this fantastic test suite,
to Jean-Pierre Andre for tirelessly working on the port and fixing
countless file system problems over the last half year and to Szeredi
Miklos for his exceptionally rapid FUSE fixes.
Enjoy,
Szaka
--
NTFS-3G: http://ntfs-3g.org
Szabolcs Szakacsits
On Thu, 3 Apr 2008, Badari Pulavarty wrote:
> On Thu, 2008-04-03 at 00:29 +0300, Szabolcs Szakacsits wrote:
>
> > The default file system type is ext3 and it passes all tests.
>
> on ext3. What am I missing ?
Your unique, consistently failing test cases for all file systems suggest
that you have a buggy private kernel or some other individual issue in your
test environment.
You could use the -x debug shell option in the test scripts, rerun the
failing ones and they will show why these test cases exactly fail.
> btrfs seems to have little more failures.
If you find the reason for the unexpected failures then the btrfs result
will be quite good. Apparently it has only a few link, truncate, and unlink
ctimes update problems. I think that's quite impressive in its state of
development.
Szaka
> ext2:
> ====
>
> Failed Test Stat Wstat Total Fail Failed List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t 58 2 3.45% 3 19
> /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153
> /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10
> /root/posix/tests/open/05.t 12 2 16.67% 5 9
> /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37
> 39 42
> /root/posix/tests/symlink/00.t 14 2 14.29% 2 5
> /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12
> /root/posix/tests/truncate/12.t 3 1 33.33% 2
> /root/posix/tests/truncate/13.t 4 2 50.00% 2-3
> Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay.
>
>
> ext3:
> ====
>
> Failed Test Stat Wstat Total Fail Failed List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t 58 2 3.45% 3 19
> /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10
> /root/posix/tests/open/05.t 12 2 16.67% 5 9
> /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37
> 39 42
> /root/posix/tests/symlink/00.t 14 2 14.29% 2 5
> /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12
> /root/posix/tests/truncate/12.t 3 1 33.33% 2
> /root/posix/tests/truncate/13.t 4 2 50.00% 2-3
> Failed 8/184 test scripts, 95.65% okay. 29/1950 subtests failed, 98.51% okay.
>
>
> jfs:
> ===
>
> Failed Test Stat Wstat Total Fail Failed List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t 58 2 3.45% 3 19
> /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153
> /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10
> /root/posix/tests/open/05.t 12 2 16.67% 5 9
> /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37
> 39 42
> /root/posix/tests/symlink/00.t 14 2 14.29% 2 5
> /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12
> /root/posix/tests/truncate/12.t 3 1 33.33% 2
> /root/posix/tests/truncate/13.t 4 2 50.00% 2-3
> Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay.
>
>
> btrfs:
> =====
>
> Failed Test Stat Wstat Total Fail Failed List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t 58 2 3.45% 3 19
> /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153
> /root/posix/tests/link/00.t 82 8 9.76% 3 5-6 8-10 56 63
> /root/posix/tests/open/05.t 12 2 16.67% 5 9
> /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37
> 39 42
> /root/posix/tests/symlink/00.t 14 2 14.29% 2 5
> /root/posix/tests/truncate/00.t 21 1 4.76% 15
> /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12
> /root/posix/tests/truncate/12.t 3 1 33.33% 2
> /root/posix/tests/truncate/13.t 4 2 50.00% 2-3
> /root/posix/tests/unlink/00.t 55 3 5.45% 17 22 53
> Failed 11/184 test scripts, 94.02% okay. 39/1950 subtests failed, 98.00% okay.
>
>
>
--
NTFS-3G: http://ntfs-3g.org
Szabolcs Szakacsits
On Fri, 4 Apr 2008, David Chinner wrote:
> On Fri, Apr 04, 2008 at 10:33:30AM +1000, David Chinner wrote:
> > On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote:
> > > The test must be run as root user and requires a few basic Perl modules.
> >
Openssl is replaced with md5sum+cut in the CVS.
It would be also nice to eliminate the Perl dependency ...
> > The current xfs-dev tree:
> >
> > Failed Test Stat Wstat Total Fail Failed List of Failed
> > -------------------------------------------------------------------------------
> > /root/posix/tests/chown/00.t 171 2 1.17% 84 88
> > /root/posix/tests/symlink/02.t 7 2 28.57% 6-7
> > Failed 2/184 test scripts, 98.91% okay. 4/1950 subtests failed, 99.79% okay.
>
> Symlink tests 6 and 7:
>
> expect 0 symlink ${name256} ${n0}
> expect 0 unlink ${n0}
>
> Test 6 is failing with ENAMETOOLONG
> Test 7 is failing (correctly) with ENOENT because test 6 failed.
>
> So there's only one failure here, and that is that that we're rejecting
> ${name256} as too long. I think that getname() is doing this. Seems sane
> to me to disallow symlinking to pathnames that can't be constructed,
> even if POSIX apparently allows it.
As Christoph noted, I also noticed XFS is unique in this behavior.
> Chown tests 84 and 88:
[...]
> So, either result is valid. Hence i suggest that test 84 and test 88
> (same failure) are special cased to "ext3" behaviour.
Done in the CVS.
> That means XFS is not failing any tests at all.
I added the xfs target but left the symlink Test 6 fail because POSIX says
"The string pointed to by path1 shall be treated only as a character
string and shall not be validated as a pathname"
and
"the length of the path1 argument is longer than {SYMLINK_MAX}"
where {SYMLINK_MAX} is typically not defined on Linux or it's {PATH_MAX}.