Sync client black-box (acceptance) testing

33 views
Skip to first unread message

mixtly

unread,
Oct 16, 2017, 4:22:10 PM10/16/17
to OSXFUSE
Did anyone try writing tests for fuse-based sync client?

I'm thinking writing black-box tests, something like:

- (void) testLocalFolderCreation
{
// assert there is no folder on the server

// create folder (either via NSFileManager or system('mkdir ...'))

// wait a few seconds

// assert there is now a relevant folder on the server
}

and bunch of other tests which test local and remote changes and sync client behavior.


I tried creating unit-testing bundle in Xcode, but that one doesn't actually start the app which I need for this kind of tests.

I'd like to test the app in one-go, not to restart the whole app for every test method or test class.

Does anyone have an idea how to approach this problem?
I guess, one option would be to create separate app (in Xcode or some completely other language), which would start the app (fuse-based sync client) and conduct tests while the app is running; but I'm not sure if this would be a proper approach.

Skyler Bock

unread,
Oct 16, 2017, 11:51:43 PM10/16/17
to osxfus...@googlegroups.com
At this point, you're pretty much testing OSXFUSE itself (rather than your client). I think the right way to test your client is to stub the FUSE api calls and verify your client calls those as expected.

Something like this:

- (void) testDirCreate

{

    MockFuse *mockFuse = [[MockFuse alloc] init];

    SyncEngine *syncer = [Syncer engineWithFuseInstance:mockFuse];

    

    // create folder (either via NSFileManager or system('mkdir ...'))

    // Force sync  (run sync logic)

    // assert mock fuse 'create directory' method was called

}


// MockFuse methods

// attr tracks changes that the client has done, so the tests can verify the correct action was taken

NSMutableDictionary *attr = [[NSMutableDictionary alloc] init];

- (void) createDir:(NSString *)path {

    // Probably better to set an attribute object here

    [attr setObject:YES forKey:path];

}

- (BOOL) fileExistsAtPath:(NSString *)path {

    return [attr objectForKey:path];

}

mixtly

unread,
Oct 17, 2017, 3:10:29 AM10/17/17
to OSXFUSE
Hi Skyler,

the kind of tests you mentioned are unit or integration tests and I already have those, but that is not what I want to achieve now.

I don't mind testing FUSE alongside my client (when conducting acceptance tests), in fact, I'm testing the whole system - even parts of the server.

I'm aware that these kind of tests would execute slowly (compared to unit or integration tests), but I'm totally fine with that too (these tests would be executed once a week or once a day and would last for ca 5 minutes).

For me, main benefit of having this kind of tests is that they are persistent even if SyncEngine changes significantly, because it tests desired behaviour and effect. One test example:
1. assert there is no local folder created
2. remotely create a folder on the server (via HTTP POST request)
3. after a few seconds (letting our client to sync), assert the folder now exists locally

Of course, this test would fail if network is down or if server is unreachable, but I'm fine with that too.

Having said all that, I'm starting to think that Xcode tests may not be a good approach for this.
This can be tested even in python or any other language which would have suitable framework (options like green/red test statuses and concept of tests in general), but I'm not sure which tool to use and whether I should stick to Xcode regarding this.

Pierre Chapuis

unread,
Oct 17, 2017, 2:41:21 PM10/17/17
to OSXFUSE
I do this kind of thing on several OSs. The test suite is composed of a set of scenarios written in Bash (in a DSL-like style).

The filesystem I test is master-master, which means it is composed of several nodes which all replicate with one another.
Typically I run several nodes on the same machine in a mode where I control synchronization manually, so that I can replicate tricky cases.

For instance, here is the scenario for this issue ("trigger_check A B" replicates modifications from node A to node B):

start_n_nodes 2

d1
="$(mountpoint_for 1)"
d2
="$(mountpoint_for 2)"

mkdir
-p "$d1/a/b"
echo
"x" > "$d1/a/b/c"

trigger_check
1 2
wait_until
[ -f "$d2/a/b/c" ]

rm_dir_rec
"$d1/a/b"

trigger_check
1 2
wait_until
[ ! -f "$d2/a/b/c" ]

check mkdir
"$d2/a/b"
trigger_check
2 1

wait_until
[ -d "$d1/a/b" ]

stop_n_nodes
2

mixtly

unread,
Oct 18, 2017, 4:08:12 AM10/18/17
to OSXFUSE
Thanks Pierre, that is really enlightening example.
Reply all
Reply to author
Forward
0 new messages