Important notice about Asst5

[Svetozar Miuchin]

Hi everyone,

We need to set up some rules for the build/run system, to make the testing of your software suitable for automation.

Here's the basic rules, pasted from the assignment spec:

"Starting your server Your server must accept three command line arguments:

the IP address on which it will listen for client connections - we provide this in case the host where your server is running has multiple network interface cards and so it can listen on several different IP addresses.

the port

the directory where it will keep the files created by the clients

The file system Your server will store files created and written by client transactions in the directory specified on startup as a command line argument. When the server starts up, that directory may already contain some files (we could put some files there for testing). You must ensure that the files in that directory are not deleted after the server exits. We will use the contents in those files to check the correctness of your system.

Since your server is statefull, it probably needs to keep a log or records helping it to remember its state. Make sure that those housekeeping logs are hidden from view on the server's file system (recall how Unix implements hidden files). Those files must be deleted every time the server exits gracefully. (If the server exits gracefully it is okay for it to forget about any transactions executed in the past.)

Committing file data to stable storage When the client asks the server to commit a transaction, the server must ensure that the transaction data is flushed to disk on the local file system. Using the"write" system call (if you are programming in C) is not enough to accomplish this. If you remember from your operating systems course, the "write" system call puts the data into file system buffer cache, but does not force it to disk. Therefore, you must explicitly flush the data to disk when committing client transactions or when writing important housekeeping messages that must survive crashes. To flush thedata to disk, use the fsync (or fflush) system call. To tell you the truth, even fsync or fflush will not guarantee that the data is forced to disk: many operating systems return from fsync before the data is flushed to disk, while flushing the data on the background. To really flush the data to disk, you must call fsync (or fflush) twice.

If you are programming in a language other than C or C++ you must find out the equivalent to the fsync system call."

In addition to this, I will expect your directory to contain a bash script called build.sh and a script called run.sh:

build.sh - Builds the server program. The output filenames of your build don't really matter, because you need to have:

run.sh - Runs the server. This script needs to accept the arguments as stated in the specification. So, I will run your scripts with the following:

<your-dir>/build.sh

<your-dir>/run.sh <ip-address> <port> <directory>

Making these scripts will take 5 minutes of your time. Failing to make them will cause you to lose some points (not too much, but it's a motivating factor).

Next up: I will provide a test harness you can use to verify your server against some simple use cases. Stay tuned.

Cheers,
Svetozar

[Svetozar Miuchin]

Hi everyone,

I've attached a python script that will be used as a simple test for your software.
To run it, unpack, install dependencies (python-levenshtein etc.), and run simple.py with the path to your project as the first argument (e.g. ./simple.py ./smiucin/)

The script wasn't written by me, but I've adapted it to conform to the requirements I gave you. Also, I've tested it with a couple of last year's assignments, and it works. If you find anything wrong with the script, please let me know ASAP.

Cheers,
Svetozar

[Svetozar Miuchin]

Also, we won't be able to extend the deadline for this assignment, so please be on time. We have a hard deadline for the grades from SFU.
Anyone who doesn't turn the assignment in on time will get zero marks.

On Tuesday, December 3, 2013 10:46:29 AM UTC-8, Svetozar Miuchin wrote:

[Dorsa Eshraghi Boroojeni]

Hi Svetozar,

is  the test case supposed to fail the second time it runs because the file (common.txt) already exits? i thought the server should not delete the completed transactions, however, the test case reads all the file contents and sees extra data from previous test runs that are left by the committed transactions and fails. is that ok?

regards.

--
You received this message because you are subscribed to the Google Groups "CMPT880_479" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cmpt880_479...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Svetozar Miuchin]

Yeah, just delete the directory contents after you run the test once.
I believe the script has a call to cleanup.sh, so you could put that in your project directory and have it delete the files.

Cheers,
Svetozar

[muirhead.todd]

Looking at "helper431Function.py":

def write(self,txn,seq,data):

sock = self.connect()

msg = " ".join(map(str,["WRITE",txn,seq,str(len(data))+nl*2+data+"\n"]))

sock.send(msg)

def writeAck(self, txn, seq, data):

sock = self.connect()

msg = " ".join(map(str,["WRITE",txn,seq,str(len(data))+nl*2+data+"\n"]))

sock.send(msg)

fields = self.getACKreply(sock)

return fields

Both functions send the same request to the server. However if I use "client.write(txn, startSequence, msg1)" in simple.py for the first test, then my output file is empty. If I replace it with "client.writeAck(txn, startSequence, msg1)", then my output file contains the contents of msg1. Any insights to why this is the case?

At the same time, my the Levenshtein distance between my output and the expected output is 1. I'm assuming this is a missing or extra character in how I am writing to the output file. The first issue is the more puzzling of the two, since my program does everything for the request, sends an ACK, then closes the connection, so I don't see why the python script accepting an ACK would change the result. There could be something more to it but that's why I'm posting it here, in case anyone else has come across it.

Thanks, 

- Todd

[Svetozar Miuchin]

Hi everyone,

I know this is kind of a late notice, but some of you have asked this, and I thought it was a given.
Anyway, you need to submit a .pdf report as usual. Don't let it take up too much of your time, as you all have other stuff to do. Just write your observations about the assignment. I would like to see stuff about the failure handling and concurrency. Don't write too much (if any) about the protocol/sockets/obvious things.

Hope I'm not putting too much strain on you guys :)

Cheers,
Svetozar

[Svetozar Miuchin]

Hi everyone,

I've started running your servers and I get a lot of failed tests, even with the simple.py script I provided you. Please test this on different machines and make sure it works.
The scripts have been tested against last year's submissions, and some of your assignments are working without problems.

I will not spend time trying to get your code to cooperate. You have until midnight today to fix these issues, I won't grade anything negatively until then.

Cheers,

Svetozar

--

[Loren Van Spronsen]

Hey Everyone,

Not sure if this has come up already, but just so everyone knows, the java test client is not perfect in what it displays. I just spent a couple hours trying to figure out where I was duplicate data that was showing up under server response on the java client, and eventually cracked open wireshark to confirm that I wasn't sending that data.

Loren Van Spronsen