java.nio.file.AccessDeniedException

[Eric Kolotyluk]

I feel stupid bringing this to the mailing list, but if I were doing
this in Java I would have a better sense of things.

I have a Scala library that finds duplicate files, and returns the
results as a list. I then wrote a simple little program that calls the
library, and tries to removed the duplicate files:

def delete(path : Path) {
try {
println("deleting " + path)
java.nio.file.Files.delete(path)
} catch {
case exception: Exception => System.err.println(exception)
}
}

val google1 =
FileSystems.getDefault().getPath("""D:\Users\Eric\Google
Drive\Music\Downloaded\Foreigner [Discography HQ]""")
val google2 =
FileSystems.getDefault().getPath("""D:\Users\Eric\Google Drive
(New)\Music\Downloaded\Foreigner [Discography HQ]""")

val duplicates = TraversablePaths(List(google1,
google2)).duplicateFilesList

println("deleting duplicate files")
duplicates.foreach(_.filter(!_.startsWith(google1)).foreach(delete))

but I get things like

java.nio.file.AccessDeniedException: D:\Users\Eric\Google Drive
(New)\Music\Downloaded\Foreigner [Discography HQ]\1977 - Foreigner\03 -
Starrider.mp3

At the heart of the library is

def identical(file1 : Path, file2 : Path) : Boolean = {

require(isRegularFile(file1), file1 + " is not a file")
require(isRegularFile(file2), file2 + " is not a file")

val size1 = size(file1)
val size2 = size(file2)

if (size1 != size2) return false

var position : Long = 0
var length = min(Integer.MAX_VALUE, size1 - position)

val channel1 = FileChannel.open(file1)
val channel2 = FileChannel.open(file2)

try {
while (length > 0) {
val buffer1 = channel1.map(MapMode.READ_ONLY, position, length)
val buffer2 = channel2.map(MapMode.READ_ONLY, position, length)
if (!buffer1.equals(buffer2)) return false
position += length
length = min(Integer.MAX_VALUE, size1 - position)
}
true
} finally {
channel1.close()
channel2.close()
}
}

Experimentation shows that I can delete a file with

java.nio.file.Files.delete(FileSystems.getDefault().getPath("""D:\Users\Eric\Google
Drive (New)\Music\Downloaded\Foreigner [Discography HQ]\1977 -
Foreigner\03 - Starrider.mp3"""))

But not when it is the result of calling my library. My conclusion is
that somewhere in my library, something is holding on to the file in
such a way that it cannot be deleted. Is there anything special about
Scala I should know that would cause my code to not close or otherwise
release a lock on the file? If not, it is the plain old game of "who the
heck is still has a lock?"

One of the things that is different is that in Scala I can call
channel1.close(), and not have to deal with a checked exception in the
finally block. Is there anything else special about Scala finally blocks
I should know about?

Cheers, Eric

[Eric Kolotyluk]

OK, problem solved, not a Scala issue.

Using memory mapped files is problematic. The mapping is not removed
until the MappedByteBuffer is finalized at garbage collection, so the
file is locked against modification.

There is nothing in the API to force the unmapping, but you can use
reflection to get at the internal 'clean' method.

Things would have been so much easier if MappedByteBuffer had a close()
method the allowed explicit unmapping.

Cheers, Eric

/**
* DirectByteBuffers are garbage collected by using a phantom
reference and a
* reference queue. Every once a while, the JVM checks the reference
queue and
* cleans the DirectByteBuffers. However, as this doesn't happen
* immediately after discarding all references to a DirectByteBuffer, it's
* easy to OutOfMemoryError yourself using DirectByteBuffers.
*
* Also, if a file is still mapped, then it is locked and cannot be
destroyed
* or possibly written to if it was mapped read.
*
* This function explicitly calls the Cleaner method of a
DirectByteBuffer.
*
* @param toBeDestroyed
* The DirectByteBuffer that will be "cleaned". Utilizes
reflection.
*
*/
public static void destroyDirectByteBuffer(ByteBuffer toBeDestroyed)
throws IllegalArgumentException, IllegalAccessException,
InvocationTargetException, SecurityException, NoSuchMethodException {

if (toBeDestroyed == null) return;

if (!toBeDestroyed.isDirect()) {
System.err.println("toBeDestroyed isn't direct!");
return;
}

Method cleanerMethod = toBeDestroyed.getClass().getMethod("cleaner");
cleanerMethod.setAccessible(true);
Object cleaner = cleanerMethod.invoke(toBeDestroyed);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.setAccessible(true);
cleanMethod.invoke(cleaner);