AtomicFileWriterTest fails

30 views
Skip to first unread message

Jonathan Bergh

unread,
Apr 14, 2018, 5:36:10 AM4/14/18
to Jenkins Developers
hi all,

i have just cloned and tried to test build all the Jenkins modules, including core, and i am getting a test failure for AtomicFileWriter complaining about permissions to generate a symlink.

I am getting this on Win10 so am a little confused?
1) Should this test even be running (do we have symlinks on Windows? I am not a everyday Windows user)
2) Where is it trying to link to? various test files in '/target/'?
3) How can i solve? Since '/target' is compile time generated, i wonder why there are any permission issues?

many thanks and regards
jonathan

Tests run: 8, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 0.064 s <<< FAILURE! - in hudson.util.AtomicFileWriterTest
symlinkToDirectory(hudson.util.AtomicFileWriterTest)  Time elapsed: 0.009 s  <<< ERROR!
java.nio.file.FileSystemException:
C:\Users\User1\Documents\NetBeansProjects\jenkins\core\target\junit4546323911809935589\junit2633652506541301542\ze_symlink: A required privilege is not held by the client.

    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(WindowsFileSystemProvider.java:585)

Mark Waite

unread,
Apr 14, 2018, 7:52:38 AM4/14/18
to jenkin...@googlegroups.com
Windows allows symbolic links as an optional behavior that must be enabled with a registry change. Does the test pass if you enable symbolic links in the windows registry?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4580b705-bfa3-4e4e-9672-b21424ec0dbc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jonathan Bergh

unread,
Apr 14, 2018, 1:47:29 PM4/14/18
to Jenkins Developers
hi Mark,

Thanks for coming back to me. This helper function in the test led me to believe this test should not be running on the Windows platform (

    private static boolean isPosixSupported(File newFile) throws IOException {
        // Check Posix calls are supported (to avoid running this test on Windows for instance)
        boolean posixSupported = true;
        try {
            Files.getPosixFilePermissions(newFile.toPath());
        } catch (UnsupportedOperationException e) {
            posixSupported = false;
        }
        return posixSupported;
    }

however when i check using gpedit.msc, i see that symlinks are allowed for Admin group users (of which i am one on the machine). So running the IDE as an Admin user allows me to pass the test.

Perhaps the comment in the unit test above is old, pre-symlinks on Windows ...

Thanks again,
regards
Jonathan

Jesse Glick

unread,
Apr 16, 2018, 2:55:46 PM4/16/18
to Jenkins Dev, Baptiste Mathus
On Sat, Apr 14, 2018 at 1:13 PM, Jonathan Bergh
<bergh.j...@gmail.com> wrote:
> This helper function in the test led me to
> believe this test should not be running on the Windows platform

The `checkPermissionsRespectUmask` test is skipped on Windows. (Well,
at least potentially—not sure about Cygwin.) `symlinkToDirectory` is
not skipped on Windows. Probably it should start with:

assumeFalse(Functions.isWindows());

Feel free to add that and file a PR claiming to follow up

https://github.com/jenkinsci/jenkins/pull/2548
https://github.com/jenkinsci/jenkins/pull/3233
https://github.com/jenkinsci/jenkins/pull/3275

Baptiste Mathus

unread,
Apr 16, 2018, 3:58:32 PM4/16/18
to Jenkins Developers
Yeah, introduced in https://github.com/jenkinsci/jenkins/commit/4f2e2615055https://github.com/jenkinsci/jenkins/pull/2548

Just checked, and though the PR history is now deleted, I'm pretty sure the CI did run on Windows for this too. 


So there must be a reason why this works or not under some circumstances I suspect.

I think I will file something tomorrow for this if nobody beats me to it. Though I'd like to understand why it succeeds sometimes or not, maybe just when running under cygwin or not indeed.
Or maybe I should just try to be smart... Ever.
And use Functions.isWindows() to disable this globally on Windows, and not just when that new isPosixSupported() method I added returns true or false.

-- Baptiste


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2YVN2s7D5QLaKZFzgHXUxBs7j1QZbWaHYzyvQAzju%2BJQ%40mail.gmail.com.

Devin Nusbaum

unread,
Apr 16, 2018, 4:14:15 PM4/16/18
to jenkin...@googlegroups.com
AIUI creating symlinks in Windows 10+ requires running as administrator unless developer mode is enabled. Blog post from Microsoft on the changes: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/. I’m not sure if there is an easy way to check that setting from Java. UtilTest currently ignores all symlink tests on Windows. (It does have one test for junctions, which are a little different.)

Since this test is unrelated to posix file permissions so I would just ignore it on Windows unconditionally. If you really wanted it to run when symlinks are supported then I guess you could run `cmd.exe /C mklink /d link target` using ProcessBuilder and if that exits successfully and the symlink exists then creating a symlink via NIO should also work.

Devin

On Apr 16, 2018, at 15:58, Baptiste Mathus <m...@batmat.net> wrote:

Yeah, introduced in https://github.com/jenkinsci/jenkins/commit/4f2e2615055https://github.com/jenkinsci/jenkins/pull/2548

Just checked, and though the PR history is now deleted, I'm pretty sure the CI did run on Windows for this too. 
<image.png>
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS6%3DxQCVqvjJbz95NTcs62mwy90C5jH0fbwCd1cWDL9exQ%40mail.gmail.com.

Jesse Glick

unread,
Apr 16, 2018, 4:31:13 PM4/16/18
to Jenkins Dev
On Mon, Apr 16, 2018 at 4:14 PM, Devin Nusbaum <dnus...@cloudbees.com> wrote:
> I would just
> ignore it on Windows unconditionally.

Yes, especially given JENKINS-37862.

Baptiste Mathus

unread,
Apr 17, 2018, 4:58:17 AM4/17/18
to Jenkins Developers
Fine by me. 

But then my only question left is: should we request INFRA to change our Windows agents on ci.jenkins.io so that they are configured differently/more like a target environment and less like a dev machine? 
Since, again, those tests did run correctly there and I guess we would like them to have failed to catch mistakes (?).

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

Baptiste Mathus

unread,
Apr 17, 2018, 5:19:40 AM4/17/18
to Jenkins Developers
Filed as https://github.com/jenkinsci/jenkins/pull/3397

@Jonathan, I would have requested your review there too, but didn't have your GitHub handle.

thanks

Jonathan Bergh

unread,
Apr 17, 2018, 5:33:36 AM4/17/18
to Jenkins Developers
@Baptiste - yes, sorry about that - i need to be properly configured on JIRA as well. Just busy with a release coming up next week and trying to get up and running with the project in between.

best
jonathan


On Tuesday, 17 April 2018 11:19:40 UTC+2, Baptiste Mathus wrote:
Filed as https://github.com/jenkinsci/jenkins/pull/3397

@Jonathan, I would have requested your review there too, but didn't have your GitHub handle.

thanks
2018-04-17 10:57 GMT+02:00 Baptiste Mathus <m...@batmat.net>:
Fine by me. 

But then my only question left is: should we request INFRA to change our Windows agents on ci.jenkins.io so that they are configured differently/more like a target environment and less like a dev machine? 
Since, again, those tests did run correctly there and I guess we would like them to have failed to catch mistakes (?).
2018-04-16 22:31 GMT+02:00 Jesse Glick <jgl...@cloudbees.com>:
On Mon, Apr 16, 2018 at 4:14 PM, Devin Nusbaum <dnus...@cloudbees.com> wrote:
> I would just
> ignore it on Windows unconditionally.

Yes, especially given JENKINS-37862.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

James Nord

unread,
Apr 17, 2018, 6:29:53 AM4/17/18
to Jenkins Developers
Symlinks do not need a reg change to be enabled on windows 10 (recent updates).

They are enabled by default for administrative users, and for non admin users you can enable them via the UI.

Either set the "developer Mode" for the user (untested but was reported to work) or via the local security policy (see screenshot).

/James
Untitled.png

Baptiste Mathus

unread,
Apr 19, 2018, 9:51:54 AM4/19/18
to Jenkins Developers
Quick heads-up: I've finally closed my small PR and explained the rationale there: https://github.com/jenkinsci/jenkins/pull/3397#issuecomment-382705447

TL;DR: @Jonathan or anyone using Windows, you're very welcome to improve this. James proposed an interesting path forward in https://github.com/jenkinsci/jenkins/pull/3397#discussion_r182026333
But as I'm personally not using Windows, doing/testing this myself would take me something like hours, when 1) it should take much less on someone using Windows able to easily change registry key values and 2) it's not even failing in our Windows CI and I believe not very often on Windows boxes.

Note: I'm happy to help Jonathan or anyone willing to tackle this. I think this is a nice improvement to work on for anyone who would like to play with Jenkins codebase.

Cheers

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1ed495c6-c427-403f-bafa-ad8a45580942%40googlegroups.com.
Message has been deleted

Baptiste Mathus

unread,
Apr 20, 2018, 8:40:25 AM4/20/18
to Jenkins Developers
@Jonathan:

Yes, but if you already cloned the repo, that's not necessary anymore.

You basically need to run that test, possibly with your IDE, or on the CLI, for instance using 

mvn clean verify -Dtest=AtomicFileWriterTest -DfailIfNoTests=false

As said before, I think you should follow James' direction in https://github.com/jenkinsci/jenkins/pull/3397#pullrequestreview-112752900 => try and refactor the existing (test, mostly) codebase to have a central method in the Functions class to be finer than Functions.isWindows(), something like Functions.isSymlinkSupported(Path).

Feel free to play with the idea and file a WIP PR so that we can provide feedback and help directly on actual code. It will be easier and more concrete for everyone.

Thanks a lot Jonathan, and again don't be shy to ask questions. We all started from scratch and learned by twiddling sometimes cluelessly for hours ;-), and there is *no* stupid questions, contributions of all kind are greatly welcome!
If needed, and depending on your TZ we can most likely organize a quick Hangout session to help you move forward (I'm on UTC+2, and available more easily starting next Monday). Just tell me.

Thanks again!

Baptiste

2018-04-19 17:08 GMT+02:00 Jonathan Bergh <bergh.j...@gmail.com>:
@Baptiste I am happy to do that.

Basically we are looking at a clone and build?

Best regards
Jonathan
(Github @trijonb)

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

Jonathan Bergh

unread,
Apr 25, 2018, 3:45:05 PM4/25/18
to Jenkins Developers
Hi @Baptiste,

No not at all - i realised i had misread your reply (that you were closing your pull request and not merging it) so i pulled by reply! I may try look at migrating that stuff out of the core, but that depends on time and whether that is pressing for our setup at work.

Best regards
Jonathan
Baptiste

Reply all
Reply to author
Forward
0 new messages