Partitions other than C:/ greyed out in file browser

[rasika randunuge]

Hi all Im on windows and theres an issue where only the C partition is selectable and others are greyed out. As you can see I cant select them. I CAN however type D:/ into the address bar and it works. However, even then I cant save bookmarks for those folders in other partitions. 

I did a quick search but couldnt find a mention of this anywhere. Is there anything I can try to workaround this issue?

thanks!

[rupert thorpe]

I experienced this,

and it seems to be down to the filesystem - NTFS seems much happier

XFAT it does not like

[eric...@gmail.com]

Interesting, thanks to both of you for bringing up the problem and helping identify the cause. Rasika, can you confirm (or not) that your D and E drives are using a FAT or exFAT file system?

If you are using FAT or exFAT, it looks like it's a bug in Visual Studio that is may be fixed in VS 2022 : https://github.com/microsoft/STL/pull/2373

We'll be moving to VS 2022 as part of the VFX platform 2023 updates, though I don't have a timeline yet for when that will happen.

But if those drives are using NTFS or something else, we can keep looking into what's going on.

- Eric

[eric...@gmail.com]

Also, it looks like you're getting bit by the missing color transform problem that causes your icons to be black. You can fix that with a couple changes to your OCIO config, described here : https://github.com/GafferHQ/gaffer/issues/4992#issuecomment-1334703688

[rasika randunuge]

That's right on the money. D:/ is an exFAT that is shared with a linux os. same goes for that e: which is external formatted to exfat.

and thanks! I didnt even notice there was an icon problem actually :P. 

cheers!

[jeremy...@gmail.com]

Hello!

Any updates on this? The latest 1.2.8.0-windows release still has the exfat problem. (Weirdly, the old 0.61.1.2-windows never had and still has no problem with exfat)

Thoughts welcome,

Jeremy

[eric...@gmail.com]

We haven't transitioned to MSVC 2022 yet, so it's not too surprising the exFAT partitions are not working. It's working in the old 0.61 because we weren't doing the filesystem operation that's causing the problem. But that operation is worth doing because it gives us file information we want to display in the browsers.

Moving to 2022 is definitely going to happen, it's just a matter of getting the momentum up to move the whole build chain, which could be anywhere from button-click-easy to throw-the-computer-out-the-window-difficult.

[jeremy...@gmail.com]

Hello!

Is there a way to watch this issue? We're holding back updating until this is in place (exFat is pretty essential for us).

Thoughts welcome,

Jeremy

[eric...@gmail.com]

I added an issue for it on Github which might be more helpful for tracking status than the groups here : https://github.com/GafferHQ/gaffer/issues/5490.

[eric...@gmail.com]

We talked about a possible workaround this morning that might fix the issue without needing to wait for 2022 support. I formatted a flash drive with exFAT and was able to reproduce the problem, and the fix at https://github.com/GafferHQ/gaffer/pull/5492 is working for me.

At the bottom of https://github.com/GafferHQ/gaffer/actions/runs/6461008934?pr=5492 there's a link ending in "windows" that will give you the build with the fix in it.If you could test it on your setup that would be super helpful. If you happen to be use shared Linux storage and make use of symlinks, give that a try too, to make sure it's not breaking symlinks.

- Eric

[jeremy...@gmail.com]

Hi Eric, thank you for looking at this.

I downloaded and tried the build and it errored on run. I let the commands echo to the command prompt and it seems to fail on the debugger lines (at line 134) of gaffer.cmd:

C:\gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows\bin>if "" NEQ "" (/debugexe "C:/gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows/bin/..\bin\python.exe" "C:/gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows/bin/.."/bin/__gaffer.py  )  else ("C:/gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows/bin/.."\bin\python.exe "C:/gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows/bin/.."/bin/__gaffer.py  )
WARNING | signal caught: error C0000005 -- access violation

C:\gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows\bin>if -1073741819 NEQ 0 (
echo "Error(s) running Gaffer"
 exit /B -1073741819
)
"Error(s) running Gaffer"

Not sure why that would be but happy to test further,

Jeremy

[eric...@gmail.com]

Hi Jeremy,

Hmm, crashing on launch is certainly not what I was hoping for! Is that correct that it happens right at the start, or are you able to do things in Gaffer before it crashes? Are you able to run any other recent Gaffer releases, like 1.3.4.0?

What version of Windows are you on? And I don't think this would matter, but is your C: that you are unzipping Gaffer onto an exFAT drive?

If you're up for doing some debugging, a stack trace leading up the the crash may be helpful for us to figure out what led to the problem. If you know how to do that already, you can paste it here or upload it to https://drive.google.com/drive/folders/1ZfjuyKIBWP6ZP5aMksS-RKPpierXL6Ap?usp=sharing

If you're not familiar with getting a stack trace and are able to install software from the Windows app store, here's a walk through for generating it.

1. Install `WinDbg` from the Windows app store: https://apps.microsoft.com/detail/windbg-preview/9PGJGD53TN86?hl=en-us&gl=US

2. Replace your `gaffer.cmd` file with this one : https://drive.google.com/file/d/1Iq2XZQxJGdmSIuG_8wngQSxnCEgOJDkX/view?usp=sharing. I'll make that change in the official Gaffer releases, but for now, manually installing it is probably easier and faster.

3. In a terminal, set a couple of environment variables (I'm using Powershell here):

  3a. `$env:GAFFER_DEBUG=1`

  3b. `$env:GAFFER_DEBUGGER="WinDbgX.exe -g"`

4. Launch Gaffer using the usual `gaffer.cmd` (the new one from step 2)

5. It should run WinDbg and you'll see a bunch of .dll modules and other stuff loading.

6. I expect if you are getting a crash on startup that it will stop at some point on it's own and output something related to your access violation. If it's not at startup, go through the steps that generate the crash and wait for it to trigger in the debugger.

7. Output a stack trace file :

  7a. See https://github.com/GafferHQ/gaffer/wiki/Debugging-on-Windows for an example. Enter something like ` .dump C:\gaffer.dmp` into the line highlighted in the screen capture on that page.

You can upload that to the Google drive link above and I can take a look at it.

- Eric

[John Haddon]

If running the debugger is problematic for any reason, it's possible that running `gaffer.cmd test GafferTest` might give us some clues. That'll run all our unit tests for the main Gaffer module (where the filesystem handlng resides) and might give us a meaningful failure, or a crash in a specific test that will help us narrow it down...

Cheers...

John

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/447b25c0-0c22-43aa-980b-a65fd5de9432n%40googlegroups.com.

[jeremy...@gmail.com]

Thank you John and Eric,

I plan to download / run the debugger, just working around a few tasks here.

In the meantime, yes, here are the results of the GafferTest:

======================================================================
FAIL: testWrapperDoesntDuplicatePaths (GafferTest.ApplicationTest.ApplicationTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows\python\GafferTest\ApplicationTest.py", line 61, in testWrapperDoesntDuplicatePaths
    self.assertEqual( value.strip(), os.environ[v] )
AssertionError: 'C:\\gaffer-pr5492-exFATfix-2023_10_09_1942-91[38502 chars]/B 0' != 'C:\\Users\\jerem\\gaffer\\startup;C:/gaffer-p[59 chars]rtup'
Diff is 37613 characters long. Set self.maxDiff to None to see it.

======================================================================
FAIL: testDurations (GafferTest.PerformanceMonitorTest.PerformanceMonitorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows\python\GafferTest\PerformanceMonitorTest.py", line 224, in testDurations
    self.assertAlmostEqual( seconds( m.plugStatistics( n2["out"] ).hashDuration ), 0.1, delta = delta )
AssertionError: 0.1104898 != 0.1 within 0.01 delta (0.010489799999999994 difference)

----------------------------------------------------------------------
Ran 1435 tests in 183.388s

FAILED (failures=2, skipped=5)

C:\gaffer-pr5492-exFATfix-2023_10_09_1942-91decc24-windows\bin>if 1 NEQ 0 (
echo "Error(s) running Gaffer"
 exit /B 1
)
"Error(s) running Gaffer"

Best,

Jeremy

[jeremy...@gmail.com]

Hi Eric,

I attached the .dmp files to your drive link, thank you.

Of note perhaps:

* 1.2.10.4 is the last version that _does_ run for me here. 1.3.0.0 and later releases fail similarly

* I've attached a .dmp file for 1.3.4.0's release as well

* debugging was a little less than straightforward because windbg was already installed as part of the Windows SDK and existed in a path with a space in it. (ugh).

* I can only guess it's something like the above (windows SDK installed) that makes our particular setup error with 1.3.x.x and up?

* some of our workstations are laptops with external monitors, and are remotely controlled with PCoIP software. I doubt this affects anything, but there are some registry keys in play to make that all work that could make our windows environments less than standard.

Best,

Jeremy

[eric...@gmail.com]

Thanks a lot for posting that crash dump Jeremy. It gives a bit of additional information to consider, but unfortunately is not as definitive as I had hoped. It does point to the crash happening in the GafferImage Python module, so that's something to go on.

On 1.3+, if you run `gaffer.cmd cli` it will (hopefully) give you a Python terminal. If you enter `import GafferImage`, what happens? Similarly, what happens if you run `gaffer.cmd test GafferImageTest`?

There were a number of upgraded dependency packages in Gaffer 1.3.0.0, like OpenEXR, OpenColorIO and OpenImageIO. It's possible there is some conflict with other versions of those or something they require. If you clear your `PATH` variable using something like `$env:PATH=''` in Powershell, then run `gaffer.cmd`, what happens?

Do you happen to have any Windows machines around that are fairly clean - no other software installed like Maya or other DCCs? Besides potential DCC conflicts it would be interesting to see if you can get it running on a system without any of the registry keys you set for the PCoIP software. I agree it doesn't sound like it should make a difference, but the more variables we can rule out, the better.

Thanks for working through this with us, it's really helpful for bug squashing and getting Gaffer running on as broad a range of hardware and configurations as possible!

- Eric

[jeremy...@gmail.com]

Hi Eric,

This helped, thank you. Gaffer did end up working on another workstation of the same spec with a new user. By running the gaffer/bin/python.exe command and inspecting the sys.path, the not-working workstations had an additional path entry: C:/Users/(user)/AppData/Roaming/Python/Python37/site-packages

In that directory were OpenImageIO and a few other things that seemed to have been the culprit. Getting those out of gaffer's paths fixed the launch problem.

Also, it looks like 1.3.5.0 has the exfat fix in place, which is great, thank you. We're up and running with the latest and greatest.

Best,

Jeremy

[eric...@gmail.com]

Hey Jeremy,

That's great! I'm glad you were able to get it sorted.There's been a ton of new additions since 0.61 that you can enjoy now.

We add the Gaffer binary paths at the beginning of `PATH` in the launch script, so it's odd that it's finding your separate OpenImageIO libraries. If keeping the path organized on your side becomes a problem, I can look into it further. We'll be doing an overhaul and organization of the binary locations on Windows as part of the necessary changes to update to a new Python version, so perhaps this will be mitigated by that change.

Best,

- Eric