Failed to create temp file - Too many open files - Could not open file for reading

[perspixe]

While loading a 2800 image project.
The pts file originate from an export from RodeonPreview. File names
ending in cr2 in the file where replaced by same ending with TIF
because I preprocess the files outside PTGui, that's what I usually do
and it works fine. Then the file was opened in PTGui Pro, path to the
Tifs given in the dialog and fov was set up and panorama was moved
25.5 deg yaw by numerical transform to sit at the center as I know my
original h fov was 51 deg and file saved, all while it was still
loading images. When I came back to the computer it had stopped on
these errors described in the title of the post.

The linked zip below contains project file as well as screen captures
of the dialog boxes telling the error.
<http://www.artincube.com/TooManyFilesOpen.zip>

I did restart the computer and reformat the scratch volume and it
repeated the error again at the same point, with around 10200 tmp
files on the scratch.
When I close the dialog boxes, Panorama editor window becomes blank
white as well as project window. Any click brings error dialog box
again. I could Command-Q Quit PTGui, it takes a little time to clean
the scratch (erase all tmp files), and then crashes.

I also tried the original export file with cr2 changed to tif only
without trying to modify anything (fov, yaw) while it is loading the
images, did the same. Panorama editor window was also open to see the
images coming in and being warped in position.

Greg

[Joergen Geerds]

ptgui had this bug sometime in the 8.x series, but I haven't seen it
since Joost fixed it, and Jeffrey didn't seem to have seen this bug
with his recent 4000 tiles library project either...

[Erik Krause]

Am 15.04.2011 23:44, schrieb perspixe:
> While loading a 2800 image project.

You didn't mention your OS, scratch disk file system, empty space, RAM
and PTGui version...

--
Erik Krause
http://www.erik-krause.de

[perspixe]

Mac OS 10.6.7
File System Mac OS Extended Journaled non case sensitive
Scratch disk is an OS X soft RAid 0 array of 3x3TB Hitachi
HDS723030ALA640 created in Disk Utility
It is completely empty (I said in the post that I tried to reformat it
to try again, so it is empty). At time of error there is around 10200
tmp files on it using 110GB (8.89GB free space out of 9TB, all base 10
figures)
Ram is 16GB with 60% to PTGui, no swapping
PTGui is 9.0.3

I have opened same kind of file with 1000 image with no issue

[Joergen Geerds]

the last time this happened, Joost was so kind to find the bug in the
code and fix it...
barely any way around it... speculative, I would suggest that this is
some kind of OS problem, having +10k files open at the same time...
you could try to specify 2 same size scratch drives as preferred in
the ptgui prefs... ptgui will automatically spread the IO traffic
across both drives, hence maybe eliminating the "too many files open"
error temporarily...

I would send Joost the pts file plus any crash report so he gets a
better idea whats happening.

joergen
360nyc.com

[perspixe]

Oh well, I surely hope this can be sorted somehow.
I'll give a try to the idea of 2 prefered volumes. Do they really need
to be same size?
I put a link to the pts file in the first post so Joost will have
that.
About crash reports, PTGui actually crashes when deleting the very
last temp files from the scratch disk, I'll grab the crash report and
send it.
Thanks!

By the way the library is 3000 so very close to mine in term of number
of images, not quyite 4000.

[perspixe]

I did set my 3 3TB HDs all as single drive and prefered.
They indeed about shared the load of temp files between them.
The issue came again with Scratch1 having 5367 temp files, Scratch2
2429 temp files and Scratch3 2402 temp files.
So a total of 10198, same number as with a single volume for Scratch.
So the issue is not the number of open files per drive/volume but the
number of open files total. and that number is just under 10200.
I hope we can sort this out.

For Joost: I updated the zip linked in the first post with 2 crash
reports (PTGui crashing at the end of deleting the temp files as in my
first post)

Going to bed now.

Greg

On Apr 16, 12:49 am, Joergen Geerds <jgee...@gmail.com> wrote:

[perspixe]

Problem: limitation on 10.6 for the number of open files per process
(10240)

Source of solution:
<http://discussions.apple.com/thread.jspa?messageID=7153640> 2nd post
page 1

Last login: Sat Apr 16 02:19:11 on console
GregMP:~ greg$ sudo sysctl -w kern.maxfilesperproc=65536
Password:
kern.maxfilesperproc: 10240 -> 65536
GregMP:~ greg$

10240 is the number given by the system as the default setting AND
corresponds to the limit in my errors.

<<
sudo sysctl -w kern.maxfilesperproc=65536 (or some other lower figure)
This change is temporary. You could set it in a file /etc/sysctl.conf
that is read at boot.
Just add the line:
kern.maxfilesperproc=65536
>>
I haven't done the permanent solution: there is no file as described
where to add the line.
Maybe I could just create the file. Not sure. And I don't want to do
something bad just trying to do it.
I am not that much of a geek really.

Joost, I have no idea how you will implement a solution that doesn't
involve your user having to enter an administrator password.
Good luck

Greg

[perspixe]

Argghhhh I posted too fast.
It passed 10240 and then did it again at 11425 what the f.
I can't believe it. Must have made a mistake somewhere.
Pfff

[perspixe]

Sorry I am messing up the subject of the thread, misubderstood the use
of "Edit Subject"

[Erik Krause]

Am 16.04.2011 00:02, schrieb perspixe:
> File System Mac OS Extended Journaled non case sensitive
> Scratch disk

I don't know about your particular file system, but some FS are limited
in the number of files per directory. See f.e. footnote 8 on
http://www.inthelight.co.nz/techo/pcfilesystem.htm

[Joergen Geerds]

On Apr 16, 7:06 am, Erik Krause <erik.kra...@gmx.de> wrote:
> I don't know about your particular file system, but some FS are limited
> in the number of files per directory. See f.e. footnote 8 onhttp://www.inthelight.co.nz/techo/pcfilesystem.htm

yes, I saw that one too, windows seems to have issues with folders
with more than 10k files... and if greg would have used windows, he
might have had hit another brickwall with his project. that was the
reason why I suggested to split the scratch into 2-3 drives... but it
turned out to be an OS issue, how many files each process can have
open... and I hope Joost can find a solution... btw, "df -i" will give
you the number of free inodes on a hfs+ system... i think each inodes
represents a file, and there are tens to hundreds of million inodes on
a hfs+ drive... so it's difficult to hit the same 10k brickwall as
with windows.

joergen
360nyc.com

[Gregoire Dupond]

Joergen,

So do you mean I should not even try using Windows 7?

Greg

> --
> You received this message because you are subscribed to the Google Groups "PTGui" group.
> To post to this group, send email to pt...@googlegroups.com
> To unsubscribe from this group, send email to ptgui+un...@googlegroups.com
> Please do not add attachments to your posts; instead you may upload files at
> http://groups.google.com/group/ptgui/files
> For more options, visit this group at http://groups.google.com/group/ptgui

[Joergen Geerds]

On Apr 16, 9:29 am, Gregoire Dupond <persp...@gmail.com> wrote:
> So do you mean I should not even try using Windows 7?

it means I do not know enough about windows to make any judgement of
this kind.
NTFS (I assume this is the current win7 file system) has a theoretical
limit of 4 billion files per drive/directory, but there could be a
similar mechanism, similar to the ext2/ext3 filesystems, where the OS
limits the amount of files to maintain some form of efficiency/
usability (also similar to the 10240 limit you found with OSX).

ptgui, in terms of performance, isn't much different on the two OS on
similar machines.

sorry, but this is something for Joost, I am just surprised that
nobody else stumbled over the 4000 tiles problem before.

joergen
360nyc.com

[Gregoire Dupond]

Quite a surprise for me too.
Again, not even 4000 but 2783 tiles!
The library took 3000 apparently...

[Joergen Geerds]

On Apr 16, 10:52 am, Gregoire Dupond <persp...@gmail.com> wrote:
> Quite a surprise for me too.
> Again, not even 4000 but 2783 tiles!
> The library took 3000 apparently...

the library was rendered under windows, as far as I know... so, if
this is a mission critical render, that can' wait for Joost's fix, I
would give it a try... even a modest €800 machine will probably do
well with some 8-12GM RAM and 3-4 sata drives (just don't use win7
home), just for rendering... you can still use your mac for setting up
all the projects and the PS work, and then simply move it over for
rendering to the PC.

joergen

[Gregoire Dupond]

I can have access to a good W7 PC from Monday this is not a problem.
And no I cannot set up projects on the Mac Pro because it is during opening of the pts file that it makes problem.
I'll get the PC Monday for trials. It probably has 16 or 24GB ram and I have hard drives to move into it.

[Joergen Geerds]

On Apr 16, 11:10 am, Gregoire Dupond <persp...@gmail.com> wrote:
> And no I cannot set up projects on the Mac Pro because it is during opening of the pts file that it makes problem.

moving the pts file to a PC is no problem. ptgui (pc) will ask you to
update the path to the source files, then simply set the new output
path, save and render...

joergen
360nyc.com

[perspixe]

I'm done now!
I think it is solved.
I spent my Saturday preparing the vegetable garden, lot of digging and
efforts, having fun with my family, and got my mind clear for the end
of the day.
And found out.

In fact there were 2 limits.
First one is max number of open files per process, 10240 default.
When I took care of that one earlier and it did make the error few
hundreds files later I was baffled.
Then I realised that there is a second limit, the total number of open
files, and that is 12288, hence the error "few hundreds file later".

I created a file sysctl.conf
put these 2 lines in it:
kern.maxfilesperproc=65536
kern.maxfiles=65536
and drop the file in /etc/

On reboot I checked the limits again and they were both at 65536. I
just hope doing so will not create unforeseen other issues later.
I opened my project and no error this time, I have 19915 tmp files in
my scratch disk.

I guess that one has to go in the Q&A.
Sure it will happen to more people as more people tend to break the
record of the biggest any other day.

Greg

[perspixe]

Bummer, as I started doing things: moving the yellow top crop line
down, the error came again.
I was at 19915 so there must be another limit around 20000.
So not solved yet.

[Gregoire Dupond]

Another update:

I manage to open the file fully now, manipulate it a bit (fov, numerical yaw and pitch change and save).
Unfortunately the file still crashes on trying to save after Generation of control points; I open another thread as I think that this might not ne related to that.

Solution for Open Files limit

To change the limits without having to come back to it (except OS upgrades maybe)
1 files is needed in /etc/
and not the sysctl.conf as I previously thought

launchd.conf
limit maxfiles 65536 65536

the first number is per user/process, the second for system.
Default for system is unlimited, but changing the first value (originaly at 256) and keeping the second unlimited return an error saying that it can't be set to unlimited (when changed from terminal, if unlimited is used in launchd.conf file the command is simply ignored and limits kept at default)

To create or open launchd.conf in /etc/ use the pico command:
sudo pico /etc/launchd.conf

These values will automatically change the sysctl limits to 65536 also

To check limits after reboot:

launchctl limit
should return
maxfiles 65536 65536

sysctl -a | grep maxfiles
should return
kern.maxfilesperproc=65536
kern.maxfiles=65536

sysctl -a | grep num_files
will give you the number of currently open files (total I think but not sure)

ulimit -a
should return
open files (-n) 65536

Needless to say I manage to find this through error and trial and reading threads.
I was lucky not to have my trials and errors making my system unusable or not to loose all my data (well, I have strong backups so that would be really unlucky).
Of course I do not warranty that someone else will be as lucky as I have been when trying this. This is someone else's own risk… so npot to be done unless needed (running into same problem).
I chose 65536 but it is possible to choose less, and probably more also, but I haven't tried it.

sysctl -a | grep num_files
will give you the number of open files

[PTGui Support]

Hi Gregoire,

Thanks for reporting the problem and figuring out the workaround, this
is very useful to know.

I'll make a next version of PTGui less 'file hungry'; currently it keeps
the temp files open as long as they exist but PTGui could close some
files and reopen them when needed.

The issue is mac only; on windows one can open hundreds of thousands of
files.

If PTGui is crashing the crash log would probably provide detailed
information. Please send it to me at sup...@ptgui.com (or contact me if
you don't know where to look for the crash log).

Joost

[PTGui Support]

Actually one of the PTGui 9 betas had the open file limit set to 256
(which is actually OS X's default for normal processes that don't to a
sysctl to change it)

Joost

[Gregoire Dupond]

Hey Joost

amother update.

I Generated control points and then 2 things happen:

I try to save simply (no dialog boxes): it begins to show the progress bar of saving and then says too many files open and can't save

I try to save as...: PTGui crashes because it can't load som interface elements:

2011-04-17 13:43:43 PTGui Pro[303] Unable to read contents in the Interface Builder file at path /System/Library/Frameworks/AppKit.framework/Resources/English.lproj/NSNavPanelContentView.nib due to error: The operation couldn\U2019t be completed. (NSCocoaErrorDomain error 256.).

2011-04-17 13:43:43 PTGui Pro[303] unable to find nib named: NSNavView in bundle path: /System/Library/Frameworks/AppKit.framework/Resources

2011-04-17 13:43:44 ReportCrash[1388] Saved crash report for PTGui Pro[303] version ??? (9.0.3) to /Users/greg/Library/Logs/DiagnosticReports/PTGui Pro_2011-04-17-134344_GregMP.crash

2011-04-17 13:43:45 com.apple.launchd.peruser.501[181] ([0x0-0x23023].com.ptgui.PTGui[303]) Job appears to have crashed: Segmentation fault

But I can open other apps and create files etc etc

So PTGui has a limit there and I'm on it. My limits for OS have been set very high (I went up to 200 000) and it does the same always, so the limit is in PTGui.

Do you have a beta or something that can be modified quickly and easily that have a higher open file limit than actually?

That would solve my issue I think.

If that is not easy and cannot be done quickly I'll have to use the slicer tool Erik talked about recently on another thread to cut my file in 2 so I can generate control points, optimise and render, putting these 2 halves together in Photoshop after. That is my last resort. I'd much prefer have all in one file for the set up and optimisation.

Thanks for helping out.

Greg

[Gregoire Dupond]

> I'll make a next version of PTGui less 'file hungry'

That sounds like a good idea ;-D
For the crash log look my other post about loading UI elements from framework.
It can't because it has reached a limit for open files, and then crashes.
I can still send you the logs if needed.

Greg

[perspixe]

Joost,

I found this post:
<http://groups.google.com/group/ptgui/msg/23525bac35fe536e>
where Hans gives detail about his setup or something. How did he get
these infos?
I am interested by the lines:
> Max number of files: current: 10240, hard limit: -1
> Number of open files: (0 vmh, 0 vmf, 0 rh)
The first line gives the open file limit I ran on at first, before
changing limits system wide.
The second line is in my dialog boxes telling me files can't be open
or read...
Now the values in the dialog boxes are (19913 vmh, 19913 vmf, 0 rh).
For information, I had at that moment 19913 tmp files on the scratch
volume.

Could that help finding how to push the limit up in PTGui?

Greg

On Apr 17, 3:19 pm, PTGui Support <supp...@ptgui.com> wrote:
> Hi Gregoire,
>
> Thanks for reporting the problem and figuring out the workaround, this
> is very useful to know.
>
> I'll make a next version of PTGui less 'file hungry'; currently it keeps
> the temp files open as long as they exist but PTGui could close some
> files and reopen them when needed.
>
> The issue is mac only; on windows one can open hundreds of thousands of
> files.
>
> If PTGui is crashing the crash log would probably provide detailed

> information. Please send it to me at supp...@ptgui.com (or contact me if

[PTGui Support]

Hi Greg,

This data is from Help - System Information in PTGui.

What does the 'max number of open files' line say on your system? Could
you send me the full contents of System Info by email?

Joost

[Gregoire Dupond]

SIMD current: SSE2, supported: SSE2

Platform: mac64/mac

OS version: 10.6.7

Number of CPUs: 8

Physical memory: 16384 MB

Virtual memory: 8388608 MB

Max number of files: current: 19922, hard limit: 165536

Number of open files: (0 vmh, 0 vmf, 0 rh)

Timer ticks per second: 1000000

Has default project: No

This is what I discovered by trying and having problems everytime when reaching 19913 open files:

The hard limit says what I did set up in the terminal but the number 19922 for current is my problem... how can we increase that. That number is nowhere in the limits I found in the terminal.

Is it within PTGui? How can we change that?

thanks

[PTGui Support]

Hi Greg,

Ah yes, PTGui doesn't attempt to increase the limit beyond 20000 (or
slightly less as you see). A classic case of '640k ought to be enough
for anyone' :-) Will be fixed in the next update.

Joost

>> <mailto:pt...@googlegroups.com>

>> To unsubscribe from this group, send email to
>> ptgui+un...@googlegroups.com

>> <mailto:ptgui+un...@googlegroups.com>

[Gregoire Dupond]

Hi Joost

mmmh yes ok. I am in BootCamp now on the same MacPro, Windows 7, reorganised my hard drives for it.
No more problems really, opened the file, generated control points, saved.
I cannot optimise yet because somehow RodeonPreview registered a column of files at the wrong position so I have to reorganise that a bit.

I said mmmh because I'd prefer to stick with OSX on my Mac Pro, but this is really for this huge pano only so as soon as its done I'll be back to my usual environment (until next time if you haven't updated before). Not the first time I'm force into Windows for some stuff so I don't mind really, and I have my MacBookPro to feel "home" ;-D.

Hoping to see a great update very soon. Carry on the good work.
cheers
Greg

[Gregoire Dupond]

Hello Joost,

this thread alive again.

> Will be fixed in the next update.

I forgot a bit what was supposed to be fixed on your side. But I still have to do the limit maxfiles trick.

Just was making some test on my new MacBook Pro 2012 2.6GHz 16GB (Retina), using SSD in thunderbolt and USB 3 for data and temp volume,
The test was to open a project made of 1579 16 bits 18 megapixel images. The project file itself has a lot of masking data.
It was created and all optimised etc on my Mac Pro, the computer I had the problem described in this thread with, 15 months ago.

And on the new machine I had the "too many open files" error again.
I applied the same recipe as 15 months ago and it worked.
To be noted /etc/launchd.conf I created under 10.6.7 was conserved all the way to 10.7.4 through all OS updates:

I repeat here the recipe

create or open launchd.conf in /etc/ use the pico command:
sudo pico /etc/launchd.conf

add this line to the file:
limit maxfiles 65536 65536

save the file and reboot

To check limits after reboot:

launchctl limit
should return
maxfiles 65536 65536


sysctl -a | grep maxfiles
should return
kern.maxfilesperproc=65536
kern.maxfiles=65536

ulimit -a
should return
open files (-n) 65536

Greg

[PTGui Support]

Hi Greg,

Actually PTGui was modified since that original posting, the soft file
limit was increased to 70000 files. But apparently there is another
limit in the OS as you describe. Will look into it.

Joost

[Gregoire Dupond]

I thought so.
just do a launchctl limit
in the terminal, and you'll be surprised to see 256 limit for open files per app, unlimited total.

xxGreg

> Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.

[PTGui Support]

Ah yes, but that 256 file limit is actually overridden by PTGui
(otherwise you would run into problems even with small panoramas).

If you do Help - System Information in PTGui, you'll see a line 'Max
number of open files: current'; this is the file limit that was granted
by the OS. On my 10.7 machine this is 10199 files. Apparently this limit
can only be raised by doing a ulimit.

Previously PTGui would only request up to 20000 max open files even if
the ulimit was set higher. Now it will attempt to go up to 65k.

I'll leave it as it is for now, there's a workaround and the ultimate
solution would be to make PTGui less file hungry, that's what I'm aiming
for.

Joost

[Greg]

On my system information from help menu it says

Max number of files: current: 65504, hard limit: 65536

This is after changing the limit with
sudo pico /etc/launchd.conf
limit maxfiles 65536 65536

I just got rid of this file and rebooted and now I've got like you

Max number of files: current: 10199, hard limit: -1

I'm fine to live with this little trick

> Previously PTGui would only request up to 20000 max open files even if the ulimit was set higher. Now it will attempt to go up to 65k.

as this makes it possible

Thanks
Greg