Drawbacks to the filesystem, and suggestions for an overhaul

[Leaf]

Hello,

I'm not an experienced animator, so I may be unaware of some of the current filesystem's benefits. But even after a fair few practice animations, I don't think OpenToonz's way of organising projects is suitable for newcomers.

The following is the present system, as I understand it:

• Projects can be created only in directories chosen beforehand (in File > Preferences > General > Additional Project Locations).
  
  
• There are no OpenToonz project files; each project is managed exclusively by the OpenToonz installation. When a new one is set up, a folder with its name is created in the selected directory, along with a file with the same name followed by "_otprj.xml".
  
  
• Project files are stored in folders organised by type (scanned drawings, resource files, exported videos, etc.).
  
  
• A project is divided into scenes, each stored in its own (.tnz) file in the "scenes" folder (by default).
  
  
• A scene may hold one or multiple animation layers; each layer also has its own file or set of files, presumably to facilitate animation reuse.

I have several problems with this filesystem:

1. It's non-portable. Because the user's projects are governed internally by OpenToonz, moving or copying them to another computer is very awkward; they will have to be re-created manually on each new computer.
   
   In addition, any file references using absolute paths may cause files to lose track of each other, even if their relative locations remain the same.
   
   
   
2. It can be messy. For example, a user might want to experiment with a large set of small scenes, and store resources in their respective scenes' folders.
   
   But right now this doesn't seem possible, as resource files can only be separated by type, and appending the $scenepath variable to paths in the project settings leads to an even more convoluted folder hierarchy.
   
   Moreover, the value of $scenepath seems to change depending on where it's placed in a path: sometimes it references the path to the TNZ file from the root folder (folder_name/ folder_name/ scene_name), other times just the scene name by itself. (In both cases, I believe the extension is removed.)
   
   
   
3. It's needlessly restrictive. Similarly to the file management point above, the user should ideally have the freedom to choose and change the root directory arbitrarily and on a per-project basis, without needing to pre-approve it in the software settings.
   

To improve OpenToonz's user experience, I'd like to make a few suggestions for a new filesystem. Some of them may demand long-term efforts, or come at the cost of backwards compatibility, but if they are tractable, I believe the result would be a major advancement.

1. Implementing OpenToonz project files. These could hold:
   
   
   • the contents of "_otprj.xml" files;
     
     
   • general data such as a title, framerate and resolution (this data could be overridden in scenes);
     
     
   • a list of paths for scenes, output files, and maybe even palettes and scripts used project-wide.
     
     
   
   Storing a project's basic information in an independent file, and decoupling it from the software installation, would immediately make it much easier to identify, maintain, and share.
   
   Incidentally, if all the scenes' information were incorporated into these project files, TNZ files themselves might no longer be needed.
   
   
   
2. Allowing a custom path for every file belonging to a project, as well as user-defined default directories, rather than rigidly enforcing a pre-determined folder tree. Leaving file management to the user's discretion would grant a much higher level of flexibility.
   
   
   
3. Permitting the user to select between a relative or absolute path in every conceivable context, maybe with a simple checkbox. This would likely lessen the need for the "Replace Level" and "Replace Parent Directory" layer functions.
   
   
   
4. This thread states that deleting a project is not possible through the interface in the 1.2 release; I haven't seen the option anywhere in the 1.3 version either.
   
   This isn't quite as problematic, but it would be a helpful feature.
   

Perhaps not all of these ideas are possible, but I hope this post at least sheds some light on what I think are sizeable issues.

I'm grateful to Shun Iwasawa and the core team for continuing to fix and improve OpenToonz. In terms of stability and usability, the program has certainly come a long way since its release a few years ago. I believe the next major step to widespread adoption is reworking the user experience to be more traditional, approachable and intuitive.

Thank you.

[DarrenT]

Hi. I'm pleased this topic has been raised. I've wondered about some of this for a while myself.

Personally, I don't find the folder structure too confusing. And moving projects or sharing them isn't hard, as all of the files are stored in the same project folder (I always use import as the option when adding images and sound files, so that keeps them together). Having so many changes to allow for relative paths for each type of file stored doesn't necessarily help and will add so much complexity to a project.

But yes, having to set the permissible folders for storing projects is incredibly outdated. I should just be able to start a new project and select a save location. I think this restriction is just historical and something that originally made the built-in file browser work better. Also, there's really no need for our own file browser. We should be using the local operating system's browser.

I think when you create a new project from the dialog and you type a name in, it should save immediately, as this is often a source of error for users.

And I agree, you should be able to delete a project from inside OpenToonz.

One of my personal pet peeves is also the project file:

1.If we need to have a file with the project name and you save your project as: "CatCartoon", the project file should be "CatCartoon.otproj". Renaming these files is fiddly, to rename the project, making sure you don't lose any of the appended extension. Again, for new users, it's not an easy proposition.

2. Why do we need to have a file with the same name as the project? Having this means they need keeping in sync. I can't think of a reason why it can't be called, "project.xml" for every project. The folder tells you the name of the project. And if you need to, you can write it in the xml. But if you use this to quickly scan for valid projects, you could always call it, "project.tnz", then you know it's a toonz project file.

Finally, as has been discussed in other user groups, a killer feature would be to have a single save file. So we could save the individual files (as we currently do or in some slightly tweeked format) in a temporary folder, but then zip them up into a single file. Then it's this single file that you can share with other users.

[Shun Iwasawa]

Hi, thank you for the suggestions.

1. Implementing OpenToonz project files. These could hold:

• the contents of "_otprj.xml" files;
• general data such as a title, framerate and resolution (this data could be overridden in scenes);
• a list of paths for scenes, output files, and maybe even palettes and scripts used project-wide.

Storing a project's basic information in an independent file, and decoupling it from the software installation, would immediately make it much easier to identify, maintain, and share.
Incidentally, if all the scenes' information were incorporated into these project files, TNZ files themselves might no longer be needed.

 Please let me point that the framerate and resolution info are already in the project settings ("_otprj.xml" file). They are saved by "Save Default Settings" command and will be default values when you create a new scene in the project.

Yes, currently the title information is missing. Episode number and take number information are also missing in the scene. I agree if the project and the scene has such meta data it will be useful for showing them in clapperboard.

Your idea of consolidating all information to one project file would  be IMO a bit risky as all your works may be spoiled once the project file is broken for some reasons.

2. Allowing a custom path for every file belonging to a project, as well as user-defined default directories, rather than rigidly enforcing a pre-determined folder tree. Leaving file management to the user's discretion would grant a much higher level of flexibility.

 

Please note that you are not restricted to store the asset under the project root. You can set any location for every aliases (i.e. setting like +drawings=C:\some\local\folder and +scenes=G:\some\shared\folder\in\network , etc.). And you can add your own alias by adding a line to "OpenToonz stuff\profiles\project_folders.txt".

3. Permitting the user to select between a relative or absolute path in every conceivable context, maybe with a simple checkbox. This would likely lessen the need for the "Replace Level" and "Replace Parent Directory" layer functions.

 I'm not sure if it will meet your demand, but every file paths used in OT can use "$scenefolder" alias, which enables to access assets with relative path from the current scene file.

Regarding the portability, I'm thinking of adding a feature to gather all assets used in the scene to the specified folder, just like "Collect Files" command in AfterEffects.

[Leaf]

Thank you both for responding.

That "Save Default Settings" button is convenient, I'm glad to find out about it.

You make a good point about file corruption, it would be an absolute catastrophe for any decently-sized project to become inaccessible because of a faulty save or file transfer. I suppose this goes back to the idea of scope; none of my OT animations so far have been ambitious enough to worry much about broken files.

I didn't mention this before since it seemed too complicated, but maybe the best solution would be to allow the import and export of scene files and animation layers. So a small project might group everything into one file, whereas a bigger one could be split up into project, scene and layer files. Admittedly, scene and "_otprj.xml" files would no longer be in human-readable XML format, but I'd prefer to edit files directly in OT than in a text editor.

Of course, that would be a massive undertaking. In any case, even a basic stand-alone file (with an .otprj extension or something similar), which could be opened automatically in OT, would simplify things.

---

About project directories: I did find your GitHub post about $scenefolder. After locating the setting (in File > Preferences > General > Path Alias Priority), choosing "Scene Folder Alias ($scenefolder)" and relaunching OT, I've found its behaviour to be similarly unpredictable to $scenepath's.

As a test, I created a project named MyProject, to which I added a scene: MyProject/scenes/MyScene/MyScene.tnz.

In the project settings, I tried different paths for animation levels, restarting OT after every change. Here is where my animation levels were saved (by default), depending on the value of +drawings:

    $scenepath\drawings:            MyProject\scenes\MyScene
    $scenefolder\drawings:          MyProject\scenes\MyScene
   
   
Even when I tried adding more subfolders, none of them were created in the right locations, if at all. Strangely, when I tried resetting Path Alias Priority to its default value ("Use Project Folder Aliases Only"), this happened:

    $scenepath\drawings:            MyProject\MyScene\MyScene\drawings
   
   
Unless I'm doing something wrong, these aliases don't appear to work as intended. However, after re-enabling $scenefolder, I noticed that when I right-clicked on the timeline and selected "New Level...", the level was in fact placed in the correct folder:

    $scenefolder\drawings:            MyProject\scenes\MyScene\drawings
   

(As a side-note, I'd recommend moving the "Path Alias Priority" option to the project settings window, where it would be less obscure.)

---

I've never used AfterEffects, but my hope is to be able to move or copy projects to other workstations without having to make extra adjustments.

[Momotron2000]

Is it planned to have some kind of a donation system in the future so more devs can work on specific improvements? This could help the development for OT a lot!

[Rodney]

Momo Tron,

Currently BountySource is used for financial contributions to support Opentoonz development.

I look forward to the day when more direct support mechanisms might be used.

[Momo Tron]

Rodney,

thank you. I don't see a link on your homepage. Please make a fat button on top :-)

Momotron

[Leaf]

In any case, OpenToonz has made good progress so far and is, for the most part, fairly usable now, after three major revisions.

I hope the matters I've called attention to are taken into consideration for the next major update, if at all possible.

[Snapai]

OpenToonz's projects share an internal structure similar to Adobe Flash/Animate: a folder with some XML specs in them, and then subfolders with different types of data in them.

The difference, is that Animate by default stashes its projects in a zip format: your modern .fla project is actually an XFL project folder zipped. (don't remember whether they're using pkzip, gzip, bzip2, xz)

Haivng a nice open editable project folder is very nice for working in an environment where you may be checking files into/out of version control like in a studio, but it does sometimes mislead people into thinking the folders and layout don't matter so much.

A zipped project folder might make more sense for sharing files - though if there are a lot of raster levels it could get VERY large, very quickly.

Certainly would be nice if project folders weren't limited to OT project roots the way they currently are though - especially since other than browsing for them, it seems the program doesn't much mind where you saved your project, it will just make it harder for you to navigate to find it again if you don't have it in your recent items.

[Leaf]

Naturally, even a modest animation project will become quite sizeable as drawings are sketched, inked and coloured. Rising project complexity is unavoidable, which is why I hope to see improved interface and filesystem flexibility in future versions of OT.