Rez how to setup caching? (and what is caching)

[Braden Jennings]

Hey I'm new to Rez, and was reading the docs about caching and memcaching.

I didn't find to many details to help set it up...

I have setup a shared drive where users can share rez packages. But in order for maximum performance, it would be good if I could also have a local rez packages path.

Does rez have a feature to generate a local cache automatically, and read from that?

Cheers
Braden

[Thorsten Kaufmann]

He Braden,

yes rez does have a feature for caching. There are two distinct payload mechanisms. Payload caching is what you describe, and you can use memcached to cache file system hits resolves and package definitions. 

Documentation is sparse currently which is something we are aware of and working to improve. You can get a lot of information by reading the rezconfig file. The docstrings are really helpful. Specifically for the payload cache: https://github.com/AcademySoftwareFoundation/rez/blob/3f9f1c995bcbf1b854e905feca6cd74359c425b3/src/rez/rezconfig.py#L217

Also make sure to look into the corresponding command line tool `rez-pkg-cache`

Also i would recommend to put these kinds of questions to the Github Project either as issues or as discussion topics as this mailing list is considered mostly legacy.

Cheers,

Thorsten

[Braden Jennings]

Hey Thorsten

So I set default_cachable to True in my config file. Which is exposed via REZ_CONFIG_FILE

{

"cache_packages_path": C:\packages,

"default_cacheable": True,

"cache_package_files": True,

"debug_memcache": True

}

And my REZ_PACKAGES_PATH is on shared drive X:\packages

My guess of how caching could have worked, is the package and directory contents are automatically copied to the cache_packages_path.

However when I do "rez-env PySide2", it grabs PySide2 from the shared drive as expected, but doesn't cache the packages locally.

Do you need to be running a memcache server to do this. Does that mean every user needs to be running a memcache server.

Thanks
Braden

[Braden Jennings]

Nevermind I didn't write the config file correctly. its working now

[Ian Henckel]

Hi Braden,

You may be seeing a behavior of the payload caching feature that seems surprising at first:

The first time you use a package with caching enabled, it won't be read from the cache.

This is because the package is being copied to the cache in the background. The wiki does a good job of explaining this.

If you "rez-env PySide2" then wait a short period of time (depending on how long the copy takes), then run the exact same command again, it should read from the cache.

> Do you need to be running a memcache server to do this. 

No. The memcached server is for resolve caching which is completely independent from payload caching. cache_package_files also relates to resolve caching, not payload caching.

Best,

 - Ian

[Thorsten Kaufmann]

Good points Ian!