Caching weirdness with Pollen

[Brendan Stromberger]

The tldr; is that in my situation, for this particular part of my code, I have to halt my Pollen server, reset the cache, and restart the server for any changes to be visible. This is necessary, despite having disabled Pollen's caching. Here's the context:

• I have disabled caching in my root pollen.rkt file like so:
• (module setup racket/base

  (provide (all-defined-out))

  (define poly-targets '(html))

  (define compile-cache-active #f)

  (define render-cache-active #f))
• It is unclear to me if I need to repeat this declaration in each subsequent pollen.rkt in each subdirectory. Whether I do or not, seems to have no effect on this problem, or anything else.
  
• A very brief explanation of my book for context for this question: the book I am working on is a translation of the I Ching which, long story short, centers around 64 hexagrams that look like ䷂ or ䷄ or ䷆ and so on (there are 64 of them). I needed a way to construct these hexagrams while retaining control of the individual lines (rather than simply using the unicode character, I will be building the hexagram out of <span>s containing ⚋ and ⚊ (yin and yang) unicode characters (this is so that I can enable mouse-over highlighting).
• Thus, I have a data structure that looks like this:
• (define (hexagrams) #hash(

  (1 . ('yang 'yang 'yang 'yang 'yang 'yang))

  (2 . ('yin 'yin 'yin 'yin 'yin 'yin))

  (3 . ('yang 'yin 'yin 'yin 'yang 'yin))

  (4 . ('yin 'yang 'yin 'yin 'yin 'yang))
• I refer to each of the values in the corresponding lists as "poles". I have the following function:
• (define (->monogram pole)

  (case pole

  [''yang (monogram (yang))]

  [else (monogram (yin))]))
• "monogram" returns a txexpr, and "yang" and "yin" both return unicode strings. So the result of calling (->monogram 'yin) yields <span>⚋</span>
• I'm not sure if this is related, but in this book, each hexagram represents a chapter and thus has its own file. For instance, Hexagram 1 lives in hexagrams/1.poly.pm, 2 lives in hexagrams/2.poly.pm, and so on. I wrote a function that takes the current document's filename and splits the path apart until I have the current hexagram number. I then use that number to look up the corresponding list from the hexagrams immutable hash (shown in the first code block). This provides some automation/centralization for an otherwise error-prone process of transcribing this book.

The problem is, when I may *any* changes to these functions, I have to halt my Pollen server, reset the cache, and restart the server for any changes to appear. I experience frequent caching problems across the board (despite disabling Pollen cache), but they are heisenbug-esque. This is the only case that is reproducible for me on a consistent basis.

[Matthew Butterick]

On Mar 22, 2019, at 11:03 AM, Brendan Stromberger <brendanst...@gmail.com> wrote:

• It is unclear to me if I need to repeat this declaration in each subsequent pollen.rkt in each subdirectory. Whether I do or not, seems to have no effect on this problem, or anything else.

Yes, if you have multiple "pollen.rkt" files in your project, you need to take care to connect them in whatever way is sensible. 

By default, Pollen searches up the directory tree for the first "pollen.rkt" it can find, and then applies that to the file. The effect is that the "pollen.rkt" in a certain directory automatically applies to the files to every subdirectory. However, once another "pollen.rkt" is put in a subdirectory, it supersedes the one in the parent directory. Of course, if you want to reuse the bindings of the parent "pollen.rkt", that's easy to do with `require` and `provide`:

#lang racket/base

(require "../pollen.rkt") ;; parent pollen.rkt

(provide (all-from-out "../pollen.rkt")) ;; or some subset of bindings

(module setup racket/base

  (require (submod "../pollen.rkt" setup))

  (provide (all-from-out (submod "../pollen.rkt" setup))))

Notice that the bindings from the `setup` submodule are propagated via another `setup` submodule. Here's code that won't trigger an error, but it won't work as expected:

#lang racket/base

(require "../pollen.rkt") ;; parent pollen.rkt

(provide (all-from-out "../pollen.rkt")) ;; or some subset of bindings

(require (submod "../pollen.rkt" setup))

(provide (all-from-out (submod "../pollen.rkt" setup)))

In this case, the `setup` bindings are moved into the main body of "pollen.rkt". Pollen doesn't look for them there, and hence won't find them.

The problem is, when I may *any* changes to these functions, I have to halt my Pollen server, reset the cache, and restart the server for any changes to appear. I experience frequent caching problems across the board (despite disabling Pollen cache), but they are heisenbug-esque. This is the only case that is reproducible for me on a consistent basis.

Keep in mind that you also need to disable browser caching (Pollen has no control over this). If you don't, then you may still see cached results (because the browser isn't making a new request of the project server). For instance in Chrome, the developer menu has a "Disable cache" button:

[Brendan Stromberger]

It seems to still be caching, even with your code snippet, or even if explicitly disabling the cache in each individual racket file. Also, browser cache is off.

./pollen.rkt ->

(module setup racket/base

(provide (all-defined-out))

(define poly-targets '(html))

(define compile-cache-active #f)

(define render-cache-active #f))

./body/pollen.rkt ->

(module setup racket/base

(require (submod "../pollen.rkt" setup))

(provide (all-from-out (submod "../pollen.rkt" setup))))

./body/hexagrams/pollen.rkt ->

(module setup racket/base

(require (submod "../pollen.rkt" setup))

(provide (all-from-out (submod "../pollen.rkt" setup))))

This is the current state of my code ^, but as I mentioned, even if I explicitly add

(define compile-cache-active #f) (define render-cache-active #f)

to each setup function individual, I still encounter cached variables, and the only way to invalidate the cache is to remove it entirely and restart the server.

Thanks!

Brendan

[Matthew Butterick]

On Mar 22, 2019, at 6:06 PM, Brendan Stromberger <brendanst...@gmail.com> wrote:

It seems to still be caching, even with your code snippet, or even if explicitly disabling the cache in each individual racket file. Also, browser cache is off.

I'm afraid it's hard for me to offer a better answer without a full example that demonstrates the failure. Cache invalidation is one of the two famously difficult problems in programming (the others being naming things, and off-by-one errors). 

For instance, if your "pollen.rkt" files import bindings from other .rkt modules, those secondary modules will be cached. Would you believe there's another setup value called `cache-watchlist` [1] that lets you specify other files that the cache should notice.

[1] https://docs.racket-lang.org/pollen/Setup.html#%28def._%28%28lib._pollen%2Fsetup..rkt%29._setup~3acache-watchlist%29%29

[Brendan Stromberger]

That suggestion made sense–I moved all my functions from my non-pollen.rkt modules into my root pollen.rkt, but it is still caching values until I restart the server and reset the cache. I opened up my repo in case you happen to have time to take a look; https://gitlab.com/bstro/richmond

If you `raco pollen start` and navigate to http://localhost:8080/body/hexagrams/3.html for instance, and then attempt to change the return value for the function ->monogram in ./pollen.rkt at line 204 (for instance, change (monogram (yang)) to (monogram "xxx"), and hit refresh, you should see the shape at the top of the page change. It won't, at least for me, until I halt the server and `raco pollen reset` the cache.

[Brendan Stromberger]

whoops, *now* the repo should be public. :)

[Matthew Butterick]

OK thanks, that was helpful. I was able to track down a bug where Pollen was disrespecting the render cache setting. I've pushed a fix for that. I tried it with your page and now get the right result. You can do `raco pkg update pollen` and see if it works for you.

On Mar 23, 2019, at 2:49 PM, Brendan Stromberger <brendanst...@gmail.com> wrote:

On Saturday, March 23, 2019 at 11:55:42 AM UTC-5, Brendan Stromberger wrote:

[Brendan Stromberger]

Thanks mb!