Royal Flush

[Jonathon Menz]

As of 3.1.7 we've got a nice new Flushable interface (thanks Sean Harvey!) but I think it's still hard to know exactly what happens when you add ?flush=1 to your URL, and it's importance in the development workflow is way underemphasised.

Here's all the documentation I could find on the subject:

• http://doc.silverstripe.org/framework/en/reference/execution-pipeline#flush-requests
  
• http://doc.silverstripe.org/framework/en/reference/flushable
  
• http://doc.silverstripe.org/framework/en/reference/urlvariabletools

I think it's unclear for existing developers how flushing is different now than it was before (not that it was clear before), and it's probably unclear to new developers just how important Flushing is.

Think there needs to be something more like this in the Docs (please correct me if I've misunderstood or missed anything):

To optimise performance SilverStripe caches data whenever it can. As you build your site the various caches can become out of date and you will occasionally need to clear them. This is called 'flushing' and you do this by adding ?flush=1 to any URL on your site. It's especially important to perform a flush every time you add new PHP classes and templates so that SilverStripe is made aware of their existence.

These are some of the things that happen when you perform a flush:

• The class manifest is rebuilt (new PHP classes are discovered)
• All .ss templates are recompiled, and new templates are discovered
• All cached combined files (e.g. CSS/JS) are deleted and rebuilt
• All cached partial cache blocks are deleted (they will be regenerated on a subsequent request)
• Cached/resampled images are regenerated for the current request (e.g. just the images on the page you're viewing)

In addition to updating the Docs I think it's important for people learning SilverStripe development that it be possible to perform a flush directly from the /dev/ menu. Ideally dev/build would automatically perform a flush, but I assume there's a practical reason for not including this functionality*. In that case I think an additional menu item 'dev/flush' would make sense. This would make this function easily accessible and highlight it as a critical step during development.

*Is there a practical reason for not including this functionality?

[Jonathon Menz]

I forgot an important one:

• YML config files are re-read and configuration cache is rebuilt

It is not obvious that you need to flush after changing configuration as Christian rightly points out. This tripped me up when I first switched to YML config.

[Fred Condo]

On Monday, November 17, 2014 4:49:33 PM UTC-8, Jonathon Menz wrote:

I forgot an important one:

• YML config files are re-read and configuration cache is rebuilt

It is not obvious that you need to flush after changing configuration as Christian rightly points out. This tripped me up when I first switched to YML config.

Not just YML, but also private static variables from PHP. 

[Jordan Koncz]

Were these updates intended to make it so that you can now do /dev/build?flush=all, and it will perform a dev/build and a full flush in a single step?

This did not work previously (the flush would be ignored as the page doesn't use SSViewer - see https://groups.google.com/d/msg/silverstripe-dev/RNWCiFAnRI0/QaUuI35wXUEJ). I started using swaiba's DevelopmentAdmin extension to try to get this working (see https://groups.google.com/d/msg/silverstripe-dev/RNWCiFAnRI0/bABhA5Otnl4J) but found it a little unreliable.

I've done a bit of testing and it doesn't seem like the updates in 3.1.7 have made it possible to perform a dev/build and a full flush in a single step, but I'm not sure if they were ever intended to do this.

[Jonathon Menz]

From what I can tell as of SS3.1.7 visiting /dev/build?flush=1 should flush everything. Note that '?flush=all' doesn't behave any differently to '?flush=1' or just '?flush' at this time. All that matters is that the flush parameter is present. I guess even '?flush=0' would work.

[Jordan Koncz]

Have you done any testing and confirmed that this is the case? I performed the following test:

1. Renamed an existing template, and updated the <% include %> for this template to reference the new name
2. Tried to load the page containing this include, and got an error about no template found for the new include name (as expected)
3. Visited site.com/dev/build?flush=1
4. Visited site.com/dev/build?flush=all (just for good measure)
5. Tried to load the page containing the updated include, and still got the same error
6. Visited site.com/?flush=all
7. Tried to load the page containing the updated include, and this time it loaded fine

 So based on that, it seems to me that the updates in 3.1.7 have not changed the fact that doing a dev/build and flush together simply ignores the flush and just does a dev/build.

[Jonathon Menz]

Just tested now and you're quite right. I can confirm that SSViewer::flush() is called on a visit to /dev/build?flush=1, but the template manifest isn't rebuilt. I think there must be a problem with the SSViewier::flush_template_cache() function.

[Jordan Koncz]

So do you think that the updates in 3.1.7 were intended to allow a dev/build and flush to be performed at the same time, but aren't quite working properly? Or have I misunderstood these updates and they were never meant to allow this?

[Jonathon Menz]

I think that was the idea but it looks like SS_TemplateManifest was forgotten. I'll raise an issue on github.

[Jonathon Menz]

There are existing issues:

• https://github.com/silverstripe/silverstripe-framework/issues/3093
  
• https://github.com/silverstripe/silverstripe-framework/issues/2325
  

[Loz Calver]

Thanks for bringing this up and working through it!

I think it's unclear for existing developers how flushing is different now than it was before

The idea is that it shouldn’t be any different to how it was before, we’re just providing a more standardised way of triggering flush-like behaviour in user code (rather than checking isset($_GET['flush'])). If something’s changed, it’s probably a regression so should be addressed.

Ideally dev/build would automatically perform a flush

It should do this for almost all core code, but I agree there’s some ambiguity between what a flush does and what a dev/build does. Perhaps we should add a Buildable interface to match Flushable that’s then triggered on a dev/build? Though we’d have two classes doing nearly the same job (unless anyone can clarify the difference between flush and dev/build?).

YML config files are re-read and configuration cache is rebuilt

These should be re-read/rebuilt on both a dev/build and a flush. If they’re not, I’d say that’s a bug, but I just ran a quick test (changing a YAML value) and it worked for me. I’ve not tested PHP statics, though.

From what I can tell as of SS3.1.7 visiting /dev/build?flush=1 should flush everything. Note that '?flush=all' doesn't behave any differently to '?flush=1' or just '?flush' at this time. All that matters is that the flush parameter is present. I guess even '?flush=0' would work.

Yep it should flush everything. To summarise:

• dev/build - should rebuild database, class manifest (detect new PHP classes/files), config manifest (YAML config), config static manifest (PHP statics config) and template manifest
• dev/build?flush - should do all of the above, plus trigger flush() in any classes that implement Flushable. 

?flush, ?flush=1, ?flush=anything, ?flush=0 should all work. I’d be reluctant to change Flushable to trigger on a dev/build: it goes against the intended purpose of the class (even if it achieves almost the same thing) - all the docs indicate it’s only triggered on a flush, not dev/build, so to change its behaviour would probably be an API change.

Loz

[Jonathon Menz]

Thanks for the reply Loz, I think it's starting to make sense.

It sounds like once this bug is fixed up, there usually shouldn't be much need to use ?flush=1 when building anymore. In fact during normal development it shouldn't be needed much at all should it? That in itself is quite different to how it was before :) and totally great.

So in fact, I take it back, the important of flushing isn't underemphasised (and probably not under-documented). It's just a tiny bit broken at the moment.

[Jonathon Menz]

Mmmph, actually, this is still an issue: https://github.com/silverstripe/silverstripe-framework/issues/2325

If you visit dev/build without ?flush in the URL, neither the config manifest nor template manifest are rebuilt as intended. Only the class manifest is rebuilt.