Thanks. I went the lazy-load route, and these confirm what I was thinking.
The filtering just doesn't work right unless you've got all the entries loaded into memory. So it's either load the entries directly onto the page itself or into a json queue at the bottom of the page, which then get pushed onto the page. ( Or make the filtered results into individual pages.)
Also, filtering issues aside, the page loads much faster with the lazy load, even the lazy loading images allows faster scrolling than the infinite scroll did.