filter=‘top’/‘bottom’ slows down DT rendering with many rows

[Alex Flyax]

Selecting filter=‘top’/‘bottom’ makes rendering a dataset with >100K rows very slow when filter=‘top’ (or ‘bottom’) is selected. I am describing the problem here, but it was described before. My dataset has numbers, characters (probably not factors at this point), and NAs. When I remove the filter, the rendering time goes from 10 minutes to seconds. When I am rendering the full dataset (~400K rows), RStudio can actually freeze with filter enabled, and it takes about 5 seconds to render without the filter.

This still doesn’t make complete sense, because I just checked, and the filter is enabled on another tab that renders the dataset quickly. But definitely removing filter from the raw_reduced_data solves the rendering delay.

Here is a copy-paste of the relevant parts of code. I should emphasize that it’s the rendering step that takes different amounts of time: it takes about 10 minutes after I see “rendering…..” printed in the console.

	output$raw_data = DT::renderDataTable( # rendering this takes ~10 sec, with 400K rows, 300 cols {
	get_select_data()
	}, options = list(orderClasses = TRUE),
	filter = 'top', rownames=FALSE
	)
	output$raw_reduced_data = DT::renderDataTable( # rendering this takes about ~10 min, with 100K rows, 4 columns
	{
	input$loadButton
	display_data <- data.table()
	isolate({
	if (input$loadButton == 0) # if loadButton hasn't been pressed yet
	return() # return nothing
	withProgress(message = 'Loading data...', {
	filtered_csv <- filter_csv()
	})
	display_data <- head(filtered_csv, 100000)
	print('filtered_csv info:')
	print(dim(display_data))
	print(class(display_data))
	print(object.size(display_data))
	})
	print(display_data)
	# 5-10 seconds to get to this point
	print('rendering.........')
	display_data # this step takes about 10 minutes for 100K rows
	}, options = list(orderClasses = TRUE),
	filter = 'top', rownames=FALSE
	)