It's true that icons take time to load but they are cached so loading time is reduced to absolute minimum. At the moment most of the loading time is spent on getting file details. While there are other options in how to load these details it's still fast enough. Directory cache will probably handle this much better.
Currently only number of items is loaded in main thread. The rest is loaded in separate thread. Program actually lists the directory first, decide on how many items we need to preload and split the list in two.
Torsten, how many files in big directory are we talking here about? Are there other program who load directory faster (not counting the CLI ones)?