Hi Daniel,
You're right. The la4j doesn't use all the advantages of the sparse entries. All the algorithms are based on primitive get/set operations and implemented in AbstractMatrix class. This looks extremely unefficient. But, the la4j now is reaching other goal then performance. I can explain my vision of the project. We're currently finishing the second step of the 3-steps methodology that I call "suffering oriented programming". Here is these steps:
1. Make it works
2. Make it good-looking
3. Make it fast
Now we're trying to make la4j API as clear and fluent as possible. And we've almost have done it. This is the second step (version 0.4.5). The next steps would affect the performance of la4j. The version 0.5.0 will contain a new entry - sequential matrix/vector engine that allows to use all the advantages of sparse entries (do not iterate through the zero values). The next version (0.6.0) will support a "parallel engine" based on F-J Framework for the same needs. There are tons of work with la4j. We have time to do that and we have fun doing all these things.
You can read more information about the engines and other ideas of how to bring la4j into parallel world at
development blog.