My attempt at a "chunk" filter doesn't work, for no obvious reason | Ole Friis Østergaard | 8/24/12 10:44 AM | Hi there! I'm trying to come up with a good solution to this StackOverflow question regarding Angular and chunking up values into rows for use in Twitter Bootstrap layouts: http://stackoverflow.com/questions/11056819/how-would-i-use-angularjs-ng-repeat-with-twitter-bootstraps-scaffolding There must be a better way to solve this problem than the current two replies, and I thought of making a filter called "chunk" that, given an array, would output an array of which contains chunks of the original array. I've put my attempt up as a jsfiddle: http://jsfiddle.net/6vNcX/3/ The problem is that in practice the filter works, but if you look at the JavaScript console, you'll see that Angular has performed 10 $digest runs, and then given up stabilizing the view. For bigger data sets, this results in a great lag in the UI. The problem suggests that my filter's outputs on each $digest are not equal in the Angular sense, but I have a unit test which shows that it ought to work just fine (the last test): 'use strict'; describe('filters', function() { beforeEach(module('chunkExampleApp')); describe('chunk', function() { it('should let non-Array input pass through', inject(function(chunkFilter) { var input = 'this is not an array'; expect(chunkFilter(input)).toBe(input); })); it('should let empty arrays pass through', inject(function(chunkFilter) { expect(chunkFilter([], 3)).toEqual([]); })); it('should chunk up arrays', inject(function(chunkFilter) { expect(chunkFilter([1, 2, 3, 4, 5, 6, 7], 3)).toEqual([[1, 2, 3], [4, 5, 6], [7]]); })); it('should create equal outputs given equal inputs', inject(function(chunkFilter) { var output1 = chunkFilter([1, 2, 3, 4], 3); var output2 = chunkFilter([1, 2, 3, 4], 3); expect(angular.equals(output1, output2)).toBeTruthy(); })); }); }); This test runs with no errors. And as far as I can see from the angular.equals method implementation, there should be no reason for it not to give "true" on two outputs of my filter (given the same input), since it handles nested arrays just fine. It would be really nice to get the "chunk" filter working, as it would make using Twitter Bootstrap grids with Angular a breeze. So if some of you clever guys can tell me what I'm missing, I'd be really grateful. Thanks in advance! |
unk...@googlegroups.com | 8/24/12 10:54 AM | <This message has been deleted.> | |
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Ole Friis Østergaard | 8/24/12 10:55 AM | Oh, I found out that it's been attempted before, but with same result: https://groups.google.com/forum/#!searchin/angular/chunk/angular/F0kNH69YqL0/mmENSj8dR1MJ And since the old thread is dead, maybe there's no hope for this one... :-( |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Pawel Kozlowski | 8/25/12 3:20 AM | Hi Ole!
What you (and others) are facing is a bit tricky combination of how angular dirty checking and ngRepeat works. Don't want to go into too much details here as we are really touching inner workings of AngularJS and ngRepeat but in short ngRepeat will keep references to elements from a previous run to change DOM elements only for new objects in subsequent runs [1]. What is happening in your filter (and other filter examples that were posted here) is that it modifies original object structure (creates a new array, but more importantly, creates new object in an array for each and every run!). Now, AngularJS figures out what to repaint based on dirty checking [2] and it will run objects comparison waiting for the model to "stabilize". It keep continue dirty checking for some time (10 iterations) and if it still see changes it will throw the error you see. So basically the issue is that for AngularJS the model changes all the time. The issue you are facing is really the combination of dirty checking mechanism with ngRepeat. Please observe that you won't see the issue if you don't use ngRepeat: http://jsfiddle.net/pkozlowski_opensource/ZADB7/1/ (and this is why your test is passing) For now I don't see how this could be solved in a filter (the only approach that would probably work would be to keep references to the filtered arrays so we are not systematically re-creating new objects but this might not be good memory-wise). It would be great to have some insight from the AngularJS core team. What I can suggest for now is to move the filter logic to a controller. This way transformed array will be changed only in response to the source array changes and this won't disturb ngRepeat. Here is the working jsFiddle: http://jsfiddle.net/pkozlowski_opensource/MBpRd/3/ Hope this helps but if you've got further questions, don't hesitate to ask. I will probable continue looking a bit into inner workings of ngRepeat to see if there is any hope of putting this logic into a filter. Cheers, Pawel [1] https://github.com/angular/angular.js/blob/master/src/ng/directive/ngRepeat.js#L83 [2] http://stackoverflow.com/questions/9682092/databinding-in-angularjs |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Ole Friis Østergaard | 8/25/12 4:43 AM | Hi Pawel Thanks for your reply! So basically you're saying that ng-repeat doesn't use "angular.equal" as normal watch expressions do, but its own heuristics for checking changes? I'll have to look at the Angular code, I guess. To me it sounds like either a bug or some sort of performance hack? I also thought of caching the earlier results from my filter, but it's just too ugly, and I'll never be able to get it right. It'll result in lots of quirks. Letting my controller chunk up the array is what I'm doing now, but it's really a view detail which is leaking into my controller. I don't like it. Besides, it means I cannot do nice filter chaining in my view and e.g. do sorting, filtering, etc. in combination with the chunking, which is a real shame. |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Ole Friis Østergaard | 8/25/12 5:19 AM | Looking at the ng-repeat code, it looks like defining a $$hashKey function on my arrays should do the trick, as ng-repeat uses a hash queue for detecting changes. (Which I believe should be considered a bug, but YMMV.) I'll try it out and get back. |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Godmar Back | 8/25/12 6:30 AM | On Sat, Aug 25, 2012 at 6:20 AM, Pawel Kozlowski <pkozlowski...@gmail.com> wrote:For now I don't see how this could be solved in a filter (the only I had completely put you on the core team, Pawel, with your helpful and insightful comments. Who is on the core team, though? Does the core team read this list? In the short time I've joined, I've seen numerous questions repeated over and over, which arise mainly due to the low quality (or total lack) of documentation. Perhaps a core team member could subscribe to the list, and whenever a frequent question arises, update the documentation?
- Godmar |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Pawel Kozlowski | 8/25/12 7:00 AM | Hi!
He, he, this is flattering but I'm not part of the angular team :-) I'm just active here since it is for me a way to learn more about angular. But glad to hear that you find my posts helpful! You can see most of them being part of the angular organization: https://github.com/angular Yes, there are responses from the core team from time to time. But I guess this list is a bit a victim of its own success as most of the questions are dealt with by the community. Regarding the documentation I really kind of like it. I mean, I've seen far worst things for frameworks. There is ongoing documentation-improvement effort so things should be getting better. Yeh, this is the part I'm missing as well, having those guys stepping in and dealing with tricky questions. Fortunately the code base of AngularJS is super-clean (IMHO), well structured and thoroughly tested so there is always an option of getting hands dirty with the source code (good learning experience!). Cheers, Pawel |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Ole Friis Østergaard | 8/25/12 7:31 AM | Oh well, I tried defining a $$hashKey function, but it has to be defined in terms of hashKey(...) of the objects in the arrays. This function is defined in apis.js in the Angular source, but apparently wrapped in a way so that my code cannot access it. My thought was something like this: (...for each of the chunks in the resulting array...) chunk.$$hashKey = function() { hash = 0; for (int i=0; i<this.length; i++) { hash = hash*31 + hashKey(this[i]); } return hash; } But my filter is not able to call "hashKey". Except if it's available through some object somewhere...? However, I guess I'd better give up now. It's a shame, because Angular is really missing something like a "chunk" filter, IMHO. |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Pawel Kozlowski | 8/25/12 9:58 AM | Hi again!
So, I've spent some time digging into what is going on with this repeater / filter stuff and made some good progress so will share here. First of all, here is the minimal reproduce scenario with some log statements to illustrate what is going on: http://jsfiddle.net/pkozlowski_opensource/nRGTX/1/ Checking the console we can see that after all the children from the inner ngRepeat are printed we do see another $digest cycle. This will result in a filter being re-evaluated, new objects created etc. etc. till 10 $digest cycles are reached and the exception thrown. The thing about ngRepeat is that this directive maintains an internal list of objects=>DOM mappings (it is more complex than this, but it doesn't matter for this particular issue). I believe that this is maintained as a perf optimization to minimize DOM elements creation / removal. But since we need a mapping from an object to a DOM element here means that angular needs to decide what to use as object identity. It could be its reference or some hash and AngularJS uses hashes. The funny thing is that those hashes are calculated in an interesting way: it is simply unique identifier. Once calculated the hash is stored as the .$$hashKey object property. Now, as for the solution to the problem you were very close when you started to look into .$$hashKey. The only missing piece was that .$$hashKey can be either a function (so each object can decide how to calculate a hash) or simply a value. Equipped with the above knowledge we could very easily fix the minimal reproduce scenario by simply coping over .$$hashKey to indicate that even if we are creating new objects those have the same identity. http://jsfiddle.net/pkozlowski_opensource/zvpVg/2/ (it uses nexuid function from angular) and here is your original jsFiddle solved using the same approach: http://jsfiddle.net/pkozlowski_opensource/6vNcX/6/ Please note that the above fiddle uses very naive method of the hash calculation that works for primitive numbers. In more general scenario we would need a more robust solution (probably using combination on nextuid and another technique for primitives). To summarize: angular does some "magic" to optimize DOM manipulations in ngRepeat. Those optimizations don't play nice with filters that re-create new objects. Fortunately there is a way to inform angular that 2 objects should be regarded as "the same" even if references are different ($$hashKey). When you know what is going on it kind of makes sense and probably those guys have measured things and figured out that all those optimizations are necessary. At the same time it is true that if you don't know it might be hard to figure out what is happening.... Hope this removes a bit mystery from this whole ngRepeat + filter story. Probably should write a blog post about it... Cheers, Pawel |
unk...@googlegroups.com | 8/25/12 10:36 AM | <This message has been deleted.> | |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Ole Friis Østergaard | 8/25/12 10:39 AM | Hi Pawel |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Jason Kuhrt | 4/4/13 7:24 AM | It appears that an upcoming feature to angular will solve the problems presented in this thread? Thanks to both of you for spending the time you have on this issue. It has been helpful to readers like myself, Jason |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Jason Kuhrt | 4/4/13 7:32 AM | Actually on a more careful read I don't think chunking could be resolved by a `track by` feature. Instead the chunking filter would have to supply its own tracking mechanism as you two have presented. |
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Jason Kuhrt | 4/4/13 7:57 AM | I've created a gh issue following the topic of this thread: If you guys could chime in with clear insight than I can that would be lovely. Jason |
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Chris Zheng | 7/9/13 9:53 PM | I asked this question on stackoverflow: but wanted to post here as well. What is happening with this issue? Its very annoying |
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Dave Barker | 10/25/13 6:39 AM | Well I've figured out a hack that gets it working, you serialise each chunk and then unserialise again as you're repeating through it. Probably terribly slow and I'm not proud of myself but here you guys go http://plnkr.co/edit/XT3tJMlBKvIodYUkoVkO?p=preview Cheers, Dave. |
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Szymon Nowak | 11/4/13 9:59 AM | I've got almost exactly the same filter and the way I've did it for now (it still sucks) is to use _.memoize function from underscore library with custom hash function that iterates over input array and joins all $$hashKey values.
|
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Szymon Nowak | 11/8/13 12:48 AM | Small update. It looks like Angular generates $$hashKey later than group filter was executed and thus item.$$hashKey returned undefined for all items. Here's an updated version (in CoffeeScript): angular.module('filters') .filter 'group', ->
grouped = [] grouped[i] = [] for i in [0...num] for item, index in array grouped[index++ % num].push(item) grouped
for item in array item.$$hashKey = _.uniqueId() unless item.$$hashKey? hash += item.$$hashKey hash No idea why nextUid() function isn't public. |
Re: [AngularJS] Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Chris Zheng | 11/8/13 2:05 AM | I had all but given up on this... You've inspired me to grit my teeth and try again :)
|
Re: My attempt at a "chunk" filter doesn't work, for no obvious reason | Алексей Грабов | 3/6/14 11:32 AM | if you want return new collection, you just need to save javascript links. app.factory('linker', function () { var links = {} return function (arr, key) { var link = links[key] || [] arr.forEach(function (newItem, index) { var oldItem = link[index] if (!angular.equals(oldItem, newItem)) link[index] = newItem }) link.length = arr.length return links[key] = link } }) you can see here how it works: http://plnkr.co/edit/2Uc5zsFgVnK3ltHOUUQx?p=preview |