Infinite Scrolling vs Virtual Scrolling

kirkjerk

Hi there!
We've come to the conclusion that our backend can't really support the 1.10 Scroller plugin -- the 2 problems are 1. each query is taking too long, so the user is left hanging and 2. our API is a little odd, that when you ask for X parents, it's return X parents + Y children (we have an adapater to flatten the data into simple rows/columns) so that there's probably no way to reliably predict where an individual scroll position should refer to, query wise.

So we'd like to return to "infinite scrolling" (meaning hit the bottom, or have the user hit a button, and more records are appended) vs the Scroller plugin's "virtual scrolling"(meaning you can jump to anywhere in the set and it tries to give you the appropriate results) But it looks like the old methods for infinite scrolling have been deprecated and removed

Can we still use this with Ajax for the initial load? Is the best policy to do that, and then us DT.append(), making the ajax call ourself? Do we need to update the other stuff we pass to the DT callback (e.g. draw/data/recordsTota/recordsFiltered) or is it ok to just use append() straight up?

allan

It sounds like you are thinking along the lines I would myself. Add a scroll event handler to the table scrolling container and when the table nears the bottom, make another call to the server and get the latest data.

The part that you might strangle with, is if you only ever have a sub-set of the data, on each filter and sort of the data, you are going to have to clear out the data that already exists at the client-side and then add the new data for the updated request.

Or you could use 1.9- - but the infinite scrolling in it had so many problems with API interaction and so on, I wouldn't really recommend it... It had to be removed! :-)

Allan

kirkjerk

So is it append() that I'm looking for, i.e. run the ajax call myself and append?
And is there a function that "remove all data because we're getting a whole new set"?

At this point my company has switched to 1.10 in general, so I don't think there's any going back now

kirkjerk

Hm - researching it looks like .append() might have been the old format, and now it's DT.row.add() ?

I did that, and because I misread the example on http://datatables.net/examples/api/add_row.html , I thought I could pass in multiple rows as an array - but the example on that page is just using a flat array, and we have named columns -- and the error I saw gave a hint, because it said we had no data for the column name we were looking for.

So i corrected it to call row.add() for each row seperately, and then called .draw() each time, but the data doesn't show up on the table. Is there another step I have to do?

allan

So is it append() that I'm looking for, i.e. run the ajax call myself and append?

No - use rows.add() to add new rows to the table.

I thought I could pass in multiple rows as an array

You can - there is a difference in the plural between row.add() and rows.add().

Is there another step I have to do?

If that isn't working, I think I'd need to be able to see the page. Only thing I can think of is if you have server-side processing enabled, in which case the client-side add functions are totally redundant.

Allan

kirkjerk

So we ARE using serverSide processing; is there a better way of manually triggering Datatables to request the next set of data, using the Ajax call we told it about? (Like I said, the virtual scrolling of Scroller isn't the behavior we want, our goal is closer to pagination, but we want to keep all loaded data in one scrolling DOM element, not bouncing from virtual page to virtual page)

So in short: would use of serverSide:true be somehow preventing an rows.add().draw() from updating the table? And better yet, is there a way of manually triggering datatables to load the next set of data?

allan

would use of serverSide:true be somehow preventing an rows.add().draw() from updating the table?

Yes, because rows.add() is a client-side method. When you have server-side processing enabled, on every draw (including a call to draw() DataTables will throw away what is at the client-side and ask the server for new data). That's the whole point of server-side processing.

There is no way, without hacking DataTables (which is what the old infinite scroll was), with server-side processing, to do an infinite scroll. The only thing I can come up with would be to lengthen the page when scrolling to the bottom and request the data again, but there is a world of hurt that way I think.

Allan

kirkjerk

(Sorry, had a vacation weekend)

Is there anything we can do to get this kind of result?

Can you have client side processing, and use appends to do ALL the construction?

I think monitoring the scrolling is a solvable problem, the trick for me right now is to have more control over adding arbitrary rows to a datatables table.

kirkjerk

Is there a way I kind find out why aiDisplay is not being updated when I do an append? It being a zero length array seems to be why the data that got added to aoData is not being displayed...

kirkjerk

Would it help to call some sort of sort?

kirkjerk

My issue turned out to be in our dt wrapper code; we had this thing with filters that used for(var i in someArray) rather than for(var i = 0; i < someArray.length; i++) and then our legacy code includes prototype.js that adds an "each" key / value to basic arrays. I had to wade through a ton of DT code to figure that out, though.