manually loading data via ajax but quickly

flippyhead

I'm trying to populate a datatable with data in two steps. First by loading in 1000 rows or so via an HTML table. This is pretty quick. When I load in more than this it takes a little too long 1) for the data to get there, and 2) to fully render (at least in IE7). Once loaded however, in most browsers the filter, sort and pagination controls are fairly snappy. In the second step I'd like to bring in the remaining few thousands rows via ajax while allowing the first thousand to be viewed, paginated, etc (perhaps not filtered though). I note however that using dataTable.fnUpdate() (or how I'm using it) is unusably slow. Maybe 100 rows per 10 or 20 seconds (all browsers ask if I should terminate the slow running JS). Is there a reason this is so slow? Is this how slow I could expect the default AJAX controls to be? Is there a way I can speed the creation of rows by JS and JSON? And why is it so much faster to process the same information but via an inline table?

The code that tags so long is like this:

var dom_id = user_profile.dom_id
var existing = jQuery('input#' + dom_id + '_id'); // would prefer: jQuery('tr#' + dom_id); BUT DataTables doesn't provide a method to set the TR id

var controls = this.templates.controls.replace(/INPUT-ID-NAME/g, dom_id+'_id').replace(/USER-PROFILE-ID/g, user_profile.id);
var admin = user_profile.admin ? 'Yes' : 'No';
var responded = user_profile.rsvp.responded ? 'Yes' : 'No';
var category = '';
var role_list = (user_profile.role_list == '') ? 'attendee' : user_profile.role_list;
var updated_at = user_profile.updated_at*1000

if (user_profile.category && user_profile.category.name) {
category = user_profile.category.name; //this.templates.category.replace(/CATEGORY-HEX_COLOR/g, user_profile.category.hex_color).replace(/CATEGORY-NAME/g, user_profile.category.name);
};

var values = [
controls,
user_profile.user.email,
user_profile.first_name,
user_profile.last_name,
user_profile.organization_name,
updated_at,
category,
role_list,
admin,
responded
];

if (existing.length) {
this.dataTable.fnUpdate(values, existing.parents('tr')[0]); //jQuery('tr#' + dom_id)[0]
} else {
this.addRow(values);
}



Thanks!

flippyhead

A thought: is there a way to send via ajax a table string and then process and it and merge it with the first table?

allan

Hi flippyhead,

If speed is of the essence (when is it not in a web-app!), I would suggest a slightly different approach from what you are currently doing. At the moment, you are loading in 1000 rows, when the majority of them simply won't be used before the others are loaded in via Ajax - which still means you have a large number of rows to process. While modern browsers won't have a problem with it, IE7 probably still will.

So if it's possible - I'd very much recommend using server-side processing. What you can do is initialise the table with the first 10 (or whatever) rows in the HTML (like the 1000 you currently have, just limited to what is actually visible), and these will be completely cleared out when the server request is completed. See this page for example on how to implement server-side processing: http://datatables.net/development/server-side/ . Overall I believe that this will be much faster and more efficient (assuming you have a reasonably fast 'net connection on your server - indexing columns on the database will help as well).

Regarding the solution you currently have, fnUpdate is so slow due to a draw being done on every call. You can reduce this overhead by using the fourth parameter: http://datatables.net/api#fnUpdate .

Regards,
Allan

flippyhead

I may have to go with server side processing. The only reason I was hoping the inline could be made usable was because of how we do real time search which I don't think will work via the server. I'll give your suggestions a shot, thank you.

allan

If requests are an issue with putting load on the server, you can reduce the number of them by using the following:

http://datatables.net/plug-ins/api#fnSetFilteringDelay - small delay for the key press for the filter
http://datatables.net/examples/server_side/pipeline.html - load more than a page at a time

Allan