Deferred rendering with DOM as dataSource

drStr

Hello Everyone,

Well directly to the point, I am developing a fileListing web application (something like a webFileBrowser).

In the server side I am creating an .xml document with all the data that I need that later is "parsed" by an .xslt , so in the return of by ajax call I already have a "DOM" html table in a variable (and some other html elements, but this is irrelevant because at the end I have the table in a variable).

The next step is that I am replacing the existing table with the new one and at the end I am calling the dataTable creation
[code] $("#mySelector").dataTable({ options.... });[/code]

The problem comes here.. In case of a big amount of data this procedure could take a really big amount of time (e.x. for 10rows ~600ms for 1.500rows ~8.000ms, 15.000 ~95.000ms and some tests that I did with >35.000rows took more than half an hour in the cases that the browsers did not crash etc) The time periods mentioned above are only from the dataTable(); function.

The limitations:
1) I can not change the form of the "incomming" data (I could just get xml data as responce, but this would cause a lot of problems while at the moment I am saving a lot of data as attributes to the tr-td-table-div etcetera html elements in the xslt file)

2)I currently (maybe it will be possible in the future) can not parts from the data (for example the first 1000) mainly for performance reasons (everytime that I do a request there are a lot of checks that have to be made, so transactions for small amount of data are time-costly)


The theoretical solution: partially rendering from the responce to the DataTable (with scrolling)

PS: The user is informed that he has to wait, and this may be for a big amount of time, and there is no problem if this time is even few minutes in the worse case.. but ofcourse not half an hour

So, is this possible? That exists for ajax-server side requests as I have seen, is there any chance for modification that it will work with dom elements also?

allan

Deferred rendering with DOM source data is fairly pointless I'm afraid. The whole point of deferred rendering is that the creation of the DOM nodes is delayed until they are needed - but it is DOM sourced, then they are absolutely needed at the very start, so you get absolutely no benefit what-so-ever.

One possible solution is to XSLT your data in a Javascript array which you can feed to aaData for the DataTables data source. That way you could use deferred rendering, since the nodes are not needed up front.

What browser are you using btw? 35'000 rows should not cause any browser to crash (with the possible exception of IE if you have the DataTable in a hidden element, or its IE8-).

Allan

drStr

Unfortunately my application must mainly support IE8 and secondary Firefox 3.6+ (the other browsers are just ignored currently, and I guess they are not going to get supported also in the future).

IE8 crashes at 40.000 (and chrome that I randomly tested without dataTables but only with enchanted tables with some jquery plugins) and works till ~35.000. Both IE and FF need a really big - non responding - period of time till they render with the classic "script not responding" popups.

My basic problem is that most of the information is stored in attributes at the TR elements (each TR contains more than 10 attr, that may be from "strings" with ~100chars till boolean values)

On recent timed tests that I did with ~15.000 elements the time needed from the server response till they where fully displayed on DataTables was splited about 40% for first DOM creation and 60% for dataTables();

I will give it a try to store the information in JS array and then pass it to dataTable.

Thanks for the suggestion and the existance of dataTables.

allan

Thanks for the explanation - yes I'm not surprised that 35'000+ rows causes IE8 and old Firefox a problem. Its all about Javascript execution speed, and Javascript engines have improved hugely since those browsers were released.

Using a Javascript array and deferred rendering should give you a nice speed boost, so that certainly sounds like a good direction to go to me.

Allan

drStr

I am currently trying to find in the forum how I should pass attributes to the TRs (it would be convenient on creation, as they are static and I already have them in the server side with no calculation).

Could you please provide me, if possible, an example of code that could do that? (the format of the data - I guess json - and how is this call done in dataTables.. I found some solutions till now but they seem deprecated..

allan

I'd suggest you have a look at this blog post: http://datatables.net/blog/Extended_data_source_options_with_DataTables . The only attributes DataTables will set for you in a TR are the `id` and `class` attributes (by using `DT_RowId` and `DT_RowClass` in the data source object respectively).

You can add custom attributes using fnCreatedRow .

Allan