DataTables 1.8 beta 2 released

allan

Hello all,

Following on from the recent released of the first DataTables 1.8 beta release, I'm really pleased to be able to update that to beta 2 now. This of course includes all the goodness of the previous beta but also now the ability for DataTables to use TH elements in the table body. This is particularly useful for progressive enhancement since the TH elements have semantic meaning, but can also be useful for styling the elements with easy CSS selctors.

In addition to this there are also a number of bug fixes from the first beta, including some improvements in the way column widths are calculated.

Beta 2 Package download: http://datatables.net/releases/DataTables-1.8.0.beta.2.zip

What's new in 1.8: http://datatables.net/new/1.8
Upgrade notes: http://datatables.net/upgrade/1.8
Original 1.8 announcement: http://datatables.net/forums/comments.php?DiscussionID=4852

As always, I'd encourage you to try this latest beta and let me know of any feedback here in the forum.

Enjoy!
Allan

sd_zuo

I am glad to see the new beta.
The deferred rendering feature is extremely great for IE. Now it displays 3000 rows of record in 1 second. Really remarkably marvelous. :)

Yet I met with a problem after the upgrade.
My data sources are JavaScript Arrays, some values in which are null.
Previous version treats null values as empty strings or 0, respect to the data type of the column.
The new version turns the null to a string "null", which is nonsense for our end users who only knows visually empty strings or the number 0.
Is it possible to revert to the previous behavior for null values?

IE 8. Opera 11.10.

allan

Hi sd_zuo,

Excellent to hear that the deferred rendering is such a huge boost :-). It's an interesting one about the null value - the reason that this has changed is that 1.7 used to cast everything that wasn't a string to a string, regardless of it's source datatype. As a result null became an empty string etc, but you lost the fact that the value should really be null. So 1.8 doesn't do that cast, it just tries to display the data as best it can. In the case of IE and Opera it appears that is done with the string 'null'. Safari and Firefox appear to do an empty string.

So... the question is what is the best way to convert that null into an empty string. It is possible to do it in the DataTables core, and that might be best for backwards compatibility - but then you are trying to display a null value, which means nothing - should it really be just an empty string. What is doing your data source generation? Perhaps that should be the point where the conversion is done?

Regards,
Allan

allan

I've just committed a change to DataTables which will take account of null data when displaying it, just like 1.7 did, but leaves the null in place for internal data processing (the cast is only done when the HTML is set). Could you give this update a go for me and let me know how you get on? It is available as the 'nightly' on the downloads page: http://datatables.net/download/

Thanks!
Allan

sd_zuo

Hello Allan,

Thanks for the reply. The deferred rendition absolutely rocks. Definitely a must have savior for IE developers and users. It also dramatically reduces the memory consumption besides the lightening speed boost, which is quite remarkable in your Update Notes in my opinion. Previously 1000 rows hogs 90MB RAM (including other RAM consumption), now the IE process takes around 50MB for the same data, which is much easier to live with. Many people here are still working with computers with 512MB RAM!!

The most horrible thing of the null value issue is that when querying data from Oracle database, it always returns null for empty string-typed values. There's no zero-length string in Oracle, and the JSON component returns null consequently. Hereby lots of "null" appear on the users' screen.

I think that the previous 1.7.6 version, treating null as empty string for string type or zero for numeric type, is reasonable.
I've downloaded the nightly build and installed it onto my development environment. Unfortunately, "null" is still used for null values. :(
Please fix the null value issue so I will put the beta into the production environment, the deferred rendition is sure to save our people's lives.

Gleeble

Hi Allan,

First of all, I'm very impressed with the library you've got here. It's highly functional out of the box and still allows for a good deal of flexibility.

I just wanted to say that I may have found a couple of bugs with the latest Nightly (grabbed that since there was the bug in B2 regarding js array of json objects and column detection). Anyway, I was trying to use null mDataProp for 2 columns in a table so that I could populate it with my own edit and delete links using fnRender, however I found that if I put null for the mDataProp then the output was nullified entirely and so the rendered output never got into the cell. Secondly, I changed the null to "null" and it worked, but then both columns produced the exact same output (the delete since it came later). I finally removed my mDataProp entirely from those columns and it is working fine but I'm just not sure if that was a problem you'd want to change or document.

Also, a couple of suggestions. I know 1.8 is a work in progress, but as far as the data object access goes, it would be nice if the object passed to the fnRender callback could get a property that is just the cell value since you can't really look it up dynamically if you've got an object as the aData and a column number that doesn't correspond to it.

Finally it would be nice if there were some sort of table callback after everything else for the table had been set up, so table finished or complete or something. I tried using the draw callback, but it ran too early for what I needed.

Thanks for the great work on this!

allan

@sd_zuo - my mistake. I've just updated the source to cope with the null display properly. Could you give the latest nightly a go again please?

@Gleeble - mDataProp being set to null should be perfectly valid and just put an empty string into the column (the top condition in _fnGetObjectDataFn()). Could you possibly put up an example showing the issue please?

> it would be nice if the object passed to the fnRender callback could get a property that is just the cell value

I was taking the approach that the rendering function would know what the object structure looked like, and therefore would be able to read it. However, I take the point that it might be quite useful to have the value that would be retrieved through the use of mDataProp. The complication of that though is that by returning a new value, the value of the parameter at mDataProp will be different if called again... I'm just wondering if this might be confusing.

> Finally it would be nice if there were some sort of table callback after everything else for the table had been set up

http://datatables.net/usage/callbacks#fnInitComplete :-)

Regards,
Allan

Gleeble

Allan,

To respond last comments first:

I think having access to the cell value would be useful for cases where you might want to use the same render function on multiple cells (i.e. 2 different date cells). Right now, each call back requires me to create the same fnRender twice with each looking up different properties, not horrible but would be nice to use an aoColumnDefs property to define it for both columns at once.

As for the fnInitComplete I had looked through all those and I think I found the "init" portion ambiguous and thought it might be prior to the end. Good to know it exists though, Thanks!

Now, as for an example:
[code]
$('#licenseCredentialsTable').dataTable( {
"aaData" : dataRows,
"aoColumns": [{"mDataProp": "licenseType.name"},
{"mDataProp": "licenseId"},
{"mDataProp": "state", "fnRender" : function(obj) {
return locationFormatter(obj.aData);
}},
{"mDataProp": "expirationDate", "stype" : "date",
"fnRender" : function(obj) {
return formatDate(obj.aData.expirationDate);
}},
{"fnRender" : function(obj) {
return credentialEditLink(obj.aData, baseUrl,
editPage);
}},
{"fnRender" : function(obj) {
return credentialDeleteLink(obj.aData, baseUrl,
deletePage);
}}
]
} );
});
[/code]

Those last 2 column definitions output some HTML creating 2 image links. When I had '"mDataProp" : null' the result was "" when I set them both to '"mDataProp : "null"' both columns wound up containing the delete link HTML. The way it is above they work fine.

One other question, since the example wasn't very descriptive (couldn't find the JSON output to look at), for the DT_RowId property is that a property that needs to be set in the JSON objects? Would it be possible to make that configurable so that I could set a table property of '"DT_RowId" : "id"' so that it would use the value of the "id" property as the row id instead?

allan

1. mDataProp - I've just tried this and it seems to work as required:

[code]
$('#example').dataTable({
"aoColumnDefs": [ {
"mDataProp": null,
"fnRender": function () {
return "hello";
},
"aTargets": [0]
} ]
} );
[/code]
i.e. the first column is filled with 'hello'. In fairness I have just made some changes to the handling for null and mDataProp - and also introduced a new parameter called 'sDefaultContent' - which might have an effect. Could you possibly try the latest nightly please?

2. DT_RowId - at the moment, this can only be configured by changing the DataTables source. The way I thought of this is that it is such a DataTables specific function, that it is correct to 'namespace' it into a certain parameter. Can you not modify the JSON data generator to add this property?

Regards,
Allan

Gleeble

Well, I just updated to the nightly and no luck, it's still producing no output. However, I have no idea if the nightly I just downloaded was different than the one I downloaded earlier since it currently says that the latest nightly was upload 4-5 hours from now (it's only 3pm on Monday for me). Below is the exact source I'm using and the generated HTML.

As far as modifying my JSON to add in that particular property, it's possible but not the best option. The JSON is generated in a Java action class serialized from a java object so I would have to manually go into the result and add in that property while I already have an "id" property that I want to use (as you can see by my row callback below).

[code]



License
ID #
Location
Expiration Date








$(document).ready(function() {
var dataRows = ${credentialJsonString};
var isLongData = dataRows.length > ${minPaginationResults};
var baseUrl = ''
var editPage = '';
var deletePage = '';
$('#licenseCredentialsTable').dataTable( {
"aaData" : dataRows,
"bInfo" : isLongData,
"bLengthChange": isLongData,
"bPaginate": isLongData,
"bFilter": isLongData,
"aoColumns": [{"mDataProp": "licenseType.name"},
{"mDataProp": "licenseId"},
{"mDataProp": "state", "fnRender" : function(obj) {
return locationFormatter(obj.aData);
}},
{"mDataProp": "expirationDate", "stype" : "date",
"fnRender" : function(obj) {
return formatDate(obj.aData.expirationDate);
}},
{"mDataProp": null, "fnRender" : function(obj) {
return credentialEditLink(obj.aData, baseUrl,
editPage);
}},
{"mDataProp": null, "fnRender" : function(obj) {
return credentialDeleteLink(obj.aData, baseUrl,
deletePage);
}}
],
"fnRowCallback": function( nRow, aData, iDisplayIndex, iDisplayIndexFull ) {
$(nRow).attr("id", dataRows[iDisplayIndexFull].id);
return nRow;
}
} );
});

[/code]

And after the page has loaded:

[code]




License


ID #


Location


Expiration Date







R.N. - Registered Nurse
12378
AR, USA
05/2011





[/code]

allan

Interesting - could you possible add the following to one of the two null data prop rendering functions and see what the output on the console is please?

[code]
console.log( obj.aData, baseUrl, deletePage, credentialDeleteLink(obj.aData, baseUrl, deletePage) );
[/code]
Thanks,
Allan

Gleeble

[code]Object country=USA expirationDate=05/2011 expired=false /this/was/the/valid/url/manageLicenseCredentials.action delete [/code]

Sorry to strip out some of the URL info, but this is for work and I'm not sure if I should be posting URLs on the web, the URLs were correct though. I think I'm just being a bit over cautious but oh well.

allan

Thanks for the info! Although I'm afraid I'm still struggling to reproduce the issue. Could you possibly try just returning a static string from the rendering function and see if that works? return 'hello world'; or something like that.

Allan

Gleeble

Strangely enough, if I change it to the below, it still isn't working. The only real difference here between mine and yours is the use of aoColumns vs. aoColumnDefs. I'll see what switching over does as well.

EDIT: Scratch that, doesn't work on the aoColumnDefs either.

[code]
{"mDataProp" : null,"fnRender" : function(obj) {
/* return credentialEditLink(obj.aData, baseUrl,
editPage); */
return "Hello"
}},
[/code]

allan

Got it :-) Thanks very much for the debugging help - I've just committed a fix for this issue, which you can grab from the downloads page (the nightly). Basically the issue was that the rendering function was being run, but the result wasn't being stored anywhere, since there was no source data property to store it into. Thus when actually creating the cell element (which is done a little bit later since you are using a Javascript array as the data source) and it tries to read the content for the cell, it finds an empty string (since the data source is null). The fix is to add this into the conditional logic for the rendering.

It's worth noting that there is one "quirk" in this area still - with this method the rendered data can't be used for sorting or filtering since it isn't stored anywhere. There is one option though, use mDataProp set to a parameter which doesn't yet exist in the source object and DataTables will use that as the storage area for the rendered result.

Thanks again,
Allan

sd_zuo

Hello Allan,

I've downloaded the new nightly build.
String typed columns no longer see "null" string any more. However, it still appears on numeric typed columns.
I find that I've got to rewrite all fnRender functions regarding to the new change too (for instance, code like this [code]return "C" + o.aData[0];[/code] should be rewritten). Ordinary end users don't understand "null" and all sites using DataTables might be affected by this change if the JSON data sources contain null values. The impact to the whole web application is, I guess, too huge.
Therefore, I guess it may be reasonable to rolling null handling things back to previous release, but add the following facilities to help:
1, provide some helper functions such as "IsNull", "IsBoolean" etc. to detect null and boolean values,
or 2, provide an initialization option to alter the way to handle null values and the default setting of it is like 1.7.6.
Maybe option 2 is a more viable way.

allan

I'll check the numbers issue when I get to my PC (out and about atm) - sorry about that, although I'm quite surprised - there is a line in the code which says if ( null ) then return '';

With regard to the upgrade issue from 1.7 - although I've tried very hard to not break backwards compatibility with 1.8, I really think that the behaviour in 1.7 was broken, and as part of the data source upgrade in 1.8, I've addressed that. Basically before DataTables was modifying your data, which I think is definitely wrong. If you are passing in nulls, then you would expect to get nulls back - otherwise when you want to use nulls (which can be useful) you'd find that you can't (like in 1.7).

As such I think this is a place where the developer needs to be aware of what is in their data source and be able to cope with it accordingly(if this means testing for null in fnRender, then so be it). I completely agree that the end user doesn't want or need to know anything about null data, but the developer using DataTables should and does.

Allan

Gleeble

Glad I could help out Allan, I just tried out the latest nightly and I'm getting the correct output so that's working great. Thanks!

sd_zuo

Hello Allan,

It is quite frustrating that Oracle always return null for empty strings.
At the first time when I was working with 1.7.6, I was surprised about the null handling. However, nowadays I found nothing wrong with it and think it reasonable. It actually brings convenience to the development. :P
I don't have to cope with null thanks to that "broken" behavior. Anyway, other developers might think it is reasonable to input null and get null like it is now in 1.8.

What's to be blamed is the darn Oracle which does not support zero-length strings. From that aspect, I think it is better to automatically turn nulls into zero-length strings if the column data type is string, and leave other nulls as is. So fnRender codes like [code]return o.aData[0].replace (/@/g, '#');[/code] won't need any boring modification. It does help a lot to avoid manually handling null.
How do you think about that? Possible to add an option which provides the previous null behavior?

allan

I totally can't understand why they do that, and Protgres does the same thing. It just doesn't make any sense since null !== '' !

I'll think about the option of doing null processing on strings, but the problem still exists then I think - and to some degree is hidden even more, since it is one way for some columns and another way for others. It might be best to have an option for it, but I'm getting a little wary about the number of options that are available - there are so many it can be a bit confusing!

Also the fnRender code could be modified to be something like:

[code]
return o.aData[0] === null ? '' : o.aData[0].replace (/@/g, '#');
[/code]
It is an extra thing to consider... I'll think on it :-)

Allan

sd_zuo

The above sample code is only one place regarding to the possible null values, however, in one of my applications there are hundreds of them. The nasty null handling can be overwhelmingly annoying and painful. :(

I would love to see an option for each column about the null value like [code]sNullText = "Representative Unassigned"
// or ...
sNullText = "Pending"[/code]. DataTables will turn each null in the specific column to that string, and the default value of that option is a zero-length string [code]sNullText = ""[/code], so the backward compatibility is achieved, meanwhile developers who insist of getting null as null can use the following option [code]sNullText = null[/code].
Since we have to take the job of displaying some meaningful information for null to the end users. This option is very helpful and makes great sense..

benjaminv

Allan,

I think I've identified a bug in 1.8 version of datatables. I'm using the latest nightly build of ColVis and Datatables. After a filter is applied (column or global) any attempt to change the column visibility results in an error:
Uncaught TypeError: Cannot read property 'parentNode' of undefined
$.fn.dataTable.fnSetColumnVisjquery.dataTables.nightly.js:2095
ColVis._fnDomColumnButtonColVis.nightly.js:469
c.event.handlejquery-1.4.2.min.js:55
c.event.add.j.handle.ojquery-1.4.2.min.js:49

What happens on screen is sort of strange...the columns disappear starting from the right but not the column header.

This only happens when "bDeferRender": true.

I hope this helps.

Ben

allan

Nice one Ben - thanks for flagging that yup - you are quite correct. There is indeed a bug there which needs to be fixed. I'll commit the fix later on :-)

Allan

benjaminv

Allan, thanks for all your hard work!

I think I may have found another bug, though this one has been a little more difficult to replicate the exact circumstances. It appears to be only an issue when bDeferRender is true.

This again has to do with column visibility. Right after I call table.fnAddData(j.data), I set some column's visibility to 'false'. This give me an error, nearly the same as above:
Uncaught TypeError: Cannot read property 'parentNode' of undefined

Frustratingly, this bug was intermittent. Now I know why. When I debugged to this section of code:
anTds = _fnGetTdNodes( oSettings );
for ( i=0, iLen=oSettings.aoData.length ; i

allan

Hi Ben,

Thanks for the extra information! I think the issue you are having with ColVis is basically the same as you are having with fnAddData here. I've just committed a change which basically changes the way that particular block of code works and it works well with ColVis and deferred rendering now - fnAddData with deferred rendering should be fine as well. If you could pick it up from the nightly on the downloads page and let me know how you get on, that would be great!

Thanks,
Allan

benjaminv

Allan,

No problem. I just tried your nightly build. Unfortunately, it seems to have solved one problem and introduced another. I suppose that is always the way with fixing bugs! The column removal seems to work perfectly fine (with and without sorting). However, attempting to make a column that was previously not visible to be visible gives me this error:
Uncaught Error: NOT_FOUND_ERR: DOM Exception 8
$.fn.dataTable.fnSetColumnVisjquery.dataTables.nightly.js:2062
filters.push.coloverview.cgi:615
c.event.handlejquery-1.4.2.min.js:55
c.event.add.j.handle.ojquery-1.4.2.min.js:49

Interestingly, sometimes you can set a column to be visible. It seems that if the column is all the way to the right it works fine, however, if the column is somewhere in the middle, setting it to visible just gives the above error, and does not change the display.

I hope this is helpful. If you need further information, let me know.

Ben

allan

Hi Ben,

I woke up this morning and just suddenly knew that I'd messed that little bit up (funny how the mind works...). Doh! I've just committed a bit which addresses it properly. fnSetColumnVis (and therefore ColVis) should now work as expected!

Regards,
Allan

sd_zuo

Hi Allan,

I had supposed to be experimenting the wonderful new features of 1.8, but finally ended up with the foiling of the null value issue.
Currently there are so many places in the code, assuming that null values will be changed to zero-length strings, before fnRender, fnRowCallback, fnDrawCallback or much other 'elsewhere'. Adding null value detection is so humdrum and finally I retreated. :( After downloading the nightly build and some days' struggle before, I rolled back to the 1.7.6 release.

benjaminv

Allan,
EDIT: Actually, I failed to actually use the latest version. It does appear to work :)

Thanks so much for fixing this!

Ben

allan

@Ben: Good to hear :-)

@sd_zuo: Yup I understand. I've been thinking about it a lot this week, and for the moment at least, I'm going to keep it the way it currently is, rather than having DataTables tampering with the data. One fairly simple option would be to loop over the data either on the server or in the client (using fnServerData) to find and replace nulls with empty strings. That would take care of the issue, and allow DataTables to not have to mess around with the data types.

Regards,
Allan