Sorting strings - danish/norwegian etc.

thomas1956

When sorting strings - danish and norwegian characters are not sorted in the correct order:
The sort order is: a, b, c, d, .......... x, y, z,

allan

Hi Thomas,

Wow - that's a good one! I would suggest that it's actually a bug in your browser rather than in DataTables however. DataTables just uses Array.sort() for strings, it's not doing anything fancy (it would be possible to write your own sorting function for this case, but I suspect it would impact on performance significantly).

What I would suggest, is trying the following in various browsers and seeing what happens - perhaps it's only one browser which does it wrong, or perhaps there is some standard they are all following (hah!).

[code]
var aLetters = [ 'b', 'd', 'y', 'a', 'z', 'c', '

thomas1956

Hi Allan

Thanks for the quick response!

I have tried the code you suggested in a variety of browsers - AND in Danish and English version of Browsers/Operating systems.

It seems the sort order is pretty much "standard" - my result is:

Firefox 3.5: "a b c d x y z

allan

Hi Thomas,

It looks like it might be something you just have to live with, unless you write your own sorting function which converts the characters in a string into their alphabetical positions and then sorts them - as I say, probably rather processor intensive!

Could it be a bug in the Unicode standard? Certainly I can see why the browsers would want to sort in Unicode ordering... Might there be another alphabet which does use these characters in the Unicode order?

Regards,
Allan

dzida

Hi guys,

I have just found that problem with my application:/

everything is sorting as axpected for english letters a-z, but everything which is not started by 'standard' letter comes at the end of searching results(or beginning). I found this problem for polish letters (like: ?, ?, ?, ?, ?, ?...).

I've done short test Allan suggested 2 posts ago and that confiremd that this is an issue with javacript not datatables.

Would someone be so kind and give a hint how to write some custom sorting function and apply it to all column in all datatables I use... please

Kind Regards,
Lukasz

allan

Hi Lukasz,

I think that this is a problem with how characters have been assigned their numerical representation (for bits/bytes) - all non-ASCII characters have a value greater than 127 for example. Getting around this is possible in Javascript, but as I suggest a couple of posts back, it's probably very processor intensive.

What I think you will need is a string with the characters in your alphabetical order, when then do a look up on each character which is being sorted upon to calculate that characters position in the array and then deal with that... If you've got a reasonable sized table... ouch... :-)

One option which might overcome this, is if you have access to a database (perhaps your data is currently coming from one?) which is locale aware and will do the correcting correctly, then you could use server-side processing.

Sorry - no magic wand with this one I fear...

Regards,
Allan

dzida

Thanks for response Allan,

I used to hear only good news from you:)

Is there any way to use custom comparer in datatbles (sth like this: arr.sort(function(a,b){return a-b;})? You mentioned about some method to write own 'comparing method', how to use it with datatbles?

Unfortunetly in my case server side sorting is not good idea.

dzida

Well,

I'm not a javascript expert, so I'm not sure about limitations on what I have done but I found solution that looks like working one.

I've overriden behavior of oSort['string-asc']and oSort['string-desc'] functions of datatbles by doing something like that (is it OK?):

[code]
jQuery.fn.dataTableExt.oSort['string-asc'] = function(a,b) {
var x = a.toLowerCase();
var y = b.toLowerCase();
return x.localeCompare(y);
};

jQuery.fn.dataTableExt.oSort['string-desc'] = function(a,b) {
var x = a.toLowerCase();
var y = b.toLowerCase();
return y.localeCompare(x);
};
[/code]

I am not sure if localeCompare is good idea (I've founds notes about some limitations of that method) but that solves the problem with polish fonts in my project. I tested it on Firefox 3.5.3 and IE8 - works on both and the performance lost is not significant (if any).

I a interested if this solution is OK in terms of good js coding practices...?

Kind Regards,
Lukasz

allan

Hi Lukasz,

I'd say this was bob on! Nice one.

The only thing to note would be that it appears that Opera support is a bit patchy ( http://www.hunlock.com/blogs/The_Complete_Javascript_Strings_Reference#localecompare and others), and probably the same for older Firefox / Safari versions as well. So two things I'd perhaps suggest:

1. Ask the question at the Opera forums ( http://my.opera.com/community/forums/ ) about if there is a method in Opera for doing this, or if Opera 9.6+ supports it (you would think it would given that they are very good with non-English language support).

2. Put in a test condition for the String.localeCompare method. For example:

[code]
if ( typeof y.localeCompare == "function" ) {
return y.localeCompare(x);
} else {
return ((x < y) ? 1 : ((x > y) ? -1 : 0));
}
[/code]
If this works well for you, it would be great to include it on the sorting plug-ins page :-)

Regards,
Allan

dzida

Hi guys,

I made some research and I found information that String.localeCompare is supported in since Opera 9.50 alpha, build 9542.

I tested on my Opera 9.51 and it is not working... I created a thread on Opera forum as you had mentioned (http://my.opera.com/community/forums/topic.dml?id=294904). we will se what they say.

Bad news is that it is not working in Google Chrome as well.

Here is simple testing page to quickly check if localeCompare is supported in certain browser: http://michael.susens-schurter.com/files/localecompare.html

Kind Regards,
Lukasz

dzida

Hi,

Unluckily nobody answered on Opera forums so this topic still needs some research.

I've done some experiments and it looks like this works OK for:
- Firefox 3.5.3
- IE 8
Not works:
- opera 9.5.1
- Chrome 3.0.195.27

Probably using test condition Allan mentioned few posts before ("2. Put in a test condition for the String.localeCompare") will be good solution. But it still is quite disappointing that many browsers do not support localeCompare method.

Regards,
Lukasz

allan

Hi Lukasz,

I've just added a reply to your Opera forums post to bump it up and try and get a response from someone. I find it very odd that Opera wouldn't support String.localeCompare...

One other option (not sure if it would suit you though) might be to use server-side processing under the assumption that the SQL database engine will be able to correctly sort UTF strings.

Regards,
Allan

dzida

Hi Allan,

Hope someone in Opera will give some hints.

My problem is solved right now. I couldn't use server side processing because of specific logic and client requirements to process sorting as fast as possible (which in my case was client side sorting without hits, to db). Fortunetly my project will be used in a small team, where everyone use Firefox (so I'm saved:)

Anyway I believe that I will face that problem again soon in other projects

Cheers,
Lukasz