Negative search/filter capability?

lisarush

Is there a way to do a negative search within the Filter box at the top of a table?
For example, with Gmail, you can enter foo to search for all emails that contain foo. Or you can enter -foo to search for all emails that do NOT contain foo. We do use the "smart search" capability, so we'd want to keep that (i.e. not using regular expressions).

This is something that I've wanted for a long time. I've seen a couple posts somewhat related, but they talk about using fnFilter in the legacy version of dataTables.

lisarush

However, typing something like !foo would be preferable over -foo. In our case, we often have dashes in our table values that we would want to match on.

kthorngren

There is nothing built into Datatables like that. I would use a search plugin for the negative search. This example shows one way to do this:
https://live.datatables.net/jorexujo/873/edit

It removes the Datatables search handler and creates its own for. the default search input. If the leading character is ! it clears the current global search and executes the search plugin with the search term - ! is removed.

The plugin loops through each column to see if the value exists. If it does then it returns false to hide the row. I believe the Datatables code combines each column into a string and performs the search on the string instead of looping each column. I don't know the details of this but Allan can tell us or you can look through the code. This might be more efficient than looping each column.

Kevin

lisarush

Question: Is more than 1 search function called when the search happens? I am confused. When does the "regular search" method get called and when does the "search plugin" get called...?

In your snippet example, how does the "smart search" get called in the 'else' clause -- if we've already specified an alternate "search plugin"?

    if ( val.substring(0, 1) === '!' ) {
          searchVal = val.slice(1);  // Remove the first character
          table.search('').draw();  // Reset search before runing plugin
    } else {
          table.search(val).draw();  // Otherwise perform normal Smart search
    }

It seems like both the "regular search" and the "search plugin" are getting called? (I see the plugin get called in both cases.) And if so, in what order?

And, is there a way to call the internal "smart search" capability of dataTables? Such that I could try to call it from my search plugin method?

kthorngren

In your snippet example, how does the "smart search" get called in the 'else' clause -- if we've already specified an alternate "search plugin"?

The Datatables search functions execute first then any search plugins. Not sure if the order really matters as all the filters applied should result in the same output - at least that is my assumption Essentially each plugin is pushed onto an array of plugins with $.fn.dataTable.ext.search.push(). The key is this:

      if ( searchVal.length === 0 ) {
        return true;
      }

If the searchVal, which contains a string only if ! is prefixed, empty then the rest of the plugin code isn't execute. Essentially its a NOP.

Some people push() and pop() the plugins as they are needed. If you have more than one this could cause issues. My preference is to init all the plugins, you can have more than one, and use if statements to control them.

is there a way to call the internal "smart search" capability of dataTables? Such that I could try to call it from my search plugin method?

Good question. This is one for @allan.

Kevin

lisarush

Essentially each plugin is pushed onto an array of plugins with $.fn.dataTable.ext.search.push()

Ah! Interesting. That wasn't clear to me, thanks.

I'll wait and see if Allan has any words of wisdom, and then maybe I'll have to roll my own search function...

allan

Hi,

Unfortunatly, DataTables does not currently have negative search abilities, although it might be possible to add it. Leave it with me and I will investigate tomorrow (I've been working on changes in filtering for DataTables 2 today so it is a well timed topic ).

The internal search mechanisim is not exposed via an API at all. If you wanted to reimplement it in a plug-in, that would be possible, but you'd need to redo the logic.

Leave it with me and I'll see what it would take to add this ability, and if it could be patched into DataTables 1.x (it almost certainly won't land as a feature in 1.x, but if what I'm thinking is doable turns out to be so, then a copy of 1.x can readily be patched.

Regards,
Allan

allan

Right - I think I've got it . I'll be committing this in for DataTables 2 as I think it is a nice extension to the existing smart filtering, however, this is how it can be done in DataTables 1.x:

1) In the function fnFilter find this condition:

        if ( invalidated ||
             force ||
             regex ||
             prevSearch.length > input.length ||
             input.indexOf(prevSearch) !== 0 ||
             settings.bSorted // On resort, the display master needs to be
                              // re-filtered since indexes will have changed
        ) {

Replace with simply:

if (true)

This makes the filtering always start from the base data - otherwise there is an optimisation in there that doesn't work with this since more typing can actually add records back in!

2) Find the function _fnFilterCreateSearch and replace with:

function _fnFilterCreateSearch( search, regex, smart, caseInsensitive )
{
    var not = [];

    if (typeof search !== 'string') {
        search = search.toString();
    }

    search = regex ?
        search :
        _fnEscapeRegex( search );
    
    if ( smart ) {
        /* For smart filtering we want to allow the search to work regardless of
         * word order. We also want double quoted text to be preserved, so word
         * order is important - a la google. So this is what we want to
         * generate:
         * 
         * ^(?=.*?\bone\b)(?=.*?\btwo three\b)(?=.*?\bfour\b).*$
         */
        var a = $.map( search.match( /["\u201C][^"\u201D]+["\u201D]|[^ ]+/g ) || [''], function ( word ) {
            if ( word.charAt(0) === '"' ) {
                var m = word.match( /^"(.*)"$/ );
                word = m ? m[1] : word;
            }
            else if ( word.charAt(0) === '\u201C' ) {
                // Smart quote match
                var m = word.match( /^\u201C(.*)\u201D$/ );
                word = m ? m[1] : word;
            }
            else if ( word.charAt(0) === '!' ) {
                // For NOT expressions we need to build a negative look around - note
                // that the word has the `!` already at the start
                if (word.length > 1) {
                    not.push('(?'+word+')');
                }

                word = '';
            } 

            return word.replace('"', '');
        } );

        var match = not.length
            ? not.join('')
            : '';

        search = '^(?=.*?'+a.join( ')(?=.*?' )+')('+match+'.)*$';
    }

    return new RegExp( search, caseInsensitive ? 'i' : '' );
}

And that should be it! You'll then be able to use ! to negate the next word.

The regex it builds is starting to get quite "fun"!

Allan

lisarush

With initial playing, this looks great! You are AWESOME!
(and I'm glad you're playing with the regex and not me)

I'm merging the changes in and will let you know if I see anything off. But with my initial searches, it looks good.

Muchísimas gracias!

allan

Awesome. Good to hear it should do the job for you. One change I'm going to make is to support the negation of a quoted phrase, which the above does not yet do.

Allan

lisarush

Sounds like a great addition. Please let me know when you have completed it.

allan

It landed in the DataTables 2 branch yesterday. Not sure when DT2 will be ready for release, there is still a lot of do, and no doubt there will be a lot left undone for future releases, but I'm making decent progress with it at the moment.

Allan