Number of DOM nodes increases with every AJAX call to refresh table

watchdogtimer · July 2014

I have a relatively complex DataTables 1.9.4 UI (although I've tested in latest 1.10.0 and the same issue occurs) that works great in modern browsers but halts to a stop in IE8.

On my Windows VM running IE I've downloaded sIEve to debug I've noticed the behavior that the # of DOM nodes climbs for each periodic AJAX call (every 10s) that is made to refresh the table. In addition, user initiated events to filter the table causes the same issue.

Being new to DT, I'm guessing that I'm using the API slightly incorrectly for this to happen. It wouldn't be an issue if the pace of increase wasn't so quick. Each AJAX request for a 100 row table adds 2,000 DOM nodes that do not get cleaned up, even during garbage collection and memory usage for just one tab and one page quickly goes from 40MB to over 500MB, which is probably more than IE7/8 users can afford.

I've pared down my code the simplest of examples, such that it should be easy to see the behavior I see in a version of Windows running IE8.

I used http://www.json-generator.com/ for the payload, using the following data to generate 100 rows of 20 cols of JSON data, so the server call is overridden with canned data:

[
  '{{repeat(100)}}',
  {
    col1: '{{guid()}}',
    col2: '{{guid()}}',
    col3: '{{guid()}}',
    col4: '{{guid()}}',
    col5: '{{guid()}}',
    col6: '{{guid()}}',
    col7: '{{guid()}}',
    col8: '{{guid()}}',
    col9: '{{guid()}}',
    col10: '{{guid()}}',
    col11: '{{guid()}}',
    col12: '{{guid()}}',
    col13: '{{guid()}}',
    col14: '{{guid()}}',
    col15: '{{guid()}}',
    col16: '{{guid()}}',
    col17: '{{guid()}}',
    col18: '{{guid()}}',
    col19: '{{guid()}}',
    col20: '{{guid()}}'
  }
]

And here is the markup. In this example, I've simplified the problem down to minimal markup. The # of DOM nodes increases by 1,000-3,000 or so for every call (shortened to every 1s for faster exhibition of the issue) and never goes down and the memory increases by 3-8MB on every call and goes from 36MB overall to over 1GB in less than 10 minutes:

<html>
<head>
    <script type="text/javascript" src="./jquery-1.10.2.min.js"></script>
    <!-- DataTables 1.9.4 OR DataTables 1.10.0, both exhibit memory growth -->
    <script type="text/javascript" src="./datatables/media/js/jquery.dataTables.js"></script>
</head>
<body>
    <table id="exampleTable" width="100%" class="asset table">
        <thead>
          <tr>
            <th class="sortable sorting" style="width: 20px; text-align:left;">col1</th>
            <th class="sortable sorting" style="width: 20px;" id="default_sort2" >col2</th>
            <th class="sortable sorting" style="width: 20px;" id="default_sort1" >col3</th>
            <th class="sortable sorting" style="width: 20px;">col4</th>
            <th class="sort-none" style='width: 20px;'>col5</th>
            <th class="sort-none" style='width: 20px;'>col6</th>
            <th class="sortable sorting" style="width:20px;">col7</th>
            <th class="sortable sorting" style="width:20px;">col8</th>
            <th class="sortable sorting" style="width:20px;">col9</th>
            <th class="sortable sorting" style="width:20px;">col10</th>
            <th class="sortable sorting" style="width:20px;">col11</th>
            <th class="sortable sorting" style="width:20px;">col12</th>
            <th class="sortable sorting" style="width:20px;">col13</th>
            <th class="sortable sorting" style="width:20px;">col14</th>
            <th class="sortable sorting" style="width:20px;">col15</th>
              <th class="sortable sorting" style="width:20px;">col16</th>
              <th class="sortable sorting" style="width:20px;">col17</th>
              <th class="sortable sorting" style="width:20px;">col18</th>
              <th class="sortable sorting" style="width:20px;">col19</th>
              <th class="sortable sorting" style="width:20px;">col20</th>
          </tr>
        </thead>
        <tbody id="assetTableBody">
        </tbody>
    </table>
<script type="text/javascript" >
    var $ = jQuery.noConflict();
    $(function() {
        $('#exampleTable').dataTable({
            "bProcessing": true,
            "bServerSide": true,
            "sAjaxSource": "../temp/assets.js",
            "fnServerData": function ( sSource, aoData, fnCallback ) {
                var json = {assets: [
                    //// COPY / PASTE this object 100 times
                    {
                        "col1": "a74feda4-239c-45ba-a808-2ec4afe8f70c",
                        "col2": "e31c2cb3-83a4-4220-bc07-0e5aa63471f5",
                        "col3": "72fde6ef-ea5d-4d38-864a-177ca81349a2",
                        "col4": "f6bc42aa-8433-40a2-81ec-a865e667806d",
                        "col5": "c76318e5-89d5-4bea-b6ca-bb60d8c1ec40",
                        "col6": "8d051e28-a455-467c-a59f-64a9d998209d",
                        "col7": "d4a52f59-ca18-4b45-a15b-0f0a77fc6e93",
                        "col8": "27a14e3f-2993-4d09-9a48-1816d791b3df",
                        "col9": "a69ca3aa-f6f2-4961-9deb-c5cabc5ccaab",
                        "col10": "fc7c9b0e-80b0-4634-8dbd-fd40d51cfd9d",
                        "col11": "bf4bb711-04ea-4c5c-91a3-92e0821820c9",
                        "col12": "90ae2428-b24c-4868-b812-fc0d32ff4c68",
                        "col13": "d45201bc-c8f8-461b-a848-03100cc1e157",
                        "col14": "8ee41a7e-7ca5-4cf7-8caf-e241259ab1a4",
                        "col15": "8176e620-fc46-49bb-8a06-2083427b686f",
                        "col16": "ecf4117b-8491-426d-a443-b570d846993f",
                        "col17": "f781365c-0177-4b3d-99bf-9bc5ac4b4b20",
                        "col18": "1b60eb27-bece-4be5-a47b-d16ba80cd26e",
                        "col19": "b8911356-503c-4469-9965-478e95e6e68f",
                        "col20": "2cdba15f-8bfa-467e-bfb2-f31c431cc9fc"
                    }
                ]}
                fnCallback({"aaData": json.assets});
            },
            "sPaginationType": "full_numbers",
            "bLengthChange": true,
            "iDisplayLength": 25,
            "bInfo": true,
            "bAutoWidth": false,
            "aoColumns": [
                { 'mData': 'col1', "sClass": "col1" },
                { 'mData': 'col2', "sClass": "col2" },
                { 'mData': 'col3', "sClass": "col3" },
                { 'mData': 'col4', "sClass": "col4" },
                { 'mData': null, "sClass": "nullcol" },
                { 'mData': null, "sClass": "nullcol" },
                { 'mData': 'col5', "sClass": "col5" },
                { 'mData': 'col6', "sClass": "col6" },
                { 'mData': 'col7', "sClass": "colx" },
                { 'mData': 'col8', "sClass": "colx" },
                { 'mData': 'col9', "sClass": "colx" },
                { 'mData': 'col10', "sClass": "colx" },
                { 'mData': 'col11', "sClass": "colx" },
                { 'mData': 'col12', "sClass": "colx" },
                { 'mData': 'col13', "sClass": "colx" },
                { 'mData': 'col14', "sClass": "colx" },
                { 'mData': 'col15', "sClass": "colx" },
                { 'mData': 'col16', "sClass": "colx" },
                { 'mData': 'col17', "sClass": "colx" },
                { 'mData': 'col18', "sClass": "colx" },
                { 'mData': 'col19', "sClass": "colx" },
                { 'mData': 'col20', "sClass": "notvisiblecol",  "bVisible": false }
            ],
            "aaSorting": [ [2,'desc'], [1,'desc'], [0,'asc'] ],
            "bDeferRender": true
        });
        function callDraw() {
            $('#exampleTable').dataTable().fnDraw(false); 
            setTimeout(callDraw, 1000);
        }
        setTimeout(callDraw, 5000);
    });
</script>
</body>
</html>

Thanks in advance, Tom

watchdogtimer · July 2014

I've been able to make some progress identifying where the potential issue in the dataTables code is.

During my research into IE8 garbage collection, I came across Douglas Crockford's very informative article: http://javascript.crockford.com/memory/leak.html

And tried applying the purge function to the tr elements that end up getting cleared out of oSettings.aoData

    /**
     * Nuke the table
     *  @param {object} oSettings dataTables settings object
     *  @memberof DataTable#oApi
     */
    function _fnClearTable( settings )
    {
        /// Run Douglas Crockford's purge
        function purge(d) {
            var a = d.attributes, i, l, n;
            if (a) {
                for (i = a.length - 1; i >= 0; i -= 1) {
                    n = a[i].name;
                    if (typeof d[n] === 'function') {
                        d[n] = null;
                    }
                }
            }
            a = d.childNodes;
            if (a) {
                l = a.length;
                for (i = 0; i < l; i += 1) {
                    purge(d.childNodes[i]);
                }
            }
        }

        for(var i = 0; i < settings.aoData.length; i++) {
            purge(settings.aoData[i].nTr);
        }

        settings.aoData.length = 0;
        settings.aiDisplayMaster.length = 0;
        settings.aiDisplay.length = 0;
    }

but this did not help in cleaning up the memory taken up by old tr elements. Still IE quickly eats up as much memory as possible and eventually becomes unresponsive or crashes.

So, I believe this is where the problem is but still not sure how to fix it. Crockford does note that his purge function does not handle breaking cycles created by events added to elements via attachEvent, so perhaps that is the issue. Not sure how to break those yet.

Daimian · July 2014

Just as a side note, here is a very well done article on this kind of subject.
http://www.smashingmagazine.com/2012/11/05/writing-fast-memory-efficient-javascript/

watchdogtimer · July 2014

Thanks, Daimian, definitely good to read up more on this subject.

I'm most familiar with Java GC and memory leak tools in the Java realm. Unfortunately the recommended tool in that article for detecting memory leaks in Chrome is no longer functional. Another article, regarding IE8 (http://blogs.msdn.com/b/gpde/archive/2009/08/03/javascript-memory-leak-detector-v2.aspx) also referenced a mem leak tool for IE that is now defunct. I can't seem to find a tool to help shed more light on this problem.

watchdogtimer · July 2014

And to follow up on Chrome, I was able to use vanilla Dev Tools to see DOM node cleanup occurring in Chrome 35, so this is a problem specific to IE8 GC where the TR elements and their children are sticking around forever

Daimian · July 2014

With Chrome now holding the worldwide market share for browsers... and IE8 has publicly known major security holes - alot of plugins/libraries are ending IE8 and older support completely.

In my jourey to better understand how the code I write get interpreted and executed, I have found that javascript GC is still not fully understood in any engine oddly enough.

Is there any way you can avoid supporting IE8?

Daimian · July 2014

I'm curious what version of jQuery you are using?

After doing some digging into the DataTables source code, I see it uses jQuery for DOM manipulation. Likely jQuery isn't properly releasing DOM elements for GC or perhaps DataTables isn't using the correct method to let jQuery do this.

watchdogtimer · July 2014

I wish we could get away from IE8, however the majority of our customers are not willing/able to upgrade to a modern browser (IT mandate).

See the code above for JQuery version: 1.10.2

Daimian · July 2014

I understand how that goes. You should make sure they are aware Microsoft is not fixing some zero-day expoits that allow full memory access in IE8 (all Windows). These exploits can give an attacker full remote access to PCs.

I wonder if there are any bug reports for IE8 w/ jQuery that are apart of this issue.

watchdogtimer · July 2014

The function in which the old TRs are replaced with new is _fnDraw is using a standard:

var body = $(oSettings.nTBody);

body.children().detach();
body.append( $(anRows) );

I've tried a couple of different approaches in addition to this:

1) Move this chunk of code into _fnClearTable which is called before _fnDraw in the call stack. I copied these lines into this function before the settings.aoData array is set to an empty array in an attempt to indicate to the garbage collector that these nodes should be GC'd
2) Update the detach() with remove() - since we don't need any of the JQuery metadata
3) Do both 1 and 2

Memory still inexorably climbs and no DOM nodes are GC'd

Now I'm thinking a completely alternate approach is to 'recycle' the old nodes, rather than creating new ones every time. This would involve patching DataTables to an extent that I'm not quite comfortable with (not being familiar with the entirety of the codebase and wouldn't be able to characterize ramifications).

allan · July 2014

What should be happening is that in server-side processing mode (which you are using) on each draw, it will clear out any reference that DataTables has to the existing nodes (i.e. the will no longer be accessible from the aoData internal array, since it is truncated) and the nodes are removed from the DOM. New nodes are created and added into the DOM.

With the old nodes no longer accessible from DOM or Javascript that should be cleaned up by the garbage collector. So there are two possibilities:

DataTables isn't releasing all links to the nodes
Old IE's GC isn't working properly

There was a problem with 1.10.0 release where by a row invalidation would cause a leak, but this has been fixed in the nightlies and will be released soon. If you are still working with 1.9, I would encourage you to switch to the nightlies of 1.10.1, since 1.9 is no longer supported and I won't be making any updates to it.

Beyond that, do you see this behaviour even in the simple test cases on this site (they would need to be slightly modified to do the draw() periodically)? (I don't have an Windows machine to hand at the moment).

Thanks,
Allan

watchdogtimer · July 2014

I'll follow up with you tomorrow after testing out my simple example above using the nightlies and attempting to simulate the issue in the test cases on this site.

By the way, Microsoft kindly provides a great site for downloading VMs to do compatibility testing:
https://www.modern.ie/en-us/virtualization-tools

I wouldn't have access to a Windows environment otherwise.

Number of DOM nodes increases with every AJAX call to refresh table

Number of DOM nodes increases with every AJAX call to refresh table

Answers

Howdy, Stranger!

Categories

DataTables

Number of DOM nodes increases with every AJAX call to refresh table

Number of DOM nodes increases with every AJAX call to refresh table

Answers

Howdy, Stranger!

Quick Links

Categories

DataTables