how to add directive attribute to element in link function?

As we known, the angular have performance if there are more than 2000 records in the page, because only some of the records need add some behavior , so I would prefer to dynamically add attribute to element according model value in link function, as there will be less watches.

So I use $compile to recompile the element like below:

mainApp.directive("popoverSetting", function ($compile) {
    return {
        restrict: "A",
        link: function (scope, element, attrs) {
            if (scope.item.isTrue) {
                element.attrs("ns-popover-trigger", "mouseenter");
                element.attrs("ns-popover-timeout", "0.01");

                $compile(element)(scope);
            }
        }
    }
})

Because there are about 1000 records, so the speed is very slow, is there some other way to add attribute and compile quickly? although there only 5 records need to add these attribute, it still increase about twofold time than before.