# Accumulator Error Feedback

(Difference between revisions)
 Revision as of 15:46, 25 February 2018 (edit)← Previous diff Revision as of 15:52, 25 February 2018 (edit) (undo)Next diff → Line 3: Line 3: ]] ]]

-                                                             function s_hat = csum(x)                                                                                                                                                                                                                                                                                                                         +                                                                function s = csum(x)
% CSUM Sum of elements using a compensated summation algorithm.                                                                                                                                                                                                                                                                                                                                                   % CSUM Sum of elements using a compensated summation algorithm.
%                                                                                                                                                                                                                                                                                                                                                                                                                 %
Line 24:                                                                                                                                                                                                                                                                                                                                                                                                       Line 24:
% end                                                                                                                                                                                                                                                                                                                                                                                                             % end

-                                                             s_hat=0; e=0;                                                                                                                                                                                                                                                                                                                                    +                                                                s=0; e=0;
for i=1:numel(x)                                                                                                                                                                                                                                                                                                                                                                                                  for i=1:numel(x)
-                                                             s_hat_old = s_hat;                                                                                                                                                                                                                                                                                                                               +                                                                s_old = s;
y = x(i) + e;                                                                                                                                                                                                                                                                                                                                                                                                     y = x(i) + e;
-                                                             s_hat = s_hat + y;                                                                                                                                                                                                                                                                                                                               +                                                                s = s + y;
-                                                             e = y - (s_hat - s_hat_old);                                                                                                                                                                                                                                                                                                                     +                                                                e = y - (s - s_old);
end                                                                                                                                                                                                                                                                                                                                                                                                               end
return                                                                                                                                                                                                                                                                                                                                                                                                            return
Line 37:                                                                                                                                                                                                                                                                                                                                                                                                       Line 37:
ones(1,n)*v  and  sum(v)  produce different results in Matlab 2017b with vectors having only a few hundred entries.                                                                                                                                                                                                                                                              ones(1,n)*v  and  sum(v)  produce different results in Matlab 2017b with vectors having only a few hundred entries.

-                                                             Matlab's variable precision arithmetic (VPA), (vpa(), sym()) from Mathworks' Symbolic Math Toolbox, cannot accurately sum a few hundred entries in quadruple precision.  Error creeps up above |2e-16| for sequences with high condition number (heavy cancellation defined as large sum|x|/|sum x|).  +                                                                Matlab's VPA (variable precision arithmetic, vpa(), sym()), from Mathworks' Symbolic Math Toolbox, cannot accurately sum a few hundred entries in quadruple precision.  Error creeps up above |2e-16| for sequences with high condition number (heavy cancellation defined as large sum|x|/|sum x|).
Use                                                                                                                                                                                                                                                                                                                                                                                                               Use
Line 58:                                                                                                                                                                                                                                                                                                                                                                                                       Line 58:
Kahan later proposed appending final error to the sum (after input has vanished).  This makes sense from a perspective of DSP because the marginally stable recursive system illustrated has persistent zero-input response (ZIR).  The modified Kahan algorithm becomes:                                                                                                                                         Kahan later proposed appending final error to the sum (after input has vanished).  This makes sense from a perspective of DSP because the marginally stable recursive system illustrated has persistent zero-input response (ZIR).  The modified Kahan algorithm becomes:

-                                                             function s = csum(x)                                                                                                                                                                                                                                                                                                                             +                                                                function s = ksum(x)
[~, idx] = sort(abs(x),'descend');                                                                                                                                                                                                                                                                                                                                                                                [~, idx] = sort(abs(x),'descend');
x = x(idx);                                                                                                                                                                                                                                                                                                                                                                                                       x = x(idx);

## Revision as of 15:52, 25 February 2018

```function s = csum(x)
% CSUM Sum of elements using a compensated summation algorithm.
%
% This Matlab code implements
% Kahan's compensated summation algorithm (1964)
%
% Example:
% clear all; clc
% csumv=0;  rsumv=0;
% n = 100e6;
% t = ones(n,1);
% while csumv <= rsumv
%    v = randn(n,1);
%
%    rsumv = abs((t'*v - t'*v(end:-1:1))/sum(v));
%    disp(['rsumv = ' num2str(rsumv,'%1.16f')]);
%
%    csumv = abs((csum(v) - csum(v(end:-1:1)))/sum(v));
%    disp(['csumv = ' num2str(csumv,'%1.16e')]);
% end

s=0; e=0;
for i=1:numel(x)
s_old = s;
y = x(i) + e;
s = s + y;
e = y - (s - s_old);
end
return
```

## Contents

### summing

ones(1,n)*v  and  sum(v)  produce different results in Matlab 2017b with vectors having only a few hundred entries.

Matlab's VPA (variable precision arithmetic, vpa(), sym()), from Mathworks' Symbolic Math Toolbox, cannot accurately sum a few hundred entries in quadruple precision. Error creeps up above |2e-16| for sequences with high condition number (heavy cancellation defined as large sum|x|/|sum x|). Use Advanpix Multiprecision Computing Toolbox for MATLAB, preferentially. Advanpix is hundreds of times faster than Matlab VPA. Higham measures speed here: https://nickhigham.wordpress.com/2017/08/31/how-fast-is-quadruple-precision-arithmetic

### sorting

Floating-point compensated-summation accuracy is data dependent. Substituting a unit sinusoid at arbitrary frequency, instead of a random number sequence input, can make compensated summation fail to produce more accurate results than a simple sum.

Input sorting, in descending order of absolute value, achieves more accurate summation whereas ascending order reliably fails. Sorting is not integral to Kahan's algorithm above because it would defeat input sequence reversal in the commented example. Sorting later became integral to modifications of Kahan's algorithm, such as Priest's (Higham Algorithm 4.3), but the same accuracy dependence on input data prevails.

### refining

Kahan later proposed appending final error to the sum (after input has vanished). This makes sense from a perspective of DSP because the marginally stable recursive system illustrated has persistent zero-input response (ZIR). The modified Kahan algorithm becomes:

```function s = ksum(x)
[~, idx] = sort(abs(x),'descend');
x = x(idx);
s=0; e=0;
for i=1:numel(x)
s_old = s;
s = s + x(i);
e = e + x(i) - (s - s_old);
end
s = s + e;
return
```

Error e becomes superfluous in the loop. Input sorting is now integral. This algorithm always succeeds, even on sequences with high cancellation; data dependency has been eliminated. Use the Advanpix Multiprecision Computing Toolbox to compare results.

### references

For fixed-point multiplier error feedback, see: