Debiasing Functions of Private Statistics in Postprocessing

Differential privacy (DP) has become widely accepted as a “gold standard” of privacy protection in statistical analysis. In particular, it has been adopted by many companies such as Google, Meta, and Apple to protect customer data and by the U.S. Census Bureau to protect respondent data in the 2020 Census [1]. DP mechanisms work by introducing randomness into the computation of all statistics published from a protected database. This added noise guarantees that no attacker can confidently determine from the published statistics whether a particular record is included in the dataset, thereby preserving its privacy.

Among DP mechanisms, additive mechanisms are canonical and widely used. These simply add data-independent, zero-mean random noise to the statistics. For example, the Laplace mechanism adds Laplace-distributed noise and is one of the first and most fundamental DP mechanisms [6]. The scale of the added noise needs to be proportional to the statistics’ global sensitivity – the greatest amount by which the statistic could change upon the addition or deletion of a single record. Intuitively, this ensures that there is enough noise to mask the presence or absence of any particular record.

Often, however, the statistics to which noise is added differ from the final statistics of interest. In these cases, the statistics of interest must be estimated using the available noisy statistics. Suppose that the noisy statistic $\tilde{q}$ is formed by applying an additive mechanism to the univariate statistic $q$, but that we want to learn $f(q)$, not $q$. Even though $\tilde{q}$ is unbiased for $q$, the plug-in estimator $f(\tilde{q})$ is not generally unbiased for $f(q)$. When unbiasedness is desired, other estimators must be used.

To address this problem, we first derive unbiased estimators for the Laplace mechanism, for a general class of twice-differentiable functions $f$ – those which are also tempered distributions (Section 3). [11] develops recursive algorithms that are unbiased estimators for polynomials in Laplace variables. Our paper provides estimators for a large class of non-polynomial functions and gives a simple, closed-form estimator for polynomials. We also provide methods to adapt functions that are not tempered distributions in a way that permits unbiased estimation over a subset of $q$’s domain (Section 4). This extension lets us provide unbiased estimators for the case when $f(q)=1/q$, which, in turn, lets us provide unbiased estimators of ratio statistics. Such cases arise frequently in practice, as discussed below. Finally, we derive unbiased estimators for a very general class of additive mechanisms when $f$ is a polynomial (see Section 7).

There are several reasons why noise may not be added directly to the statistic of interest and bias in the plug-in statistic must be considered. A leading case occurs when $f(q)$ has a much higher global sensitivity than $q$. For example, when the domain of $q$ includes 0 or values arbitrarily close to 0, the global sensitivity of $f(q)=1/q$ is typically infinite and no amount of noise provides a finite DP guarantee. This same problem affects the many statistics which can be expressed as ratios of low-sensitivity statistics. For example, the mean is the ratio of a sum and a sample size. Likewise, in a simple linear regression of the regressand $y$ on the regressor $x$, the ordinary least squares (OLS) estimator of the slope coefficient is the ratio of the empirical (co)variances $\text{Cov}[x,y]$ and $\text{Var}[x]$. It is common, then, to add noise to the low-sensitivity statistics that form these ratios and use the plug-in estimator for the ratio statistic of interest. See, for example, [2] for such a treatment of the OLS estimator.

Noise may also be added to statistics that are not of direct interest because data curators, such as government agencies, may publish noisy microdata or a noisy predetermined set of aggregates for general-purpose use. For example, a researcher trying to learn the proportion of the population with doctoral degrees may only have access to published noisy totals of the general population and the population of degree holders. The plug-in estimator of the mean is, as above, the ratio of these noisy totals.

This situation may also arise because, under DP, there is a limited “privacy budget” which is drawn upon every time we use the raw data to release another (noisy) statistic. Splitting the budget among more statistics requires that more noise be added to each of them. This makes it beneficial to “re-use” statistics whenever possible. For example, in Section 5, we develop a DP mechanism that uses our results to provide private unbiased estimates of a mean and sample size. This mechanism obtains a noisy sample size via the Laplace mechanism and then re-uses it to estimate the denominator of the mean statistic. We find that this approach outperforms an alternative mechanism that uses an existing method from [12] to add noise directly to the mean query, without re-using the noisy sample size.

Again, these scenarios all have in common that the plug-in estimator $f(\tilde{q})$ will typically be biased.¹¹1In fact, in the case with the Laplace mechanism and $f(q)=1/q$, the expectation and all higher moments of the plug-in estimator $f(\tilde{q})$ fail to even exist, implying that the estimator has extremely fat tails and is very prone to returning extreme outliers. This affects the ratio statistics discussed above, as well. The unbiased estimator we develop for this case in Section 4.1 possesses finite moments of all orders. To see why unbiasedness is desirable, recall that the bias and variance of a sum of $n$ uncorrelated estimates are respectively the sums of the estimates’ biases and variances. Accordingly, the sum’s bias increases at the rate $O(n)$ while its standard deviation grows at the rate $O(\sqrt{n})$. The sum’s overall RMSE therefore grows at the $O(\sqrt{n})$ rate if the estimates are unbiased, but at the faster $O(n)$ rate otherwise.

For example, consider the following simple example: suppose the true value of some quantity of interest is $1$, but each time we try to learn the value of this quantity, we get a fresh draw from the distribution $𝒩(1,100)$. A mechanism that ignores the data and returns $0$ has bias $1$ and variance $0$, resulting in an overall RMSE of $1$. On the other hand, reporting the value of any single draw would have bias $0$ and variance $100$. On average, this will be off by around $10$. Thus, on individual draws, the first mechanism is more accurate. However, if we take the mean of $10,000$ such draws, the mechanism that always returned $0$ still gives a mean of $0$, which is still off by $1$. On the other hand, the mean of the $10,000$ draws now has variance $\frac{100}{10,000}=\frac{1}{100}$, resulting in an RMSE of $\frac{1}{10}$. That is, we would now expect this estimate to be off by around $0.1$, which is a significant improvement over the biased estimator.

This makes unbiasedness very important in meta-analyses, which aggregate multiple estimates. It is also important when adding noise to a large number of quantities whose sums are of independent interest. This situation commonly arises in the local model of DP, where an extra layer of privacy is obtained by adding noise to every record even before entrusting it to the data curator. Sums or means using these noisy records could then be subject to severe error if the record-level estimates being summed are biased. For example, with network data, the count of $k$-stars (i.e., sets of $k$ edges sharing a node) is a sum of polynomials in each node’s degree. In experiments with network data protected by local DP, [11] find that a mechanism that simply sums unbiased estimates of these polynomials outperforms the L2 error of prior work by factors as high as 5 orders of magnitude.

Likewise, unbiasedness is key when noise is added to disaggregate sums with the expectation that they can be aggregated further to obtain sums for larger groups. For example, [13] and [8] develop mechanisms for use with per-record DP – a variant of DP whose privacy guarantees differ between records, and which is being considered by the Census Bureau for use with its County Business Patterns (CBP) data product [3]. [8] develop transformation mechanisms for this purpose, which improve privacy guarantees by adding noise to concave functions of $q$ rather than to $q$ itself. Estimates of $q$ must then be obtained from these noisy transformed values. The CBP data includes sums of employment and payroll, grouped by finely divided geographies and industry codes. If these transformation mechanisms were used for the CBP and the estimator of $q$ for these sums were biased, further aggregates of these estimates to obtain, say, state-level sums would be subject to severe biases.

In Section 6, we apply our estimators to create variants of these transformation mechanisms that satisfy a stronger type of per-record DP guarantee than the ones originally proposed in [8].

In this paper, we make the following contributions:

1. 1.

   We derive closed-form unbiased estimators for a large class of functions – twice-differentiable functions that are tempered distributions – when the Laplace mechanism is used. We also develop estimators that are unbiased for subsets of the statistic’s domain for functions that are not in this class.

2. 2.

   We exposit the deconvolution method from the statistics literature (e.g., [15], page 185) for deriving unbiased estimators. This could be used to derive estimators for further functions and further mechanisms, and we believe its use in DP is novel and of independent interest.

3. 3.

   We apply our unbiased estimators to create novel unbiased privacy mechanisms for per-record DP, a new variant of DP being considered for use by the Census Bureau [3].

4. 4.

   We derive closed-form unbiased estimators for polynomial functions of statistics privatized using any of a large class of additive mechanisms.

The following is the definition of differential privacy, introduced in [6]:

Most of our work uses Fourier transforms [9]. The following definitions and theorems are adapted from the textbook treatment in [14].

There are many important functions we wish to compute unbiased estimates of which are not absolutely integrable. In particular, polynomials and the function $f(q)=1/q$ are not absolutely integrable, so we must define the Fourier transform over a more general family of functions, tempered distributions.²²2Technically, $1/q$ is not a tempered distribution either, but only because it is poorly behaved at 0. This will be addressed in Section 4. Importantly, in this use, the term “distribution” does not refer to a probability distribution. Rather, it refers to a class of objects also known as “generalized functions.” For the purposes of this work, we can largely restrict ourselves to working with tempered distributions which are also functions, though there exist tempered distributions which are not functions, such as the Dirac delta “function”. Below, we specialize the relevant theory to the case of tempered distributions that are also functions, but the interested reader should see Appendix A of this paper’s full version (linked to on the title page) for the more general case.

For our purposes, then, tempered distributions can be thought of as functions that may not be absolutely integrable, but which grow no faster than a polynomial. Formally, this is expressed by the condition that the product of a tempered distribution and any function in the Schwartz space (defined below) is integrable.

Note that all the functions $s\in S(ℝ)$ are absolutely integrable, so their Fourier transforms $\widehat{s}$ exist. With the Schwartz space so defined, we introduce tempered distributions below.

Definition 3 introduces the Fourier transform only for absolutely integrable functions. The following definition extends it to all tempered distributions.³³3To see that Definition 3 implies Definition 6 for absolutely integrable functions, note that the $e^{-2\pi ixy}$ term does not depend on the function $f$, so swapping the order of integration by Fubini’s theorem immediately gives us the equality in Definition 6.

Technically, the Fourier transform of a tempered distribution always exists and is a tempered distribution, but may not also be a function, even when the distribution being Fourier-transformed is a function. See Appendix A of this paper’s full version for details.

The deconvolution method we use to derive unbiased estimators in Section 3 is applicable because, as explained below, the requirement that an estimator be unbiased can be expressed in terms of a convolution.

Critically, the Fourier transform of a convolution is the product of the convolved functions’ Fourier transforms.

We will also need the following theorem to derive unbiased estimators for the case of Laplace noise.

With the query $q$ and its privacy-preserving noisy estimate $\tilde{q}$, we say that an estimator $g$ is unbiased for $f(q)$ if

By conditioning on the true query value, $q$, we treat the database as fixed. Our estimators, then, are unbiased with respect to the randomness in the 0-centered noise being added for privacy. Throughout the rest of this paper, all expectations are conditional on $q$ unless otherwise noted and we suppress the extra conditioning notation so that $𝔼[.]\equiv 𝔼[.|q]$.

Let the noise added for privacy be independent of the database and denote its PDF by $r$. The deconvolution method, as seen, for example, on page 185 of [15], starts by noting that if $g$ is unbiased for $f(q)$, then Equation 1 can be reexpressed in terms of a convolution:

With the unbiasedness equation in this form, Theorem 8 lets us Fourier-transform both sides to turn the convolution on the right-hand side into a simple multiplication. Finally, we simply solve for the Fourier transform of $g$ in terms of the Fourier transforms of $f$ and $r$ and inverse-Fourier-transform the result. Formally,

assuming the existence of all the involved Fourier and inverse Fourier transforms.

A standard mechanism for differential privacy perturbs the query with Laplace noise scaled to the global sensitivity of a query, which is the maximum difference between the query values on neighboring databases. That is, to achieve $(ϵ,0)$-DP when releasing the value of a query $q$ with global sensitivity $\mathrm{\Delta }$, we can simply release $q+\mathrm{Lap}(0,\frac{\mathrm{\Delta }}{ϵ})$ [6].

Our primary contribution is deriving unbiased estimators for functions of $q$ when we only have access to the value of $q+\mathrm{Lap}(0,b)$, for some noise scale parameter $b$. These estimators are unique (up to their values on a set of measure zero).

If the function $q\mapsto f(q)$ is not a tempered distribution, we can often bound the domain such that it is continuous and twice-differentiable in that domain. That is, suppose we know a priori that $q\ge L$ for some lower bound $L\in ℝ$. This is often the case in differential privacy, as DP-protected queries are commonly sums of nonnegative variables. Likewise, counts can often be lower bounded by $1$. Then, suppose we replace the function $f$ with some function

where $h(L)=f(L)$ and $h$ is twice differentiable with $h^{\prime }(L)=f^{\prime }(L)$ and $h^{\prime \prime }(L)=f^{\prime \prime }(L)$. The function $\tilde{f}$ is thus twice differentiable. Assuming that $h(q)$ and $f(q)$ and their derivatives grow no faster than a polynomial as, respectively, $q\to -\mathrm{\infty }$ and $q\to \mathrm{\infty }$, $\tilde{f}$ is a tempered distribution, as well. We can then apply Theorem 10 to get an unbiased estimator of $\tilde{f}$, i.e.

With the assumption that $q\ge L$, we have $\tilde{f}(q)=f(q)$, making this estimator unbiased for $f(q)$, as well.

We now focus on optimizing this method over polynomial extensions for a particular function of interest: $f(q)=1/q$.

In this section, we illustrate the utility of our results by numerically comparing two mechanisms designed to return unbiased estimates of the sample size $n$ and the mean $m$ of an attribute $c\in [0,1]$ in the database $D$. Sample sizes are published alongside any reported means in most research applications, making this a realistic use case. One mechanism, $M_U$, returns an unbiased estimate of the mean using the results from Section 4.1. The other, $M_{SS}$, uses the unbiased mean mechanism from [12] (see their Theorem D.6 and proof). To the best of our knowledge, this is the only published unbiased mechanism for means when the sample size is not treated as known. Both mechanisms use the Laplace mechanism with privacy budget ${ϵ}_1$ to obtain the noisy sample size $\tilde{n}$. Each mechanism then allocates a separate privacy budget ${ϵ}_2$ to obtain a noisy mean. Both mechanisms have a total privacy budget of ${ϵ}_0={ϵ}_1+{ϵ}_2$.

Denote attribute $c$ of record $r$ by $r.c$ and let $g(\tilde{q};k,L)$ be the unbiased estimator of $1/q$ from Section 4.1 with the generic query $q$ and polynomial extension of order $k$ for $q\le L$. Algorithm 1 lays out $M_U$. This algorithm applies the Laplace mechanism to the sum $s\equiv {\sum }_{r\in D}r.c$ and forms an unbiased estimate ${\tilde{m}}_U$ of the mean $m=s/n$ by multiplying the noisy sum $\tilde{s}$ by $g(\tilde{n};k,L)$. This is unbiased for $m$ as long as $n\ge L$.

For $n\ge L$, the variance of Algorithm 1 is

For this section’s numerical results, we calculate $𝕍[g(\tilde{n};k,L)]$ numerically. Finally, we note that it is straightforward to show that all moments of ${\tilde{m}}_U$ exist and are finite.

Let $T_3$ denote a random variable distributed according to a standard $t$ distribution with 3 degrees of freedom. $M_{SS}$ is laid out in Algorithm 2. This algorithm first forms a version $m_{SS}$ of the mean query that simply equals 1 if $n=0$. It then scales the noise variable $T_3$ in proportion to an upper bound on the query’s smooth sensitivity [12]. The final noisy mean ${\tilde{m}}_{SS}$ is obtained by simply adding the scaled noise variable to $m_{SS}$.

The scaling factor for the noise is $\tau \max (e^{-\beta (n-1)},1/\max (n,1))$, where $\tau$ and $\beta$ satisfy ${ϵ}_2=4\beta +2/(\sqrt{3}\tau )$. The standard $t$ distribution with $d$ degrees of freedom has variance $d/(d-2)$ giving ${\tilde{m}}_{SS}$ a variance of

Because the $t$ distribution is symmetric, this mechanism is unbiased for $s/n$ as long as $n\ge 1$. Unlike ${\tilde{m}}_U$, however, the third and higher moments of ${\tilde{m}}_{SS}$ are infinite or do not exist. This is due to the $t$ distribution’s very fat tails and implies that ${\tilde{m}}_{SS}$ is more liable than ${\tilde{m}}_U$ to produce extreme outliers.

In our numerical evaluation of these mechanisms, we fix ${ϵ}_1={ϵ}_2=m=.5$ for both mechanisms. For $M_{SS}$, we follow [12] in setting $\beta ={ϵ}_2/12$ and $\tau =\sqrt{3}/{ϵ}_2$. For $M_U$, we set $L=1$ so that both mechanisms are unbiased for $n\ge 1$ and set $k=10$.

With these settings, we compare the standard deviations (SDs) of the two mechanisms’ mean estimates for a range of sample sizes. Because the mechanisms are unbiased, this is equivalent to their root mean squared error. The sample size estimates of both mechanisms are the same, so we do not report their properties.

Figures 1 and 2 plot the mechanisms’ SDs as functions of $n$, with Figure 2 zooming in on larger values of $n$ for clarity. It is immediately clear that the $M_U$ has a larger SD than $M_{SS}$ for , and that this pattern reverses for larger $n$. For $n\le 19$, however, both mechanisms have SDs greater than 1, making both unfit for most purposes at these sample sizes, given that the mean $m$ has the domain [0,1].

Figure 3 shows the relative SD - that is, the ratio $SD(M_{SS})/SD(M_U)$ - as a function of $n$. For $n\ge 13$, the relative SD rises to a peak near 15 before settling down to an apparent constant of about 1.9 for $n\ge 115$.

Ultimately, $M_U$ appears to be the better mechanism for this setting; for any sample size where either mechanism returns useful results, $M_U$ has a substantially lower SD. The thinner tails of $M_U$ also recommend it as the better choice.

In this section, we use our estimators to develop versions of unbiased privacy mechanisms from [8] that enjoy stronger privacy guarantees. Our estimators allow us to do so while maintaining the mechanisms’ unbiasedness.

$(ϵ,0)$-DP guarantees the upper bound $ϵ$ on the privacy loss between any pair of neighboring databases. [8] develops mechanisms for a related privacy concept, per-record DP (PRDP) [13]. PRDP generalizes ($ϵ,0$)-DP by letting the privacy loss bound be a function of the record on which a given pair of neighbors differs. Semantically, this allows different records to have different levels of protection.

Denote a record in the database $D$ by $r$ and denote $r$’s attribute $c$ by $r.c\in [0,\mathrm{\infty })$. PRDP was originally motivated by the need to protect data used to compute the sum query $q(D)={\sum }_{r.c\in D}r.c$. Because the domain of $c$ is unbounded, this sum can change by an arbitrarily large amount when a record is added or deleted. That is, the sum’s global sensitivity is infinite. This prevents commonly used privacy mechanisms, such as the Laplace mechanism, from providing a differential privacy guarantee with finite $ϵ$.

The traditional fix for this is to clip attribute $c$ to lie in a bounded set before taking the sum. $(ϵ,0)$-DP can then be guaranteed by perturbing the sum with noise scaled in proportion to the width of the clipped data’s domain. Unfortunately, when the sum is dominated by a small number of large outliers, the outliers typically need to be clipped to drastically smaller values to preserve a reasonable balance of privacy loss and noise variance. This can induce catastrophic bias, rendering the clipped sums essentially useless. One might expect to see this type of behavior with income data, for example.

PRDP allows us to take a finer look at the privacy-utility tradeoff by recognizing that, even though outliers may suffer extreme privacy loss, the rest of the dataset may still enjoy strong privacy protections. Intuitively, a particular record’s privacy loss is proportional only to the amount by which the addition or deletion of that record can change the query. Queries may be highly sensitive to the presence of outliers while being relatively insensitive to typical records, leading different records to have different levels of privacy loss. The reassurance that the vast majority of the data may enjoy strong privacy guarantees whether or not the data is clipped may allow a data curator to reasonably decide against clipping if the resulting bias outweighs the enhanced privacy protection for a small number of records.

Below, we define PRDP.

Ensuring strong privacy protection corresponds to ensuring that $P$ is, in some sense, small. $(ϵ,0)$-DP is recovered by making the constant privacy guarantee $P(r)=ϵ$ for all $r$, and strong privacy protection follows from a small $ϵ$. We cannot always make a guarantee this strong. Take the example where we want to publish a sum query on an unbounded attribute $c$ that we are unwilling to clip. In this case, the privacy loss of the mechanisms that we will consider here is growing in $r.c$. Even though we cannot prevent $P(r)$ from growing without bound in $r.c$, we can use mechanisms for which the growth rate is slow. [8] call such mechanisms “slowly scaling.” A slowly growing $P$ narrows the gap in privacy losses between records with large and small values of $c$, letting a data curator more easily provide a desired level of protection for the bulk of the data without compromising too much on the privacy of outliers.

[8] introduces slowly scaling mechanisms, called transformation mechanisms, that work by adding Gaussian noise to a concave transformation $f$ of the query (plus offset term) $q(D)+a$ and then feeding the noisy value of $f(q(D)+a)$ to an estimator $g$ of $q(D)$. By adding Gaussian noise, these mechanisms satisfy per-record zero-concentrated DP (PRzCDP), which is a weaker privacy guarantee than PRDP. PRzCDP relates to zero-concentrated DP [4, 7] in the same way that PRDP relates to $ϵ$-DP. The use of Gaussian noise also allowed [8] to draw on existing unbiased estimators from [16] to make their mechanism unbiased for a variety of transformation functions $f$.

Using the unbiased estimators from Theorem 10, we strengthen the transformation mechanisms to provide PRDP guarantees by adding Laplace, rather than Gaussian, noise, and we do so without losing the mechanism’s unbiasedness. Algorithm 3 lays out our transformation mechanism.

To obtain the PRDP guarantee of Algorithm 3, we first need to define the per-record sensitivity [8], a record-specific analog of the global sensitivity.

Theorem 19 gives our most generic result on the PRDP guarantees of Algorithm 3. Theorems 19 and 20, their proofs, as well as Algorithm 3 are minimally modified from their analogs in [8], which use Gaussian, rather than Laplace noise. This is to facilitate the interested reader’s comparison of our results with theirs.

See Appendix D of this paper’s full version (linked to on the title page) for the proof.

Probably the most common query encountered in applications of formal privacy, even as a component of other, larger queries, is the sum query. We now use the above result to derive the policy function for the transformation mechanism applied to a sum query.

See Appendix D for the proof.

Critically, the policy function from Theorem 20 grows in $r.c$ more slowly when the transformation function $f$ grows more slowly. In the case where $a=0$ and $f(x)=\sqrt[k]{x}$ for some $k\ge 1$, the policy function is simply $\sqrt[k]{r.c}$. Choosing larger values of $k$, then, forces the privacy loss to grow more slowly in $r.c$, reducing the gap in privacy losses between records with large and small values of $c$.

Applying our main results, we can obtain estimators such that the transformation mechanism gives us an unbiased estimate of $q(D)$. In particular, polynomials⁵⁵5In the case of polynomials, [11] derived unbiased estimators which could also be used here. The estimator obtained from our Theorem 10 merely simplifies computation in this setting. are twice differentiable functions which are tempered distributions, so the following holds for $f(x)=\sqrt[k]{x}$:

Additive mechanisms other than the Laplace mechanism, such as the discrete Gaussian or the staircase mechanisms, may be preferable in practice due to achieving higher accuracy while having similar privacy loss [5, 10]. In contrast to the Laplace case, these mechanisms may not admit tractable Fourier transforms, and hence unbiased estimators are generally not available in closed form. One exception is when the query of interest is a polynomial in one or many queries.

Using the following results to obtain an unbiased estimator of a polynomial that approximates a non-polynomial estimand may also allow users to obtain approximately unbiased estimators with great generality.

In this work, we have shown how to compute unbiased estimators of twice-differentiable tempered distributions evaluated on privatized statistics with added Laplace noise. In addition, we have proposed a method to extend this result to twice-differentiable functions which are not tempered distributions in a way that achieves approximately optimal expected squared error.

As the Laplace mechanism is simple and commonly used, these results are widely applicable to obtain unbiased statistics for free in postprocessing, which is particularly valuable due to the fact that aggregating unbiased statistics accumulates error more slowly than aggregating biased statistics. We have applied our results to derive a competitive unbiased algorithm for means and to derive unbiased transformation mechanisms for per-record DP mechanisms that enjoy stronger privacy protection than do analogs in previous work. Finally, we have derived an unbiased estimator for polynomials under arbitrary noise distributions with known moments, such as the discrete Gaussian mechanism or the staircase mechanism [5, 10].

We believe this paper opens several avenues for future research. These include the use of the deconvolution method to obtain unbiased estimators for other estimands and noise distributions. We believe a deconvolution method using multivariate Fourier transforms could also be used to obtain unbiased estimators of functions of multivariate queries. Although we did not attempt to optimize the numerical implementation in Section 5 of the integration in Section 4, we believe that an improved implementation could enable the practical use of higher-order polynomial extensions and further reduce error. In Section 7, we developed estimators that are exactly unbiased for polynomials that could approximate other functions of interest. Further work could elaborate on this process, developing concrete procedures for picking the approximating polynomial and deriving bounds on the resulting bias. Finally, future work could attempt to derive noise distributions that are optimal in the sense of minimizing the variances (or other utility metrics) of their unbiased estimators.