A Differentially Private Approximation of the Width Problem
Abstract
The width of a point set – the minimum distance between two parallel hyperplanes enclosing the data – is a fundamental geometric measure that captures how “flat” or “fat” a dataset is. As such, it serves as a basic shape descriptor used in visualization, convex hull approximation, and geometric data analysis. Despite its importance, width is highly sensitive to single-point changes, and no differentially private algorithm for approximating it was previously known.
We present the first pure -differentially private algorithm that approximates the width of a dataset. Our algorithm is a private adaptation of Chan’s approximation scheme [8] and operates by privately approximating the solution to a collection of suitably formulated linear programs. In addition to estimating the width, our method privately identifies a corresponding direction, enabling a private “fattening” transformation of the dataset – a basic structural preprocessing step for many geometric algorithms. This work advances the understanding of how geometric shape descriptors can admit good approximations even under the constraints of differential privacy.
Keywords and phrases:
Differential privacy, computational geometry, width approximation, private algorithms2012 ACM Subject Classification:
Theory of computation Approximation algorithms analysisEditor:
Huijia (Rachel) LinSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
This work is motivated by the problem of data-visualization, and is aimed at finding the data’s “geometric shape” using differentially private techniques.
A key geometric measure we focus on is the directional width, which captures how spread out a point set is along a given direction. Formally, for a unit vector , the directional width of a point set in direction is
The collection of directional widths over all directions characterizes the global geometric shape of the data and its convex hull. For any fixed direction , computing the maximal and minimal projection reduces to a one-dimensional range query, a problem that has been extensively studied in differential privacy [5, 9, 15]. However, extending such guarantees simultaneously to all directions or finding representing -directions for the data is substantially more challenging.
This issue is best demonstrated by the two extremal directional widths: the diameter, which is the maximum directional width over all directions; and the width of the set, defined as the minimum directional width over all directions. We denote that data’s width as . The width captures how “flat” or “fat” a dataset is. When the width is comparable to the diameter, the dataset is well-rounded or fat. In contrast, when the width is significantly smaller than the diameter, the dataset is nearly degenerate and concentrated around a low-dimensional structure. This distinction has significant algorithmic consequences. Many geometric approximation schemes – such as those for convex hull simplification, geometric packing, coresets, extent measures, bounding volumes, and LP-type problems – become substantially simpler and more efficient when the input is fat [14, 7, 10]. Conversely, when the width is very small, these problems often require more complex constructions to obtain comparable guarantees.
The main contribution of this work is thus a (pure) -DP algorithm that outputs an approximation to the data’s width as well as a direction on which it is obtained. Once those are found, it is fairly straight-forward to identify an interval in the direction of of length that holds most of the data, and then linearly stretch this interval so that the spread of the data along direction will be roughly as large as the diameter. So by repeating this process at most times we are able to create a “fat” dataset.
Beyond its structural role, the width problem also has direct geometric implications. For example, a ball of radius is guaranteed to fit inside the convex hull of the data, a property that is useful in algorithms that seek a point in the convex hull [19, 17]. Thus, width provides both structural and algorithmic insight into the dataset.
Non private width algorithms
In the non-private setting, the width problem on a dataset of size can be solved exactly in two dimensions in time [25] and in three dimensions in time [1]. In higher dimensions, an algorithm running in time is noted in [8]. More importantly, there are known -time approximation schemes for the width problem developed by Duncan, Goodrich, and Ramos [11] and by Chan [8] in his seminal work. The latter forms the basis of our work.
The Setting: Low-dimensional data with an exponentially large grid
It is evident that the algorithms that approximate the width run in -time, and so in this work we assume that the data is low-dimensional. Indeed, there exists datasets that hold sensitive information and are intrinsically low-dimensional, such as 2D-locations, trajectories or medical records composed of only a few features. And so we assume the given input is composed on points in the -dimensional cube . However, as proven by Bun et al. [5], no DP-algorithm is feasible if the data’s range is infinite, so we are also provided with a finite grid that the data resides on. We assume this grid has granularity , yet this granularity is exponentially small. (As a rule of thumb, think of , as if each coordinate in the data is given in C’s ints.) And so our algorithms must avoid brute-search over all points in .
Our contribution
It is fairly obvious to see that adding or removing a single datum to a data can change its width by an arbitrarily large amount, rendering the DP-techniques that rely on the global sensitivity of the width unapplicable. However, note that for any fixed pair of parallel hyperplanes the number of points between them changes by at most one across neighboring datasets. Thus, while the optimal width is highly unstable, the score function that counts how many points lie within a given slab has sensitivity one.
Building on this observation, we present a differentially private version of Chan’s approximation algorithm [8]. Chan reduces the width problem to solving a collection of linear programs (LPs), one for each direction in a (finite) cover of the unit sphere. We give a detailed explanation of Chan’s non-private algorithm in Section 2.2.1, where we formalize these LPs. However, roughly speaking, these LPs seek a vector (of largest possible magnitude) where by taking the dot-products of all points in with , these dot-products fall inside an interval of length .
And so in this work we introduce the notion of a score of a possible LP solution , defined as the maximal number of points from whose dot-product with reside inside an interval of length . (See Section 3 for the formal definition.) Similarly, the score of a region (a closed rectangle in ) is the maximal score of all s in the region. These definitions enables us to use simple SVT-based algorithm to conduct our private width approximation, seeing as the sensitivity of the score is at most .
This approach yields the first pure -differentially private algorithm for approximating width. Given a privacy parameter , an approximation parameter and a failure parameter , our algorithm achieves a -bi-criteria approximation: with probability at least the returned width satisfies: , where denotes the minimum width of the dataset. Moreover, there exists a pair of parallel hyperplanes at distance that sandwich all but at most points of .
Formally, our main result is as follows.
Theorem 1.
Fix , , and . Let and let be the grid in of granularity . There exists an -DP algorithm that, given a set of size , -approximates the width problem, where
The algorithm runs in time
where denotes the time required to solve linear equations in variables.111Currently, the best known bound is .
In addition, we give an -DP algorithm that outputs both an approximate width and two parallel hyperplanes at distance that enclose at least points, where
Organization.
In the remainder of the introduction we survey the few results that do exist in the intersection of DP and computational geometry. We then survey background material in Section 2, detailing also Chan’s non-private approximation algorithm. In Section 3 we give our -DP algorithms for approximating the width, and detail the main definitions of a score of a point and of a region. The main technical hurdle, which is how to find the score of a region without brute-force iterating over all grid-points in this region, is given in Sections 4 and 5. First, in Section 4 we compute it for 2D data (which is arguably the only case where iterating over a cover is practical); and in Section 5 we show how to compute it in -dimensions. Finally we conclude with open problems in Section 6.
Related Work.
We note that no prior work focuses specifically on the width problem and its DP-approximation. However, the problem of privately approximating geometric structures has been studied from several angles. In [16] the authors aim to learn a shape from the data, assuming there are many points inside and outside the shape. The works of Beimel, Moran, Nissim and Stemmer [2] and of Kaplan, Sharir and Stemmer [19] provide differentially private algorithms for finding a point that lies inside the convex hull of a dataset, using the notion of Tukey-depth [26]. Building on this notion of Tukey-depth, the work of Gao and Sheffet [13] investigates how to privately approximate the convex hull and the structure of Tukey-regions. Among the algorithms proposed by Gao and Sheffet is an algorithm that approximates the width of a Tukey-region, but only under the premise that the width is proportional to the diameter (as they simply approximate the directional width for each direction is a cover of the unit-sphere). Several works [23, 22] look at the -cluster problem under differential privacy, which can be applied to approximate the data’s diameter (even in a high-dimensional setting).
Chan’s approximation method focuses on solving multiple linear programming problems. Our algorithm addresses how to solve these problems privately and efficiently. A related work is [17], which provides a DP-algorithm for solving linear problems, but whose efficiency depends on the “roundness” parameter, defined as the radius of the smallest ball that fits inside the LP polytope. This quantity is related to the width, since the roundness cannot be smaller than the minimal width of the point set. (See more discussion in Section 2.3.)
2 Preliminaries
2.1 Differential Privacy
A dataset is a multi-set of points in . Two datasets and are called neighbors if they differ on a single point. An algorithm is said to be -differentially private (-DP) if for any two neighboring and and any measurable set of outputs it holds that
One of the basic mechanisms that preserves -DP is the Laplace mechanism. First, given a query from datasets to the reals , we say that the query is -sensitive if . Given a -sensitive query, it is known that the mechanism that on input outputs is -DP, where denotes a random variable drawn from the Laplace distribution with . Another basic property of DP is the basic composition of two DP algorithms, where for any the joint output of a -DP mechanism and of a -DP mechanism is -DP.
Sparse Vector Technique (SVT)
One of the tools we use repeatedly in this work is the Sparse Vector Technique (SVT) (also known as the Above-Threshold algorithm in [12]). In this setting we are given a sequence of queries, where all queries are -sensitive, and our goal is to identify the first query whose value exceeds a publicly known threshold .
Fact 2.
The SVT is a -DP algorithm which satisfies the following. Set . Then, if SVT halts on query then w.p. we have that , and for every it holds that .
2.2 Width of a Set
The width of a set of points is defined as the smallest distance between two parallel hyperplanes that enclose all points in . Formally, if we denote the set of points by and given a unit vector , the directional width of along direction , denoted . The width of dataset is thus .
2.2.1 Chan’s Approximation Algorithm of the Width
In [8], Chan reformulates an algorithm originally presented by Duncan et al. [11] to approximate the width of a set.
A rather natural “first attempt” at obtaining a -approximation of the width would work as follows. Discretize the unit sphere into a -cover, and find the minimum width among all directions in the cover. However this approach guarantees only an additive approximation to the width, which may be very large for datasets with diameter width. In other words, in order to obtain a multiplicative guarantee for the width approximation problem, this approach requires a very fine discretization of the unit-sphere (proportional to the data’s width). This, in turn, renders the naïve approach of applying the Exponential Mechanism [21] over a cover of the unit-sphere infeasible. Chan’s approach avoids this issue by adapting to the geometry of the data via linear programs, one per each direction in the cover.
We now reiterate Chan’s algorithm for clarity. Recall that the width problem is: . We can write the problem as:
| s.t. |
Equivalently, we can write:
Note that in the latter formulation, for any we have that both and for any scalar lead to the same minimal value. So we restrict the solution by requiring that . This constraint fixes the norm of . Thus, we obtain the following problem:
| s.t. |
which is equivalent to the problem
| (1) | ||||
| s.t. | (2) |
In this formulation of the problem all constraints are linear, yet the objective isn’t a concave function and so we cannot obtain its maximization easily. Instead, Chan suggestes using a cover of all directions. Let be a set of unit-length vectors with the property that for any unit-length there exists some such that . It is known that a cover of size exists. It now follows that the optimal solution to the above-mentioned optimization problem becomes “almost aligned” with some grid direction – namely, we set , and have that for some it holds that:
And so, Chan suggestes we solve all LPs of the form:
| LP(): | (3) | |||
| s.t. | (4) |
and take the maximum over all solutions, and this guarantees a solution whose width is .
Min. Possible Width.
In our setting, all points in are known to reside on a predefined grid of granularity . We wonder as to the smallest possible value of . clearly, if all points lie on some strict subspace of then their width is . So, we pose the question: what is the smallest non-zero value can take assuming the data isn’t degenerate?
We answer this question using Chan’s formulation. Based on (4), we know that the solution to LP() is attained at a vertex of its feasible polytope. Such a vertex is determined by a system of linear equations in variables, where each non-zero coefficient in the system lies between and . By Cramer’s rule, each coordinate of the solution is given by a ratio of two determinants. By Hadamard’s inequality, the numerator is at most (since the norm of each row is bounded by ), while the denominator is at least . Hence each coordinate of is bounded by .
It follows that the LP objective value is at most . Since this objective only approximates the inverse width, we incur an additional constant factor of when translating the bound to the true objective. Inverting this bound, the smallest possible non-zero width satisfies .
The analysis on the min-possible width is required for our -DP algorithm for two reasons. First, it is required as part of the SVT-analysis, whose utility bound depends logarithmically on the number of queries posed – and we pose one for every (approximated) value of the LP value. Second, this analysis plays an even more important role in Section 3.1, where we give the algorithm that finds an approximated LP solution (and not merely the LP-value). The algorithm merely conducts DP-binary search over the grid of possible solutions to the LP, and this grid needs to be of granularity . As a result, in the processes of conducting said binary search, we may lose additional many points. Needless to say, if we replace this worst-case grid with a grid of larger granularity, then the cost of the latter part can significantly improve.
2.3 Baseline
An off-the-shelf DP-algorithm for approximating the width would therefore be to run a DP-LP solver for the problems , and apply private selection algorithms [20, 24] to choose a good solution. The DP-LP solver of Kaplan et al. [18] runs in exponential time, as it requires convexifying the set of points that satisfy precisely constraints of the LP for any value of . The current best DP-LP solver is given in a work of Kaplan et al. [17] where they bound the number of violated LP constraints at where is the granularity of the given grid and is a new parameter known as the width of the LP, which we don’t know how to apriori control.222Kaplan et al. bound it at . In contrast, our work yields the bound , so we enjoy both a better dependency on and a double-log dependency on . This unfortunately comes at the expense of the runtime – our algorithms run in time whereas the runtime of the LP solver of [17] runs in time fully polynomial in . But recall that Chan’s approximation algorithm solves many LPs anyhow, so a runtime with exponential dependency on is unavoidable (to the best of our knowledge).
3 Private Width Approximation Algorithms
We begin with a high-level description of the full private width-approximation pipeline. At a conceptual level, the algorithm proceeds in two stages. First, it privately estimates the width value by searching over candidate values and checking, for each one, whether there exists a direction and an associated LP solution that places almost all projected points inside an interval of length . Second, once such a width value is found, the algorithm privately recovers a corresponding direction and LP solution, from which one can also derive a pair of enclosing hyperplanes. The detailed subroutines used in these stages are given later in this section and in the following sections. The high-level procedure:
-
Run Algorithm 1 on to obtain an approximate width .
-
Run Algorithm 3 on input to obtain a vector witnessing this approximate width.
-
Project the points of onto and privately identify an interval of length containing many projected points.
-
Convert this interval into a pair of parallel hyperplanes orthogonal to , at distance .
We now describe the first stage of the algorithm, namely the private estimation of the width value. As outlined above, this stage reduces to searching over candidate width values and identifying the largest value for which there exists a direction and an associated LP solution that captures almost all points. At its core, the algorithm is a simple SVT: we are conducting a search for the optimal value of the width , where we start with a guess of and reduce it from there all the way down to the lowest possible value of . Thus, the number of guesses of is upper bounded by . We denote our guessed values as .
For each guess , our SVT asks if there exists at least one LP() (as in Equation (4)) for which there exists some that satisfies and when projecting points onto , then many points fit inside an interval of length . More formally, we are looking for some of the form for some perpendicular to , and ask for the maximal number of points whose projection onto fits in an interval of length .
Definition 3 (Score at a Point).
Let be a set of points. Fix a scalar and a direction . Given a point perpendicular to we denote the score of with respect to and , denoted , as the maximum number of points in whose dot-product with lie inside an interval of length .
Formally, given and , we denote . For any consider the interval and denote as . Then . (We omit the superscript from the score function when the context is clear.)
In the 2-dimensional setting, we often fix the orthogonal direction to , , and identify the point with the scalar such that , namely define . We use a similar brevity in -dimensions as well. Given an orthonormal basis to the subspace perpendicular to , we denote the score of as the score of the point , i.e. .
Efficiently computing the score of a point
We argue that the score of a given point , , can be efficiently computed in time (for any dimension ). Indeed, all we need to do is the following.
-
1.
Compute the dot-product of each with .
-
2.
Sort the resulting dot-products to obtain .
-
3.
Iterate over the intervals to check how many points fall inside this interval.
-
4.
Return the maximal number of points found in Step 3.
This takes -time (Step 3 can be done in -time per interval by simply holding two indices where index points to the first dot-product and points to the first dot-product ).
Score at a region
The main hurdle is that we do not know which point in the subspace orthogonal to yields the highest possible score or how to find such a . And this is the main technical contribution of this paper: we show that in order to find a of large score, there are points of interest in whose score we need to check and will be among them. But we table the discussion on runtime analysis to Sections 4 and 5, and continue with the formal definitions.
Again, we first consider the 2-dimensional case. Our goal is to find , or , of large score. So we thus define the score of any closed interval to be the maximal score of any . The same idea is generalized to any in the following definition.
Definition 4 (Score at a region).
Let be closed intervals in (including perhaps the case that for some ), and let be a closed rectangle in . Let be a set of points. Fix a scalar and a direction . Fix to be some orthonormal basis of the subspace perpendicular to in . Then the score of the region is defined as
The rest of the definitions we give are simply notations for maximal scores. We now define the score of a direction as , and the score of a guess as
| (5) |
(Again, we omit the superscript from all score-functions when the context is clear.)
It should be clear that for any two neighboring and and any and , we have
or, alternatively, that the sensitivity of the score at a point is at most . It follows that for any -dimensional rectangle the sensitivity of the query is also , as a maximum over queries of sensitivity 1. (See [4] for a proof of this fact.) Similarly, we also have that the query is also of sensitivity 1, and so does . And so our algorithm simply uses SVT for the sequence of queries, where , and halts on the first guess whose score is too low, and returns the next-to-last query. The details appear in Algorithm 1.
Theorem 5.
Proof.
Privacy.
The fact this algorithm is -DP follows directly from Theorem 2. The entire algorithm is composed of queries, each is -sensitive, for which we run the SVT algorithm, therefore it is -DP. Moreover, simple tail bounds on the Laplace distribution give that w.p. all random variables drawn from in the execution of Algorithm 1 are all of magnitude .
Utility.
We argue that for each guess satisfying , it holds that . Let be an optimal solution to Equation (2), before restricting to the discretized directions , and let be the closest direction to in . Decompose as , where . By the grid approximation guarantee, we have . Since , it follows that .
In the algorithm, the coefficient of is required to be exactly . Thus, consider the scaled vector , where and therefore . Since the projections of all points onto lie within an interval of length , scaling by implies that the projections onto lie within an interval of length at most . Hence, .
Therefore, for each guess we get that . It follows that the noisy score must be whereas the noisy threshold and so the SVT skips on this guess and continues to the next guess. Therefore we can only halt once , which implies we return (assuming ). Finally, observe that the noisy score at guess was greater than . It follows we return guess for which the score must be at least , hence as required.
Runtime.
Lastly, the runtime is composed of iterations in which we do -work plus the time of computing . In order to compute as per Equation (5) we need to iterate over all vectors , of which there are -many, and for each one find the max score of any point in the subspace orthogonal to – which takes -time. So overall our runtime is .
3.1 Finding the Direction that Yields the Width
Next we turn our attention to finding an approximated solution to the , and not just its value. Assume Algorithm 1 has returned some . The main idea is to do the DP-version of tracing-back our steps in computing the score to find a good solution for our problem.
In more detail, given , we first run another SVT to find the direction with a large score – i.e., where only a few points do not fall into an interval of size 1. This SVT involves many queries. Once is found, we conduct a binary search over the region of possible solutions: we equi-partition the region along some axis to obtain two rectangular regions and , query to see whether the score of is large – and if so we continue to , otherwise we continue to . We halt our binary search once our region holds a single grid point and return that point. Given the solution to the LP, the last step is to conduct a search along the direction of for an interval of width that contains many points, thereby finding the two hyperplanes that yield the width.
There’s just one subtlety. While the given points of do reside on the given grid , it doesn’t assure as that the solution of the LP() lies on the same grid. Moreover, in our binary search we look for approximated solutions of LP() (one that only places the projection on points of some subset inside an interval of length ). Luckily, we already established in Section 2.2.1 that each coordinate of any vertex in such a polytope is upper bounded by a ratio of two determinants of matrices whose entries are integer multiplications of . Thus, each coordinate of a solution we seek is upper bounded by , and any non-zero coordinate is lower bounded in magnitude by . So let be the grid in of granularity . (Thus, the new grid has many points.) Our binary search is conducted on this grid. The full details of the algorithm are given in Algorithm 3, which is deferred to Appendix A along with its proof of correctness.
4 Computing the Score in 2D
We now turn our attention to the question of computing the score of a region . In this section, we detail the algorithm that finds a score for a region when the input is two-dimensional. This means that the region in which we search for a point of maximal score is simply a closed interval .
Recall that our goal is to find which maximizes the score , which is defined as the maximal number of points whose dot-product with the vector reside in an interval of length . So we represent any as , and it follows that . Algorithm 2, we iterate over all possible choices of two distinct points, and , which sends to the bottom and top of the interval of length 1 resp.
Claim 6.
Algorithm 2 returns in time .
Proof.
Fix . Let be the point of maximal score. Let be the set of points whose dot-products with lie inside an interval of length . Thus .
For each pair of distinct points in we say it is degenerate if , and otherwise admissible. Note that pairs that are degenerate have a fixed distance between their dot-products with regardless of the value of . It is only the distance between admissible pairs that is affected by the value of . For a set and vector , define the maximal distance within ’s dot-products with as
Our focus in this proof is on . In the extreme case there are no admissible pairs in , which makes ill-defined. But in this case it follows that all points in have the same value, and so the distance between their dot-products with isn’t affected by . So computing the score on either ends of should allow us to find .
So assume is well-defined. By the definition of we know that all dot-products lie inside an interval of length , so . Now if , then there exists an admissible pair so that . Clearly, define , and so when the for-loop of our algorithm reaches this admissible pair, we add to the list of candidate values, and the algorithm therefore returns .
Otherwise, . As we increase the quantity varies linearly for each pair . Since is the maximum over finitely many such linear functions, it varies continuously in . Define
Note that this set of larger -s isn’t empty. Fix some pair of admissible and with , and we can see that the function is increasing in . As at , by assumption, all admissible pairs satisfy , then we must increase to obtain the value that sets .
Having established that is well defined, denote . By construction, we have , and moreover, for every we have that . Thus all points in have dot-products with that fall inside an interval of length , and so for every . Now, in its for-loop, our algorithm reaches the admissible pair that yields . And so if we add it to , and we have that is returned by the algorithm. Otherwise, , but in this case we test the score on , and so is returned by the algorithm.
Finally, observe that our algorithm does iterations to find the values, and then does work per score computation, so overall the runtime of our algorithm is . This completes the proof.
We comment that the algorithm can be “massaged” so that it computes the score of all possible -values in advance, and given an interval finds the max-score by looking at the appropriate intersection. This is especially useful when Algorithm 3 runs its binary search and repeatedly invokes Algorithm 2 over the same choice of and .
5 Computing the Score in General Dimension
In this section we generalize the previous 2D-algorithm to a general dimension . The algorithm’s main idea is again to brute-force search over a pairs of points that will determine the end points of the interval of length and then guess additional constraints: either points at the interval extremes or region bounding constraints. For each choice of tight constraints we solve the linear system to obtain a candidate point. This fixes at most candidate points. Due to brevity, we defer this section in its entirety to Appendix B.
6 Open Problems
In this work we give the first DP algorithm that bi-approximates the width of a dataset , one of the first measures of a shape of . Our work leaves many questions unanswered. First, we ponder as to a lower-bound for the problem: can we argue that any -DP algorithm that approximates the width must omit many points? Or does there exists a better bi-criteria approximation of the width?
A second question we raise is regarding the efficiency of the binary search for the point of highest score inside a region. In our work we used the most straight-forward binary search, using pure-DP. However, is it possible to adapt the -DP technique of Bun et al. [6] for binary search to this setting (over a general function)?333The score function isn’t quasi-concave, so the recursive technique of [3] is inapplicable in this setting. This should allow us to conduct a binary search with queries while omitting only many points.
Thirdly, we ponder as to the applicability of our approach to general LPs. Can the ideas from this work be successfully applied to any low-dimensional LP? Is there a simpler and more efficient algorithm than the algorithm of [17] for solving LPs in lower dimensions?
Lastly, we would like to extend our work to other notions and properties studied in computational geometry, and devise DP algorithms for estimating those as well. (This includes other notions of responsible computing – like fair algorithms or reproducible algorithms.) Of course, this requires us first to devise robust analogs of these notions and properties, analogs that generalize. Namely, if it is the case that is drawn i.i.d. from some distribution , then the estimator over should be indicative as to the same property over . Ultimately, we would love to describe ’s shape using DP algorithms – afterall, if ’s shape is “robust,” then adding or removing a single datum shouldn’t change its overall shape, and we should be able to say that ’s in the shape of say, a rectangle, using DP techniques.
References
- [1] Pankaj K. Agarwal and Micha Sharir. Efficient randomized algorithms for some geometric optimization problems. In Proceedings of the Eleventh Annual Symposium on Computational Geometry, SCG ’95, pages 326–335, New York, NY, USA, 1995. Association for Computing Machinery. doi:10.1145/220279.220314.
- [2] Amos Beimel, Shay Moran, Kobbi Nissim, and Uri Stemmer. Private center points and learning of halfspaces. In Alina Beygelzimer and Daniel Hsu, editors, Conference on Learning Theory, COLT 2019, 25-28 June 2019, Phoenix, AZ, USA, volume 99 of Proceedings of Machine Learning Research, pages 269–282. PMLR, 2019. URL: http://proceedings.mlr.press/v99/beimel19a.html.
- [3] Amos Beimel, Kobbi Nissim, and Uri Stemmer. Private learning and sanitization: Pure vs. approximate differential privacy. In Prasad Raghavendra, Sofya Raskhodnikova, Klaus Jansen, and José D. P. Rolim, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques - 16th International Workshop, APPROX 2013, and 17th International Workshop, RANDOM 2013, Berkeley, CA, USA, August 21-23, 2013. Proceedings, volume 8096 of Lecture Notes in Computer Science, pages 363–378. Springer, 2013. doi:10.1007/978-3-642-40328-6_26.
- [4] Avrim Blum, Katrina Ligett, and Aaron Roth. A learning theory approach to non-interactive database privacy. In Cynthia Dwork, editor, Proceedings of the 40th Annual ACM Symposium on Theory of Computing, Victoria, British Columbia, Canada, May 17-20, 2008, pages 609–618. ACM, 2008. doi:10.1145/1374376.1374464.
- [5] Mark Bun, Kobbi Nissim, Uri Stemmer, and Salil P. Vadhan. Differentially private release and learning of threshold functions. In Venkatesan Guruswami, editor, IEEE 56th Annual Symposium on Foundations of Computer Science, FOCS 2015, Berkeley, CA, USA, 17-20 October, 2015, pages 634–649. IEEE Computer Society, 2015. doi:10.1109/FOCS.2015.45.
- [6] Mark Bun, Thomas Steinke, and Jonathan R. Ullman. Make up your mind: The price of online queries in differential privacy. In Philip N. Klein, editor, Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2017, Barcelona, Spain, Hotel Porta Fira, January 16-19, pages 1306–1325. SIAM, 2017. doi:10.1137/1.9781611974782.85.
- [7] Vítor Gomes Chagas, Elisa Dell’Arriva, and Flávio Keidi Miyazawa. A framework for efficient approximation schemes on geometric packing problems of -dimensional fat objects. arXiv preprint, 2024. arXiv:2405.00246.
- [8] Timothy M Chan. Approximating the diameter, width, smallest enclosing cylinder, and minimum-width annulus. In Proceedings of the sixteenth annual symposium on Computational geometry, pages 300–309, 2000. doi:10.1145/336154.336216.
- [9] Edith Cohen, Xin Lyu, Jelani Nelson, Tamás Sarlós, and Uri Stemmer. Optimal differentially private learning of thresholds and quasi-concave optimization. In Proceedings of the 55th Annual ACM Symposium aon Theory of Computing, pages 472–482, 2023. doi:10.1145/3564246.3585148.
- [10] Minati De, Saksham Jain, Sarat Varma Kallepalli, and Satyam Singh. Online geometric covering and piercing. Algorithmica, 86(9):2739–2765, 2024. doi:10.1007/S00453-024-01244-1.
- [11] Christian A Duncan, Michael T Goodrich, and Edgar A Ramos. E cient approximation and optimization algorithms for computational metrology. In Proc. 8th ACM-SIAM Sympos. Discrete Algorithms, pages 121–130, 1997. URL: http://dl.acm.org/citation.cfm?id=314161.314196.
- [12] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., 9(3–4):211–407, August 2014. doi:10.1561/0400000042.
- [13] Yue Gao and Or Sheffet. Differentially private approximations of a convex hull in low dimensions. In Stefano Tessaro, editor, 2nd Conference on Information-Theoretic Cryptography, ITC 2021, Virtual Conference, July 23-26, 2021, volume 199 of LIPIcs, pages 18:1–18:16. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2021. doi:10.4230/LIPIcs.ITC.2021.18.
- [14] Sariel Har-Peled. Geometric approximation algorithms. Number 173 in Mathematical Surveys and Monographs. American Mathematical Soc., 2011.
- [15] Haim Kaplan, Katrina Ligett, Yishay Mansour, Moni Naor, and Uri Stemmer. Privately learning thresholds: Closing the exponential gap. In Conference on learning theory, pages 2263–2285. PMLR, 2020. URL: http://proceedings.mlr.press/v125/kaplan20a.html.
- [16] Haim Kaplan, Yishay Mansour, Yossi Matias, and Uri Stemmer. Differentially private learning of geometric concepts. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 3233–3241. PMLR, 2019. URL: http://proceedings.mlr.press/v97/kaplan19a.html.
- [17] Haim Kaplan, Yishay Mansour, Shay Moran, Uri Stemmer, and Nitzan Tur. On differentially private linear algebra. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing, pages 2362–2373, 2025. doi:10.1145/3717823.3718274.
- [18] Haim Kaplan, Yishay Mansour, Uri Stemmer, and Eliad Tsfadia. Private learning of halfspaces: Simplifying the construction and reducing the sample complexity. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 13976–13985, 2020.
- [19] Haim Kaplan, Micha Sharir, and Uri Stemmer. How to find a point in the convex hull privately. In Sergio Cabello and Danny Z. Chen, editors, 36th International Symposium on Computational Geometry, SoCG 2020, Zürich, Switzerland, June 23-26, 2020, volume 164 of LIPIcs, pages 52:1–52:15. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2020. doi:10.4230/LIPIcs.SOCG.2020.52.
- [20] Jingcheng Liu and Kunal Talwar. Private selection from private candidates. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019, pages 298–309, New York, NY, USA, 2019. Association for Computing Machinery. doi:10.1145/3313276.3316377.
- [21] Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2007, Providence, RI, USA, October 20-23, 2007, Proceedings, pages 94–103, 2007. doi:10.1109/FOCS.2007.66.
- [22] Kobbi Nissim and Uri Stemmer. Clustering algorithms for the centralized and local models. In Algorithmic Learning Theory, pages 619–653. PMLR, 2018. URL: http://proceedings.mlr.press/v83/nissim18a.html.
- [23] Kobbi Nissim, Uri Stemmer, and Salil Vadhan. Locating a small cluster privately. In Proceedings of the 35th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, pages 413–427, 2016. doi:10.1145/2902251.2902296.
- [24] Nicolas Papernot and Thomas Steinke. Hyperparameter tuning with renyi differential privacy. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022. URL: https://openreview.net/forum?id=-70L8lpp9DF.
- [25] F.P. Preparata and M. Shamos. Computational Geometry: An Introduction. Monographs in Computer Science. Springer New York, 1993. URL: https://books.google.co.il/books?id=gFtvRdUY09UC.
- [26] Tukey J. W. Mathematics and the picturing of data. Proceedings of the International Congress of Mathematicians, Vancouver, 1975, 2:523–531, 1975.
Appendix A A Differentially Private Algorithm for Finding the Hyperplanes that Yield the Width
Theorem 7.
Proof.
Privacy.
The algorithm is composed of two subalgorithms: one for finding a direction with a large score (Lines 2-12); and another that finds a grid point of large score using binary search (Lines 13-25). The former is -DP as it simply a SVT over -many queries, all of sensitivity . The latter is composed of many queries, each of sensitivity ; so we partition the privacy budget to and add noise of to each – which preserves -DP overall. Moreover, w.p. , all r.v.s we draw in the former part (from ) are of magnitude ; and w.p. , all r.v.s we draw in the latter part (from ) are of magnitude . So w.p. both event holds. We continue our analysis assuming all r.v.s are bounded in magnitude.
Utility.
Based on Theorem 2, it is simple to see that the direction we find satisfies that This implies that some point satisfies for the same defined in Line 14.
We now argue inductively, that in each iteration of the binary search we have a region of score . The base case (iteration ) was already established. The induction step is also straight forward once we assume all r.v.s in the binary search have bounded magnitude – at most . If the induction hypothesis holds in the beginning of the iteration, then the point of score lies either in or . If it is in then we ought to pass the threshold-query in Line 19, and continue to work with and then in the next iteration as required. Whereas this point of high score lies in we either fail or pass the threshold-query of Line 19: if we fail it then we continue to and as required; and if we pass it then we continue to , where because of passing the query we know that .
It follows that end of the binary search, namely, after iterations, we’re left with a single grid point of score . And so .
Runtime.
Lastly, we detail the runtime of Algorithm 3. The former part (SVT on all directions) takes time, and the binary search takes . So overall we get a runtime of .
We comment that Algorithm 3 finds a vector so that the dot-product of many point with lie in an interval of length . (So the projection onto the direction of at least many points fits inside an interval of width .) If it is our goal to find a pair of parallel hyperplanes that hold points between them, then we need to return in addition to some scalar so that many points satisfy that their dot-product with falls inside the interval . This requires us to conduct another binary search for this using the 1-dimensional dataset of projected points . This binary search of course may omit a few more points from , but only many. We thus omit the details of this last binary search as they are subsumed by the -dimensional binary search for .
Appendix B Computing the Score in General Dimension
In this section we generalize the previous 2D-algorithm to a general dimension. Recall, we are given a scalar and a direction and we fix some basis of the subspace orthogonal to . In addition, we are given a rectangular region . This means that our goal is to find scalars so that when taking dot-product of the points in with
we maximize the number of points that fit into an interval of length . The algorithm for computing the score of the region (which is the score of the best ) is given in Algorithm 4.
Lemma 8.
Algorithm 4 returns .
Proof.
Fix a vector of maximal score, and let be the maximal set of points whose dot-products with lie in an interval of length . We show that we can find a vector satisfying constraints from with the property that the dot-products of all points in the same with fall inside an interval of length .
We proceed by induction on the number of linearly independent constraints currently satisfied by , and show how to iteratively shift to a new vector that satisfies more and more constraints in . Starting from the base case: . This implies that all dot-product differences for are strictly less than . Fix a direction in the -dimensional parameter space, and move continuously from along : in the affine subspace . Because all dot-product differences vary continuously and only finitely many pairs exist, there is a smallest such that either (i) some pair satisfies , or (ii) we reach the boundary of the feasible rectangle. In either case a new constraint becomes tight, so set , which means now .
Now assume . The set of vectors satisfying the current constraints in is an affine space of dimension . As long as , this space has positive dimension, so fix some nonzero direction that preserves all currently tight constraints. We now move to , where is the smallest value for which either a new pair reaches projection gap , or a new boundary of the rectangle is hit. (Here “new pair” means that their projection gap at was strictly less than , i.e., .) Since shifting along preserves all previously tight equalities and never increases any projection gap beyond , all points of remain inside some interval of length . Thus one new constraint is added and increases. We argue that this additional constraint leaves linearly independent: it is the only constraint with a component in direction whereas all other constraints originally in were orthogonal to .
Since at most constraints can become tight, repeating this process finitely many times yields a vector satisfying exactly linearly independent constraints, while still preserving the property that all points of project into an interval of length .
Now, as written, Algorithm 4 runs in time where is the time required to solve a system of linear equations in variables. From the proof it follows that we can refine our search – rather than trying all pairs of points for each constraint, our search can work as follows. First we pick one pair of points that go to the two endpoints of the interval (namely, so that ). Once and are chosen, iterate over the choice of other points and all options of assigning them to the same end as or the same end as . This gives additional equations of the form (or ), and now solve the system of equations in variables in time . This requires time . Now, each one of those might yield a vertex on which we need to test the score, so this costs overall .
