Vantage Point Selection Algorithms for Bottleneck Capacity Estimation
Abstract
Motivated by the problem of estimating bottleneck capacities on the Internet, we formulate and study the problem of vantage point selection. We are given a graph whose edges have unknown capacity values that are to be discovered. Probes from a vantage point, i.e, a vertex , along shortest paths from to all other vertices, reveal bottleneck edge capacities along each path. Our goal is to select vantage points from that reveal the maximum number of bottleneck edge capacities.
We consider both a non-adaptive setting where all vantage points are selected before any bottleneck capacity is revealed, and an adaptive setting where each vantage point selection instantly reveals bottleneck capacities along all shortest paths starting from that point. In the non-adaptive setting, by considering a relaxed model where edge capacities are drawn from a random permutation (which still leaves the problem of maximizing the expected number of revealed edges NP-hard), we are able to give a approximate algorithm. In the adaptive setting we work with the least permissive model where edge capacities are arbitrarily fixed but unknown. We compare with the best solution for the particular input instance (i.e. by enumerating all choices of tuples), and provide both lower bounds on instance optimal approximation algorithms and upper bounds for trees and planar graphs.
Keywords and phrases:
Bottleneck capacity, Approximation algorithms, Instance optimalityFunding:
Rezaul Chowdhury: Partially supported by NSF (CCF-2318633).Copyright and License:
2012 ACM Subject Classification:
Networks Network algorithmsAcknowledgements:
We thank an anonymous reviewer who caught an error in a claimed result that has now been omitted from this paper; we discuss and pose the question as the first open problem in Section 4. We also thank Srinivas Narayana for discussions on capacity discovery.Editors:
Pat Morin and Eunjin OhSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
Network tomography [31, 9, 10] concerns the problem of inferring the internal topology and parameters of a network based on end-to-end measurements. For example, collecting IP, router, and provider-level network topologies has been an active research topic for more than 20 years. Long-term continuous measurement efforts such as CAIDA’s Ark infrastructure [21] provide important information useful to many longitudinal analyses and network events of interest. One of the basic measurement tools is TraceRoute, which obtains the sequence of router interface IP addresses along the forward path to a destination by sending probe packets with varying time to live (TTL) values and examining the Internet Control Message Protocol (ICMP) responses. By using the transmission timestamp of each probe, TraceRoute can report the round trip time from the source to each node on the forward path. Although TraceRoute was initially designed for network administrators to make diagnoses on a small scale, recent efforts applied TraceRoute for Internet scale topology probing [20, 6].
Beyond network topology, estimating network bottleneck capacity is a classical research topic [1, 16, 5, 17]. The capacity of a link (edge) is the highest bit rate possible to transmit over the edge. The capacity of a path is determined by the bottleneck edge, the one on the path with the lowest capacity [28]. Estimating the bottleneck or available capacity between a pair of vertices is useful for many applications, such as routing management, intrusion detection and improving performance of transport protocols. In particular, knowledge of network bandwidth can be instrumental to client-side optimization in real-time throughput sensitive applications, such as video conferencing. A list of existing measurement tools for available bandwidth estimation is summarized in [29]. Existing techniques mostly focus on end-to-end capacity, using tools such as variable packet size, packet pair/train dispersion, or periodic streams. Common to these different implementations is the central idea of examining the jumps in round trip time (RTT) from the source to each hop of a path. Since the bottleneck edge determines the available capacity of the entire path, the bottleneck capacity can be discovered by examining packet latency in probes.
The common practice in large scale measurements of the Internet issues probes from a narrow range of vantage points (VPs), typically cloud systems or research universities, due to cost, convenience, accessibility and scalability [22]. For example, RIPE Atlas [3], a platform that supports user specified measurement requests, has over active measurement vantage points. But a user is limited to the total number of vantage points used in a measurement. Therefore, a user would naturally seek to maximize the amount of information one could obtain by optimizing the choice of vantage points [19].
In this paper we formulate mathematically the problem of choosing vantage points for network probing. Our goal is to use probing tools for network-wide capacity discovery. We consider a network with publicly known topology, where each edge/link has an unknown capacity. We would like to select (possibly from a given set of vertices) a set of vantage points from which probing messages are sent out to other vertices in the network. We use a model of probing in which probing from a source along a path to a destination reveals the bottleneck capacity of the sub-path from to , for every intermediate vertex on the path. In this model, if the bottleneck capacity drops from to , in comparing paths and with being the immediate downstream vertex after , then the edge has capacity . Our objective is to reveal the link capacity for as many edges in the network as possible.
In this paper we assume that the network topology is known, e.g., learned using the relatively mature methods for topology discovery. We also work mainly with an assumption that all paths along the probings are unique shortest paths. Thus the set of shortest paths from one vertex to all other vertices in the network forms a shortest path tree. To examine the connection to reality, we first remark that modern Internet uses multi-path routing, where traffic to a destination can be spread on multiple paths for throughput and redundancy [4, 32]. On the other hand, the assumption that routes originating from a vantage point to the Internet form a tree-like structure is heavily exploited to reduce probing redundancy in modern TraceRoute tools, e.g., Doubletree [12] and its variants [11, 24]. Therefore it is reasonable to assume that using existing tools the routes probed starting from a vantage point indeed form a tree.
There has been a number of empirical work understanding vantage points on the Internet in terms of their characteristics and influence on data collected for network probing [8, 7, 25, 30]. None of the prior work is directly related to our problem formulation.
The problem of vantage point selection is not limited to the Internet domain. There are a number of other network scenarios where estimating bottleneck on the network is important. This problem is natural for transportation network for estimating traffic bottleneck although probing on the transportation network may have to rely on opportunistic inputs. In the traffic engineering literature, the traffic sensor location problem (TSLP) [27] is to determine how many sensors are required and where should they be deployed in order to best understand the traffic bottlenecks in road and transportation networks. For a blood vessel network, a medical procedure known as angiography considers probing of the blood vessel network of a patient by a dye injected into the bloodstream through a catheter, in order to determine the blocked vessel (the bottleneck). On an abstract level, determining the site for the injection(s) is similar to the bottleneck discovery problem in this paper.
1.1 Problem Definition
Here we formulate the problem of vantage point selection for bottleneck capacity discovery. We are given an undirected graph with each edge associated with an unknown positive capacity . The goal is to discover these unknown capacities by using bottleneck queries from vantage points , which reveals bottlenecks on the shortest path between and for all . The shortest path may be defined by the path of minimum number of hops or by another weight metric which is known (the only unknown is the capacities). Unless mentioned otherwise, we will assume (perhaps with slight perturbation of the edge weights) that all shortest paths are unique. The problem is then to select a set of vertices as vantage points such that when we issue queries from each vantage point, we can reveal a maximum number of the unknown edge capacity values.
We first discuss what edges are revealed from selecting as a set of vantage points.
Definition 1 (Bottleneck Edges).
Denote by the shortest path from to , and let be the edges along this path. We say that, for , is a bottleneck edge along if for all , and is a bottleneck capacity. The bottleneck edges from a set to are edges that are bottleneck along for some and some .
Note that while we state the definition of a bottleneck edge with respect to a path , in accord with the standard definition of a bottleneck edge, we extend the definition to a set to match our query model: for every , a single query comprises of subqueries finding the bottleneck edge in for all . For example, on the path graph where edges have monotone increasing capacities, querying results in the revelation of all capacities whereas querying only reveals the capacities of the last edge.
With this background, we can loosely define the problem of bottleneck discovery.
Problem 1 ().
Given a non-negatively weighted (weights ) undirected graph , and a positive integer , find a subset of vertices with such that the number of revealed bottleneck edges is maximized.
Non-Adaptive Setting.
In this setting, we work with the assumption that the ordering of the edge capacities is uniformly randomly chosen from a permutation of . Thus, each edge has equal chance of being the one with the smallest capacity among all edges in the network. Here we consider algorithms in a non-adaptive manner, where the vantage points are selected all at once and probes are issued afterwards. We seek to maximize the expected number of edges revealed.
Adaptive Setting.
In the adaptive setting, we work in the worst-case model where we assume the edge capacities to be arbitrarily fixed distinct real numbers, unknown to the algorithm. We also allow algorithms that run in an adaptive manner, where the vantage points are selected one by one, with probes from a vantage point issued immediately upon selection, before we select the next vantage point.
Here we aim to perform well on any specific input. In other words, we study the instance-optimal setting. Traditionally, instance optimality is studied by comparing the output of an algorithm to the optimal solution for the input instance. In this spirit, we assume an algorithm that is already aware of all the capacities, and for a specific input instance , let denote the maximum number of edges revealed by vantage points on instance , i.e., obtained by enumerating all subsets of vantage points and keeping the best choice. In contrast, a capacity-unaware algorithm will be called -instance optimal if after selecting at most vantage points (possibly in an adaptive manner) on the input instance , it can reveal at least many edge capacities. We can think of as the resource augmentation factor and as the approximation factor for the objective function.
We close this subsection with the following observations on the problem.
Observation 2.
Let be the shortest path tree rooted at a vantage point .
-
The capacities of all edges in that are incident to are revealed.
-
In any root to leaf path from to on , the capacities of the edges revealed form a record setting decreasing subsequence, where the capacity of the th revealed edge is strictly smaller than all edges earlier on the path (and of course smaller than that of the st revealed edge).
-
If is on the shortest path from to , all edges that can reveal along the path from to can instead be revealed if is selected as a vantage point.
1.2 Overview of Results
Non-Adaptive Setting (with Stochastic Capacities).
We start with the nonadaptive setting with stochastic capacity assumption, where vantage points are selected before any information on the bottleneck capacities is revealed. Here, the ordering of the capacities is assumed to be a random permutation of , and we look for an algorithm that maximizes the expected performance. This problem is NP-hard, since the vertex cover problem is a special case. We therefore aim for approximation algorithms and compare with the optimal algorithm which works under the same assumption.
A first observation is that the expected number of revealed edges obtained using a subset of vantage points is a monotone and submodular function with respect to . That is, adding a new vantage point is always beneficial and adding to a set rather than , with , has a diminishing return. This observation allows us to use a greedy algorithm to select the next vantage point that maximizes the expected marginal increase of the objective function, which yields a -approximation algorithm, by known results for maximizing coverage for submodular functions [26]. To obtain our approximation result, then, we must address the non-trivial calculation of the expected marginal increase of adding a vantage point to the set being constructed greedily. A crucial step here is to efficiently count the number of capacity assignments to the graph such that an edge is revealed as the result of a new vantage point. We show that the algorithm can be implemented in time , with as the number of vertices, as the number of edges and as the total number of vantage points selected.
We remark that one can also resort to a sampling-based approach (e.g., see Proposition 3 in [23]), which does not provide better running time in general and can only provide an approximate solution which leads to a approximation factor. Our deterministic algorithm provides more insight into the structure of the problem.
Adaptive Setting (with Worst-Case Capacities).
In the adaptive setting, we prove the following results. First, we show interesting lower bounds for the tradeoffs of parameters and for -instance optimal algorithms, where selects vantage points (with the full knowledge of capacities). One trivial algorithm is to select all vertices as vantage points, as opposed to the vertices selected by , obtaining and . We thus ask if we can lower with respect to the other parameters. Our lower bounds suggest that cannot be too small. For deterministic adaptive algorithms, we show that ; for randomized adaptive ones, . These lower bounds apply for algorithms that can perform any finite computation between the selection of the vintage points.
We then show that both bounds are tight when the underlying graph is a tree, i.e., we give vantage point selection algorithm achieving the same tradeoffs. For planar graphs, we give a deterministic algorithm that achieves a tradeoff of . Finally, while we leave open the question of existence of an algorithm for general graphs, we show that if the shortest path trees are not unique, and each shortest path tree may break ties independent from others, there is a stronger lower bound of , for all .
We present the algorithmic results in the main body of the paper. We postpone the hardness proof for both settings and some proof details to the Appendix.
2 in the Non-Adaptive Setting (with Stochastic Capacities)
In this section we show a constant approximation to in the non-adaptive setting with stochastic capacities, which is NP-hard (Appendix A).
Theorem 3.
There is an algorithm that gives a approximation to in time , where is the base of the natural logarithm.
Define to be the set of edges that are revealed with vertices selected, and let . We prove Theorem 3 in two parts. First, we show that is monotone submodular, and hence a greedy algorithm for selection into yields a approximation by the result of [26]. This is the less interesting part and so the details are defered to Section B.1. We then design a value oracle for each iteration of the greedy algorithm, which is implemented by using polynomial multiplications.
Setup.
Recall that is the set of edges whose capacities are revealed from selecting vantage points , and we would like to select vertex to maximize . By linearity of expectation, . It is thus enough to show how to compute for an arbitrary subset , after which we can sum over all the edges at the expense of a multiplicative factor blowup in the running time.
(Overview) Computing .
For the remainder of this section, let us fix an edge for which we want to compute , and let us also have fixed. First, we show what vertices contribute to .
Definition 4 ((Non-)essential vantage points).
A vantage point is said to be non-essential if at least one of the following holds:
-
does not go through and does not go through .
-
If goes through , there is some where .
-
If goes through , there is some where .
If none of the above hold, we say that is essential.
It is reasonably clear that non-essential vantage points do not have any bearing on the value of . We now state a few observations with proofs in Appendix B.
Observation 5.
Let be the set of all essential vantage points. Then .
Now we define the minimal subgraph required to compute .
Definition 6 (, the tree rooted at ).
Let be the set of all essential vantage points. We define with (edge-)root (see Figure 1(a)).
Observation 7.
is a tree, with all its leaves belonging to . Moreover, can be computed in time.
Observation 8.
is a bottleneck edge for if and only if is the smallest capacity on some (edge-)root to (vertex-)leaf path on .
Put otherwise, using Observation 8, we have reduced the problem of computing to that of counting labellings of the edges of with such that no two edges receive the same label, and has the smallest label on some root to leaf path of (recall that is an edge-rooted tree). These labellings can be thought of as the order statistic of edge capacities in .
Counting Labellings.
For convenience, we work on the “edge-tree” rooted at
which allows us to think in the parlance of labelling vertices instead of edges (see Figure 1(b)). One can see that counting vertex-labellings on such that there is some root to leaf path where the label of is smallest is the same as counting similarly constrained edge-labellings on . This leaves us with the following problem.
Problem 2 ().
Given a rooted tree with root , how many bijective labellings of with result in the label of being the minimum label of some root-to-leaf path?
To solve , however, we first consider a more constrained auxiliary problem.
Problem 3 ().
Given a rooted tree with root , compute, for each , how many black-white colorings of there are such that:
-
Exactly vertices are colored black;
-
There is no root-to-leaf path comprising of only vertices colored white.
Conceptually, we can think of vertices colored black as those whose labels are smaller than the root’s, and vertices colored white as those whose labels are no smaller than the root’s. A solution to counts bad events, since, under this viewpoint we want some root-to-leaf path with all vertices colored white.
Observation 9.
Let be a solution to on . Then
gives a solution to .
Proof.
There are colorings such that exactly vertices are colored black and there is some white root-to-leaf path. For each such coloring, there are exactly labellings of the vertices such that white vertices have larger labels than the root’s label; black vertices have smaller labels than the root’s label; and there is some root-to-leaf path where the root has the smallest label. The result then follows from adding up good labellings where exactly vertices have a smaller label than that of the root.
We now supply a polynomial time algorithm for , which returns a polynomial .
Claim 10.
CountBlackWhiteColorings returns a polynomial such that the coefficient of is the number of colorings of there are such that exactly vertices are colored black and there is no white root-to-leaf path.
Proof.
The claim is certainly true for trees with one vertex. More generally, consider any vertex . If is colored black, we can color its descendants in any way and there would be no white root-to-leaf path. The coefficient of in counts the number of ways to color vertices black, with being colored black. If, on the other hand, is colored white, there must be no root-to-leaf path in all of the subtrees rooted at all of its children. We can recursively compute the polynomials with respect to each of ’s children , and multiply them to get the polynomial which counts colorings in this case.
Claim 11.
CountBlackWhiteColorings runs in time .
Proof.
There are polynomial multiplications involved in the total computation, where each polynomial has degree at most . Using [18] to bound the running time of polynomial multiplication completes the proof.
Armed with CountBlackWhiteColorings, we may now proceed to state complete the algorithm CountGoodLabellings. By Observation 9 and Claim 10, CountGoodLabellings solves and has running time .
Back to Computing .
3 Adaptive Setting (with Worst-Case Capacities)
We use notation developed in Section 1.1. We want to select vantage points from in a given graph with unknown, but fixed, worst-case capacities . Recall that an algorithm is called -instance optimal if it selects at most vantage points and reveals at least many bottleneck edges, where is the number of edges revealed (after selecting vantage points) by an algorithm that is also given all the capacities as input. Here, and .
3.1 Lower bounds
In the following lower bounds, we do not limit the computation time of an algorithm – it can be adaptive, and can do any finite computation between rounds.
Theorem 12 (Deterministic Algorithm Tradeoff).
For any , there exists a graph for which any deterministic, adaptive, -instance optimal algorithm must satisfy
Proof.
We will assume divides for simplicity. Let be the graph with connected components , , where each is a path on vertices. Let be a deterministic and adaptive algorithm. Consider the following adversarial strategy. For any , whenever selects a vantage point that is the first vantage point on , the adversary reveals only the two neighboring edges on , giving them the lowest two capacities out of the edges in . At any point in the execution of , let be the vantage points selected on so far, and assume selects a new vantage point on . If is not a neighbor of for any , the adversary reveals the two edges incident to , giving them the next lowest capacities of all the edges revealed in so far. Otherwise, the adversary reveals either one or zero edges (depending on whether one or both neighbors of was already selected). We observe that the adversary’s strategy is consistent: for any , all edges yet to be revealed have a higher capacity than those revealed.
Let be the number of vantage points selected on at the end of . We have that , where is the resource augmentation factor. The adversary now reveals the remaining edges in the following way. Fix a path . If , the adversary gives all edges in decreasing capacities from the first vertex in (and therefore selecting that vertex would have revealed all edges). Otherwise, if , there is a contiguous set of edges such that none of the endpoints have been selected by as vantage points. The adversary gives them decreasing weights, starting from, say, the first vertex in this set of edges. Selecting this first vertex would have revealed not only the edges with decreasing capacities, but also the two “outer edges”, since their capacities are lower. Thus selecting this vertex reveals edge capacities. We now have that .
Clearly reveals at most edges, at most two per vantage point selected. Thus
The second equation implies that there exists a set such that and for all . We will use to lower bound the sum in the first inequality. We have
and thus , proving the theorem.
Remark.
Note that the above theorem is vacuous when , as is always at least 1. We leave open the existence of a lower bound that implies when . As we will see, the trade off in Theorem 12 is tight for trees, so such a lower bound (if it exists) would involve an instance with cycles.
Next, we consider a randomized algorithm that selects at most vantage points. Given an instance , different runs of may reveal different numbers of edge capacities. We define for a randomized algorithm to be OPT divided by the expected number of edge capacities revealed by over all runs of on (expectation over random coins of ). The proof of the following theorem can be found in Appendix C.
Theorem 13 (Randomized Algorithm Tradeoff).
For any , there exists a graph for which any randomized, adaptive, -instance optimal algorithm must satisfy .
Remark.
-
1.
As with Theorem 12, note that Theorem 13 is vacuous when , as is always at least 1. We leave open the existence of a lower bound that implies when . As we will see, the tradeoff in Theorem 13 is tight (upto the log factor) for trees, so such a lower bound (if it exists) would involve an instance with cycles.
-
2.
A comparison of the tradeoffs in Theorems 12 and 13 reveals that the tradeoff in Theorem 12 is larger than that in Theorem 13 as long which by the above remark is the setting for Theorem 13. For example, when , the first tradeoff gives whereas the second tradeoff gives . This indicates that randomization may help. Indeed, it does, as we show next.
3.2 Upper bounds
For general graphs, observe that there is always an algorithm that achieves the tradeoff of , simply because (each vantage point reveals at most edge capacities in its shortest path tree), and the trivial algorithm that selects any set of vantage points reveals many capacities. Obviously, is much worse than the tradeoffs in Theorems 12 and 13. We first show that Theorems 12 and 13 are tight for trees, thus implying that strengthening Theorems 12 and 13 will require instances with cycles.
Theorem 14 (Tight tradeoffs for trees).
Let be a weighted tree on vertices with fixed, unknown capacities on edges.
-
1.
For any , there exists a deterministic algorithm that for any given , selects vantage points in such that is -instance optimal achieving the exact tradeoff in Theorem 12, .
-
2.
For any , there exists a randomized algorithm that for any given , selects vantage points in such that is -instance optimal with . This tradeoff is only a factor worse than that in Theorem 13.
Proof.
Both claims will require the notion of a “cover” , which we describe first. We take the tree rooted at an arbitrary vertex . Take the lowest vertex with at least descendants with inclusive (i.e., any child of has fewer than descendants). We put in set . We remove and its descendants and repeat until we are left with at most vertices. Clearly there are at most vertices in . We call this set a cover of the tree. Sometimes we will need to vary the number of descendants: we will denote by the cover of size at most , generated in the same way as above starting with the lowest vertex with at least descendants. The covers satisfy the following lemma.
Lemma 15.
For any vertex taken as the root of the tree, all but vertices are descendants of vertices in .
Proof.
satisfies the requirement for the root chosen during the construction of by design. Now suppose we have a different root . We argue that again covers all but vertices in the descendants of for rooted at . Consider the tree rooted at . We run a case study.
If is not a descendant of any vertex in , then all descendants of in remain to be descendants of in tree . Thus we are done.
If is a descendant of a vertex in . Wlog we take as the lowest such ancestor of . We take the child of who is the ancestor of (or itself). All vertices that are not in the subtree of are now descendants of in . Thus they are “covered”. Now we focus on the vertices in the subtree of in . Among these vertices, those that are descendants of some other vertex of in this subtree will continue to be descendant of in – since is not in the subtree with root . Thus the only vertices left would be those that are not in subtree of other vertices in . There can only be at most vertices – otherwise would have been selected to in the computation of .
Proof of Theorem 14 Claim 1.
The algorithm selects a cover, with any as the root in the above cover-selection procedure and . Thus the size of the cover is at most . If the size of the cover is strictly smaller than , we select some arbitrary vantage points to make the total number of selected vantage points equal to .
Let be the total number of edge capacities revealed by . Consider a vertex chosen by the OPT solution. By Lemma 15, in the tree rooted at , all but vertices of are descendants of . Thus all the edges that are revealed by in the OPT solution, except those that are not descendants of , will be revealed as well by our algorithm . Since OPT chooses vantage points, algorithm would only miss at most edges and thus reveal at least edges.
Now we do a case analysis. If , . Thus . , since . On the other hand, if , algorithm selects vantage points, just the edges incident to these points reveal at least many edge capacities (in the worst case, these vertices form a matching). Therefore This immediately gives . Thus the exact tradeoff in Theorem 12 is achieved, and this finishes the proof of Claim 1.
Proof of Theorem 14 Claim 2.
The randomized algorithm is very simple: it selects a set of vantage points uniformly at random (without repetition) from . The proof is similar to the deterministic case. Let be the number of edge capacities revealed in . Let us consider the expected number of edges covered by the vantage points. Note that has vertices; if all selected as vantage points, would reveal at least many edges revealed by OPT. By linearity of expectation the expected number of edges that are revealed by a random choice of vantage points from is at least Again we consider two cases. If , . Thus , or, equivalently, . If , our algorithm reveals at least edges since we select vantage points. Thus . Or, . This finishes the proof.
We remark that all algorithms are non-adaptive with polynomial run time. This is slightly surprising, since the lower bounds were on algorithms not limited by computation. We conjecture that while for trees such non-adaptive algorithms suffice, adaptivity may be needed to achieve the tradeoffs (if it is possible at all) for general graphs.
Planar Graphs.
We observed in the beginning of this section that for general graphs, the trivial algorithm achieves , and if this is tight, a matching lower bound would involve instances with cycles. However, we show next that such an instance cannot be planar. We leave as an interesting open question whether one can design a deterministic algorithm for general graph with .
Theorem 16.
Given a weighted planar graph with fixed unknown capacities, and a , there exists a deterministic -instance optimal algorithm with and , and hence .
3.3 Stronger lower bound on general graph: Multiple shortest paths
In all of the discussion so far, we have assumed that shortest paths are unique. If there are ties and each shortest path tree may break ties independently of other trees, we can have a stronger lower bound of when , for any , between the instance optimal solution and the best solution by any (possibly randomized, adaptive) algorithm.
We first consider a grid graph of vertices. We wrap it around to be a torus – the top boundary is identified as the bottom boundary and the left boundary is identified as the right boundary. The shortest path trees at all vertices, topologically, are identical – each tree includes the horizontal edges on the row containing the root , and for each vertex with the same -coordinate with , there are two chains of length each. See Figure 2 (left).
Consider one specific vertex (in blue) and the shortest path tree rooted at . We take the largest capacity values and assign them on the tree rooted at , sorted in decreasing capacities in a round robin manner moving away from . Specifically, the immediate neighbors of are given the highest capacities, those two hops away from in the tree are given the next highest capacities, and so on. This way, all edges in the tree rooted at (called ) can be revealed if the optimal solution chooses . For all other edges we assign the remaining capacity values randomly.
Now consider another vertex . If does not have the same -coordinate as (not on the same row as ), the shortest path rooted at has all edges on the same row as (shown as edges in red in Figure 2 (left)), which have lower capacity than the edges on tree . Therefore, all edges other than the vertical edges on the same column as cannot be revealed. Thus, reveals at most edges. If stays on the same row of , may be able to reveal up to order edges but there are only such vertices. Since all shortest paths look exactly the same topologically, any algorithm trying to guess the optimal tree reveals in expectation edges111Specifically, suppose an algorithm chooses a vertex with probability . An adversary would place the vertex on the row with the minimum total probability (which is no greater than ) among all possible rows..
This example can be generalized to a -dimensional cube of vertices in each dimension. See Figure 2 (right) for an example for . The tree rooted at a vertex takes all edges with the same coordinate of as . For each vertex of the same coordinate as , take the chain of vertices that share the same coordinate as , and so on. Use the same allocation of capacities, the optimal tree can reveal edges while any algorithm without the knowledge of capacity distribution reveals edges. This leaves a gap of , for any , between the instance optimal solution and the best oblivious algorithm.
4 Open Problems
We conclude the paper with a couple of open problems.
The Adaptive Setting (with Stochastic Capacities).
There is an intermediate model that is not addressed in this paper, which is the adaptive setting with stochastic capacities, where the ordering of capacities are a random permutation. There are various notions of stochastic submodularity (e.g. [2, 14]) for adaptive algorithms that admit good approximations via myopic algorithms. Unfortunately, they do not seem to hold for this problem.
Adaptive Setting with Worst-Case Capacities.
A notable open problem is to find algorithms that achieve the lower bounds (Theorems 12 and 13) for a general graph. One special parameter range is when , i.e., there are vertices that, if chosen as vantage points, can reveal number of edges. The lower bounds for both deterministic and randomized algorithms suggest that . Now, can we find vertices as vantage points that reveal at least edge capacities? Our results show that for a tree graph our algorithm with vantage points suffice, and for a planar graph our algorithm with vantage points suffice. We do not know any non-trivial bound of (and ) for a general graph nor a lower bound example that requires vantage points.
References
- [1] Mark Allman and Vern Paxson. On estimating end-to-end network path properties. SIGCOMM Comput. Commun. Rev., 29(4):263–274, August 1999. doi:10.1145/316188.316230.
- [2] Arash Asadpour, Hamid Nazerzadeh, and Amin Saberi. Stochastic submodular maximization. In Internet and Network Economics: 4th International Workshop, WINE 2008, Shanghai, China, December 17-20, 2008. Proceedings 4, pages 477–489. Springer, 2008. doi:10.1007/978-3-540-92185-1_53.
- [3] RIPE Atlas. User-defined measurements. https://atlas.ripe.net/docs/getting-started/user-defined-measurements.html.
- [4] Brice Augustin, Timur Friedman, and Renata Teixeira. Multipath tracing with paris traceroute. In 2007 Workshop on End-to-End Monitoring Techniques and Services, pages 1–8, May 2007. doi:10.1109/E2EMON.2007.375313.
- [5] S Banerjee and A K Agrawala. Estimating available capacity of a network connection. In Proceedings IEEE International Conference on Networks 2000 (ICON 2000). Networking Trends and Challenges in the New Millennium, pages 131–138, September 2000.
- [6] Robert Beverly. Yarrp’ing the Internet: Randomized High-Speed active topology discovery. In Proceedings of the 2016 Internet Measurement Conference, IMC ’16, pages 413–420, New York, NY, USA, November 2016. Association for Computing Machinery. URL: http://dl.acm.org/citation.cfm?id=2987479.
- [7] Jan Böttger. Understanding benefits of different vantage points in today’s Internet. PhD thesis, Technische Universität Berlin, 2017.
- [8] Valentin Burger, Matthias Hirth, Christian Schwartz, Tobias Hoßfeld, and Phuoc Tran-Gia. Increasing the coverage of vantage points in distributed active network measurements by crowdsourcing. In Measurement, Modelling, and Evaluation of Computing Systems and Dependability and Fault Tolerance, pages 151–161. Springer International Publishing, 2014. doi:10.1007/978-3-319-05359-2_11.
- [9] Rui Castro, Mark Coates, Gang Liang, Robert Nowak, and Bin Yu. Network tomography: Recent developments. SSO Schweiz. Monatsschr. Zahnheilkd., 19(3):499–517, August 2004.
- [10] A Coates, A O Hero, III, R Nowak, and Bin Yu. Internet tomography. IEEE Signal Process. Mag., 19(3):47–65, May 2002. doi:10.1109/79.998081.
- [11] Benoit Donnet, Bradley Huffaker, Timur Friedman, and K C Claffy. Increasing the coverage of a cooperative internet topology discovery algorithm. In NETWORKING 2007. Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet, pages 738–748. Springer Berlin Heidelberg, 2007.
- [12] Benoit Donnet, Philippe Raoult, Timur Friedman, and Mark Crovella. Efficient algorithms for large-scale topology discovery. In Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, SIGMETRICS ’05, pages 327–338, New York, NY, USA, June 2005. Association for Computing Machinery. doi:10.1145/1064212.1064256.
- [13] Greg N Federickson. Fast algorithms for shortest paths in planar graphs, with applications. SIAM J. Comput., 16(6):1004–1022, December 1987. doi:10.1137/0216064.
- [14] Daniel Golovin and Andreas Krause. Adaptive submodularity: Theory and applications in active learning and stochastic optimization. Journal of Artificial Intelligence Research, 42:427–486, 2011. doi:10.1613/JAIR.3278.
- [15] Michael T Goodrich. Planar separators and parallel polygon triangulation (preliminary version). In Proceedings of the twenty-fourth annual ACM symposium on Theory of Computing, STOC ’92, pages 507–516, New York, NY, USA, July 1992. Association for Computing Machinery. doi:10.1145/129712.129762.
- [16] Cesar D Guerrero and Miguel A Labrador. On the applicability of available bandwidth estimation techniques and tools. Comput. Commun., 33(1):11–22, January 2010. doi:10.1016/J.COMCOM.2009.08.010.
- [17] K Harfoush, A Bestavros, and J Byers. Measuring bottleneck bandwidth of targeted path segments. In IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428), volume 3, pages 2079–2089 vol.3, March 2003.
- [18] David Harvey and Joris Van Der Hoeven. Integer multiplication in time . Annals of Mathematics, 193(2):563–617, 2021.
- [19] Thomas Holterbach, Emile Aben, Cristel Pelsser, Randy Bush, and Laurent Vanbever. Measurement vantage point selection using a similarity metric. In Proceedings of the Applied Networking Research Workshop, ANRW ’17, pages 1–3, New York, NY, USA, July 2017. Association for Computing Machinery. doi:10.1145/3106328.3106334.
- [20] Yuchen Huang, Michael Rabinovich, and Rami Al-Dalky. FlashRoute: Efficient traceroute on a massive scale. In Proceedings of the ACM Internet Measurement Conference, IMC ’20, pages 443–455, New York, NY, USA, October 2020. Association for Computing Machinery. doi:10.1145/3419394.3423619.
- [21] Y. Hyun and K. Claffy. Archipelago measurement infrastructure. http://www.caida.org/projects/ark/, 2015.
- [22] Jordan Jueckstock, Shaown Sarker, Peter Snyder, Panagiotis Papadopoulos, Matteo Varvello, Benjamin Livshits, and Alexandros Kapravelos. The blind men and the internet: Multi-Vantage point web measurements, May 2019. arXiv:1905.08767.
- [23] Mohammad Karimi, Mario Lucic, Hamed Hassani, and Andreas Krause. Stochastic submodular maximization: The case of coverage functions. Advances in Neural Information Processing Systems, 30, 2017.
- [24] Bo Li, Jingsha He, and Henghua Shi. Improving the efficiency of network topology discovery. In 2008 The 3rd International Conference on Grid and Pervasive Computing - Workshops, pages 189–194, May 2008. doi:10.1109/GPC.WORKSHOPS.2008.34.
- [25] Franziska Lichtblau. From the edge to the core : towards informed vantage point selection for Internet measurement studies, 2021.
- [26] G L Nemhauser, M L Fisher, and L A Wolsey. An Analysis of Approximations for Maximizing Submodular Set Functions. Center for Operations Research & Econometrics, 1976.
- [27] Mahmoud Owais. Traffic sensor location problem: Three decades of research. Expert Systems with Applications, 208:118134, 2022. doi:10.1016/j.eswa.2022.118134.
- [28] R Prasad, C Dovrolis, M Murray, and K Claffy. Bandwidth estimation: metrics, measurement techniques, and tools. IEEE Netw., 17(6):27–35, November 2003. doi:10.1109/MNET.2003.1248658.
- [29] Dixon Salcedo, Cesar D. Guerrero, and Roberto Martinez. Available bandwidth estimation tools metrics, approaches and performance. International J. Commun. Netw. Inf. Secur., December 2018.
- [30] Yuval Shavitt and Udi Weinsberg. Quantifying the importance of vantage point distribution in internet topology mapping (extended version). IEEE Journal on Selected Areas in Communications, 29(9):1837–1847, 2011. doi:10.1109/JSAC.2011.111008.
- [31] Y Vardi. Network tomography: Estimating Source-Destination traffic intensities from link data. J. Am. Stat. Assoc., 91(433):365–377, March 1996.
- [32] Kévin Vermeulen, Justin P Rohrer, Robert Beverly, Olivier Fourmaux, and Timur Friedman. Diamond-miner: Comprehensive discovery of the internet’s topology diamonds. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), pages 479–493, 2020.
- [33] Andrew Chi-Chin Yao. Probabilistic computations: Toward a unified measure of complexity. In 18th Annual Symposium on Foundations of Computer Science (SFCS 1977), pages 222–227, 1977. doi:10.1109/SFCS.1977.24.
Appendix A Proof of hardness
Theorem 17.
The decision version of is -hard.
Proof.
We give a reduction from to . Let be an instance of . We transform it to the following instance of : where the weights are of the form for edge so as to ensure unique shortest paths. We claim that has a vertex cover of size at most if, and only if, there exists a set of at most vantage points that reveal edges in expectation (and hence edges no matter what the capacities are).
- ()
-
If does indeed have a vertex cover of size , then setting reveals every edge as a bottleneck (since edges incident to any are assured to be revealed) no matter what the edge capacities are.
- ()
-
If, conversely, has no vertex cover of size , then consider any of size . Since is not a vertex cover, there is at least one edge whose endpoints are not in . With probability at least , is the highest capacity among all the edges, and cannot be revealed unless one of its endpoints was selected. The expected number of revealed bottlenecks for is thus strictly less than .
Next, we prove hardness for the adaptive setting with worst-case capacities.
Theorem 18.
There does not exist a deterministic adaptive polynomial time algorithm satisfying for all graphs and all capacities , unless .
Proof.
In the following, we show that if such an exists, then we can solve vertex-cover in polynomial time. Assume we are given a connected graph on vertices and edges, and a number , and asked if has a vertex cover of size exactly .
The idea is to run on , starting from unknown capacities, but reveal the capacities adaptively to in such a fashion that is left with no choice but to solve the decision version of vertex cover. Let the current set of vantage points selected by be . Say in the next round selects a vantage point . For all the edges between and a vertex not in , we assign them the next highest many capacities (from some arbitrarily selected alphabet) arbitrarily. This results in a consistent set of capacities: any edge capacities not yet revealed must be higher than those already revealed, and our strategy satisfies this invariant.
At the end of the execution of , we check if the set of vantage points selected by is a vertex cover. If so, we answer YES to the decision version of vertex-cover; otherwise, we answer NO.
Let be the size of a minimum vertex cover in . If , then for any permutation of the capacities, in particular the capacities arrived at the end of ’s execution, . Therefore, in this case, also, and that the set of vantage points selected must be a vertex cover, for if an edge is not covered by , then its capacity is larger than all revealed edges, and could not have been revealed to .
On the other hand if , then the set of vantage points selected by cannot be a vertex-cover by definition of , and so we answer correctly. Hence, the theorem is proved.
Appendix B Omitted Proofs: in the Stochastic, Non-Adaptive Setting
B.1 Submodularity of
Lemma 19.
is monotone and submodular as a function of :
-
1.
Monotonicity: ;
-
2.
Submodularity: If , then .
Proof.
We prove each claim separately.
-
1.
Let (resp. ) be the indicator for being revealed from selecting (resp. ). Then, by linearity, . It is clear that , and so is monotone.
-
2.
Let (resp. ) be the indicator for being revealed from selecting (resp. ). Then
(Linearity of expectation) (Law of total probability) and so is submodular.
By the results on maximizing submodular functions [26], incrementally building up a set by adding the vantage point maximizing until yields a approximation algorithm.
Observation 5. [Restated, see original statement.]
Let be the set of all essential vantage points. Then .
Proof.
In short, this follows from Observation 2.
Let .
Querying can only reveal bottleneck edges in , the shortest path tree rooted at . Consequently, if does not go through and does not go through , then .
On the other hand, note that at most one of goes through and goes through . Without loss of generality, say goes through . Since , there must be a vertex where such that . The event that querying reveals as a bottleneck is a subset of the event that querying reveals hence .
Finally, observe that a vertex being non-essential only depends on and , and since was chosen arbitrarily, .
Observation 7. [Restated, see original statement.]
is a tree, with all its leaves belonging to . Moreover, can be computed in time.
Proof.
being a tree follows immediately since, in our model, shortest paths are unique. That all its leaves are in follows from Definition 4, the definition of essential vantage points.
To compute , we need only find (resp. ), the shortest path tree rooted at (resp. ), which we can compute in time using Dijkstra’s algorithm. We can then find by running a breadth first search in and , truncating at
-
Subtrees which do not have as an ancestor;
-
Vertices ;
and finally iteratively removing leaves that are not in . Since , this produces . The running time follows.
Observation 8. [Restated, see original statement.]
is a bottleneck edge for if and only if is the smallest capacity on some (edge-)root to (vertex-)leaf path on .
Proof.
This follows, in short, from Observation 2.
-
Suppose is a bottleneck edge for . Let be the vantage point that is closest to , for which is a bottleneck edge on, say, . Since is the closest vantage point, it is essential and so . In particular, is a leaf with being the smallest capacity on .
-
The converse is proved similarly. Suppose for a leaf , the capacity is smallest in say . Then is a bottleneck edge of .
Proof of Theorem 3.
The correctness and approximation factor of GreedyNonAdaptive have been established earlier. For running time, GreedyNonAdaptive takes rounds, and computing each step takes inner-iterations, with each iteration computing for some and . Each such computation takes time to compute (Observation 7) and time to run CountGoodLabellings.
Appendix C Omitted Proofs: Adaptive setting with worst-case capacities
C.1 Lower Bound Proofs in Section 3.1
Proof of Theorem 13.
We will first invoke Yao’s principle [33], and try to lower bound the performance of a deterministic algorithm on the following distribution of inputs. In fact, our distribution inputs will all have the same underlying graph, but different set of capacities.
We will assume divides for simplicity. Let be the disconnected graph with components , , where each is a path on vertices. Consider assigning capacities in the following randomized procedure. First, select a random set , . For any , assign the edges in decreasing capacities, starting from the leftmost vertex. We will call these the “good” paths. For any , assign the edges in a random permutation of integers in the range . We will call these the “bad” paths. The optimal adaptive algorithm will select the left endpoints of all good paths and thus reveal roughly edges.
Our hard distribution will be the uniform distribution over all capacities obtained following the above procedure on . We now upper bound the expected number of edges revealed by any deterministic algorithm over the distribution of inputs . Recall that is allowed to select many vantage points. First, observe that if , the theorem is trivially true since . So we will assume , which means that , the total number of paths. Thus, does not have enough vantage points to cover all paths.
We can assume that does not select two vantage points on the same bad path, as this will only boost the chance of hitting the good paths. To argue this assumption, first, we boost by giving it for free all edge capacities if it selects any (not necessarily the first) vantage point on a good path, so there is never a need to select another vantage point on a good path. If selects a vantage point on a bad path, in expectation it reveals many edge capacities. This is because on a random permutation of length there is a decreasing sequence of record-setting edge capacities, in expectation. This follows from Lemma 2: the probability that the th edge has a capacity lower than the previous edges is at most , and the expected number of edges revealed can be shown to be upper bounded by . does not have enough vantage points to cover all paths, and therefore it is always optimal to not place another vantage point on a bad path, and instead place it on an uncovered path.
We have that the expected number of good paths revealed by is , and thus the expected number of good edges revealed is .
Now observe that the expected number of bad edges revealed is , at most per vantage point on bad path. Thus the expected number of total edges revealed is still , whereas . Thus , proving the theorem.
C.2 Upper Bound Proofs in Section 3.2
Proof of Theorem 16.
We use the -division of a planar graph [15, 13]. Specifically, an -division is a subdivision of into pieces satisfying three conditions: Each piece has vertices; Each piece has boundary vertices (i.e., vertices shared with other pieces); Each piece has holes (faces of the piece that are not faces of . Such a -division can be computed in time [15].
Suppose the optimal solution chooses vertices , our algorithm basically chooses all vertices on the boundary of all pieces in an -division, for a value to be decided later. There are boundary vertices. .
For any vertex in a piece without any vantage point from the optimal solution , the shortest path from any vantage point (which is outside the piece) to has to go through one of the boundary vertices of the piece , say . Thus, all edges that are revealed along the path , which is a subpath of , are also revealed by our algorithm (by in particular). Therefore the only edges our algorithm may have missed would be the edges of the shortest path tree rooted at that stay inside , for each . There are at most such edges for each vantage point and a total of such edges for all vertices in .
Suppose that the optimal solution reveals edges. We will reveal at least edges, for some constant . If , . Thus and . If , we reveal at least edges since we select vertices. Thus . Or, . Now we take to balance the two terms. This gives an upper bound for .
