Online Multistage Subset Maximization Problems

Numerous combinatorial optimization problems (knapsack, maximum-weight matching, etc.) can be expressed as \emph{subset maximization problems}: One is given a ground set $N=\{1,\dots,n\}$, a collection $\mathcal{F}\subseteq 2^N$ of subsets thereof such that $\emptyset\in\mathcal{F}$, and an objective (profit) function $p:\mathcal{F}\rightarrow\mathbb{R}_+$. The task is to choose a set $S\in\mathcal{F}$ that maximizes $p(S)$. We consider the \emph{multistage} version (Eisenstat et al., Gupta et al., both ICALP 2014) of such problems: The profit function $p_t$ (and possibly the set of feasible solutions $\mathcal{F}_t$) may change over time. Since in many applications changing the solution is costly, the task becomes to find a sequence of solutions that optimizes the trade-off between good per-time solutions and stable solutions taking into account an additional similarity bonus. As similarity measure for two consecutive solutions, we consider either the size of the intersection of the two solutions or the difference of $n$ and the Hamming distance between the two characteristic vectors. We study multistage subset maximization problems in the \emph{online} setting, that is, $p_t$ (along with possibly $\mathcal{F}_t$) only arrive one by one and, upon such an arrival, the online algorithm has to output the corresponding solution without knowledge of the future. We develop general techniques for online multistage subset maximization and thereby characterize those models (given by the type of data evolution and the type of similarity measure) that admit a constant-competitive online algorithm. When no constant competitive ratio is possible, we employ lookahead to circumvent this issue. When a constant competitive ratio is possible, we provide almost matching lower and upper bounds on the best achievable one.


Introduction
In a classical combinatorial optimization setting, given an instance of a problem one needs to find a good feasible solution. However, in many situations, the data may evolve over the time and one has to solve a sequence of instances. The natural approach of solving every instance independently may induce a significant transition cost, for instance for moving a system from one state to another. This cost may represent e.g. the cost of turning on/off the servers in a data center [16,7,4,1], the cost of changing the quality level in video streaming [15], or the cost for turning on/off nuclear plants [23]. Gupta et al. [14] and Eisenstat et al. [12] proposed a multistage model where given a time horizon t = 1, 2, . . . , T , the input is a sequence of instances I 1 , I 2 , . . . , I T , (one for each time step), and the goal is to find a sequence of solutions S 1 , S 2 , . . . , S T (one for each time step) reaching a tradeoff between the quality of the solutions in each time step and the stability/similarity of the solutions in consecutive time steps. The addition of the transition cost makes some classic combinatorial optimization problems much harder. This is the case for instance for the minimum weighted perfect matching problem in the off-line case where the whole sequence of instances is known in advance. While the one-step problem is polynomially-time solvable, the multistage problem becomes hard to approximate even for bipartite graphs and for only two time steps [5,14].
In this work, we focus on the on-line case, where at time t no knowledge is available for instances at times t + 1, . . . , T . When it is not possible to handle the on-line case, we turn our attention to the k-lookahead case, where at time t the instances at times t + 1, . . . , t + k are also known. This case is of interest since in some applications like in dynamic capacity planning in data centers, the forecasts of future demands may be very helpful [17,18]. Our goal is to measure the impact of the lack of knowledge of the future on the quality and the stability of the returned solutions. Indeed, our algorithms are limited in their knowledge of the sequence of instances. Given that the number of time steps is given, we compute the competitive ratio of the algorithm after time step T : As we focus on maximization problems, we say that an algorithm is (strictly) α-competitive (with competitive ratio α) if its value is at least 1 α times the optimal value on all instances. As it is usual in the online setting, we consider no limitations in the computational resources available. This means that at every time step t, where instance I t is known, we assume the existence of an oracle able to compute the optimal solution for that time step. Notice also that our lower bounds do not rely on any complexity assumptions. Some recent results are already known for the on-line multistage model [6,14], however all these results are obtained for specific problems. In this work, we study multistage variants of a broad family of maximization problems. The family of optimization problems that we consider is the following.
◮ Definition 1. (Subset Maximization Problems.) A Subset Maximization problem P is a combinatorial optimization problem whose instances I = (N, p, F ) consist of A ground set N ; A set F ⊆ 2 N of feasible solutions such that ∅ ∈ F; A positive weight p(S) for every S ∈ F. The goal is to find S * ∈ F such that p(S * ) = max{p(S) : S ∈ F }.
We will consider that the empty set is always feasible, ensuring that the feasible set of solutions is non empty. This is a very general class of problems, including the maximization Subset Selection problems studied by Pruhs and Woeginger in [22] (they only considered linear objective functions). It contains for instance graph problems where N is the set of vertices (as in any maximization induced subgraph problem verifying some property) or the set of edges (as in matching problems). It also contains classical set problems (knapsack, maximum 3-dimensional matching,. . . ), and more generally 0-1 linear programs (with non negative profits in the objective function).
Given a problem in the previous class, we are interested in its multistage version [14,12]. The stability over time of a solution sequence is classically captured by considering a transition cost when a modification is made in the solution. Here, dealing with maximization problems, we will consider a transition bonus B for taking into account the similarity of two consecutive solutions. In what follows, we will use the term object to denote an element of N (so an object can be a vertex of a graph, or an edge,. . . , depending on the underlying problem). for any t ∈ T , an instance I t of the optimization problem. We will denote: p t the objective (profit) function at time t F t ∈ 2 N the set of feasible solutions at time t B ∈ R + a given transition profit. the value of a solution sequence S = (S 1 , . . . , S T ) is is the transition bonus for the solution between time steps t and t + 1. We will use the term profit for p t (S t ), bonus for the transition bonus b(S t , S t+1 ), and value of a solution S for f (S); the goal is to determine a solution sequence of maximum value.
There are two natural ways to define the transition bonus. We will see that these two ways of measuring the stability induce some differences in the competitive ratios one can get. Note that by scaling profits (dividing them by B), we can arbitrarily fix B = 1. So from now on, we assume B = 1.
In this article, we will consider two possible ways for the data to evolve.

Related Work
A series of papers consider the online or semi-online settings, where the input changes over time and the algorithm has to modify (re-optimize) the solution by making as few changes as possible (see [3,8,11,13,19,20] and the references therein). The multistage model considered in this paper has been introduced in Eisenstat et al. [12] and Gupta et al. [14]. Eisenstat et al. [12] studied the multistage version of facility location problems. They proposed a logarithmic approximation algorithm. An et al. [2] obtained constant factor approximation algorithms for some related problems. Gupta et al. [14] studied the Multistage Maintenance Matroid problem for both the offline and the online settings. They presented a logarithmic approximation algorithm for this problem, which includes as a special case a natural multistage version of Spanning Tree. They also considered the online version of the problem and they provide an efficient randomized competitive algorithm against any oblivious adversary. The same paper also introduced the study of the Multistage Minimum Perfect Matching problem for which they proved that it is hard to approximate even for a constant number of stages. Bampis et al. [5] improved this negative result by showing that the problem is hard to approximate even for bipartite graphs and for the case of two time steps. When the edge costs are metric within every time step they proved that the problem remains APX-hard even for two time steps. They also showed that the maximization version of the problem admits a constant factor approximation algorithm, but is APX-hard. Olver et al. [21] studied a multistage version of the Minimum Linear Arrangement problem, which is related to a variant of the List Update problem [24], and provided a logarithmic lower bound for the online version and a polylogarithmic upper bound for the offline version. The Multistage Max-Min Fair Allocation problem has been studied in the offline and the online settings in [6]. This problem corresponds to a multistage variant of the Santa Klaus problem. For the off-line setting, the authors showed that the multistage version of the problem is much harder than the static one. They provided constant factor approximation algorithms for the off-line setting. For the online setting they proposed a constant competitive ratio for SSFS-type evolving instances and they proved that it is not possible to find an online algorithm with bounded competitive ratio for GE-type evolving instances. Finally, they showed that in the 1-lookahead case, where at time step t we know the instance of time step t + 1, it is possible to get a constant approximation ratio.
Buchbinder et al. [10] and Buchbinder, Chen and Naor [9] considered a multistage model and they studied the relation between the online learning and competitive analysis frameworks, mostly for fractional optimization problems.

Summary of Results and Overview
The contribution of our paper is a framework for online multistage maximization problems (comprising different models), a characterization of those models in which a constant competitive ratio is achievable, and almost tight upper and lower bounds on the best-possible competitive ratio for these models.
We increase the complexity of the considered models over the course of the paper. We start with the arguably simplest model: Considering a static set of feasible solutions clearly restricts the general model of evolution; while such a straightforward comparison between the Hamming and intersection bonus is not possible, the Hamming bonus seems simpler in that, compared to the intersection model, there are (somewhat comparable) extra terms added on the profit of both the algorithm and the optimum. As we show in Subsection 2.1, there is indeed a simple 2-competitive algorithm: At each time t, it greedily chooses the set S t that either maximizes the transition bonus w.r.t. S t−1 (that is, choosing S t = S t−1 , which is possible in this model) or maximizes the value p t (S t ). We complement this observation with a matching lower bound only involving two time steps.
We then toggle the transition-bonus model and the data-evolution model separately and show that constant competitive ratios can still be achieved. First, in Subsection 2.2, we consider intersection bonus. We show that, after modifying the profits to make larger solutions more profitable, a (2+1/(T −1))-competitive algorithm can be achieved by a greedy approach again. We also give an (almost matching) lower bound of 2 again. Next, we toggle the evolution model. In Subsection 3.1, we adapt the greedy algorithm from Subsection 2.1 by reweighting to obtain a (3 + 1/(T − 1))-competitive algorithm using a more complicated analysis. We complement this result with a lower bound of 1 + √ 2. In Subsection 3.2, we finally consider the general-evolution model with intersection bonus, where we give a simple lower bound showing that a constant-competitive ratio is not achievable. This lower bound relies on forbidding to choose any item in the second step that the algorithm chose in the first step. We circumnavigate such issues by allowing the algorithm a lookahead of one step and present a 4-competitive algorithm for that setting. A similar phase transition has been observed for a related problem [6], but our algorithm, based on a doubling approach, is different. We also give a matching lower bound of 4 on the competitive ratio of any algorithm in the same setting. We summarize all results described thus far in Table 1.
We note that the lower bounds mentioned for the Hamming model are only shown for a specific fixed number of time steps, and that in general there is no trivial way of extending these bounds to a larger number of time steps. One may however argue that the large-T regime is in fact the interesting one for both practical applications and in theory, the latter because the effect of having a first time step without bonus vanishes. At the end of the respective sections, we therefore give asymptotical lower bounds of 3/2 and roughly 1.696 for the cases of a static set of feasible solutions and general evolutions, respectively. These bounds are non-trivial, but we do not know if they are tight.
It is plausible that the aforementioned upper bounds can be improved if extra assumptions on characteristics of the objective function and the sets of feasible solutions are made. In Subsubsection 3.1.2, we show that already very natural assumptions suffice: Assuming that at each time the feasible solutions are closed under taking subsets and the objective function is submodular, we give a (21/8 + o(1))-competitive algorithm for the model with a general evolution and Hamming bonus, improving the previous (3 + o(1))-competitive ra-tio. Our lower bounds for general evolution and Hamming bonus in fact fulfill the extra assumptions.
In Section 4, we summarize our results and mention directions for future research that we consider interesting.

Model of a Static Set of Feasible Solutions
We consider here the model of evolution where only profits change over time: F t = F for any t. We first consider the Hamming bonus model and show a simple 2-competitive algorithm. We will then show that a (asymptotic) competitive ratio of 2 can also be achieved in the intersection bonus model using a more involved algorithm. In both cases, this ratio 2 is shown to be (asymptotically) optimal.

◮ Theorem 5. In the SSFS model with Hamming bonus, there is a 2-competitive algorithm.
Proof. We consider the very simple following algorithm. At each time step t, the algorithm computes an optimal solution S * t with associated profit p t (S * t ). At t = 1 we fix (which is possible thanks to the fact that the set of feasible solutions does not change).
Let f * be the optimal value. Since any solution sequence gets profit at most p t (S * t ) at time t, and bonus at most n between two consecutive time steps, we get By construction, at time t > 1, either the algorithm gets profit p t (S * t ) when p t (S * t ) > n, or bonus (from t − 1) n when n ≥ p t (S * t ). So in any case the algorithm gets profit plus bonus at least pt(S * t )+n 2 . At time 1 it gets profit at least p 1 (S * 1 ). So which completes the proof. At time 1, A chooses S 1 : at time 2 we give profit 0 to object 1, and profit 1 to all other objects. Then, if the algorithm takes S 0 (resp, S 1 , S 2 ), at time 2 its gets value n − 1 (resp, n, n − 1) while the solution consisting of taking S 2 at both time steps has value 2n − 1.
At time 1, A chooses S 1 : at time 2 we give profit n to object 1, and 0 to all other objects. Then if the algorithm takes S 0 (resp, S 1 , S 2 ) at time 2 its gets value n − 1 (resp, n, n), while the solution consisting of taking S 1 at both time steps has value 2n. In any case, the ratio is at least 2n−1 We complement this lower bound with an asymptotical result for large T . The remaining time horizon is partitioned into contiguous phases. Consider a phase that starts at time t. The invariant at the beginning of the phase is that both the algorithm and the optimum have picked the same item in the previous time step t − 1. Let this item be w.l.o.g. item 2; the other case is symmetric. Then p t ({2}) = 1 and p t ({1}) = 3. By the same reasoning as above, we can assume the algorithm chooses an item at t. Let i ∈ {1, 2} be that item. Then p t+1 ({i}) = 0 and p t+1 ({3 − i}) = 1. As long as the algorithm is still not Once the algorithm picks item 3 − i at some time, the phase ends regularly; otherwise it ends by default.
Now consider a phase of length ℓ that ends regularly (note ℓ ≥ 2). We claim that the values of the algorithm and the optimum have a ratio of at least 3/2. This is because of the following estimates on the algorithm's and optimum's value: In either case for i, the algorithm obtains a value of 3 in time step t. Furthermore, the total bonus in all subsequent time steps is (ℓ − 2) · 2, because the algorithm has to switch from item i to item 3 − i. There is an additional profit of 1 at time t + ℓ − 1. Therefore, the total value is 4 + (ℓ − 2) · 2 The value of the optimum is at least 6 + (ℓ − 2) · 3: It chooses item 3 − i already at time t and keeps it until time t + ℓ − 1, obtaining a value of 3 in that time step and another 3 in each subsequent time step. This proves the claim and thereby the theorem as a phase that ends by default can be extended to one that ends regularly by modifying the optimum's and algorithm's values by constants. ◭

Intersection-Bonus Model
In the intersection-bonus model things get harder since an optimal solution S * t may be of small size and then gives very small (potential) bonus for the next step. As a matter of fact, the algorithm of the previous section has unbounded competitive ratio in this case: take a large number n of objects, F = 2 N , and at time 1 all objects have profit 0 up to one which has profit ǫ. The algorithm will take this object (instead of taking n − 1 objects of profit 0) and then potentially get bonus at most 1 instead of n − 1.
Thus we shall put an incentive for the algorithm to take solutions of large size, in order to have a chance to get a large bonus. We define the following algorithm called MP-Algo (for Modified Profit algorithm). Informally, at each time step t, the algorithm computes an optimal solution with a modified objective function p ′ t . These modifications take into account (1) the objects taken at time t − 1 (2) an incentive to take a lot of objects. Formally, MP-Algo works as follows: 1. At t = 1: let p ′ 1 (S) = p 1 (S) + |S|. Choose S 1 as an optimal solution for the problem with modified profits p ′ 1 . Proof. Let (Ŝ 1 , . . . ,Ŝ T ) be an optimal sequence. Since S t is optimal with respect to p ′ t , for t = 2, . . . , T − 1 we have:

For
Since S t−1 is also a feasible solution at time t, we have: Similarly At t = 1 p ′ t (S) = p t (S) + |S|, so Now, note that |S t ∩ S t−1 | is the transition bonus of the computed solution between t − 1 and t. By summing Equation (1) for t = 2, . . . , T − 1, Equation (3) and Equation (5), we deduce: Since in the optimal sequence the transition bonus between time t and t + 1 is at most |Ŝ t |, we get: Now we sum Equation (2) for t = 2, . . . , T − 1 and Equation (4): From this we easily derive: By summing Equations (6) and (7)  We note that competitive ratio 2 can be derived with a similar analysis when the number of time steps is 2 or 3. We show a matching lower bound (which is also valid in the asymptotic setting). Proof. Let ǫ > 0 and T = 1 ǫ . We consider T time steps, and a set N of n = T objects. The objective function is linear, and feasible solutions are sets of at most 1 object. At t = 1, the profit of each object is 1. Then, at each time step, if the algorithm takes an object, this object will have profit 0 until the end. While an object is not taken by the algorithm, its profit remains 1.
Since the algorithm takes at most one object at each time step, there is an object which is never taken till the last step. The solution of taking this object during all the process has value 2T − 1. But at each time step the algorithm either takes a new object (and gets no bonus) or keeps the previously taken object and gets no profit. So the value of the computed solution is at most T . The ratio is 2

Model of General Evolution
We consider in this section that the set of feasible solutions may evolve over time. We will show that in the Hamming bonus model, we can still get constant competitive ratios, though ratios slightly worse than in the case where only profits could change over time. Then, we will tackle the intersection bonus model, showing that no constant competitive ratio can be achieved. However, with only 1-lookahead we can get a constant competitive ratio.

Hamming-Bonus Model
In this section we consider the Hamming bonus model. We first show in Section 3.1.1 that there exists a 3 + 1 T −1 -competitive algorithm. Interestingly, we then show in Section 3.1.2 that a slight assumption on the problem structure allows to improve the competitive ratio. More precisely, we achieve a 21/8 (asymptotic) competitive ratio if we assume that the objective function is submodular (including the additive case) and that a subset of a feasible solution is feasible. These assumptions are satisfied by all the problems mentioned in introduction. We finally consider lower bounds in Section 3.1.3.

General Case
We adapt the idea of the 2-competitive algorithm working for the Hamming bonus model for a static set of feasible solutions (Section 2.1) to the current setting where the set of feasible solutions may change. Let us consider the following algorithm BestOrNothing: at each time step t, BestOrNothing computes an optimal solution S * t with associated profit p t (S * t ) and compares it to 2 times the maximum potential bonus, i.e to 2n. It chooses S * t if the associated profit is at least 2n, otherwise it chooses S t = ∅. A slight modification is applied for the last step T . Formally, BestOrNothing works as follows: 1. For t from 1 to T − 1: a. Compute an optimal solution S * t at time t with associated profit p t ( We shown an upper bound on the competitive ratio achieved by this algorithm. The global profit of an optimal solution up to time t 1 is at most 2n(t 1 −|J|)+ t∈J p t (S * t ). Its bonus (including the one from time t 1 to t 1 + 1) is at most nt 1 . So its global value is at most n (3t 1 − 2|J|) + t∈J p t (S * t ). The solution computed by BestOrNothing gets profit at least t∈J p t (S * t ). Note that it chooses the empty set always but |J| times, so it gets transition bonus n at least t 1 − 2|J| times (each step in J may prevent to get the bonus only between t − 1 and t, and between t and t + 1). So the global value of the computed solution up to time t 1 is at least n max{0; t 1 − 2|J|} + t∈J p t (S * t ). Up to time t 1 , the ratio r between the optimal value and the value of the solution computed by BestOrNothing verifies where we used the fact that t∈J p t (S * t ) ≥ 2n|J|. Since max{0; t 1 − 2|J|} + 2|J| ≥ t 1 the ratio is at most 3 up to time t 1 . Now, let us consider the end of the process, from time t 1 + 1 (or 1 if J is empty) up to time T . If t 1 = T − 1 then we take the best solution at time T and get no extra loss, so the algorithm is 3-competitive in this case. Now assume t 1 < T − 1. We know that BestOrNothing chooses the empty set up to T − 1. Let us first assume that p T (S * T ) < n. Then on the subperiod from t 1 + 1 to T BestOrNothing gets value n(T − t 1 − 1) (bonuses), while the optimum gets bonus at most n(T − t 1 − 1) and profit at most 2n(T − t 1 − 1) + n. The optimal value is then at most Now suppose that p T (S * T ) ≥ n. On the subperiod from t 1 + 1 to T BestOrNothing gets value n(T − t 1 − 2) + p T (S * T ), while the optimum gets bonus at most n(T − t 1 − 1) and profit at most 2n(T − t 1 − 2) + p T (S * T ). The worst case ratio occurs when p T (S * T ) = n. In this case, as before, the value of the computed solution is n(T − t 1 − 1), while the optimal value is at most n (3T − 3t 1 − 2) ≤ 3n(T − t 1 − 1) + n.
Then, in all cases we have that the optimal value is at most 3f (S 1 , . . . , S n ) + n. But f (S 1 , . . . , S n ) ≥ (T − 1)n (the computed solution has value at least t 1 n up to t 1 , and then at least n(T − t 1 − 1)), and the claimed ratio follows. ◭

Improvement for Submodularity and Subset Feasibility
In this section we assume that the problem have the following two properties: subset feasibility: at any time step, every subset of a feasible solution is feasible. submodularity: for any S, S ′ , any t, p t (S ∩ S ′ ) + p t (S ∪ S ′ ) ≤ p t (S) + p t (S ′ ). Note that this implies that, if a feasible set X is partitioned into (disjoint) subsets X 1 , . . . , X h , then X 1 , . . . , X h are feasible and p t (X) ≤ i p t (X i ).
We exploit this property to devise algorithms where we partition the set of objects and solve the problems on subinstances. As a first idea, let us partition the set of objects into into 3 sets A, B, C of size (roughly) n/3; consider the algorithm which at every time step t computes the best solutions S A t , S B t , S C t on each subinstance on A, B and C, and chooses S t as the one of maximum profit between these 3 solutions. By submodularity and subset feasibility, the algorithm gets profit at least 1/3 of the optimal profit at each time step. Dealing with bonuses, at each time step the algorithm chooses a solution included either in A, or in B, or in C so, for any t < T , at least one set among A, B and C is not chosen neither at time t nor at time t + 1, and the algorithm gets transition bonus at least n/3. Hence, the algorithm is 3-competitive.
We now improve the previous algorithm. The basic idea is to remark that if for two consecutive time steps t, t + 1 the solution S t and S t+1 are taken in the same subset, say A, then the bonus is (at least) 2n/3 instead of n/3. Roughly speaking, we can hope for a ratio better than 1/3 for the bonus. Then the algorithm makes a tradeoff at every time step: if the profit is very high then it will take a solution maximizing the profit, otherwise it will do (nearly) the same as previously. More formally, let us consider the algorithm 3-Part. We first assume that n is a multiple of 3. x ∈ [0, 1] will be defined later.  (2) occurs. It means that Case (2) or (3) occurred at the previous step, so S t−1 is included in A, B or C. Suppose w.l.o.g that algorithm took S t−1 ⊆ A. Then S A t gives a bonus at least 2n/3 (between t−1 and t), and S B t and S C t gives a bonus at least n/3. By computing max{a t + 2n/3, b t + n/3, c t + n/3}, we derive:

Partition N in three subsets
where S * t is a solution maximizing the profit at time t, using the fact that p t (S * t ) ≤ a t + b t + c t by subset feasibility and submodularity. Since in Case (2) p t (S * t ) ≤ xn, we derive: with r 2 = 3x+4 9(1+x) . Now, consider a time step t ≥ 2 where Case (3) occurs. Then necessarily Case (1) occurs .
If n is not a multiple of 3, adding one or two dummy objects add T − 1 or 2(T − 1) to solution values, inducing a loss which is a fraction at most O(1/n) of the optimal value. ◭

Lower Bounds
We complement the algorithmic results with a lower bound for two time steps and an asymptotical one. Interestingly, these bounds are also valid for the latter restricted setting with subset feasibility and submodularity.
◮ Theorem 12. Consider the GE model with Hamming bonus. For any ǫ > 0, there is no Proof. We consider a knapsack problem with n objects (n = 2 suffices to show the result, but the proof is valid for any number n of objects), and T = 2 time steps. At time 1, all objects have weight 1 and profit α = √ 2 − 1; the capacity of the bag is n. Let S 1 be the set of objects chosen at step 1 by the algorithm (possibly S 1 = ∅). At t = 2 the algorithm receives the instance I 2 (S 1 ) where: the capacity is c 2 = n − |S 1 |. each object not in S 1 receives weight and profit 1. each object in S 1 has a weight greater than c 2 . Then at step 2 the algorithm receives value 1 for each object not in |S 1 | (either by transition bonus from step 1, or by taking it at step 2). The value of its solution is α|S 1 |+n−|S 1 |. Now, the solution consisting of taking S 1 at both time steps has value α(n − |S 1 |) + n − |S 1 | + n = n(2 + α) − |S 1 |(1 + α). The chosen α is such that 2 + α = 1+α 1−α , so the solution (S, S) has value 1+α 1−α (n − |S|(1 − α)). The ratio is 1+α Proof. Consider some ǫ > 0 and some online algorithm A. The ground set only consists of the single item 1, that is, N = {1}. At time 1, it is not feasible to pick the item, that is, F 1 = {∅}. We partition the remaining time horizon {2, 3, . . . , T } (with T yet to be specified) into phases. Hence, the first phase starts in time step 2. In any phase, as long as A has not included item 1 in its solution until time t < T , both including and not including it is feasible at t + 1, that is, F t+1 = {∅, {1}}. Once A includes the item in its solution at time t < T (meaning S t = {1}), including it becomes unfeasible at the next time, that is, F t+1 = {∅}. The current phase also ends at this time. In this case, we say that the phase ends regularly. At t = T , the current phase ends by default in any case. If a phase however ends regularly at time t + 1 < T , a new phase starts at time t + 2.
There is no profit associated with the empty set, that is, p t (∅) = 0; the profit p t ({1}) is β whenever t is the first time step of a phase, and it is γ in all other cases (note that, however, it may be unfeasible to include item 1 in the solution). The remaining part of the proof is concerned with finding β, γ so as to maximize the competitive ratio.
For the analysis, denote by S * = (S * 1 , S * 2 , . . . , S * T ) the optimal solution, and denote by S = (S 1 , S 2 , . . . , S T ) the solution that A finds. We consider phases separately. First consider a phase of length ℓ starting at time t 0 ending regularly (at time t 0 + ℓ − 1). Note that ℓ ≥ 2 and that the initial situation is independent of t 0 and ℓ in that 1 / ∈ S t−1 . For each time t that is part of the phase, we count b(S t−1 , S t ) + p t (S t ) and b(S * t−1 , S * t ) + p t (S * t ) towards the values of the optimum and algorithm, respectively. If ℓ = 2, the resulting values of the algorithm and optimum are max{β, 2} ≥ 2 and β, respectively. If ℓ > 2, the value are max{β + (ℓ − 2)(1 + γ), ℓ} ≥ β + (ℓ − 2)(1 + γ) and (ℓ − 2) + γ, respectively. Hence, in phases of length at most 2, the optimum does not pick the item; in longer phases, it picks the item at all times when it can.
To express the lower bound that we can show, first note that assuming that each phase ends regularly is only with an additive constant loss in both the algorithm's and the optimum's value, so we may make this assumption for the asymptotical competitive ratio considered here. Since the algorithm chooses the phase lengths, the lower bound α that we can show here is equal to the largest lower bound on the ratio between the optimum's and the algorithm's value within any phase, which is lower bounded by according to the above considerations. Note that the infimum in (9) is minimized when its argument is identical across all γ. This is the case when Furthermore, (9) is minimized when both its arguments are identical, meaning and therefore This shows the claim. ◭

Intersection-Bonus Model
We now look at the general-evolution model with intersection bonus. This model is different from the ones considered before: We first give a simple lower bound showing that there is no constant-competitive algorithm. Note that in this model, by adding dummy time steps giving no bonus and no profit, the previous lower bound extends to any number of time steps. This lower bound motivates considering the 1-lookahead model: at time t, besides I t , the algorithm knows the instance I t+1 . It shall decide the feasible solution chosen at time t. We consider an algorithm based on the following idea: at some time step t, the algorithm computes an optimal sequence of 2 solutions (S * t,1 , S * t,2 ) of value z * t for the subproblem defined on time steps t and t + 1. Suppose it fixes S t = S * t,1 . Then, at time t + 1, it computes (S * t+1,1 , S * t+1,2 ) of value z * t+1 .
We have t + 1 > T ǫ , implying t = T ǫ (otherwise A could not have confirmed at t). Then the total value of the optimum is For convenience, we will now describe a sequence a ′ 1 , a ′ 2 , . . . , a ′ Tǫ of rational numbers; a 1 , a 2 , . . . , a Tǫ can then be obtained by multiplying all numbers in the former sequence with a suitable natural number. Let ǫ ′ ∈ (0, ǫ) be rational. Now the goal can be reformulated to be to choose a ′ 1 , a ′ 2 , . . . , a ′ Tǫ such that the ratios and (corresponding to the above ones) are all at least 4 − ǫ ′ . To do so, we start by setting a ′ 1 := 1. Now we inductively define a ′ t for t ≤ T ǫ − 2 (note T ǫ is yet to be defined). Assuming all a ′ 1 , . . . , a ′ t−1 are defined, we set a ′ t to be such that (10) for t is precisely We claim there exists a (first) t 0 such that (meaning the (t 0 + 1)-st element of the sequence a 1 , a 2 , . . . would become smaller than a ′ t0 ). Then we set T ǫ := t 0 + 2 and a ′ t0+2 = a ′ t0+1 = a ′ t0 . Note that then indeed, by (12) and a ′ t0+1 = a ′ t0 , (10) is at least 4 − ǫ ′ for t = T ǫ − 1 (and therefore, by the previous argument, for all). Furthermore, since a ′ t0+2 = a ′ t0+1 , (11) is identical to (10) for t = T ǫ − 1 and therefore also at least 4 − ǫ ′ .
So it remains to show the claim. Define for all t including the first element where the fraction becomes at most 1 (if it exists; otherwise the sequence is infinite). Further note that for all such t ≥ 3 we have t j=1 a ′ j = (4 − ǫ ′ ) · a ′ t−1 (by using the definition for a t−1 ). Therefore b t = (4 − ǫ ′ ) · (1 − a t−1 /a t ) = (4 − ǫ ′ ) · (1 − 1/b t−1 ).
We now show two properties of the sequence b 1 , b 2 , . . . : If b t ≥ 1 for t ≥ 3, then b t+1 < b t . Note that this expression simplifies to to b 2 t − (4 − ǫ ′ ) · b t + (4 − ǫ ′ ) > 0, which is true for all b t ≥ 1. The sequence b 1 , b 2 , . . . does not converge to a value at least 1. Suppose it did. This would imply there exists x ≥ 1 with x = (4 − ǫ ′ ) · (1 − 1/x), which however does not have a real solution. By basic calculus, this proves that there exists a t with b t ≤ 1, implying the claim, which in turn implies the theorem. ◭

Conclusion
In this paper, we have developed techniques for online multistage subset maximization problems and thereby settled the achievable competitive ratios in the various settings almost exactly. Disregarding asymptotically vanishing terms in the upper bounds, what remains open is the exact ratio in the general-evolution setting with Hamming bonus (shown to be between 1 + √ 2 and 3 in this paper) and exact bounds for the models with Hamming bonus when T → ∞. Furthermore, it is plausible that the ratios can be improved for (classes of) more specific problems.
We emphasize that we have focussed on deterministic algorithms in this work. Indeed, some of our bounds can be improved by randomization (assuming an oblivious adversary): In the general-evolution model with Hamming bonus assuming submodularity and subset feasibility, there is a simple randomized (2 + o(1))-competitive algorithm (along the lines of the algorithms in Subsubsection 3.1.2): Initially partition N uniformly at random into two equal-sized sets (up to possibly one item) A and B. At each time, select the optimal solution restricted to A. Again, the algorithm is (2 + o(1))-competitive separately on both profit and bonus. While the strong lower bound without lookahead in the general-evolution model with intersection bonus still holds, we can get a simple 2-competitive algorithm for lookahead 1: Inititally flip a coin to interpret the instance as a sequence of length-2 instances either starting at time 1 or 2. Thanks to lookahead 1, the length-2 instances can all be solved optimally. The total value of all these length-2 instances adds up to at least the optimal value, and the expected value obtained by the algorithm is half of that.
While we believe that we have treated various of the most natural ways of defining value in multistage subset maximization problems, other ways can be thought of, to some of which our results extend. For instance, Theorem 5 also works for time-dependent or object-dependent bonus without major modifications (whereas, e.g., Theorem 8 does not).
We have not worried about computational complexity in this work (and therefore neither about the representation of the set of feasible solutions); indeed, often we use an oracle providing the optimal solution to instances of a potentially hard problem. However, we mention that, if only an approximation algorithm to the problem at hand was known, we would be able to obtain similar online algorithms whose competitive ratio would depend on the approximation guarantee of the approximation algorithm.