eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
17
10.4230/DagSemProc.06051.1
article
06051 Abstracts Collection – Kolmogorov Complexity and Applications
Hutter, Marcus
Merkle, Wolfgang
Vitanyi, Paul M.B.
From 29.01.06 to 03.02.06, the Dagstuhl Seminar 06051 ``Kolmogorov Complexity and Applications'' was held in the International Conference and Research Center (IBFI),
Schloss Dagstuhl. During the seminar, several participants presented
their current research, and ongoing work and open problems were
discussed. Abstracts of the presentations given during the seminar
as well as abstracts of seminar results and ideas are put together
in this paper. The first section describes the seminar topics and
goals in general. Links to extended abstracts or full papers are
provided, if available.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.1/DagSemProc.06051.1.pdf
Information theory
Kolmogorov Complexity
effective randomnes
algorithmic probability
recursion theory
computational complexity
machine learning knowledge discovery
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
13
10.4230/DagSemProc.06051.2
article
Application of Kolmogorov complexity and universal codes to identity testing and nonparametric testing of serial independence for time series.
Ryabko, Boris
Astola, Jaakko
Gammerman, Alex
We show that Kolmogorov complexity and such its estimators as
universal codes (or data compression methods) can be applied for
hypothesis testing in a framework of classical mathematical
statistics. The methods for identity testing and nonparametric
testing of serial independence for time series are described.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.2/DagSemProc.06051.2.pdf
Algorithmic complexity
algorithmic information theory
Kolmogorov complexity
universal coding
hypothesis testing
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
23
10.4230/DagSemProc.06051.3
article
Automatic Meaning Discovery Using Google
Cilibrasi, Rudi
Vitanyi, Paul M.B.
We survey a new area of parameter-free similarity distance measures
useful in data-mining,
pattern recognition, learning and automatic semantics extraction.
Given a family of distances on a set of objects,
a distance is universal up to a certain precision for that family if it
minorizes every distance in the family between every two objects
in the set, up to the stated precision (we do not require the universal
distance to be an element of the family).
We consider similarity distances
for two types of objects: literal objects that as such contain all of their
meaning, like genomes or books, and names for objects.
The latter may have
literal embodyments like the first type, but may also
be abstract like ``red'' or ``christianity.'' For the first type
we consider
a family of computable distance measures
corresponding to parameters expressing similarity according to
particular features
between
pairs of literal objects. For the second type we consider similarity
distances generated by web users corresponding to particular semantic
relations between the (names for) the designated objects.
For both families we give universal similarity
distance measures, incorporating all particular distance measures
in the family. In the first case the universal
distance is based on compression and in the second
case it is based on Google page counts related to search terms.
In both cases experiments on a massive scale give evidence of the
viability of the approaches.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.3/DagSemProc.06051.3.pdf
Normalized Compression Distance
Clustering
Clasification
Relative Semantics of Terms
Google
World-Wide-Web
Kolmogorov complexity
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
20
10.4230/DagSemProc.06051.4
article
Binary Lambda Calculus and Combinatory Logic
Tromp, John
We introduce binary representations of both lambda calculus
and combinatory logic terms, and demonstrate their simplicity
by providing very compact parser-interpreters for these binary
languages.
We demonstrate their application to Algorithmic Information Theory
with several concrete upper bounds on program-size complexity,
including an elegant self-delimiting code for binary strings.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.4/DagSemProc.06051.4.pdf
Concrete
program size complexity
ambda calculus
combinatory logic
encoding
self-delimiting
binary strings
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
5
10.4230/DagSemProc.06051.5
article
Combinatorial proof of Muchnik's theorem
Shen, Alexander
Original proof of Muchnik's theorem on conditional descriptions can be modified and split into two parts:
1) we construct a graph that allows large online matchings (main part)
2) we use this graph to prove the theorem
The question about online matching could be interesting in itself.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.5/DagSemProc.06051.5.pdf
Matching conditional descriptions Kolmogorov complexity
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
20
10.4230/DagSemProc.06051.6
article
Complexity Monotone in Conditions and Future Prediction Errors
Chernov, Alexey
Hutter, Marcus
Schmidhuber, Jürgen
We bound the future loss when predicting any (computably) stochastic sequence online. Solomonoff finitely bounded the total deviation of his universal predictor $M$ from the true distribution $mu$ by the algorithmic complexity of $mu$. Here we assume we are at a time $t>1$ and already observed $x=x_1...x_t$. We bound the future prediction performance on $x_{t+1}x_{t+2}...$ by a new variant of algorithmic complexity of $mu$ given $x$, plus the complexity of the randomness deficiency of $x$. The new
complexity is monotone in its condition in the sense that this complexity can only decrease if the condition is prolonged. We also briefly discuss potential generalizations to Bayesian model classes and to classification problems.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.6/DagSemProc.06051.6.pdf
Kolmogorov complexity
posterior bounds
online sequential prediction
Solomonoff prior
monotone conditional complexity
total error
future loss
ra
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
5
10.4230/DagSemProc.06051.7
article
Error in Enumerable Sequence Prediction
Hay, Nick
We outline a method for quantifying the error of a sequence prediction. With sequence predictions represented by semimeasures $
u(x)$ we define their error to be $-log_2
u(x)$. We note that enumerable semimeasures are those which model the sequence as the output of a computable system given unknown input. Using this we define the simulation complexity of a computable system $C$ relative to another $U$ giving an emph{exact} bound on their difference in error. This error in turn gives an exact upper bound on the number of predictions $
u$ gets incorrect.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.7/DagSemProc.06051.7.pdf
Sequence prediction
Solomonoff induction
enumerable semimeasures
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
15
10.4230/DagSemProc.06051.8
article
Learning in Reactive Environments with Arbitrary Dependence
Ryabko, Daniil
Hutter, Marcus
In reinforcement learning the task
for an agent is to attain the best possible asymptotic reward
where the true generating environment is unknown but belongs to a
known countable family of environments.
This task generalises the sequence prediction problem, in which
the environment does not react to the behaviour of the agent.
Solomonoff induction solves the sequence prediction problem
for any countable class of measures; however, it is easy to see
that such result is impossible for reinforcement learning - not any
countable class of environments can be learnt.
We find some sufficient conditions
on the class of environments under
which an agent exists which attains the best asymptotic reward
for any environment in the class. We analyze how tight these conditions are and how they
relate to different probabilistic assumptions known in
reinforcement learning and related fields, such as Markov
Decision Processes and mixing conditions.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.8/DagSemProc.06051.8.pdf
Reinforcement learning
asymptotic average value
self-optimizing policies
(non) Markov decision processes
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
12
10.4230/DagSemProc.06051.9
article
Multisource Algorithmic Information Theory
Shen, Alexander
Multisource information theory is well known in Shannon setting. It studies the possibilities of information transfer through a network with limited capacities. Similar questions could be studied for algorithmic information theory and provide a framework for several known results and interesting questions.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.9/DagSemProc.06051.9.pdf
Kolmogorov complexity multisource information theory
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
1
10.4230/DagSemProc.06051.10
article
Natural Halting Probabilities, Partial Randomness, and Zeta Functions
Calude, Christian S.
Stay, Michael A.
We introduce the {it natural halting probability} and the {it natural complexity} of a Turing machine and we relate them to program-size complexity and Chaitin's halting probability. A classification of Turing machines according to their natural (Omega) halting probabilities is proposed: divergent, convergent and tuatara. We prove the existence of universal convergent and tuatara machines. Various results on randomness and partial randomness are proved. For example, we show that the natural halting probability of a universal tuatara machine is c.e. and random. A new type of partial randomness, asymptotic randomness, is introduced. Finally we show that in contrast to classical (algorithmic) randomness---which cannot be characterised in terms of plain complexity---various types of partial randomness admit such characterisations.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.10/DagSemProc.06051.10.pdf
Natural halting probability
natural complexity
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
7
10.4230/DagSemProc.06051.11
article
On impossibility of sequential algorithmic forecasting
V'Yugin, Vladimir
The problem of prediction future event given an individual
sequence of past events is considered. Predictions are given
in form of real numbers $p_n$ which are computed by some algorithm
$varphi$ using initial fragments $omega_1,dots, omega_{n-1}$
of an individual binary sequence $omega=omega_1,omega_2,dots$
and can be interpreted as probabilities of the event $omega_n=1$
given this fragment.
According to Dawid's {it prequential framework}
%we do not consider
%numbers $p_n$ as conditional probabilities generating by some
%overall probability distribution on the set of all possible events.
we consider partial forecasting algorithms $varphi$ which are
defined on all initial fragments of $omega$ and can
be undefined outside the given sequence of outcomes.
We show that even for this large class of forecasting algorithms
combining outcomes of coin-tossing and transducer algorithm
it is possible to efficiently generate with probability close
to one sequences
for which any partial forecasting algorithm is failed by the
method of verifying called {it calibration}.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.11/DagSemProc.06051.11.pdf
Universal forecasting
computable calibration
Dawid's prequential framework
algorithmic randomness
defensive forecasting
eng
Schloss Dagstuhl – Leibniz-Zentrum für Informatik
Dagstuhl Seminar Proceedings
1862-4405
2006-07-31
6051
1
11
10.4230/DagSemProc.06051.12
article
Recent Results in Universal and Non-Universal Induction
Poland, Jan
We present and relate recent results in prediction based on
countable classes of either probability (semi-)distributions
or base predictors. Learning by Bayes, MDL, and stochastic
model selection will be considered as instances of the first
category. In particular, we will show how analog assertions
to Solomonoff's universal induction result can be obtained for
MDL and stochastic model selection. The second category is
based on prediction with expert advice. We will present a
recent construction to define a universal learner in this
framework.
https://drops.dagstuhl.de/storage/16dagstuhl-seminar-proceedings/dsp-vol06051/DagSemProc.06051.12/DagSemProc.06051.12.pdf
Bayesian learning
MDL
stochastic model selection
prediction with expert advice
universal learning
Solomonoff induction