In this post I would like to comment on two papers I “stumbled upon”, one in regularization theory and one in image processing.
The first one is A regularization parameter for nonsmooth Tikhonov regularization by Kazufumi Ito, Bangti Jin and Tomoya Takeuchi. As the title announces, the paper addresses the problem of determining suitable regularization parameter for some kind of Tikhonov regularization. In particular, the authors propose a new heuristic method, i.e. method which does not use any estimate of the noise level in the data. This is an interesting and important topic for several reasons:
- Practically, estimates on the noise level are rarely available and if they are, they are not too reliable.
- Strictly speaking, these kind of rules are “bad” since there is the “Bakushinksii Veto”: Such rules only provide regularizations (e.g. in the sense of Engl, Hanke, Neubauer for problems which are well-posed (as a great service, the authors state and prove this statement as Theore 3.2).
- Despite this veto, several heuristic rules produce excellent results in practice.
Note that the last second points are not in contradiction. They merely say that the notion of “regularization” may be too strict. Usually, it uses a worst case estimate which may practically never observed.
The paper contributes a new rule and state that it is applicable to a broad range of problems. They use very general Tikhonov functional:
and do not assume that or are smooth. They use the value function
and propose the following rule for : For some choose such that
is minimal. I do not have any intuition for this rule (however, from they proofs you see that they work, at least for “partially smooth cases”, see below). Lacking a name for this rule, one may use the term “weighted value function rule”.
They prove several nice properties of the value function (continuity, monotonicity and concavity) with loose assumptions on and (especially they do not even need existence of minimizers for , only that the minimum exists). However, when it comes to error estimates, they only obtain results for a specific discrepancy measure, namely a squares Hilbert space norm:
It seems that, for general convex and lower-semicontinuous penalties they build upon results from my paper with Bangti Jin on the Hanke-Raus rule and the quasi-optimality principle.
Another contribution of the paper is that it gives an algorithm that realizes the weighted value function rule (a thing which I omitted in my paper with Bangti). Their numerical experiments demonstrate that the weighted value function rule and the proposed algorithm works well for academic examples.
The next paper I want to discuss is the preprint Properties of : The one-dimensional case by Kristian Bredies, Karl Kunisch and Tuomo Valkonen. There the authors analyze the somehow recent generalization “total generalized variation” of the omnipresent total variation. The TGV has been proposed by Bredies, Kunisch and Pock in this paper recently and Kristian and me also briefly described it in our book on mathematical image processing. Loosely speaking, the TGV shall be a generalization of the usual total variation which does not lead to “staircasing”. While one may observe from the construction of the TGV functional, that staircasing is not to be expected, the authors in this paper give precise statements. By restricting to the one dimensional case they prove several interesting properties of the TGV functional, most notably that it leads to an equivalent norm of the space .
Maybe I should state the definitions here: The total variation of a function is
leading the the -norm
The seminorm for a parameter tuple is
and the associated norm is
In Lemma 3.3 they prove that and are equivalent norms on . In Section 4 they show that minimizers of
obey staircasing of degree 0, i.e. the solution is piecewise constant when it is not equal to . For the minimizers of
one has staircasing of degree 1: is affine linear where it is not equal to .
These two facts combined (norm equivalence of and and the staircasing of degree 1) seem quite remarkable to me. They somehow show that staircasing is not related to the space of functions of bounded variation but only to the specific semi-norm. This is somehow satisfying since I still remember the thorough motivation of L. Rudin in his 1987 thesis for the usage of the space in image processing: If there where images which are not in we could not observe them. (He even draws an analogy to the question: How many angles can dance on the point of a needle?) Moreover, he further argues that is not too large in the sense that its elements are still accessible to analysis (e.g. in defining a weak notion of curvature although they may be discontinuous). The -model shows that it is possible to overcome the undesired effect of staircasing while staying in the well founded and mathematically sound and appealing framework of .
The paper contains several more interesting results (e.g. on preservation of continuity and “affinity” and on convergence of with respect to which I do not collect here.