Abstract
In the end, computer vision learning is built from instances. When picture costs are low, much lower than documenting an instance, it pays to get the most out of each instance. Where searching for an instance in a large database is a relevant and hard problem by itself, tracking also can be perceived as locating the instance over time, as much as re-identification can be perceived as establishing whether the instance reappears in the scene again. Instance analysis is therefore key to many computer vision problems. The problem of instance analysis is hard. From the one instance one needs to get hold of all accidental variations in the target's appearance for this view as as well as one has to derive information on all seen and unseen views of the instance. Instance search started as generalized matching salient points, equivalent to searching for matching sets in the image, in feature space, and by emphasis of the similarity function. The matching approach is suited for rigid targets: logos, buildings and scenes, who are effectively flat and one-sided. The approach is unsuited for rigid 3D-objects as the variability in view point variations is exceeds the matching capacity cannot handle large differences in viewpoints. To solve that 3D-attributes needs to be learned so big that one needs to learn about other viewpoints in general before one can start with searching from one example, especially when searching for known object-types. For structured objects like shoes attributes can to be learned to identify targets also from previously unseen views. The approach gives satisfactory results for structured 3D-objects yet fails for arbitrary objects. In such cases, first it needs to be learned how views of objects appear in general given one arbitrary view of the object. We do so by employing Siamese networks optimized for this task and learned on tracking datasets. The network can then be applied to establish the yes-or-no similarity of different views of an arbitrary, previously unseen object in general. The success of the algorithm suffices to reach state of the art results in tracking yet is not good enough for instance search. Current research is on learning more about object variations in general by structuring the network or structuring the data input for the purpose of knowing all there is to know about an instance.
Biography
Arnold W.M. Smeulders is retired professor at the University of Amsterdam for research in the theory and practice of multimedia and computer vision, part of AI. The group's video search engines have received a top-3 performance for all 14 years in the NIST TREC-vid competition. He was recipient of a Fulbright fellowship at Yale University, and visiting professor in Hong Kong, Tuskuba, Modena, Cagliari and Florida. Additionally, he is in charge of COMMIT/, a nation-wide, public-private ICT-research program of 100M. He is fellow of the International Association of Pattern Recognition, member of the Academia Europaea, associate editor of IJCV, recipient of the ACM SIGMM outstanding contribution award, and the Korteweg medallion. He is currently director of the Qualcomm - UvA and the Bosch - UvA labs. He was co-founder of Euvision Technologies BV, a company spin off from the UvA and eventually sold to Qualcomm.