Interactive Image Mining Annie Morin 1, Nguyen-Khang Pham 1,2 TEXMEX/IRISA. Cantho University, Vietnam

Size: px

Start display at page:

Download "Interactive Image Mining Annie Morin 1, Nguyen-Khang Pham 1,2 TEXMEX/IRISA. Cantho University, Vietnam"

Marcia Crawford
5 years ago
Views:

1 Interactive Image Mining Annie Morin 1, Nguyen-Khang Pham 1,2 1 TEXMEX/IRISA 2 Cantho University, Vietnam

2 Outline Image retrieval and motivation Examples Image topic discovery FCA and adaptation on images Image retrieval Image indexing using indicators of FCA Numerical results Conclusion and future works CARME

3 Image retrieval Image query Sorted list of images Image retrieval System Decreasing similarity Image database CARME

4 Starting Point: Video Google - Interest point detection - SIFT computation Hessian - Affine detector [Mikolaczyk & Schmid, 2004] SIFT computation [Lowe, 2004] Quantification SIFT R 128 Bag of visual words SIFT K-means clustering visual words CARME Video Google [Sivic & Zisserman, 2003]

5 SIFT descriptor R 128 SIFT = Scalable Invariant Feature Transform 4 x 4 x 8 (directions) = 128 (dimensions) CARME

6 Visual words SIFT Visual words R 128 R 128 Clustering (for inst.., k-means) CARME

7 Contingency table word 1 word 2 word 3 word N image 1 image 2 image M CARME

8 Data Analysis Techniques Used Video Google tf*idf weighting Probabilistic Latent Semantic Analysis (Sivic et al., 2005) Our goal Replace these techniques by Factorial Correspondence Analysis (FCA) CARME

9 Motivation Success of application of Factorial Correspondence Analysis on textual data CARME

10 Some examples : the Caltech4 database Caltech4 database 4090 images divided into 5 categories faces : 435 motorbikes : 800 airplanes : 800 cars (rear): 1155 backgrounds : 900 Vocabulary : 2224 visual words (Sivic et al., 2005) CARME

11 Caltech4 dataset CARME

12 Factorial Correspondence Analysis CARME

13 Factorial Correspondence analysis images visual words Image with all its visual words Characteristic visual words for this image in this plane CARME

14 Cars CARME

15 Motorbikes CARME

16 Faces CARME

17 Airplanes CARME

18 Backgrounds CARME

19 Image information extraction Red:neg Blue:pos CARME

20 Example : Categorization of images Alogic database (961 images, 1000 visual words) CARME

21 Categorization of images CARME

22 Numerical results: databases Caltech4 database 4090 images divided into 5 categories Vocabulary : 2224 visual words Nister database 2250 groups of 4 images images in total 5000 visual words CARME

23 Nistér dataset CARME

24 Experimental results : methods comparison CARME

25 Experimental results Nistér-Stewénius Exhaustive search CARME

26 Speeding up the retrieval by using image categorization One factorial axis two different topics One image may belong to several topics Topics determination: Contribution to the inertia of an axis Quality on an axis CARME

27 Image Indexing using FCA Hypothesis: Two similar images share some common properties Indicators of FCA have relevant properties Representation quality of images on axes An image is well represented on some axes Contribution to the inertia of axes An image highly contributes to the inertia of some axes CARME

28 Image Indexing using FCA Inverted file property Contains images which possess this property For an axis we construct two inverted files: Negative part/positive part Each file contains images well represented on this axis (or images which highly contribute to the inertia of this axis) CARME

29 Image Indexing using FCA Retrieval schema For a given image query Sort axes by their representation quality (contribution to the inertia) Determine relevant properties of the query (i.e. some first axes) and take the correspondent inverted files Merge the inverted files by majority vote candidate list Search in the candidate list CARME

30 Inverted files axis α Topics F 1 + F 1 - F 2 + F 2 - image Topics determination image 2 image image i z iα image image Inverted files Coordinates of the images in the factorial space CARME

31 Experimental results Approximative search with inverted files Method 1: Inverted files based on the contribution Method 2: Inverted files based on the quality of representation Exhaustive search CARME

32 FCA with indexing technique Precision and time results Methods #images 5 img 10 img 50 img 100 img Times (ms) FCA, exhausted search Representation quality-based indexing Contributionbased indexing Tf*IDF, exhausted search Caltech4 dataset - Number of axes kept: 15 - #images: size of candidate list times faster than FCA without indexing - 20 times faster than TF*IDF CARME

33 FCA with indexing technique Methods #images precision Times (ms) FCA, exhausted search FCA with indexing nf= 21, nf_thres = 11 FCA with indexing nf: auto, nf_thres: auto TF*IDF, exhausted search Nistér dataset #image: size of candidate size precision: precision at 4 first returned images - 10 times faster than FCA without indexing - 30 times faster than TF*IDF CARME

34 Results Methods Inverted files based on the contribution Inverted files based on the quality of représentation Acceleration gain min max mean CARME

Parallelization of the filtering step Filtering of non relevant images Use inverted files to filter the images sharing few topics with the query Refinement step Sequential

35 Parallelization of the filtering step Filtering of non relevant images Use inverted files to filter the images sharing few topics with the query Refinement step Sequential search among the images candidates Most of the time is for filtering Computation time distribution for filtering step and refinement step for 1 million images CARME

36 Results with Nister-Stevenius database +1 million images from FlickR Methods Response time (ms) Acceleration gain Exhaustive Filtering on CPU Filtering on GPU Method with a parallel filtering on GPU 10 times faster than the method without parallelization of the filtering step 100 times faster than the exhaustive search CARME

37 Conclusion Adaptation of FCA on images for indexing and retrieval Hierarchical CA to focus on some areas of a factorial plane CARME

38 Work of J.B Sivic and A Zisserman using probabilistic latent semantic analysis and visual words in images CARME

Comparables Sales Price (Old Version)

Chapter 486 Comparables Sales Price (Old Version) Introduction Appraisers often estimate the market value (current sales price) of a subject property from a group of comparable properties that have recently