Semantic embedding: scene image classification using scene-specific objects

Visual scene understanding is a hot and challenging topic in image processing that aims to understand the general (global) concept of a scene image. In this paper, we propose a novel image embedding algorithm using a learned embedded space, which introduces a high-level semantic representation of the scene images. The learned embedded space as a suitable semantic framework for visual concepts can be used in most applications such as image captioning, Visual Question Answering (VQA), and scene recognition or classification. Inspired by the human inference mechanism in visual scene understanding, the proposed method learns a semantic embedded space of visual concepts using prior semantic knowledge. Prior knowledge is extracted from ConceptNet as one of the most comprehensive knowledge graphs in the form of semantic vectors and is transformed to the learned embedded space with a transformation function. The transformation function is learned by solving a minimization problem. To evaluate our proposed approach, we introduce a scene image dataset called “Scene23”, which is based on the VisualGenome dataset. A non-linear SVM classifier is utilized to classify the representations of images to the scene categories. The experimental results show 99.44% classification accuracy on the “Scene23” dataset. Also, we evaluated our proposed method by the “UIUC Sports” and “MIT67” datasets. Experimental results indicate that our proposed method outperforms other state-of-the-art methods on the “UIUC Sports” dataset and achieves competitive results on the “MIT67” dataset.