In the data analytics world, data has been proliferated in different ways. Multimedia data is one of such data that has been recently generated in huge volumes in the Internet. It is being captured by various multimedia computing devices like computers, tablets, mobile phones, and cameras. The nature of the data being captured is also ubiquitous. Initially, started with audio, video it has now reached animations in the form of gifs. The advances in multimedia data are summarized as follows.
• Visual data:
The most common form of multimedia data found is the video data. A visual data consists of a sequence of image sequences that has to analyze one by one. Most of the unstructured information exists in the form of visual data and contains very rich information. Visual data analytics process involves extracting meaningful information from the different image sequences that are present in the visual data. However, the main challenge lies in the size of the visual data. Recent technologies such as cloud computing, high performance computing have enabled the visual data and analytics research in areas such as video surveillance systems, autonomous systems, and healthcare. The advances in visual data and analytics are challenging the human brain and its computation. In one of the competitions held, machine outperformed the humans in image classification.
• Audio data:
One more type of multimedia data that is mostly used is audio/speech data. Real-time audio analytical applications are needed in social media, healthcare, and industry. Audio analytics involves extracting useful information from the different pieces of the information present in the audio data. Call centers are one of industry applications that need audio analytics for interaction with the customer and training the persons involved in the call center. Big data platforms such as Spark, Hadoop, and different libraries are widely used for such audio analytical applications.
• Text data:
Multimedia data may be embedded in the textual context in the form of web pages, surveys, feeds, and metadata. The analysis of such data helps to gain interesting insights. Multimedia data in the form of text can be structured or unstructured. The structured kind of data can be analyzed with the help of traditional relational database techniques of query retrieval. However, the multimedia data in the form of feeds are unstructured and needs to be transformed into a structured format for further analysis. One of the example applications of multimedia text analytics is based on a particular situation like election, natural disaster, stock market, etc. The emotions behind the text can be analyzed with the help of multimedia data analysis. In this way, different kinds of information can be extracted from different sources of multimedia textual data.
• Sensor data:
IoT is playing a significant role nowadays and sensors are present almost everywhere. The sensors are equipped with not only capturing the data but also apply analytics in real time. With the advances in the hardware and the technologies of cloud computing, sensor data are increasing enormously. It is highly challenging to analyze such data and develop an analytical application based on that. Sensor data applications are highly seen in astronomical sciences for meteorological patterns, satellite conditioning, and monitoring. In healthcare, most of the applications are based on sensor data. Therefore, with the advances in the sensor data, it is highly essential to develop analytical applications based on it.
• Social networks:
The main source for multimedia data is social networks. The advances in social networking and sharing that started from a normal text have now reached to image, video, live video, public groups, etc. Recommendation applications widely use the multimedia content available in the social networks to provide recommendations by analyzing the shared messages, video, audio, and text. Personalization services are more widely used by the users of smartphones based on the subscriptions made by them. The main characteristic nature of the multimedia data is variety. It exists in different forms and at various sources. These advances in multimedia data and analytics have enabled various challenges and technologies for developing different applications.
Though, there are various technologies that exist for multimedia analytics the challenges that are put forth for analysis are more and have to be addressed carefully.
Literature; Multimedia Big Data Computing for IoT Applications, Concepts, Paradigms and Solutions, Editors, Sudeep Tanwar, Sudhanshu Tyagi, Neeraj Kumar, 2020