Multiple kernel-based multimedia fusion for automated event detection from tweets

A method for detecting hot events such as wildfires is proposed. It uses visual information as well as textual information to improve detection. It starts with picking up tweets having texts and images. The data is then pre-processed to eliminate unwanted data and transform unstructured data into structured data. Then features are extracted. Text features include term frequency-inverse document frequency. Image features include histogram of oriented gradients, grey-level co-occurrence matrix, color histogram, and scale-invariant features transform. Next, text features and image features are input to the multiple kernel learning (MKL) for fusion which can automatically combine both feature types to achieve the best performance. Finally, event detection is done. The method was tested on Brisbane hailstorm 2014 and California wildfires 2017. It was compared with methods that used text only or images only. With the Brisbane hailstorm data, the proposed method achieved the best performance, with a fusion accuracy of 0.93, comparing to 0.89 with text only, and 0.85 with images only. With the California wildfires data, a similar performance was recorded. It has demonstrated that event detection in Twitter is enhanced and improved by combination of multiple features. It has delivered an accurate and effective event detection method for spreading awareness and organizing responses. The research presents a breakthrough in terms of risk management strategies, improving public health preparedness and leading to better disaster management.