Automated Weapon Detection in Banks and ATMs
[1]Harshad Nichat, [2]Prerana Bhole, [3]Samiksha Pandhare, [4]Sapna Parshuramkar , [4]Prof. Pranali Dhawas
[1]Student, [2]Student, [3]Student, [4]Student , [4]Professor
[1]Hars20042002@gmail.com, [2]bholeprerna@gmail.com, [3]samikshapandhare07@gmail.com, [4]sapnaparshuramkar52@gmail.com, [4]pranalidhawas10@gmail.com
Abstract- Every year, gun-related violence affects a significant amount of the global community. It has major impacts on psychology, public health, and economy. Early weapon detection allows us to respond quickly to stop violence. As a result, a completely automated computer-based system called Weapons Detection has been created to recognize common weapons and equipment, with a primary emphasis on pistols, sharp objects, rifles, etc. There is a need for an automated weapon detection system in banks and ATMs to strengthen security protocols and protect against possible risks. In process to rising issues about violent crimes and armed robberies in such situations, this research paper presents the development of an automatic weapon detection application for real-time surveillance, images, and videos. Our method uses advanced deep learning algorithms to reliably recognize weapons. The deep learning algorithm used is YOLOv8 to improve the speed and accuracy in detecting weapons and other potentially harmful items. The solution has been developed to function seamlessly with the current security framework. The key features such as image processing and object detection enhance robust systems which may encounter challenges like self-occlusion and similarities between object and background structure. This can be reduced by minimizing the false positive value.
Keywords- Deep Learning (DL), Computer Vision (CV), Weapon Detection, Threat Detection, Security, Image Processing.
I. Introduction
Today, public safety and security are of paramount importance especially in locations that have a possibility of an assault such as shopping malls, schools and financial institutions like banks and automated teller machines (ATMs). This has made it necessary to find imaginative ways to counter the dangers and protect public property since these places become targets for armed robberies and cases of assault happening on a daily basis. The traditional approaches used in enhancing the security level like cameras installation and hiring guards could not prevent these kinds of criminal activities. Hence, there is an increasing interest in using modern technology like object detection algorithms [24] which can enhance security in such situations.
Weapons are objects designed to hurt, harm, or injure others. There are many items in this class which include knives, firearms, and different contraptions. Protecting public safety and protection is a top concern for governments, regulation enforcement agencies, and groups at some stage in the sector because of the increase in criminal charges and converting nature of crook interest. The technique of locating and detecting weapons(such as guns, knives, etc) in a fine environment is weapon detection [15]. Several methods are used to carry out this work, together with optical inspection, sensor-primarily based systems, or the use of deep gaining knowledge of and contemporary imaginative and prescient algorithms. The foremost purpose of weapon detection is to identify capacity threats-including guns, knives, explosives, or different items [21].
The advancement in deep learning has seen constant change in algorithms, methods and architecture that have a great impact on efficiency and performance. Starting from convolution neural networks to development of recurrent neural networks architectures and more advanced algorithms such as long short-term memory (LSTM), generative adversarial network (GANs), etc. in terms of time series, complex structure and sequential data. This advancement in algorithms and techniques not only affected performance in deep learning but also in other fields like robotics, natural language processing, computer vision, etc. and it brings a new phase for Artificial Intelligence application and research. Many other fields of deep learning and computer vision include various detection, classification, recognition problems like object detection [12], image classification [20], image segmentation [14], object/image recognition, weapon classification,etc. In which image classification tends to identify or classify the categories or classes of an object in an image while object detection identifies as well as locates the object in the image. Weapon detection works on the principle of object detection [13]. Object detection, as discussed identifying and locating the object (like car, animal, people,etc). On the other hand, weapon detection specifically identifies weapons like guns [23], knives [6] and other harmful weapons.
In the past year, India has seen a worrying increase in criminal activity, including violent gun-related events and armed robberies. The percentage of cases of weapon-related crime in India shows variations in crime rates over time and across various regions and locations. Socio-economic factors, law enforcement efforts and cultural influences can all contribute to an increase in gun-related crime in a particular area. The crime rate of armed robberies at ATMs and banks varies from year to year, and law enforcement and financial organizations have shown concern over these incidents.
As of 2022, five states with the most number of Arm Act in India are: Uttar Pradesh, Madhya Pradesh, Rajasthan, Delhi and Bihar.
The data of Arm act from 2017-2020:
-In 2017, the total number of 58,053 cases registered under the arm act , of which 36,292 is the number of firearms used.
-In 2018, the total number of 66,305 cases registered under the arm act, of which 39,192 is the number of firearms used.
-In 2019, the total number of 73,122 cases registered under the arm act, of which 44,823 is the number of firearms used.
-In 2020, the total number of 68,463 cases registered under the arm act, of which 44,789 is the number of firearms used.
fig.1.0 Data of number of cases registered under Arm Act 2020
Some unsafe weapons have been used in public places in recent years. Starting with the last year in Jaipur ATM, on December 31, 2023, unidentified criminals stole thousands of rupees from an ATM. In the event, the cash inside the ATM was accessed by using a gas cutter to crack it open. The offenders managed to flee with a large sum of money around 29 Lakhs despite efforts to capture them. On Sept 19, 2023, a group of 6 to 7 unidentified robbers looted the bank. The total amount of looted money was 8.5 Cr in which 7 Cr were cash and 1.5 Cr was gold. In this incident the bank manager was hurt by a sharp object during the robbery. On December 30, 2021, at SBI bank Mumbai. One SBI bank employee unfortunately died after being shot during a heist. Armed attackers targeted the bank in the event, which resulted in violence and the deadly gunshot. The robbers were able to get away from the site before the authorities could capture them.
This research paper offers a thorough analysis and implementation of a weapon detection system designed especially for banks and Automated Teller Machines (ATMs) in response to this urgent concern. This system aims to deliver danger detection and warning by merging cutting-edge algorithms like deep learning and computer vision with the current security infrastructure and improves overall security measures and reduces risks to staff and people. Our application aims to system that employ advanced object detection techniques and algorithms instead of using traditional sensor-based techniques, to boost the efficiency, speed and accuracy.
II. Problem Statement/ Limitation
For security and safety of the public the identification of weapons is important.Despite advances in deep learning algorithms and advanced closed-circuit television (CCTV) systems, accurate identification of firearms remains difficult. This research paper aims to provide a effective weapon detection model which is based on deep learning method i.e. Yolov8. [15]
Some shortcomings of existing Weapon Detection methods are listed below.
False Positive and Negative: Due to similar presence of object(weapon or non-weapon) in an image and video which lead to increase in false positive and false negative rate.
Difficult practical situations: In real-world sighting conditions, reliable ammunition can be challenging to identify due to its complexity .
The main objectives are as follows to overcome the issues:
Implemented a weapons detection model with YOLOv8(You only look once), which performs faster and more accurately than YOLOv5, HoG (Histogram Of Oriented Gradients ), CNN (Convolution Neural Network) and other deep learning methods.
It produces a number of model parameters that need to be sent in order to operate quickly without compromising accuracy.
III. Literature Survey
The study [16] by Sanam et al in 2021, explored the enhancement and improved accuracy of Yolov3 compared to Yolov2 in weapon identification. Integrated the system with cctv to minimize the incident and criminal activity.
CCTV plays an important role at financial institutions such as banks and ATMs, to prevent crime and used for safety purposes and other visualization basically for object and human movement detection. Closed-circuit television(CCTV) is used to monitor, observe and record the activities around a certain area. It is used to capture events which can be used for future needs or requirements. The work of Jesus et al[19] designed a system that detects harmful weapons by using Deep Learning algorithms and object inclusion concept to reduce the false negative and false positive rate of detection in real-time. In this, they integrated the CCTV with the application which detects the weapon. The Yolov4 algorithm stands out the best among all other algorithms by giving the highest accuracy which is 91%. In 2022, author detect weapons in surveillance videos by enhancing deep learning algorithm i.e. Yolov4 which had an excellent accuracy of 92.1% and processed 185.7 frames per second. The use of GPU such as Jetson Nano for real-time CCTV weapon detection in and for high performance RTX2080TI GPU which resulted in improved mean average precision (mAP). The study examines the faster interface time and higher mAP scores compared to previous approaches.
In [17] shows the efficacy of Deep Learning in the area of weapon detection. The report identifies the Faster RCNN gives better performance. The comparison between two methods, the sliding window applied on the HoG with VGG-16 which gives speed 0.07 frame per second, not suitable for real-time object detection and 14 second per image and region approach applied on Faster RCNN with VGG-16 which gives speed 7 frame per second and 140 millisecond per image. Alarm activates when a true positive occurs. In reference to[10] the author, implemented the weapon detection using two deep learning methods, which are CNN based SSD and FasterRCNN. The study found that the Faster RCNN method gives better outcome than the SSD method with an accuracy of 84.6%, but it lacks speed 1.606 second per frame. The research highlights the effectiveness of Deep Learning for firearm detection in videos, cctv, etc. In the paper[11], Volkan et al presents a method for detecting and classifying weapons using streamlined deep learning algorithms. The model is implemented using VGGNet architecture, which gives efficient output and performance. The proposed model compared to other methods such as VGG-16 (89.75%), ResNet-50 (93.70%) and ResNet-101 (83.33%) achieve extremely high accuracy of 98.04%. The proposed methodology shows excellence in weapon detection and classification.
In [25] original work from 1997, introduced enhanced concealed weapon detection using image fusion by combining the feature-level fusion with the pixel-level fusion and utilizing wavelet transforms. Edge and region-based approach applied over the image, in which region-based approach gives clear and more accurate output. The 95GHz millimeter wave (MMW) shows the existence of the weapon. The study of fusion techniques to achieve advanced concealed weapon detection and works in every condition. [4] Rick et al (2004) presented the multiresolution mosaic approach combined with the visual and IR or MMW images through which concealed weapons are detected using k-mean clustering by featuring the weapons location. Efficiency of the model is showcased by the decomposition level, in which the two-level decomposition achieves better visuals. Furthermore, applying the algorithm to all images generates a synthesized result. In Smriti et al paper[1], the study focuses on the development and implementation of millimeter wave imaging radar systems for non-invasion and non destructive concealed weapon detection. Techniques used like discrete convolution, mean and standard deviation based thresholding, and canny edge detection for effective output in weapon detection. The author proposed the 60 GHZ active MMW (Millimeter Wave) image radar for detecting concealed weapons. This research represents the improvement in the security measures and advancement in the area of concealed weapon detection. In reference[8] to paper, an image processing method was introduced for weapon detection. The work on concealed weapon detection based on a new framework provides a robust system. Obtained efficient fused image by applying color image with IR image using fusion method for detecting weapon. The approach of converting IR image to HSV color space resulted in the most reliable, accurate and efficient framework for weapon detection. The study concluded that concealed weapon detection offers a more fruitful output than sensor methodology.
Collins et al[9] proposed a hybrid weapon detection system in 2019 which combines the metallic test and image test arm plus. The metallic test integrates more parameters to enhance the accuracy, minimizing the false alarm and efficient security system. In addition to the research, the author implemented the fuzzy logic system for effective detection and improving the decision making process. Significantly, the algorithm achieved the accuracy of 94.64% by image processing method. The review of [22] by Nodia et al implemented the weapon detection model with the advancement to existing model by the use of false alarm while detecting the weapons, particularly focusing on handgun detection. The primary goal of this model, to address the issue regarding false alarms and use it as the main source to train a deep autoencoder and model them. The result shows the significant decrease of false positive up-to 37.9%. The model achieves both high accuracy and low false positive rates using Faster RCNN mAp of 79.33%. Tufal et al [7] compared the model implemented using Yolov3 and Yolov4 for weapon detection in surveillance videos. They manually collected the data for the dataset from google images. The result shows the Yolov4 model outperformed the Yolov3 model in terms of processing time and sensitivity, with mAP of 77.36% (Yolov3) and 84.85% (Yolov4). On the other hand, Adual et al [3] worked on the framework based on Yolov5 with stochastic gradient descent and Gaussian blur to remove the background of the images. In case of performance Yolov5 provides a fast speed of 0.010 s/f and high recall rate of detection, compared to Faster RCNN’s 0.17 s/f. The author highlighted this by expanding the dataset to reduce the false positive and negative rate of weapon detection. [18] paper significantly focuses on detecting knives and pistols in surveillance videos. Muhammad used Yolov3 with Darknet, enhancement in the model by optimizing network balance by additional inclusion of fourth prediction layer and customize anchor box according to small object i.e for detection. The experiment shows that the Yolov3 gives good accuracy of 90.20% and fast detection speed 12.32 f/s. Even in low brightness and cluttered backgrounds, the model is able to identify objects.
Hence the literature review study concludes the advancement and techniques used in building weapon detection systems.
IV. Methodology
This section is divided into 4 parts, section 1 Data collection: The data to be collected for training the model, section 2 Processing: The process through which data undergoes, section 3 AI algorithm and model used : The algorithm, and techniques used for detecting weapons. It consists of methodology for weapon detection, the process through which the model undergoes to give the resultant output. Here is the flowchart of the process:
fig 4.0 Methodology of Weapon Detection
A. Data Collection
In our study, we get data from the internet for training, validation, and testing purposes in our project. We utilized the weapon detection dataset, which contained Handgun and Knife. We manually gathered data from websites like Google, GitHub, YouTube, and others to improve our model. Handguns (such as pistols and guns), sharp weapons (such as swords and knives), shotguns, rifles, etc. are all included in our dataset. After that, the labeling program was used for marking each image. With the labeIimg tool, we create a bounding box based on the weapon's position in the picture.[3]
B. Processing
We use LabelImg to label our data since it makes processing tasks like image orientation, scaling, and contrast adjustment easier. To maintain default proportions when augmenting with additional photos, our dataset is divided into three segments: Train, Valid, and Test. We identify probable quality problems in security camera photos and correct them by applying random blurring to mimic problems such as focus defects, camera distance, or motion of the subject. We also improve color, brightness, and rotation. By doing this, we can optimize our model's performance in weapon detection tasks by ensuring that our weapon detection dataset is reliable and realistic for actual situations.
C. AI Algorithms and Model Used
i. YOLOv8: An enhanced object detection method is called YOLOv8. It belongs to the group of real-time object detection techniques called YOLO[16]. Because of their reputation for accuracy and speed, these algorithms are often used in robots, autonomous cars, and surveillance systems. By combining advancements in training methods, design, and performance, YOLOv8 expands on its predecessors. To identify objects in pictures or video frames, deep convolutional neural networks are usually used. The capacity of these algorithms to concurrently estimate class probabilities and bounding boxes for several objects in an image during a single network pass is one of its primary characteristics.
Architecture
fig 4.1 YOLOv8 Architecture
The architecture of YOLOv8, is designed to efficiently detect objects in images or video frames with high accuracy and speed. Here's a simplified explanation of its architecture:
a] Input: YOLOv8 takes an image as input. The fed image's size is usually split into a grid.
b] Backbone: YOLOv8’s backbone network is responsible for extracting features from the input image. It is a deep convolutional neural network (CNN) such as Darknet-53 or CSPDarknet-53. This network is pre-trained on a large dataset (such as ImageNet) to identify some common features that can be useful for different computer vision tasks.
c] Neck: The Yolov8 neck is responsible for further refining the features extracted by the backbone network. It usually consists of operations like upsampling or concatenation of feature maps from different layers and additional convolutional layers. This helps to capture more abstract and context-rich information from the input image.
d] Dense Prediction: Dense prediction is used at several scales in Yolov8. It densely predicts class probabilities and bounding boxes across the whole input image. This indicates that YOLOv8 predicts numerous bounding boxes for each grid cell in the input image, together with the likelihood that each class will be present within those bounding boxes. Convolutional layers and a collection of anchor boxes in various sizes and shapes are used to accomplish this.
e] Sparse Prediction: In addition to dense predictions, YOLOv8 also makes sparse predictions at higher-resolution feature maps. This helps to capture finer details and smaller objects in the input image. These sparse predictions are typically made at multiple stages of the network, allowing YOLOv8 to detect objects of varying sizes and scales.
ii. Benefits of YOLOv8 over YOLOv5
fig. 4.2 Difference between the Model Size of Yolov5 and Yolov8
YOLOv8 brings some neat improvements over its older siblings that make it stand out in the world of object detection:
a) Sharper Eyes and Faster Reflexes: Think of YOLOv8 as the upgraded superhero in our object detection universe. It's faster and more precise at spotting objects in images or video frames compared to its predecessors.
b) Smarter Decision-Making: Just like a seasoned detective, YOLOv8 has learned from its past experiences. It's better at making sense of complex scenes, ensuring fewer mistakes in identifying objects, even in tricky situations.
c) Smoother Performance: It moves swiftly, like a seasoned athlete. It processes images in real-time, which is crucial for applications where every millisecond counts, like surveillance systems or autonomous vehicles.
d) Adaptable and Flexible: YOLOv8 is like a chameleon, effortlessly adjusting to different tasks and environments. Whether it's detecting people in crowded streets or items on store shelves, it's versatile enough to handle various scenarios with ease.
e) User-Friendly Interface: The method simplifies the whole detection process, just like upgrading to a sleeker smartphone. It streamlines the workflow, to use and integrate in the projects for developers and researchers
D. Evaluation Metrics
IoU: Intersection over Union (IoU) is a crucial metric that measures the degree of overlap between a predicted bounding box and the ground truth bounding box, providing a key assessment of object localization accuracy.
Average Precision (AP) gives one number that shows how well the model does in both being accurate and finding all the right items.
Mean Average Precision (mAP): Averaging the AP values across several object classes, mAP expands on the idea of AP. This is helpful to give a thorough assessment of the model's performance in multi-class object detection settings.
Precision and Recall: Precision allows to assess how effectively the model avoids false positives by calculating the proportion of real positives among positive predictions, whereas recall shows how effectively the model detects every occurrence of a particular type in an image.
F1 Score: By integrating accuracy and recall in a balanced manner and taking into consideration both false positives and false negatives, the F1 Score provides an accurate representation of a model's efficacy.
V. Result
We compared the performance of the model Yolov8 weapon detection with the yolov5 detection model. We trained the model by collecting the images and annotating them using a labeling tool and this annotated dataset shows the performance of the model in terms of speed and robustness. As a result, Yolov8 received better accuracy in identifying and localizing the weapon like handgun, knives and other weapons in image and video as compared to Yolov5. Overall, the implementation of our model demonstrates the advantages over the existing models, providing improved accuracy, robustness and performance for strengthening the safety and security measure.
VI. Conclusion
In conclusion, our implementation of the yolov8 model for weapon detection in images and videos has shown a significant improvement. Based on the strength of the previous yolo model for detection purposes, it demonstrates high performance and efficiency, the application is well suited for surveillance and security. In short, this system enhances the safety of banks and its staff and customers by detecting weapons and notify the authorities. The concept behind this is to guarantee a safe environment and banking experience while limiting the possibility of risks and illegal activities. It grows by investment and enhancement, strengthening the bank reputation and secure financial institute.
VII. Future Scope
Object detection models have made extensive improvements currently. Enhancing the overall performance of those models, particularly YOLOv8, is essential. Improvements may be made through growing the velocity of item detection and accuracy in identifying objects. Expanding the dataset with more images and classes can also boost the model’s capabilities. However, false positive or false negative can still occur, partly due to the limitations of the infrared (IR) image sensor. Future work could involve information fusion with additional sensors, such as a millimeter wave imager, to enhance the chances of accurate detection.
VIII. References
[1] S. Agrawal, B. Kumar “Non-invasive concealed weapon detection and identification using V band millimeter wave imaging radar system” in 2015 National Conference on Recent Advances in Electronics & Computer Engineering, IEEE.
[2] S. Ahmed, M. Bhatti and M. Khan “Development and optimization of deep learning models for weapon detection in surveillance videos” Applied science, 2022, vol. 22, issue.12.
[3] A. Ashraf, M.Imran “Weapons detection for security and video surveillance using cnn and YOLO-v5s” 2021, vol. 70, pp. 2761-2775.
[4] R.Blum, Z. Xue “Multisensor concealed weapon detection by using a multiresolution mosaic approach” in 60th Vehicular Technology Conference, IEEE, 2004, vol. 7, pp. 4597-4601.
[5] N. Dwivedi, D. Singh and D. Kushwaha “Weapon classification using deep learning convolution neural network” in 2019 IEEE Conference on Information and Communication Technology, pp. 1-5.
[6] M. Galab and A. Taha “Adaptive technique for brightness enhancement of automated knife detection in surveillance videos with deep learning” in Arabian Journal for Science and Engineering, 2021, vol. 46, pp. 4049-4058.
[7] T. Hashmi, N. Haq “Application of deep learning for weapon detection in surveillance videos” in 2021 international conference on digital futures and transformative technologies, pp. 1-6.
[8] N. Hussein and F. hu “An alternative method to discover concealed weapon detection using critical fusion image and infrared image” in International conference on Computer Communication and Internet, 2016, vol. 16.
[9] C. Ineneji and M. Kusaf “Hybrid weapon detection algorithm, using material test and fuzzy logic system”2019, vol. 78, pp. 437-448.
[10] H. Jain, A. Vikram, Mohana “Weapon detection using artificial intelligence and deep learning for security applications” in International Conference on Electronics and Sustainable Communication Systems, IEEE, 2020.
[11] V. Kaya, S. Tuncer, A. Baran “Detection and classification of different weapon types using deep learning” Applied science section of Robotic and Automation, 2021, vol. 11, issue 16.
[12] C. Liu, Y. Tao “Object detection based on YOLO network” in 2018 IEEE 4th information technology and mechatronics engineering conference, pp. 799-803.
[13] P. Malhotra and E. Garg “Object detection techniques: A comparison” in 2020 IEEE 7th International Conference on Smart Structures and Systems, pp. 1-4.
[14] S. Minaee, Y. Boykov “Image segmentation using deep learning: A survey” IEEE transaction on pattern analysis and machine intelligence, vol. 7, pp. 3523-3542, 2021.
[15] A. Nazeem, X. Bei “Real-time weapon detection in videos” in 11th International Conference on Pattern Recognition Applications and Methods, 2022, pp. 497-504.
[16] S. Narejo, B. Pandey “Weapon detection using YOLOv3 for smart surveillance system” Mathematical Problem in Engineering, 2021.
[17] R. Olmos, S. Tabik and F. Herrera “Automatic handgun detection alarm in videos using deep learning” Neurocomputing 275, 2017, pp. 66-72.
[18] M. Quyyum and M. Abdullah “Weapon detection in surveillance videos using deep neural networks” 2023, pp. 183-195.
[19] Ruiz-Santaquiteria, A. Velasco-Meto “Handgun detection using combined human pose and weapon appearance” IEEE, 2021, pp. 99.
[20]\W. Rawat and Z. Wang “Deep convolutional neural networks for image classification: A comprehensive review” Neural Computation, vol. 29, pp. 2352-2449, 2017.
[21] A. Toet “Color Image fusion for concealed weapon detection” in Sensors, and command, control, communications, and intelligence technologies for homeland defense and law enforcement II, 2003, pp. 372-379.
[22] N. Vallez, A. Velasco-Mata “Weapon detection for particular scenarios using deep learning” in 9th Iberian Conference, 2019, vol. 9, pp. 371-382.
[23] A. Warsi, M. Abdullah “Gun detection system using YOLOv3” in 2019 IEEE International Conference on Smart Instrumentation, Measurement and Application, pp. 1-4.
[24] Z. Zhao, P. Zheng “Object detection with deep learning: A review” IEEE transactions on neural networks and learning systems, vol. 11, pp. 3212-3232.
[25] Z. Zhang and R. Blum “Region-based image fusion scheme for concealed weapon detection” in 31st annual conference on information sciences and systems, pp. 168-173.
[26] T. Hande, P. Dhawas “Yoga Postures Correction and Estimation using OpenCV and VGG 19 Architecture ” International Journal of Innovative Science and Research Technology, vol. 8, issue 4, 2023.
[27] P. Kolhe, P. Dhawas “Document Analyser using Deep Learning” International Research Journal of Engineering and Technology, vol. 10, issue 1, 2023.
[28] P. Dhawas, M. Ramteke “Big Data Analysis Techniques: Data Preprocessing Techniques, Data Mining Techniques, Machine Learning Algorithm, Visualization” Big Data Analytics Techniques for Market Intelligence, 2024, pp. 183-208.
[29] S. Barse, D. Bhagat “Cyber-Trolling Detection System” SSRN, 2023.
[30] M. Agarwal, A. Kotecha “Deep Learning Approaches for Plant Disease Detection: A comparative Review” IEEE International Student’s Conference on Electrical, Electronics and Computer Science, pp. 1-6, 2023.