The most paramount step before training any convolutional neural network is the acquisition of a dataset. A dataset may consist of images which belong to several classes of interest. The dataset used for this project is a custom dataset with 12 classes of unwanted objects in a baggage, the dataset is known as Prohibited Item Detection Dataset (PIDray). For dataset annotation, a platform known as RoboFlow is used to annotate the images. Since the PIDray had 12 classes of object, every class was differentiated by a set of colors; red for the gun class, blue for the bullet class, yellow for the knife class, aqua for the pliers class, light brown for the power bank class, violet for the baton class, green for the lighter class, dark green for the sprayer class, sky blue for the hammer class, dark brown for the handcuffs class and a lighter shade of green for scissors. The annotations were saved in the YOLO format and exported to Google Colab for model training. The PIDray dataset which consists of 47, 677 images are used for training the models. All the images in the dataset are resized to 416 × 416 as an image pre-processing step to help improve the performance metrics of the module. The development of the neural network is firstly trained with optimized hyperparameters and an incremental epoch of 50, 100, 150, and 200. The second part of the development phase is deploying the trained neural network locally. The performance metrics used in evaluating the model performance involves the precision, recall, f1-score, mAP@0.5, and mAP@0.5:0.95. Google Colab is used for the training phase of the model and for the deployment process is performed by using PyCharm, command prompt and the local computer camera.Upon completion of detailed comparison between YOLOv3 and YOLOv5, it is safe and sound to conclude that YOLOv5 is the best fit for x-ray baggage object detection using neural networks for safety purposes. This is a reason being that YOLOv5 has the highest recorded performance metrics (precision, recall, f1-score, mAP@0.5, and mAP@0.5:0.95): 90.4%, 84.6%, 87.4%, 90% and 66.7% respectively. Furthermore, the testing phase of the model proofs to be flawless: the proposed model can detect accurately all objects of all the 12 classes in the PIDray dataset and detect objects when they are clustered on top of each other. To conclude, the YOLOv5 model was deployed locally on a computer to run real time, and the inference speed was recorded to be ±97.8ms.