Developing an Alternative Math Assessment Tool Using Speech Recognition

The world today is now in the era of Information Technology. The development of ICT-based processes specifically in the area of assessment in school is now visible. Project LISTEN (Literacy Innovation that Speech Technology ENables) is an inter-disciplinary research project at Carnegie Mellon University to develop a novel tool to improve literacy – an automated Reading Tutor that displays stories on a computer screen, and listens to children read aloud. This study does not provide right or wrong answers for they let the user evaluate the answer. The main objective of this study is to develop an Alternative Math Assessment Tool for Preschoolers using Speech Recognition. These software aims to assist teachers in the review of Math lessons for preschooler using speech recognition. The development of the system utilizes the System Development Cycle approach that includes data gathering to identify system’s expected functionalities, designing the system using Use-Case Diagram, integration of JSAPI for Voice Recognition, using Synthesizer software for reading the questions out loud, a graphical display of teacher representation and a graphical display for every questions in the review. Along in the development of this assessment tool is the implementation of the system. The system was developed using Java Programming language. It also uses MySql database to store data for preschooler, review questions and text answers. In the conduct of the review digital microphone and a speaker is needed. The developed system is capable of creating questions for a particular review, activating a review for the preschooler to take, and record the preschooler’s scores at every end of the review. The system also includes graphical display of questions. In the conduct of the review, the system was able to read out loud the questions, and a 5-second time span for the pupil to answer the review questions. The system will listen and the feedback from the study will display the correctly uttered answer. User testing results indicates an 83% correct response of system against the correct uttered answer of the preschooler.

Road Obstacle Detection

Autonomous vehicle systems can be divided into two main parts, the perception system and the decision-making system. In this project, the goal was to focus on the perception system’s subsystem which was an object detector for road obstacle detection on the roads of Karachi. To do this, firstly a dataset of 3000 images was annotated with bounding box annotation for 12 different kinds of road obstacles. These images were extracted frames (every 10th second) from about 10 hours of dashcam footage from different areas of Karachi. In parallel three different models were trained on the Berkeley Deep Drive Dataset (BDD100k), which were YOLOv3, RetinaNet and Faster R-CNN. Due to computational constraints the models were trained on only 5,000 out of 70,000 images and validated on 1,000 out of 10,000 images present in the BDD100k dataset. The models trained had the following mAP on BDD100k; YOLOv3(29.47), RetinaNet(37.34) and Faster R-CNN(35.78). These models were then used as pretrained models for transfer learning on Karachi Dataset to create three new models. The models trained on this new dataset had the following mAP; YOLOv3(41.67), Retina Net(67.26), Faster R-CNN(65.80). Analysis suggests that transfer learning using the BDD100k dataset, is the most optimum technique for training an object detection model on Karachi Dataset