Breast cancer (BC) is the highest cause of deaths in ladies around the globe. Woman are unaware in the remote and backward areas of under developed and developing states, that treatment of breast cancer is possible if it is found at an early stage. The casualties of BC can also be reduced, if demographic risk factors of female are evaluated a prior. Due to its nature of complexity, identifying breast irregularity through mammography and/or ultrasonography is a challenging job for radiologists. A more consistent and precise imaging based computer aided diagnosis (CAD) system assists in recognition of breast cancer at initial stage and play a noteworthy role in the classification of suspicious breast lesions. Ultrasonography of breast is acknowledged as the utmost significant support to mammography for patients with palpable masses and unsatisfying results of mammograms especially in case of young female. Therefore, a CAD system is required for breast ultrasound (BUS) images to distinguish malignant and benign cases. This dissertation has two main modules: the first one is CAD system and second one is the risk assessment of BC. In the proposed CAD framework, pre-processing is executed to remove the unwanted area and suppress the noise from the mammography and ultrasonography images. Then segmentation detects the lump in mammograms and BUS images using cascading of Fuzzy C-Means (FCM) and region-growing technique called FCMRG method and marker-controlled watershed transformation respectively. Hyrbrid features extraction technique employing local binary patterns and gray level cooccurance matrix (LBP-GLCM) along with local phase quantization (LPQ) is used for mammography to extract significant information from segmented masses. Morphological features of ultrasound breast lesion are designed to extract various statistical parameters from contour and shape properties. These features are then used to differentiate benign masses from malignant one using support vector machine (SVM), decision tree (DT), K nearest neighbors (KNN), linear discriminant analysis (LDA) and ensemble classifier. The goodness of the proposed CAD model is evaluated through performance measures on Mammographic Image Analysis Society (MIAS), Digital Database for Screening Mammography (DDSM) and Open Access Series of Breast Ultrasonic Data (OASBUD) datasets. The proposed CAD system achieved remarkable accuracy (=98.2%) with hybrid features on MIAS dataset and (=96%) with morphological features on transverse scan of OASBUD dataset. The proposed CAD system can also be implemented for the patients residing in the rural and backward areas to diagnose the scanned images of mammography and ultrasonography and to detect breast anomalies in the nonavailability of expert radiologists and weak cellular coverage. In second module, demographic risk factors of female have been employed to evaluate the risk grade (that is low, moderate, high) in a specific lady under investigation. For this purpose, Adaptive neuro fuzzy inference system (ANFIS) with sub-clustering and FCM is used and achieved high accuracy on the patient data gathered through questionnaire. The outputs of the CAD system can also be used to merge with demographic risk factors of the patients to find the future prediction of possibly occurring breast cancer risk.