Cognitive Radio has gained worldwide attention from research communities and is expected to be a revolutionary technology for the next generation (4G) wireless systems. In this dissertation, Amplify-and-Forward (AF) based relay-assisted cognitive radio networks (RCRNs) are studied in an underlay spectrum sharing environment. The primary issue faced by underlay networks is the limited transmit power ability of the secondary users (SUs) due to the interference constraints towards the primary users (PUs), which reduces secondary throughput and allows only short- range communication. Thus, performance enhancement of secondary communication in the frequency bands allocated to the PUs is a major design challenge faced by the underlay RCRNs. It requires relay selection along with the fine tuning and adjustment of the transmit power of the secondary relays. In this thesis, we proposed advanced multiple relay selection schemes for secondary network in the Rayleigh flat-fading scenario considering the availability of perfect instantaneous channel state information (CSI). The effects of variations in the instantaneous CSI, transmit power of source and relays, interference threshold of the primary network, signal-to-noise ratio (SNR) threshold of the secondary network and size of potential relay network on multiple relay selection in underlay RCRNs are the main issues that are analyzed in depth in this research. Furthermore, the performance analysis of multiple relay selection has been carried out and closed form expressions for the outage probability and average probability of error have been derived through the cumulative distributive function (CDF) of the received SNR at secondary destination, which is a new contribution to the AF-based underlay RCRNs. The optimization tool used in this study is Artificial Bee Colony (ABC) global optimizer. Another novel idea proposed in this dissertation is Fuzzy Rule Based System (FRBS) for multiple relay selection and transmit power allocation (RSTPA), which is a new contribution to the underlay RCRNs. The proposed FRBS assisted RSTPA schemes aim to perform intelligent multiple relay selection for performance enhancement of secondary communication in power constrained RCRNs. It is proved through simulations that FRBS is an optimal choice to solve the non-linear optimization problems of SNR maximization and transmit power minimization. Another contribution of this research is in the field of spectrum sensing in CRNs. Spectrum sensing faces a lot of challenges in terms of reliability and accuracy of information for detection and estimation of primary transmissions in CRNs. The advantages and limitations of different cooperative and non-cooperative spectrum sensing schemes have been studied in detail, and a novel spectrum sensing scheme based on uniform linear array (ULA) of sensors is proposed, which not only detects the number of sources, but also estimates their parameters such as frequency, Direction-of-Arrival (DOA) and power strength. The effectiveness and reliability of the proposed scheme is proved under low SNR conditions. Genetic Algorithm (GA) hybridized with Pattern Search (PS) is used to optimize the results. All the proposed algorithms have been investigated through simulations under different design requirements, constraints and a well-defined range of different parameters to validate their significance and effectiveness.