The primary orientation of this thesis is to explore some interesting linear and nonlinear instabilities of low frequency waves in magnetized plasmas including the dust dynamics. These waves may be electrostatic or electromagnetic in nature. These investigations find significance and applications in small and large scale plasmas. This thesis contains several novel collective modes and instabilities in both classical and quantum plasmas. The low-frequency long wavelength electro- magnetic waves, viz., shear Alfv ́n waves in a cold dusty plasma, are examined e employing two-potential theory and plasma fluid model. It is observed that the presence of the unmagnetized dust particles and magnetized plasma components give rise to a new ion-dust lower hybrid cutoff frequency for the electromagnetic shear Alfv ́n wave propagation. e Subsequently, we consider the parametric instabilities for two regimes. First, the parametric decay instability of an Alfv ́n wave into low-frequency electro- e static dust-lower-hybrid and electromagnetic shear Alfv ́n waves in a classical e dusty magnetoplasma. Second, the parametric decay instability of a dust ion acoustic wave into low-frequency electrostatic dust-lower-hybrid and electromag- netic shear Alfv ́n waves is also investigated in detail in an inhomogeneous cold e quantum dusty plasma in the presence of an ambient uniform magnetic field. Magnetohydrodynamic fluid equations and the quantum magnetohydrodynamic model of plasmas with quantum effect arising through the Bohm potential and the Fermi degenerate pressure are employed in order to find the linear and nonlin- ear response of the plasma particles for three-wave nonlinear coupling in a dusty magnetoplasma. Here, the two cases are discussed, the first when the pump taken to be a relatively high frequency electromagnetic Alfv ́n wave and in the second e case the pump is an electrostatic dust ion acoustic wave. It couples with other two low-frequency internal possible modes of the dusty magnetoplasma, viz., the dust-lower-hybrid and shear Alfv ́n waves. The nonlinear dispersion relation of e the dust-lower-hybrid wave is solved to obtain the growth rate of the parametric decay instability. The growth rate of the dust-lower-hybrid wave is derived and solved graphically for different cases. Finally, the drift waves and their instabilities are investigated in detail in a nonuniform dusty magnetoplasma using the quantum hydrodynamic model of plasmas with the ion streaming. It is found that in the presence of a nonuniform ambient magnetic field, the drift waves grow in amplitude by taking energy from the streaming ions and density inhomogeneity.