Agriculture is the primary source of income in India. Paddy is grown almost everywhere in the world but is most common in Asian nations where it serves as the main source of food to world's population. Various diseases attack at different stages of plant growth. The biotic & abiotic stresses that affected plant growth are temperature, viruses, bacteria, fungi & various environmental issues. Brown spot, Sheath rot, bacterial blight and Leaf blast are all important paddy leaf diseases that destroy rice and drastically reduce yield. By using various image processing techniques farmers can identify leaf diseases. In this research paper by integrating CNN with edge detection mechanism paddy leaf disease cab be identified. Various images can be captured from farm using camera. These images include disease like brown spot, bacterial blight, blast diseases and sheath rot. During preprocessing RGB images can be converted into HSV images. Then various color and texture features have been extracted using GLCM. After this edge-based CNN have been applied to improve the accuracy of the model. To train the model 70% images have been categorized as training set, 20% images as testing set and remaining 10% have been considered for validation set. The accuracy of the proposed model is 98%.