This paper presents an algorithm for contrast enhancement of medical images since the same is highly desirable for better medical diagnosis. The current methods use local enhancement techniques but suffer with the drawback of high complexity and over enhancement of the input image. Over enhancement essentially adds some information which may lead to a wrong diagnosis and the cost can be human life. In this paper we are making a very careful use of Human Visual Characteristics (HVC) for enhancement of such images. This is done by first studying characteristics of medical images and then segmenting it using HVC into the image into several overlapping blocks. These segmented blocks are enhanced based on the characteristics of the medical images. We also propose an algorithm for enhancement of color medical images. For this we first decompose the color image into Hue, Saturation and Intensity (HSI) component and propose two different algorithms to be applied on S and I components. In order to validate the effectiveness of the proposed method we compared our results with conventional histogram equalization method and found the promising results. Copyright © 2011 ACTA Press.