The image processing algorithm is using the methods of matching template images, masking with different range of BGR values and so on. All of this have been arranged as 'filter'. Opencv, an open source Python library is the core of completing the tasks in image processing. The prototype of image based solar tracker is made of polyform to save the cost. Image processing algorithm primarily using the method of converting an image or frame into HSV format and filtering the noises and unwanted objects to track the sun precisely. To know which direction the motor should be moving to, we separate the frame into 9 regions, each region represented one direction. Besides of direction predictions, the algorithm we have also included how to calculate the efficiency, and this is the main parts of our system. In addition to being a solar tracking system in and of itself, image processing algorithms can also be used to stack other solar tracking systems, such as active based solar tracking systems, with the goal of evaluating other solar tracking systems' efficiency. The efficiency is basically the displacement between the center location of the frame and the current sun location in the frame. Besides, I was planned to stack image processing algorithm into active based solar tracking system, so whenever the active based system finds the highest light intensity location, the image processing will wind up the rest to position the solar panel to the center of the sun in frame. The order of the active based and image based can be reversed, which might produces different results.