Automatic detection of the angle and position of saffron flowers is an essential step for automatic processing of saffron flowers. Due to the diversity of saffron flowers, traditional algorithms can not yield acceptable accuracy. In this paper, an algorithm based on the deep convolutional networks is proposed. In the proposed architecture, the image is divided into a number of small square areas, and in each region, the existence of the flower is estimated. Also, the relative location suitable for cutting the saffron flower is estimated relative to the center of the region. To estimate the angle, the 360-degree range is first divided into several parts and the angle of the flower is classified to one of the values. Then, the relative angle with the center of the region is estimated. In order to evaluate the performance of the proposed algorithm, a dataset of 163 images and 3035 flowers has been collected, and the parameters for each flower are annotated by an expert. The evaluation of the proposed algorithm shows that more than 95% of the flowers are correctly estimated.