Automatic driving technology is regarded as a powerful tool to solve traffic problems such as traffic accidents, congestion, and exhaust pollution. In this paper, we propose a new control strategy training method for self-driving vehicles, based on the deep reinforcement learning model. The method involves a Q-learning algorithm with filtered experience replay and pre-training with experiences from professional drivers, which accelerates the training process due to reduced exploration spaces. The method also involves resampling the state input after clustering, which improves the generalization ability of the strategy effectively, due to the individual and independent distribution of the samples. Experimental results demonstrate that the proposed model could reduce the time consumption of training by 92%, and the control stability increases by about 34%, compared with the existing neural fitted Q-iteration algorithm. In addition, procedure on resampling after clustering could increases the average travel distance by 73.4%, compared with the Q-learning algorithm with filtered experience replay that is based on the testing track which slightly more complicated than training set.