The best time to go to Saudi Arabia is definitely after the summer months and before the winter. These are days from mid-April until October. The temperatures in these days average to 40 degrees Celsius, couple notches lower than summer temperatures. Try to avoid Ramadan months as this can be a very dangerous time. The country is most crowded and everyone is very busy.
For people who can only vacation on winter months, they can try staying in Jeddah, where temperature remains warmer than other cities.