How to obtain accurate travel time predictions is among the most critical problems in Intelligent Transportation Systems (ITS). Recent literature has shown the effectiveness of machine learning models on travel time forecasting problems. However, most of these models predict travel time in a point estimation manner, which is not suitable for real scenarios. Instead of a determined value, the travel time within a future time period is a distribution. Besides, they all use grid structure data to obtain the spatial dependency, which does not reflect the traffic network's actual topology. Hence, we propose GCGTTE to estimate the travel time in a distribution form with Graph Deep Learning and Generative Adversarial Network (GAN). We convert the data into a graph structure and use a Graph Neural Network (GNN) to build its spatial dependency. Furthermore, GCGTTE adopts GAN to approximate the real travel time distribution. We test the effectiveness of GCGTTE with other models on a real-world dataset. Thanks to the fine-grained spatial dependency modeling, GCGTTE outperforms the models that build models on a grid structure data significantly. Besides, we also compared the distribution approximation performance with DeepGTT, a Variational Inference-based model which had the state-of-the-art performance on travel time estimation. The result shows that GCGTTE outperforms DeepGTT on metrics and the distribution generated by GCGTTE is much closer to the original distribution.