GPS trajectories serve as a significant data source for travel mode identification along with the development of various GPS-enabled smart devices. However, such data directly integrate user private information, thus hindering users from sharing data with third parties. On the other hand, existing identification methods heavily depend on the respective manual travel mode annotations, whose production is economically inefficient and error-prone. In this paper, we propose a Semi-supervised Federated Learning (SSFL) framework that can accurately identify travel modes without using users' raw trajectories data or relying on notable data labels. Specifically, we propose a new identification model named convolutional neural network-gated recurrent unit model in SSFL to accurately infer travel modes from GPS trajectories. Second, we design a pseudo-labeling method for the clients to set pseudo-labels on their local unlabeled dataset by using a small public dataset at the server. Furthermore, we adopt a grouping-based aggregation scheme and a data flipping augmentation scheme, which can boost the convergence and performance of the proposed framework. Comprehensive evaluations on a real-world dataset show that SSFL outperforms centralized semi-supervised baselines and is robust to the non-independent and identically distributed data commonly seen in practice.