Location information is very important in the real world. How to detect users’ locations and further movement trajectories automatically is significant for many location-based services such as dietary recommendation and tourism planning. With the rapid development of social media such as Sina Weibo and Twitter, more and more people publish posts at any time which contain their real-time location information. This makes it possible to detect users’ movement trajectories automatically by social media. In this paper, we propose a novel method to detect city-level locations and further identify movement trajectory for a given user in social media. Our efforts are made in two phases: (1) location detection, and (2) movement trajectory identification. Phase (1) combines two components, i.e. a Bayes model based on words distribution over locations and the social relationships of users in social media, to detect or infer users’ locations. The former one is utilized to judge whether the content of a post contains explicit/implicit location information and further to detect the location. The latter one assumes that friends tend to gather together when they talk about the same thing. Thus, it infers a user’s locations from his friends’ detected locations in social media. Phase (2) constructs the user’s movement trajectory by smoothing these detected locations in first phase according to both semantic context of posts and transfer time between locations. Experiments on real dataset from Sina Weibo demonstrate that our approach can outperform baseline methods significantly in terms of Precision, Recall, F1, Error Distance (ErrDist), Detect Ratio (DeteRatio), T-Precision, T-Recall and T-F1, respectively.