In this paper, we define the freestyle object localization problem where a model is expected to autonomically learn and locate an arbitrarily specified object given only the name. However, learning the semantic features of an object requires corresponding training data which are not always available in existing datasets. Obviously, Internet provides massive off-the-rack data for learning given only the keyword while few existing methods are able to directly use them. To directly use them, a localization model should firstly overcome two difficulties: (1) learning from single-class annotated images and (2) from images with unpredictable sizes. For single-class annotation, we establish a novel probabilistic model to capture the distribution of the object in the background data space. Based on a deep hierarchical architecture with alterable depth that is designed for unpredictable sizes, we propose an association module and design an energy function to drive the probabilistic model. By maximizing the log-likelihood, the proposed model is able to highlight the semantic features of object that has the highest probability in training images, i.e., the specified object. In experiments, the proposed model achieves state-of-the-art performance among weakly-supervised localization methods on public datasets and freestyle localization of objects that are rare in public datasets.