Neural networks have been gaining popularity in recent years as one of the most powerful machine learning toolsets to solve real life, practical problems. One type of neural network that has become the go-to choice of many machine learning practitioners for image-related problems is Convolutional Neural Network (CNNs). Given an image recognition problem, one may build their CNNs from the ground up and fine tune the parameters such that it can successfully achieve the required functionality to solve that problem. However, this requires a great amount of empirical observations and therefore takes up large costs in time and resources for training models. Another alternative to this would be to leverage the generalization of award winning CNNs trained on huge multi-class datasets and conduct transfer learning to specialize those models to the domain problem that require solutions. Our project will be investigating these two approaches, their pros and cons in solving the gender classification problem of in-the-wild human images.