Multi-layer perceptron (MLP) is a classical example of neural networks, simple and provide us great understanding of the concepts in this field of machine learning. While MLPs work well for simple datasets and tasks, they become insufficient for more complex tasks such as computer vision and natural language processing. The former often required learning features of image, which can explode the number of parameters needed to train MLPs, while the later requires sequence processing that classical MLPs do not possess. Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are better choice for these two high level tasks. CNNs are the go-to method for predictive modelling of images as they can extract patterns, location invariantly, in images and understand the spatial relations in data well (pixels in images). CNNs might be used for text-based tasks and this will be shown in this project. RNNs on the other hand, are geared with persistence unit that allow good modeling of temporal relations, which gives it good performance in sequence predicting like NLP applications. In this assignment, the construction, and effects of various hyperparameters on the performance of these 2 types of network will be investigated.