In recent years, researchers have actively proposed tools to automate testing for Android applications. Their techniques, however, still encounter major difficulties. First is the difficulty of achieving high code coverage because applications usually have a large number of possible combinations of operations and transitions, which makes testing all possible scenarios time-consuming and ineffective for large systems. Second is the difficulty of achieving a wide range of application functionalities, because some functionalities can only be reached through a specific sequence of events. Therefore they are tested less often in random testing. Facing these problems, we apply a reinforcement learning algorithm called Q-learning to take advantage of both random and model-based testing. A Q-learning agent interacts with the Android application, builds a behavioral model gradually and generates test cases based on the model. The agent explores the application in an optimal way that reveals as much functionalities of the application as possible. The exploration using Q-learning improves code coverage in comparison to random and model-based testing and is able to detect faults in applications under test.