Human monitoring within health centers, residential care, and assisted living facilities, as well as nursing or retirement homes, is usually in charge of health care professionals. Frequently, they rely on surveillance cameras to identify the person's location and physical condition. However, this task requires a tedious continuous evaluation and demands the facilities' staff full attention. This task increases the attentional demand when working with patients with neurological conditions due to incremental risks related to sensory disturbances, cognitive and perceptual dysfunction, mobility, and communication disorders, visual, personality, and intellectual changes, and a complex range of speech and associated language disorders. In some cases, 24/7 monitoring is needed. The use of artificial intelligence shows a great performance for detecting objects and people by images or videos. This article aims at developing a methodology based on deep learning for the detection and monitoring of people in closed and open environments using video. The proposed method is non-invasive, low-cost, and evaluates a person's physical activity and inactivity in real-time. Preliminary results in public databases present outstanding results in the monitoring and estimation of caloric expenditure in indoor and outdoor environments.