Portal de Periódicos da CAPES

Did Anyone Say BINGO: A Socially Assistive Robot to Promote Stimulating Recreational Activities at Long-Term Care Facilities

2013; ASM International; Volume: 7; Issue: 3 Linguagem: Inglês

10.1115/1.4024511

1932-619X

Wing-Yue Geoffrey Louie, Runqi Han, Goldie Nejat,

Context-Aware Activity Recognition Systems

One of greatest health threats to our aging population is cognitive decline, which can be a result of the natural aging process as well as more severe disorders such as dementia and hypertension [1]. Recent studies have demonstrated that non-pharmacological interventions such as participation in leisure activities, including general socialization, physical exercise, and/or cognitively stimulating activities (reading, playing board games and Bingo, playing musical instruments, doing crossword puzzles, etc.), are associated with a reduced risk of cognitive decline in the elderly [2]. For example, the group activity of Bingo has been shown to improve confrontational naming, memory, recall, and recognition in cognitively impaired older adults in adult day care centers [3]. Bingo also has physical and social benefits as it focuses on implementing accurate motor patterns when placing game pieces on the card and can also facilitate social interactions between the players themselves [3].However, the current state of long-term care facilities has shown that there is a need for increased staff-to-resident ratios to support and sustain recreational activities due to residents being understimulated [4]. Increased activity levels have been shown to reduce functional decline and mortality, and improve the moods of the elderly [5]. Although there exists a demand for more staff to incorporate such activities, it is projected that the supply of health care workers will substantially decrease as they too are aging and a large number of them will retire in the next several years [6]. Therefore, there is a need to investigate the use of technological solutions as alternative measures in order to improve the quality of care and quality of life of residents in long-term care facilities. For example, assistive robots can be integrated into long-term care facilities to assist with conducting and monitoring group recreational activities. Robot guided activities have already shown to aid in improving memory, social skills, cognitive attention, and the moods of the elderly, i.e., [7,8].Our work focuses on developing a unique autonomous socially assistive robot to facilitate group recreational activities such as Bingo in long-term care facilities to provide cognitive stimulation to the elderly, Figure 1(a). With respect to the game of Bingo, the socially assistive robot will monitor the actions of multiple elderly users to provide assistance when necessary or requested. In particular, the robot's assistive behaviors include: 1) calling out the Bingo numbers, 2) repeating to players, when needed, the numbers that have been already called out, 3) prompting players to mark the correct numbers on their card, and 4) verifying winning Bingo cards and celebrating with the Bingo winners. For the game, we have designed multiple number cards each marked with a unique identification symbol and a 5 × 5 grid of large easy to read numbers, Figure 1(b). Players mark the numbers that are called out by the robot by placing red circular markers on these numbers on their cards. The objective of the game is to obtain 5 consecutive numbers either in a row, column, or diagonal configuration in order to win the game and call out BINGO!A Creative 10MP webcam mounted on the head of the robot is used to monitor game progress and verify winning cards via a card identification and localization approach. The robot is being designed to provide game assistance by either: 1) actively navigating the game area and monitoring the Bingo cards of players, or 2) by directly responding to a player's raised hand. Each identification symbol on the cards has unique Speeded-Up Robust Features (SURF) [9], which can therefore be used by the robot to recognize a card of interest. Namely, the SURF feature vectors are determined on a captured image of the Bingo card and matched to a database which contains the SURF features for each of the identification symbols using a k-Nearest Neighbor algorithm [10]. A homography matrix between the database of symbol features and the features obtained on the cards (i.e., image features) is determined, and Random Sample Consensus (RANSAC) [11] is used to filter out outliers from the matched image features. The advantage of using SURF features are that they are both scale and rotation invariant, allowing the robot to identify the Bingo cards from different locations around the card. In order to localize a card, a perspective transform from the symbol in the database to the robot's view of the card is applied using the estimated homography matrix in order to localize a Bingo card (Fig. 2(a)). Once a card is identified, the grid lines surrounding the number squares are recognized using a Hough transformation [12] based method (Fig. 2(b)). In order to determine if a player has correctly marked each Bingo number, the location of each red circular marker is determined using color (red blob filter) and geometric (ellipse fitting) features. Marked Bingo squares can then be identified by determining the square closest to each circular marker using the centroid locations of the two (Fig. 2(c)). The robot can then actively prompt users to mark the correct numbers on a card if they have not been marked or have been marked incorrectly, or verify if a player has gotten Bingo.The robot is also able to recognize if a player raises his/her hand for assistance during the game using a KinectTM 3D sensor mounted along its chest and the KinectTM 3D skeleton model [13]. In particular, the robot is able to simultaneously identify hand-raising for a total of 6 players by tracking the skeletons of these individuals over 20 depth images using the following relationship:(1)Raised Hand={True,False,[(yw>yh)∨(yw>ye+a)]∧[‖xw−xh‖>b]where w, h, and e are the wrist, head and elbow joints of a player, and x and y are the world coordinates of these skeleton joints, respectively. a is the minimum length from the elbow joint to the wrist joint of adults and b represents the minimum distance the wrist should be from the head to be considered raised. A voting procedure is then applied over the last 20 frames to classify whether a player has raised his/her hand.Prior to a user study with elderly users, we conducted a performance study of the robot with 6 healthy adults playing multiple Bingo games. During the study, the robot called out the Bingo numbers, and the participants were asked to raise their hand when they had a winning card or needed assistance with the game. A Lenovo Intel® CoreTM i7 tablet computer with a 2.7 GHz CPU and 8.0 G of RAM was used onboard the robot for the Bingo games.Experimental results for both the card and raised hand monitoring systems are presented in Tables 1 and 2. Our results show high detection rates for identifying Bingo cards and marked number squares on a player's card. In addition, the robot was also successful at determining when a player had raised his/her hand.The overall goal of our research is to develop robots that can be used as assistants in long-term care facilities to provide cognitive stimulation via the facilitation of proven effective recreational activities. Such robots can improve the quality of life of elderly residents by making recreational activities more accessible. In this paper, we presented a socially assistive robot to be used to facilitate the group activity of Bingo. Our performance study results showed that our proposed robotic system can be designed to identify Bingo cards and monitor game activities such as the marking of appropriate numbers as well as identifying if players have raised their hands for assistance. These results are promising. Our future work will include developing additional robot behaviors for this activity and performing a pilot user study with elderly players.