Abstract

This research aims to realize a knowledge-based system recognizing teenagers’ behaviours in the lifestyle domain. Since various heterogeneous data sources are involved in this recognition process (e.g. wearable sensors, etc.), it is essential to face the harmonization of the data produced from these disparate sources and expressed under the form of both structured and unstructured content. The herein proposed approach leverages a common semantic reference model which represents the individual as a whole, permitting classification and integration of behavioural and contextual information. In this regard, an automatic semantic enrichment of the data produced by the involved sources is carried out, thus allowing to support a behaviour recognition process handled in a two-phase process (real-time and long-term detection and analysis of behaviours and trends).

Introduction

Presently we are witnessing a growing demand of customized services which are tailored to the specific needs and characteristics of users. A key success factor for the implementation of these services is the user profiling, which consists of creating an explicit representation of a person's identity, i.e., a conceptual understanding of the user [1] [2] that is commonly employed to enhance usability as well as to support personalization, adaptivity and other user-centric features [3] . In addition, as a user might be found in various contexts, it is essential to construct a conceptual representation of a user model which is context-aware, thus able to infer which context the user is in a given moment in time, and consequently adapts the intervention to the selected context.

This paper reports major features of an approach to support the process of user profiling which leverages the combination of Semantic Web technologies (SWT) [4] with the mobile and wearable technologies. The potential of this approach has been verified in the context of the European research project PEGASO ¹ [5] which aimed at promoting healthy lifestyle among teenagers. In particular, it provides personalized interventions based on the recognition of users’ behaviours with the objective of triggering an adjustable behaviour change process. In this regard, an ontology-based approach (Fig. 1) [6] is adopted to capture and cover the knowledge related with the so called Virtual Individual Model (VIM) [7] , characterizing the user on different aspects (i.e. physical, functional and behavioural parameters) [8] . Such a common reference model, which represents the individual as a whole, can contribute to harmonize data from different sources, providing in this way a systematic manner to classify and integrate the valuable knowledge. In fact, the common data model collects all the information and defines the reciprocal relationships between individual's different aspects, aggregating and unifying all the information. The definition of VIM stems from a conceptual framework of relations linking status, behaviours of the individual in different domains as well the context and the relation [9] .

This paper is structured as follows. Section 2 introduces the behaviour recognition in PEGASO project, while Section 3 illustrates the semantic approach followed in this project to support behaviour recognition. Finally, Section 4 draws the conclusions, summarizing the main outcomes.

**Figure 1.** PEGASO approach based on Semantic Web Technologies

The behaviour recognition in PEGASO

The data sources

Leveraging the VIM, a Behaviour Recognition System (BRS) has been implemented to conduct an individualized behaviour detection. The BRS is implemented as a two-dimensional system (short and long term) to be able to measure the two timescales considered in the project. In particular, the BRS is based on a mobile platform, where the smartphone is a key element that acts as the main contact point with the user. The mobile device also acts as communication gateway towards the sensors, running reasoning processes that allows inferring in real-time the necessary information to personalize the interventions delivered through the apps. This analysis is performed upon the available data extracted from various sources. Specifically:

Physical activity and sleep data is retrieved from available wearables (e.g. bracelet, etc.) and smartphone accelerometer.
Nutritional information is gathered through a mobile food app.
Context data includes different kind of information that involves users’ situation in a certain moment (e.g. day and weather information, user location).

Overview of the Knowledge Based System

The VIM ontology

The definition of a common reference model, which represents the individual as a whole, is proposed as a solution to harmonize data from the different heterogeneous sources, providing in this way a systematic manner to classify and integrate the valuable knowledge. The common data model defines the reciprocal relationship between the individual's different aspects, aggregating and unifying all this information. In particular, the definition of such virtual individual model stems from a conceptual model of relations linking status, behaviours of the individual in different domains as well the context surrounding the individual. This reference model is represented through a set of ontologies by adopting the SWT. An ontology based approach offers the significant key advantages of allowing to represent a formal semantics which contributes to enhance the interoperability of different software applications. In addition, it allows to entail and infer new knowledge about the concepts and their linking relationships, in addition to those initially already asserted. Finally, it allows to efficiently model and manage data to be distributed over the network, under the form of Linked Open Data (LOD) [10] . The latter is a proven way to publish and interlink public set of data in the scientific community (and also other fields). Considering the relevance of the open model, we could imagine another connection to the LOD diagram through a circle containing the VIM meta model which is paired with a data set of example.

The architecture

The proposed technological architecture is capable to handle and expose the VIM ontology (expressed in OWL DL ² ). It consists of two main pillars hosted on AWS cloud: the Semantic Repository (SR) and the Connector (Fig. 2). This latter represents the one single gateway, which play the role of Integration Service [11] , by transforming and combining structured and unstructured legacy data into valuable information according to the common data model behind VIM ontology. After this conversion process, the extracted information are also persisted and stored in a specific database (Semantic Repository). Specifically, the managed data comprise (Figure 3):

the domain ontology (TBOX), as representation of the knowledge about VIM;
the ontological population (ABOX), compliant to the domain ontology;
the derivation rules (RBOX) needed to properly entail the implicit knowledge and specified through the syntax of SWRL ³ .

The Connector, exposed as a RESTFUL web-service, is in charge of translating received HTTP requests into corresponding queries, formulated in SPARQL ⁴ which is the standard language to compose the semantic query. Thus, SPARQL queries play the role of mediator between the application-driven models and the ABOX, TBOX and RBOX of the VIM model (Fig. 3).

The Semantic Repository

One of the strengths of the SWT is that they allow to virtually express any type of information. The flip side is that this expressivity and flexibility imply the need to revise for SWT most part of the classical data management problems, including efficient storage and query optimization. Thus, as the request of semantic applications in real scenarios is continuously growing, the issue of efficiently storing and retrieving RDF data is getting more and more relevant. This need is posing significant challenges for researchers and technologists in terms of scalable and efficient stores. Under these conditions, a significant study of the PEGASO project has regarded the identification of a valid RDF store [12] , which finally brought to exploit an instance of the Stardog database ⁵ .

Interaction of the data sources with the Semantic Repository

BRS takes advantage of both mHealth (smartphone apps and wearables) and cloud solutions to perform a two-stage processing of the user-generated data at different timespans to provide different levels of feedback and user interaction. Collected data, which encompasses indicators of detectable target behaviours, is cleaned and then fused into daily aggregated summaries (24h timespan) that are finally stored in the SR deployed in the cloud for further knowledge management. Additionally, collected and inferred data in those 24h timespan is continuously analyzed to provide real-time response to detected events such as behaviour abnormalities.

On the other side, the PEGASO cloud consumes the inferred information from the semantic repository to build a longer timespan summary (1 month) to be further analyzed. This long-term analysis is in charge of identifying user behaviour trends and evaluate the user progression on the goals set in the platform. As a result, it provides behavioural feedback and recommendations on personalized behaviour change interventions.

**Figure 2.** Technological platform hosting VIM ontology

**Figure 3.** Ontology querying mechanism

Conclusions

This paper has introduced an approach to support teenager behaviour recognition which is based on a semantic architecture. The latter offers interesting advantage, in particular allowing to enhance the semantic interoperability of the data sources monitoring and collecting teenager data. In addition, the architecture provides a SPARQL endpoint, which is the preliminary step to expose the meta-model of the VIM as Linked Open Data.

Future developments will address three main goals. First of all, the VIM-based approach will be validated also with other categories of users, different from teenagers (e.g. elderly people or people with impairment). In fact, even the PEGASO approach has been conceived for teenager, VIM ontology can be tailored for other categories of users, while the semantic architecture is agnostic to the meta-model of the used semantic data. The second goal will concern the integration of other sensors to extend the functionalities of the already implemented knowledge based system. Finally, the proposed architecture will be extended to enable a new model of collaboration among the involved sensors and apps, which can support their synergistic and smarter interaction and cooperation [13] .

References

M. Sacco, E. G. Caldarola, G. Modoni, and W. Terkaj. Supporting the design of AAL through a SW integration framework: the D4All project. International Conference on Universal Access in Human-Computer Interaction. Springer International Publishing, 2014.
G. Fischer. User Modeling in Human-Computer Interaction. User Modeling and User-Adapted Interaction. 11: 65-68, 2001.
M. Golemati et al.. Creating an ontology for the user profile: Method and applications. Proceedings of the first RCIS conference, 2007.
T. Berners-Lee, J. Hendler and O. Lassila, "The Semantic Web", Scientific American, vol. 284, n. 5, 2001.
R. Guarneri, and G. Andreoni. PEGASO Fit for Future. European Project Space on Computational Intelligence. Knowledge Discovery and Systems Engineering for Health and Sports.
T. R. Gruber. A translation approach to portable ontology specifications. Knowledge acquisition, 5(2), 1993.
C. Lafortuna, J. C. Serrano, et al.. The building of a virtual individual model (VIM): multi domain characterisation of health status in the PEGASO project. Advances in Human Aspects of Healthcare, 2014.
F. Velickovski, S. Orte, et al.. Detection and Assessment of Behaviours Associated with the Risk of Obesity in Adolescents. In eHealth 360°. Springer International Publishing, 2017.
A. Sojic, W. Terkaj, et al.. Modularising ontology and designing inference patterns to personalise health condition assessment: the case of obesity. Journal of Biomedical Semantics, 7(1), 12, 2016.
C. Bizer, T. Heath, and T. Berners-Lee. Linked data-the story so far. Semantic Services, Interoperability and Web Applications: Emerging Concepts, 2009.
G. E. Modoni, M. Veniero, and M. Sacco. Semantic Knowledge Management and Integration Services for AAL Applications. Proceedings ForItAAL2016, Lecture Notes in Electrical Engineering, 2016.
G. E. Modoni, M. Sacco, and W. Terkaj. A survey of RDF store solutions. Proceedings of the 20th International Conference on Engineering, Technology and Innovation, 2014.
G.E. Modoni, M. Veniero, A. Trombetta, et al.. Semantic based events signaling for AAL systems. Journal of Ambient Intelligence and Humanized Computing, 2017.

http://www.w3.org/TR/owl-features/ ^[back]

A semantic approach to recognize behaviours in teenagers