The overall goal of the project is the development of principled methods to link fundamental
sensing tasks using multiple modalities, and automated cognition regarding the
understanding of human behaviour in complex indoor environments, at both individual and
collective levels. Given the two above principles, the consortium will conduct research on
three core scientific and technological objectives:
- sensor modelling and information fusion from multiple, heterogeneous perceptual
- modelling, localization, and tracking of multiple people
- modelling, recognition, and short-term prediction of continuous complex human
Overall strategy and general description
Prometheus is a project scheduled for duration of 36 months, and involves 7 partners from 5 countries. The main outcome of the project will be a concrete framework that allows the interpretation of human behavior based purely on the integration of sensors’ information and statistical models of human appearance, motion and behavior. The work in PROMETHEUS is subdivided into 8 work packages (WP1-8), including project management activities, research and technological development, demonstration and innovation-related activities, which are organized in a modular way. They will be carried out both in a parallel and
sequential way. The work packages and sub-sequential sub-tasks are as follows:
WP1: Project management
WP2: Scenarios definition, data collection and annotation
o Task 2.1: Requirements analysis and scenarios description
o Task 2.2: Design specification of hardware components
o Task 2.3: System set-up and data collection
WP3: Sensor modeling and multi-sensor fusion techniques
o Task 3.1: Sensors’ error patterns and uncertainty modeling
o Task 3.2: Sensor online adaptation and self-calibration
o Task 3.3: Bayesian network structures for multi-modal fusion
WP4: Multi-person detection, localization and tracking
o Task 4.1: Derivation of human abstractions
o Task 4.2: Motion models and multi-person tracking under occlusion
o Task 4.3: Online adaptation and learning
WP5: Behavior modeling
o Task 5.1: Particle filtering techniques applied to the learning process of Bayesian network structures
o Task 5.2: Learning and recognition of human action & interaction patterns
o Task 5.3: Short time prediction of human intention
WP6: System integration and optimization
o Task 6.1: System integration – Initial demonstrator
o Task 6.2: System optimization – Final demonstrator
WP7: System assessment & evaluation
o Task 7.1: System component assessment
o Task 7.2: Integral system assessment
o Task 7.3: Project evaluation
WP8: Dissemination, exploitation and standardization
o Task 8.1:Dissemination
o Task 8.2: Exploitation & patent filling
WP1 will run throughout the duration of the project. Among the 5 technical work packages (WP2-WP6), WP2 will form the basis for the development work to be carried out during the project by defining the data collection process and providing the multi-modal data, while WP3-WP5 are the work packages on which most of the development work will be focused. WP6 will build upon the results of the previous technical WPs. WP7 will start early in the project lifetime and will last throughout the whole project.
PROMETHEUS contributes to consolidate interdisciplinary research currently spread across various communities, including signal processing, computer vision, and machine learning. In addition, towards the end of the project, PROMETHEUS will take part to relevant international workshops, contributing to sessions on multi-sensor networks and multi-person behavior modeling, associated to one of the leading conferences in which the consortium partners regularly participate (ICASSP, ICCV, ECCV, NIPS). More specifically, the dissemination of theoretical results will take place through publications at competitive
journals, workshops and conferences.
Specifically we will publish results in prestigious conference as:
• International Conference on Multimodal Interfaces, ICMI (systems that are designed
to recognize events,*
activities, and their relationships in interaction scenarios)
• IEEE International Conference on Acoustic Speech and Signal Processing, ICASSP
(signal processing techniques as applied to tracking)
• International Conference on Signal Processing Proceedings, ICSP, 2006.( speaker
• International Conference on Information Fusion (information fusion of heterogeneous
Research results are expected to appear one year after the beginning of the project.
Publicity / confidentiality strategy
Given the security constraints, special care will be given to managing the knowledge and IPR issues. Generated knowledge of commercial interest will be safeguarded and protected for exploitation by the owner, i.e., the organisation investing into its research. On the other hand, the partners of this project have come together in order to collaborate and benefit from their respective resources and competencies. Thus, added value through sharing knowledge and promoting exploitation are the project’s clear objectives and driving forces. The two aspects described above lead to the principle that generated knowledge is made available to the partners, to the extent needed, in order to fulfil their technical work in the required level of quality. All partners will be ensured from the beginning of the project that their own background information (pre-existing know-how) to be used in the project, is identified and recognised by the other participants up front. The background information will relate to information developed before the starting of the project, whether it is patented or not, secret or not, as well as to results obtained outside the project after it has started, i.e., in parallel to it.
The exploitation of knowledge will involve all activities related with the protection of the foreground and the plans for its use. In view of the partners’ specific legitimate interests, intellectual property protection (including patent searches, filing of patent or other IPR applications, etc.) will be undertaken to safeguard the efficient use of results.
Any knowledge developed in the project that has potential industrial and/or commercial applications will be protected with due regard to the legitimate interests of the partners concerned. It is expected though, that there will be cases when journal publication or other means of putting knowledge in the public domain will constitute appropriate alternatives, taking into account the nature of the concerned results and the interests of the participants. This approach to knowledge and the IPR management will be regulated in detail in the
Consortium Agreement and will be signed by all partners before the project commences.
Some of the aspects covered include:
- Confidentiality: Each partner will treat information from other partners as confidential and will not disclose it to third parties, unless it is obvious that the information is already publicly available.
- Ownership of Knowledge: Knowledge is owned by the partners who carried out the work, generating the knowledge, or on whose behalf such work was carried out. If a partner wishes to assign any knowledge to a third party, he should inform the other partners and request their consent.
- Patents: Partners who own patentable knowledge may (and are encouraged to) at their own expense, make application for patent or a similar form of protection and shall supply details of such an application to the other partners.
- Access Rights: Partners grant themselves royalty-free access rights to any knowledge, generated in the project, to the extent needed for the project to be successfully performed. Access rights to a partner’s pre-existing knowledge that is needed for use outside the project, and only to the extent necessary for him/her to make use of the project results, shall be given on preferential conditions to the other
The consortium is composed of academic partners, research institutes and industrial companies providing highly complementary input with respect to the system set-up, evaluation, and exploitation of the results obtained. The Coordinator of the project is Swedish Defence Research Agency (FOI), which has demonstrated experience in security and surveillance systems for many years.
The PROMETHEUS project is characterized by a balanced consortium in terms of:
a) Actors: 7 organizations stemming from the public academic and/or research sector and the private industrial sector. The project sortium consists of:
i. Academic/educational technology providers (UOP, TUM, FCTUC, TEIC), that are interested in integrating their research achievements so-far in a concrete project and to also further their research on topics that match the aims of PROMETHEUS.
ii. A research institute (FOI) that is activated in research, method and technology development, as well as studies for the use of defence, security and safety. The institute proposes problem solving based on new knowledge and new technology and provides its customers with leading expertise in a large number of fields of application through in-house competence and wide-ranging networks.
iii. Industrial partners (MARAC, PROBAYES), that are strongly interested in high-level autonomous cognitive systems based on emerging technologies, capable of performing in demanding environments. Especially there is a vital concern in the area of sensor networks, monitoring, multi-person detection, localization and tracking.
b) Experience: in various and complementing fields including multi-sensor management and
tracking (FOI), modeling and tracking articulated motion using multiple cameras (TUM, FCTUC), signal processing using Bayesian analysis with emphasis on Markov Chain Monte Carlo techniques (PROBAYES), machine learning techniques, acoustic surveillance and audio processing in general (TEIC, UOP).
c) Geographical distribution: developments and validation are distributed in Greece, Germany, Portugal, France and Sweden.
All of the consortium members have worked successfully in past and on-going research projects which provide solid collaboration grounds and guarantees the high cohesion and
commitment of the whole working team for this project as well.