the activities

What happened last year?

Stemming from the Partners activities carried out in 2009 and early 2010, on the basis of the outcomes of 2008, some main results display a feasibility of achieving the final Jumas system.

# JUMAS system development: starting from the design developed (user requirements and system design @wp1), the system architecture for Italy and Poland has been confirmed and development of SW modules performed @wp5

# Implementation of the enhanced courtroom module based on the outcomes of WP2 (Research on Audio and speech recognition and deception) and WP3 (video processing), with new acquisition chain to cope the requirements of audio and video processing algorithms.

# Implementation of the baseline for the Automatic Speech Recognition, and Video Analysis engines to the judicial domain through collection of audio-video trial documents and their analysis.

# Implementation of several modules coming from R&D activities.

# R&D activities about Automatic Speech Recognition, Deception detection and Emotional state recognition on audio recordings from criminal trials and definition of the audio acquisition chain.

# Implementation of the components for multimedia and semantic information retrieval: semantic annotation, template filling, ontology, retrieval components, multimedia summarisation, user based annotation @wp4

# Definition and specification of integrated prototype @wp5

# Definition of the validation plan @wp6

# Analysis of audio-video recordings from criminal trials (all WPs).

# Dissemination activities in conferences and organisation of an international workshop in Skopje (ICT4JUSTICE 2009) @wp7

# Redefinition of the Consortium activities due to the partner joining the project @wp8.

WP1 user & system requirements

objectives. JUMAS envisages a system for the embedded semantic extraction from multimedia data that join into an advanced knowledge management system. Moreover JUMAS is tailored at managing those situations in which multiple cameras and audio-sources are used to record assemblies in which people debate and event sequences need to semantically reconstruct for future consultation.

spotlight. JUMAS system aims to serve the categories of users identified as Judges, Court Clerks, Prosecutors, Lawyers, IT consultants. The composition of the Group of Interest is heterogeneous, according to the following properties: Country and territorial jurisdiction; Role of the interlocutor, inside his organisation and in respect to the proper subject matters (i.e. judicial jurisdiction); Dimension (in terms of annual amount of hearings and trials) of the area of activity; Previous experiences with advanced IT solutions for Justice.

A specific "User Needs and Requirements Questionnaire" was elaborated in order to survey current requirements and practices related to the integration of new IT systems within Criminal Law Courts. The Questionnaire was submitted to technical and non-technical interlocutors, identified as JUMAS potential users. Download it on the right here.

objectives. The objective is to improve the performance of present automatic transcription technology in the specific application domains. To do this suitable language and acoustic models will have to be trained in order to reduce the effects on the word error rate of phenomena such as: spontaneous speech, noise, distortions of various types (room echo, far microphones, etc). A specific task is devoted to identify problems and to set the baseline for successive improvements. Other activities address to deception recognition and classification of the emotional state of the speaker.

spotlight. Definition and implementation of the audio baselines for Italy and Poland. First enhancement in terms of performances of ASR and emotional annotation component. Issue of deliverable D.2.2. WP2 research activities are carried out for improving the performance of Automatic Speech Recognition (ASR), emotion state annotation and deception detection in the judicial domain. While in deliverable D.2.1 the main purpose is to set up some baselines and benchmarks for the audio processing in the judicial domain, the work reported in D.2.2 mostly focuses on the development and test of models and algorithms, that allow improving the overall performance of the systems. In general, it was difficult to obtain training data well collected in this own domain (both text and audio data) and in particular for deception detection and emotion state annotation. The main achievements are the refinement and tuning of statistical models and algorithms used in all tasks. Some audio data have been acquired both in Poland an Italy and have been added to the test sets.

objectives. The main focus of this work package is to analyse video data and facilitate intelligent search and retrieval services. State of the art techniques will be employed to allow optimal content interpretation at a semantic level. Initially, the audio-visual content will be collected from the judicial and public administration domain. Auxiliary schemes such as automatic camera registration and calibration will be utilized for casting video analyses efficient in realistic environments. Both supervised and unsupervised video segmentation methods will be used to detect interesting areas within the captured scenes. Subsequently, the semantic annotation task will be applied on top of these segments in order to semantically characterize them based on appropriately learned knowledge models. At the final stage the recognition of a restricted amount of human behaviours and interesting events will be attempted. Thus, the prime objective of this work package is to semantically annotate multimedia content and provide adequate support to the envisaged system in fullfiling the content management and sharing goals. In video technology there are no methods practically usable in similar environment: automatic camera calibration real-time, shot-detection for cross modal events, motion detection in complex scenario. The aim is to modify and design algorithms that have least restrictions and could work in real circumstances and will promote more precise semantic annotation and video-interpretation at a higher level. The problems are not evident and their solution for a prototype is a fine challenge.

spotlight. wp3 has defined video requirements for video analysis. Video has been obtained from Polish courts for its analysis. A prototype providing spatiotemporal segmentation, motion-based annotations, detection of upper body motion, gestures, and the delivery of the related annotations has been developed. Annotations are provided for wp4 for information retrieval, and collaboration with wp2 is developed in order to extract annotations based on fusion of audio annotations (transcripts, emotion and deception detection), leading to higher level annotation of audiovisual content.

objectives. The main objective of WP4 is the development of the query engine that integrates and aligns audio, video, text and user tagging and annotation. Part of this work needs interactions with WP2 and WP3. Providing multimedia summarization and search capabilities beyond full text search, including media similarities and concepts as well as multilingualism. As a specific feature, the retrieval system will be capable of providing feedback to audio and video components (WP 2—3). This variety of access methods is served by an object oriented database system and an interface to support full document collection analysis and cross-relation detection methods.

spotlight. Partners carried out the development and tests with the JUMAS EXchange (JEX) library to share annotations across partners. The interface of the retrieval engine was redesigned due to the change in partnership. The retrieval engine synchronizes with OODB via JEX and serves from its own full text index. Tests with methods that combine relevance scores based on text term matches and video events have been performed.

objectives. The main objectives of this work package can be described as follows: # Integration of the specific modules in a first kernel prototype to be tested. # On site integration and Customization of the prototype to be tested. # Refinement of the kernel prototype in order to obtain a final system.

spotlight. Pilot system based in Italy and Poland have been defined on the basis of SIDIP system baseline (even if a document management is enough for the project purposes) in order to increase the final acceptance of the users. The first enhanced Courtroom Infrastructure Module, with new camera and enhanced audio acquisition module, has been deployed in the Court of Naples and new recordings have been made available to the research groups for WP2 and WP3 activities. JUMAS full systems will be installed in the Court of Naples and in the Court of Wroclaw.

objectives. Concerning the Show Case the objectives are: # To prepare the field for the system demonstration # To run the platform in the pilot for a full validation by the user point of view # To receive feedback for final refinement of the platform. Concerning the Evaluation, our objectives are: # The development of an agreed evaluation plan for the JUMAS project across the demonstration sites, and which fully meets the requirements of determining the European added value of the project. # Evaluation of the demonstrated products from a technical and operational perspective. This will follow the evaluation plan to ensure that the subsequent validation results are sufficiently rigorous to enable sound common understandings of the technical, operational and other performance criteria to be developed across the sites and in the broader European context. The link between WP6 and WP7, and so the assessment and dissemination activities, will enhance the involvement of people working in organisations external to the JUMAS consortium.

spotlight. The pilot demonstration and validation in Jumas are managed in different locations: two Courts of Law in Italy, coordinated by the the Office for Automated Information Systems DG-SIA of the Ministry of Justice. And the Circuit of the Court of Wroclaw in Poland. The continuous collection of audio and video materials from Courts allowed the implementation of the improvements needed to judicial ICT infrastructure and to information extraction algorithms.

The new infrastructure has been implemented and tuned in the Court of Naples (Italy) and following the indication from the first run it will be implemented in the Court of Wroclaw (Poland).

The JUMAS project was presented to judicial stakeholders in a major event ICT4JUSTICE2009 (Skopje, FYR Macedonia), gaining a wide interest from judicial stakeholders. Ministries of Justice and Public authorities of Italy, The Netherlands, Montenegro, FYR Macedonia, Turkey, Poland attended the conference. In 2010 JUMAS will be tested in the courtrooms of Naples and Wroclaw.

objectives. The purpose of this work package is to ensure the dissemination of JUMAS project on both a European and international scale and a sustainable exploitation of the technologies and methodologies beyond the duration of the project. In particular, this work package will include the organisation of a wide range of regional and European seminars, workshops, conferences, kick-off meetings and to promote the latest research and results on the innovation market.

spotlight. Partners has been prepared scientific papers and presentations through different means (see download section), organised workshops and meetings addressed to both scientific and promotion purposes. Taking into account the social, environmental and practical impacts that the project might have, it is very important the involvement of any of users. Jumas plugs, logo, dedicated website, deliverables, papers, targeted promotion, are the main tools for dissemination.The wp7 activity is defined in a Dissemination Plan D.7.1.

objectives. The overall objective is co-ordination and management of the project. In deeper detail, it means: # To keep the project schedule and guarantee the execution of the work plan and the achievement of the project goals in time and within budget constraints # Overall quality assurance # Liaison between EC and partners # To maintain technical control over the project # To prepare Progress and Management Reports # To review and finalise all Internal Reports and Deliverables.

spotlight. Within these activities the Partners and the Coordinator have been organising the project general meetings in order to check, refine, present, explain, gather, exchange and manage the activities and the deliverables produced and carried out until now. The Partners set-up an internal management web-portal within Wikimedia tool, hosted in RWTH Aachen machines with password access control. Its direct link is available on Jumas home page. We suggest to get visiting the download and events sections for more infos.