Big Data in the field of automotive testing requires Test Data Governance

Big Data in the field of automotive testing is mainly about the analysis of huge amounts of data that result from a wide range of test benches and driving tests. The aim is to identify new patterns and correlations in these test data in order to describe a complex system of different components and to predict the future behavior of this system. In this way, e.g. it is possible to gain important insights for the development of new simulation models and to achieve considerable savings in the execution of cost-intensive tests. Among other things, the basis for this are tools for Data Mining, Predictive Analytics or Machine Learning, which are nowadays available in a large number on the market – in some cases even free of charge.

However, the use of these tools requires that well-documented test data is available. In practice, this is often the point at which the problem begins: On the one hand, many test engineers do not consider the maintenance of meta information as “sexy”. On the other hand, extensive knowledge of the test context is crucial for interpreting measurements correctly in the long term and independently of individual persons. If the systematic documentation of the test context is missing, the measured data is only a column of figures that lose value very quickly.

Up to now, many test departments neglect metadata management – if one can speak of it at all. In particular, its embedding in the test processes is at best deficient. Guidelines for dealing with metadata, which prescribe uniform processes and responsibilities for data entry and maintenance (= Test Data Governance), are often looked for in vain. In addition, many of the systems used for measurement data management (MDM) do not offer sufficient functionalities to support and enforce such Test Data Governance. Unfortunately, the companies still give not enough attention to this fact during the planning and selection of MDM platforms.

This results in numerous, often department-specific and isolated solutions with individual and mostly rudimentary documented test data stocks. These are only little suitable as a basis for the implementation of Big Data analyzes deployed across many domains.

Conclusion: Test departments that want to tap the potential of Big Data in the near future are doing well to deal with the topic of Test Data Governance today. Standards such as ASAM ODS provide the appropriate basis for this. Software platforms, such as openMDM or Peak Resource Planner, help to anchor the topic firmly in the company-specific test processes.

ASAM ODS goes Big Data

The ASAM e.V. is currently working on a project that prepares the expansion of ASAM ODS to Big Data. Phase 1 of the project was finished successfully last year with a collection of use-cases, features and non-functional requirements of end users for processing large amounts of data throughout the automotive development process. Now, ASAM Technical Steering Committee (TSC) has approved phase 2 of the project.

Continue reading “ASAM ODS goes Big Data”

ASAM e.V. has released ODS 6.0

The ASAM ODS base standard in version 5.x offers clients access to ODS servers by means of CORBA and RPC. Since these communication protocols are technically outdated and either badly or not supported by newer development tools, version 6.0 of ASAM ODS now additionally provides a streamlined API.

Continue reading “ASAM e.V. has released ODS 6.0”

ToDo´s to extend ASAM ODS to Big Data

Driving tests produce an enormous amount of measured data for different development disciplines. Some companies have therefore begun to collect these data in a so-called “Data Lake”.

At the heart of such a “Data Lake” is usually the open source platform Hadoop. It provides a variety of frameworks that enable to process and analyze the incoming data volumes flexible in manifold ways.

Continue reading “ToDo´s to extend ASAM ODS to Big Data”

Standardization and transparency in the test environment

Measurement data must be quickly retrievable, comparable and interpretable for different purposes and by different persons even after a long time. Companies can only achieve this by precisely documenting under which conditions (= context) the respective measurement data have been created. Conclusion 1: Transparency on test data becomes problematic, if measurement files are stored on the hard disk of a local computer or server just based on individually selected names without a detailed description.

That´s why the market offers a whole series of software tools with which this file-based storage of measurement data can be better organized.

Continue reading “Standardization and transparency in the test environment”

openMDM® 5 Full-Text Search – Technology Proposal

An important function of openMDM® 5 is the full-text search. It allows test engineers to find measurements based on arbitrary search terms and without knowing the exact location.

When choosing a suitable platform, Peak Solution investigated the two servers SolR and ElasticSearch in detail. This article gives a brief overview on how their features match with the requirements of the openMDM® 5 Eclipse Working Group and what recommendation was given.

Continue reading “openMDM® 5 Full-Text Search – Technology Proposal”

BigODS project examples

Automotive companies generate large amounts of test data from various sources such as test benches, simulations, driving and field tests. This data volume is growing exponentially day-by-day. Because traditional solutions reach their performance limits, the industry is dealing for some time with the question of how Big Data technologies can be beneficially used in the field of test data management and analysis. Frequently the companies desire to enhance established standards like ASAM ODS gradually.

The development of index-based search methods as well as new query and analysis techniques for ODS data are two concrete project examples.

Continue reading “BigODS project examples”