Overview of the transformation from file based data to a data target

Description of the data sources in general. Is an aggregation of specific data source types. This generic datasource is added to the model to make associations possible with generic requirements and principles

Data entities saved in a relational (staging) database

Data saved in a file for example semi structured like: XML, XLS, JSon, edifact etc Unstructured like Word, Text etc.

Transformation function from the specific source data model to the standardized target data model. This can be a model or a protocol transformation. Often this transformation is divided in sub steps and these sub steps will be analyzed and modeled in the architecture of a specific data pipe implementation and components

Transformation of a datafile (mostly semi structured) for example an XML file with an internal datamodel that has to be transformed to the standardized model.

Transformation of relational data entities (stored in a relational database) to the standardized data target.

Data target based on a standardized data model. Based on this model and the included data entities one or more interfaces can be implemented for the consumers.

Physical model for the description of a technical implementatin model. This is platform specfic and is used for the creation of SQL DDL and ETL scripts and eventually for technical documentation.

Transformation functionality for transforming the data content from a (application) specific datamodel to a generic datamodel based on a standardized datamodel (see the service diagrams). This can be impllemented either in an ETL transform process or in a message handler for example in a service bus.

Workflow management for the scheduling and iteration of transformation tasks

For the transformation of source to target datamodels a description of the data elements has to be modeled.

The logfical or conceptual model is abstract in nature and is therefore independent from the implementation platform. In most cases this model must be readable for non IT stakeholderrs.

Extract functionality for receiving data from the source systems. This has a XML format and the source is either a webservice or a XML file. The data is extracted and prepared to be transformed

Load functionality for transferring the data to a data target and load it there. This can either be an ETL process or the implementation in a message handler