JBoss.orgCommunity Documentation

Chapter 1. Introduction

1.1. What is Teiid Designer?
1.2. Metadata Overview
1.2.1. What is Metadata
1.2.2. Business and Technical Metadata
1.2.3. Design-Time and Runtime Metadata
1.2.4. Source and View Metadata
1.3. It's all in the Modeling...
1.3.1. What Are Models?
1.3.2. How is a Model Defined?
1.3.3. Guiding through the process
1.3.4. Model Classes and Types
1.3.5. Models and VDBs
1.3.6. Model Validation
1.3.7. Testing Your Models
1.3.8. Model Object Extensions

The Teiid Designer User's Guide provides detailed descriptions of Teiid Designer features and functionality.

Teiid Designer is an Eclipse-based graphical modeling tool for modeling, analyzing, integrating and testing multiple data sources to produce Relational, XML and Web Service Views that expose your business data.


Why Use Teiid Designer?

Teiid Designer is a visual tool that enables rapid, model-driven definition, integration and testing of data services without programming. With Teiid Designer , not only do you map from data sources to target formats using a visual tool, but you can also:

  • resolve semantic differences

  • create virtual data structures at a physical or logical level

  • use declarative interfaces to integrate, aggregate, and transform the data on its way from source to a target format which is compatible and optimized for consumption by your applications

This allows you to abstract the structure of the information you expose to and use in your applications from the underlying physical data structures. With Teiid Designer, data services are defined quickly, the resulting artifacts are easy to maintain and reuse, and all the valuable work and related metadata are saved for later reference.

You can use Teiid Designer to integrate multiple sources, and access them using the common data access standards:

  • Web Services / SOAP / XML

  • JDBC / SQL

  • ODBC / SQL

Teiid Designer is an integral part of the Teiid Designer enterprise-class system for providing data services for service-oriented architectures.

Metadata is data about data. A piece of metadata, called a meta object in the Teiid Designer, contains information about a specific information structure, irrespective of whatever individual data fields that may comprise that structure.

Let’s use the example of a very basic database, an address book. Within your address book you certainly have a field or column for the ZIP code (or postal code number). Assuming that the address book services addresses within the United States, you can surmise the following about the column or field for the ZIP code:

This definition represents metadata about the ZIP code data in the address book database. It abstracts information from the database itself and becomes useful to describe the content of your enterprise information systems and to determine how a column in one enterprise information source relates to another, and how those two columns could be used together for a new purpose

You can think of this metadata in several contexts:

Editing Metadata vs. Editing Data

The Teiid Designer helps you to create and describe an abstract graphic representation of your data structure of your data in the original data sources. It also describes whether those data sources are composed of Relational databases, text files, data streams, legacy database systems, or some other information type.

The Teiid Designer allows you to create, edit, and link these graphically-represented meta objects that are really a description of your data, and not the data itself.

So when this documentation describes the process of creating, deleting, or editing these meta objects, remember that you are not, in fact, modifying the underlying data.

Metadata Models

A metadata model represents a collection of metadata information that describes a complete structure of data.

In a previous example we described the field ZIPCode as a metadata object in an address book database. This meta object represents a single distinct bit of metadata information. We alluded to its parent table, StreetAddress. These meta objects, and others that would describe the other tables and columns within the database, would all combine to form a Source Metadata model for whichever enterprise information system hosts all the objects.

You can have Source Models within your collection of metadata models These model physical data storage locations. You can also have View Models, which model the business view of the data. Each contains one type of metadata or another. For more information about difference between Source and View metadata, (see Section 1.2.4, “Source and View Metadata”).

NOTE: For detailed information about creating models from your metadata, see Section 1.3, “It's all in the Modeling...”

Metadata can include different types of information about a piece of data.

Note that the terms technical and business metadata, refer to the content of the metadata, namely what type of information is contained in the metadata. Don’t confuse these with the terms “physical” and “view” metadata that indicate what the metadata represents. For more information, (see Section 1.2.4, “Source and View Metadata”).

Technical Metadata

Technical metadata represents information that describes how to access the data in its original native data storage. Technical metadata includes things such as datatype, the name of the data in the enterprise information system, and other information that describes the way the native enterprise information system identifies the meta object

Using our example of an address book database, the following represent the technical metadata we know about the ZIP code column:

  • Named ZIPCode

  • Nine characters long

  • A string

  • Located in the StreetAddress table

  • Uses SQL Query Language

These bits of information describe the data and information required to access and process the data in the enterprise information system.

Business Metadata

Business metadata represents additional information about a piece of data, not necessarily related to its physical storage in the enterprise information system or data access requirements. It can also represent descriptions, business rules, and other additional information about a piece of data.

Continuing with our example of the ZIP Code column in the address book database, the following represents business metadata we may know about the ZIP code:

  • The first five characters represent the five ZIP code numbers, the final four represent the ZIP Plus Four digits if available, or 0000 if not

  • The application used to populate this field in the database strictly enforces the integrity of the data format

Although the first might seem technical, it does not directly relate to the physical storage of the data. It represents a business rule applied to the contents of the column, not the contents themselves.

The second, of course, represents some business information about the way the column was populated. This information, although useful to associate with our definition of the column, does not reflect the physical storage of the data.

Teiid Designer software distinguishes between design-time metadata and run-time metadata. This distinction becomes important if you use the Teiid Designer Server. Design-time data is laden with details and representations that help the user understand and efficiently organize metadata. Much of that detail is unnecessary to the underlying system that runs the Virtual Database that you will create. Any information that is not absolutely necessary to running the Virtual Database is stripped out of the run-time metadata to ensure maximum system performance.

Design-Time Metadata

Design-time metadata refers to data within your local directory that you have created or have imported. You can model this metadata in the Teiid Designer, adding Source and View metadata.

Runtime Metadata

Once you have adequately modeled your enterprise information systems, including the necessary technical metadata that describes the physical structure of your sources, you can use the metadata for data access.

To prepare the metadata for use in the Teiid Designer Server, you take a snapshot of a metadata model for the Teiid Designer Server to use when resolving queries from your client applications. This run-time metadata represents a static version of design-time metadata you created or imported. This snapshot is in the form of a Virtual Database definition, or VDB.

As you create this runtime metadata, the Teiid Designer:

You can continue to work with the design-time metadata, but once you have created a runtime metadata model, it remains static.

In addition to the distinction between business and technical metadata, you should know the difference between Source Metadata and View Metadata.

Source and View metadata refer to what the metadata represents, not its content.

Source Metadata directly represents metadata for an enterprise information system and captures exactly where and how the data is maintained. Source Metadata sounds similar to technical metadata, but Source Metadata can contain both technical and business metadata. When you model Source Metadata, you are modeling the data that your enterprise information systems contain.

View Metadata, on the other hand, represent tailored views that transform the Source Metadata into the terminology and domain of different applications. View Metadata, too, can contain both technical and business metadata. When you model View Metadata, you’re modeling the data as your applications (and your enterprise) ultimately use it.

Modeling Your Source Metadata

When you model the Source Metadata within your enterprise information systems, you capture some detailed information, including:

The Source Metadata captures this detailed technical metadata to provide a map of the data, the location of the data, and how you access it.

This collection of Source Metadata comprises a direct mapping of the information sources within your enterprise. If you use the Teiid Designer Server for information integration, this technical metadata plays an integral part in query resolution.

For example, our ZIPCode column and its parent table StreetAddress map directly to fields within our hypothetical address book database.

To extend our example, we might have a second source of information, a comma-separated text file provided by a marketing research vendor. This text file can supply additional demographic information based upon address or ZIP code. This text file would represent another Enterprise Information System (EIS), and the meta objects in its Source Model would describe each comma-separated value.

Modeling Your View Metadata

When you create View Metadata, you are not describing the nature of your physical data storage. Instead, you describe the way your enterprise uses the information in its day-to-day operations.

View Metadata derives its classes and attributes from other metadata. You can derive View Metadata from Source Metadata that describes the ultimate sources for the metadata or even from other View Metadata. However, when you model View Metadata, you create special “views” on your existing enterprise information systems that you can tailor to your business use or application expectations. This View Metadata offers many benefits:

Our example enterprise information sources, the address book database, and the vendor-supplied comma-delimited text file, reside in two different native storage formats and therefore have two Source Metadata models. However, they can represent one business need: a pool of addresses for a mass mailing.

By creating a View Metadata model, we could accurately show that this single View Table, the AddressPool, contains information from the two enterprise information systems. The View Metadata model not only shows from where it gets the information, but also the SQL operations it performs to select its information from its source models.

This View Metadata can not only reflect and describe how your organization uses that information, but, if your enterprise uses the Teiid Designer Server, your applications can use the View Metadata to resolve queries.

To create this View Metadata, you create a view and define a transformation for that view, a special query that enables you to select information from the source (or even other view) metadata models. For more information, see “Section 5.2.1, “Transformation Editor”.”

Metadata Transformations

By modeling View Metadata, you can illustrate the business view of your enterprise information sources. View Metadata models not only describe that business view, but also illustrate how the meta objects within the View Metadata models derive their information from other metadata models.

Let’s return to the example of our address book database and the vendor’s comma-separated list. We want to generate the View Metadata model, Address Pool, from these enterprise information systems.


The transformation that joins these metadata models to create the virtual Address Pool metadata model contains a SQL query, called a union, that determines what information to draw from the source metadata and what to do with it.

The resulting Address Pool contains not only the address information from our Address Book database, but also that from our vendor-supplied text file.

SQL in Transformations

Transformations contain SQL queries that SELECT the appropriate attributes from the information sources.

For example, from the sources the transformation could select relevant address columns, including first name, last name, street address, city, state, and ZIP code. Although the metadata models could contain other columns and tables, such as phone number, fax number, e-mail address, and Web URL, the transformation acts as a filter and populates the Address Pool metadata model with only the data essential to building our Address Pool.

You can add other SQL logic to the transformation query to transform the data information. For example, the address book database uses a nine-character string that represents the ZIP Plus Four. The transformation could perform any SQL-supported logic upon the ZIPCode column to substring this information into the format we want for the Address Pool View metadata model.

Mapping XML Transformations

When you model View Metadata, you can also create a View XML Document model. This View Document lets you select information from within your other data sources, just like a regular View Metadata model, but you can also map the results to tags within an XML document.


In this example, the Address Pool View Metadata model still selects its information from the Address Book Database and the Vendor Text File, but it also maps the resulting columns into tags in the Address XML document.

A model is a representation of a set of information constructs. A familiar model is the relational model, which defines tables composed of columns and containing records of data. Another familiar model is the XML model, which defines hierarchical data sets.

In Teiid Designer, models are used to define the entities, and relationships between those entities, required to fully define the integration of information sets so that they may be accessed in a uniform manner, using a single API and access protocol. The file extension used for these models is .xmi ( Example: NorthwindOracle.xmi ) which adheres to the XMI syntax defined by the OMG.

Below is an example of the partial contents of a model file.


Note

Model files should never be modified "by hand". While it is possible to do so, there is the possibility that you may corrupt the file such that it cannot be used within the Teiid Designer system.

The fundamental models in Teiid Designer define the structural and data characteristics of the information contained in data sources. These are referred to as source models (represented by ). Teiid Designer uses the information in source models to federate the information in multiple sources, so that from a user's viewpoint these all appear to be in a single source.


In addition to source models, Teiid Designer provides the ability to define a variety of view models(represented by ). These can be used to define a layer of abstraction above the physical (or source) layer, so that information can be presented to end users and consuming applications in business terms rather than as it is physically stored. Views are mapped to sources using transformations between models. These business views can be in a variety of forms:

  • Relational Tables and Views

  • XML

  • Web services

  • Functions

For full list of supported model types see Chapter 3, New Model Wizards

A third model type, logical, provides the ability to define models from a logical or structural perspective.

To make the process of using Teiid Designer to build models more as easy as posssible, a guides view (Section C.2.13, “Guides View”) has been introduced. It provides action sets which bring together the actions necessary to develop models for specific use-cases. Action sets are available for the following scenerios:

  • Consuming a SOAP Web Service
  • Creating a REST WAR archive
  • Creating a SOAP WAR archive
  • Modelling from a Flat File Source (a text file)
  • Modelling from a JDBC Data Source
  • Modelling from a Local XML File Source
  • Modelling from a Remote XML File Source
  • Connecting to a Teiid Server

Teiid Designer in conjunction with Teiid provides an extensible framework to define custom properties for model objects over-and-above what is defined in the metamodel. These custom property values are added to your VDB and included in your runtime metadata. This additional metadata is available to use in your custom translators for both source query manipulation as well as adjusting your result set data being returned.

In the 7.6 release, Teiid Designer introduces a new Model Extension Definition (MED) framework that will replace the current EMF-based Model Extension metamodel in a later 8.0 release.

This new MED framework provides the following improvements:

Also see: Section 5.3, “Managing Model Object Extensions” and Section C.3.3, “Model Extension Definition Editor”.

The purpose of a MED is to define one or more sets of extension properties. Each set of extension properties pertains to one model object type (or metaclass). Each MED consists of the following:

A MED file is an XML file with an extension of "mxd." A MED schema file (see attached modelExtension.xsd file) is used to validate a MED file. Here is a sample MED file:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<modelExtension xmlns:p="http://org.teiid.modelExtension/2011"
		xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	metamodelUri="http://www.metamatrix.com/metamodels/Relational"
    	namespacePrefix="mymodelextension" namespaceUri="org.my.extension.mymodelextension"
    	version="1"
    	xsi:schemaLocation="http://org.teiid.modelExtension/2011 modelExtension.xsd"
    	xmlns="http://org.teiid.modelExtension/2011">
    <p:description>This is my model extension</p:description>
    <p:extendedMetaclass name="com.metamatrix.metamodels.relational.impl.BaseTableImpl">
        <p:property advanced="false" index="true" masked="false" name="copyable" required="false" type="boolean">
            <p:description locale="en_US">Indicates if table can be copied</p:description>
            <p:display locale="en_US">Copyable</p:display>
        </p:property>
    </p:extendedMetaclass>
</modelExtension>
				

The MED Registry is where the MEDs used by Designer are stored. MED files can be edited by opening the .mxd file in the Section C.3.3, “Model Extension Definition Editor”.