JBoss.orgCommunity Documentation

Teiid - Scalable Information Integration

Teiid Administrator's Guide

7.4

Legal Notice

1. Installation Guide
1.1. Installation
1.2. CXF Installation
1.3. Directory Structure Explained
1.3.1. /deploy/teiid/teiid-jboss-beans.xml
1.3.2. /deploy/teiid/connectors
1.3.3. /conf/props
1.3.4. /conf/jboss-teiid-log4j.xml
1.3.5. admin-console.war
1.3.6. /deployers/teiid.deployer
1.3.7. lib
1.3.8. teiid-examples
1.3.9. teiid-docs
2. Deploying VDBs in Teiid 7
2.1. Deploying a VDB
2.1.1. Direct File Deployment
2.1.2. Admin Console Deployment (Web)
2.1.3. AdminShell Deployment
2.1.4. Admin API Deployment
2.2. Deploying VDB Dependencies
2.2.1. JDBC Data Sources
2.2.2. File Data Sources
2.2.3. Web Service Data Sources
2.2.4. Salesforce Data Sources
2.2.5. LDAP Data Sources
2.3. VDB Versioning
2.3.1. Deployment Scenarios
2.4. Migrating VDBs from 6.x
3. Teiid Security
3.1. Authentication
3.1.1. Pass-through Authentication
3.2. Authorization
3.3. Encryption
3.4. LoginModules
3.4.1. Built-in LoginModules
3.4.2. Security at Data Source level
3.5. Configuring SSL
3.5.1. SSL Authentication Modes
3.5.2. Encryption Strength
4. Logging
4.1. General Logging
4.1.1. Logging Contexts
4.2. Command Logging
4.3. Audit Logging
5. Clustering in Teiid
6. Performance Tuning
6.1. Memory Management
6.2. Threading
6.3. Cache Tuning
6.4. Socket Transports
6.5. LOBs
6.6. Other Considerations
7. Teiid Admin Console
7.1. What can be monitored and/or configured?
7.1.1. Configuration
7.1.2. Metrics
7.1.3. Control (Operations)
7.1.4. Deploying the VDB
8. AdminShell
8.1. Introduction
8.1.1. Download
8.2. Getting Started
8.2.1. Essential Rules
8.2.2. Help
8.2.3. Basic Commands
8.3. Executing a script file
8.4. Log File and Recorded Script file
8.5. Default Connection Properties
8.6. Handling Multiple Connections
8.7. Interactive Shell Nuances
A. AdminShell Frequently Asked Questions
B. Other Scripting Environments
C. System Properties

Starting with the 7.0 release Teiid needs to be installed into an existing JBoss AS installation, which is entirely different from previous versions.

Note

Teiid does not support the "embedded" mode in 7.4 version. ("embedded" will be coming in a future release).

Steps to install Teiid

  1. Download the JBoss AS 5.1.0 application server. Install the server by unzipping into a known location. Ex: /apps/jboss-5.1.0

    Note

    You may also choose to use an existing AS installation. However if a previous version of Teiid was already installed, you must remove the old teiid distribution artifacts before installing the new version.

  2. Download Teiid 7.4. Unzip the downloaded artifact inside any "profile" in the JBoss AS installation. Teiid 7.4 uses a JBoss AS service called the "profile service" that is only installed in "default" and "all" profiles, so installing into one of these profiles is required. The default profile is the typical installation location, for example "<jboss-install>/server/default". The Teiid runtime directory structure matches JBoss profiles directly - it is just an overlay.

    The all profile is recommended in a clustered environment to take advantage of clustered caching and cluster safe distribution of events.

  3. Start the JBoss AS server by executing "<jboss-install>/bin/run.sh" if you installed in the "default" profile. Otherwise use "<jboss-install>/bin/run.sh -c <profilename>"

  4. That it!. JBoss AS and Teiid are now installed and running. See below instructions to customize various settings.

  5. Once VDBs have been deployed, users can now connect their JDBC applications to Teiid. If you need help on connecting your application to the Teiid using JDBC check out the "Client Developer's Guide".

The usage of CXF is expected for utilizing Salesforce and Web Services connectivity through Teiid. If you do not plan on integrating either of these features, then you may leave JBoss AS with the default "native" web services stack.

  1. Download JBossWS-CXF 3.1.2 and unzip to a temporary location.

  2. From the jbossws-cxf-bin-dist directory, save the ant.properties.example file as ant.properties and change the values for jboss510.home, jbossws.integration.target, jboss.server.instance, jboss.bind.address accordingly:

    ...
    jboss510.home=<jboss-install>
    
    # The JBoss server under test. This can be [jboss500|jboss501|jboss510|jboss600]
    jbossws.integration.target=jboss510
    
    # The JBoss settings
    jboss.server.instance=<profile>
    jboss.bind.address=<bind address>
    

    The jboss-install location should be the root directory of your AS installation, profile (typically default) should indicate the profile selected for your Teiid installation, and the bind address should be the bind address used when launching JBoss AS (use the value localhost if you do not set the bind address when launching JBoss AS).

  3. From the jbossws-cxf-bin-dist directory, install JBossWS-CXF by running the ANT build script:

    $ant deploy-jboss510
  4. Optionally run tests to verify that there are no errors with the installation:

    $ant tests

A VDBis the primary means to define a Virtual Database in Teiid. A user can create a VDB using Teiid Designer or follow the instructions in the Reference Guide to create a "Dynamic VDB" without Teiid Designer.

Once you have a "VDB" built it can be deployed/removed in Teiid runtime in different ways.

Warning

If VDB versioning is not used to give distinct version numbers, overwriting a VDB of the same name will terminate all connections to the old VDB. It is recommended that VDB versioning be used for production systems.

Note

Removing an existing VDB will immediately clean up VDB file resources, but will not automatically terminate existing sessions.

Teiid provides a groovy based AdminShell scripting tool, which can be used to deploy a VDB. Check out the "deployVDB" method. Consult the AdminShell documentation for more information. Note that using the AdminShell scripting, you can automate the deployment of artifacts in your environment.

Apart from deploying the VDB, the user is also responsible for providing all the necessary dependent libraries, configuration for creating the data sources that are needed by the models (schemas) defined in "META-INF/vdb.xml" file inside your VDB. For example, if you are trying to integrate Oracle and File sources in your VDB, then you are responsible for providing the JDBC driver for the Oracle source and any necessary documents and configuration that are needed by the File Translator.

Data source instances may be used by only one VDB, or may be shared with as many VDBs or other applications as makes since for your deployments. Consider sharing connections to sources that have heavy-weight and resource constrained connections.

With the exception of JDBC, each of the data sources supported by has a corresponding .rar (zip format) file in <jboss-install>/server/<profile>/deploy/teiid/connectors. If not using JOPR or other tooling to create your -ds.xml files, you can consult the .rar files META-INF/ra.xml for a full description of how the source can be configured.

Some -ds.xml files may contain passwords or other sensitive information. See the WIKI article EncryptingDataSourcePasswords to not store passwords in plain text.

Once the VDB and its dependencies are deployed, then client applications can connect using the JDBC API. If there are any errors in the deployment, a connection attempt will not be successful and a message will be logged. You can use the admin-console tool or check the log files for errors and correct them before proceeding.

The following is an example highlighting configuring an Oracle data source. The process is nearly identical regardless of the vendor. Typically only the client jar and the setting in the -ds.xml file change.

There are templates for all the data sources in the "<jboss-install>/docs/examples/jca" directory.

  1. Copy the Oracle JDBC JAR file into "<jboss-install>/server/<profile>/lib" directory

  2. Create a "data source" to the Oracle instance in the JBoss container. This typically done by creating "xxx-ds.xml" file and copying this file to the "<jboss-install>/server/<profile>/deploy" directory. The following shows a "-ds.xml" file template for Oracle. You can also use admin-console to create this data source.

    <?xml version="1.0" encoding="UTF-8"?>
    <datasources>
      <xa-datasource>
        <jndi-name>OracleDS</jndi-name>
        <!-- uncomment to enable interleaving <interleaving/> -->
        <isSameRM-override-value>false</isSameRM-override-value>
        <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
        <xa-datasource-property name="URL">jdbc:oracle:oci8:@tc</xa-datasource-property>
        <xa-datasource-property name="User">scott</xa-datasource-property>
        <xa-datasource-property name="Password">tiger</xa-datasource-property>
        <!-- Uses the pingDatabase method to check a connection is still valid before handing it out from the pool -->
        <!--valid-connection-checker-class-name>
        	org.jboss.resource.adapter.jdbc.vendor.OracleValidConnectionChecker
        </valid-connection-checker-class-name-->
        
        <!-- Checks the Oracle error codes and messages for fatal errors -->
        <exception-sorter-class-name>
        	org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter
        </exception-sorter-class-name>
        
        <!-- Oracles XA datasource cannot reuse a connection outside a transaction once enlisted in a global transaction and vice-versa -->
        <no-tx-separate-pools/>
          <!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml (optional) -->
          <metadata>
             <type-mapping>Oracle9i</type-mapping>
          </metadata>
      </xa-datasource>
    </datasources>

Template files [xxx-ds.xml] for different databases can found at {jboss-as}/docs/examples/jca, and also additional sources like ingres, mondrian, intersystems-cache etc that Teiid suports can be found at {jboss-as}/server/{profile}/teiid-examples/jca directory.

Web service data sources use a Teiid specific JCA connector. You need to create "-ds.xml" file and copy it to the "<jboss-install>/server/<profile>/deploy" directory.


To enable the use of WS-Security, the SecurityType should be set to WSSecurity. At this time Teiid does not expect a WSDL to describe the service being used. Thus a Spring XML configuration file is not only required, it must instead contain all of the relevant policy configuration. And just as with the general configuration, each data source is limited to specifing only a single port configration to use.

Example 2.3. Example WS-Security enabled data source

<?xml version="1.0" encoding="UTF-8"?>
<connection-factories>
   <no-tx-connection-factory>
      <jndi-name>somewhere-ws-source</jndi-name>
      <rar-name>teiid-connector-ws.rar</rar-name>
      <connection-definition>javax.resource.cci.ConnectionFactory</connection-definition>      
      <config-property name="EndPoint">http://somewhere.com</config-property>
      <config-property name="ConfigFile">${jboss.server.home.dir}/server/default/conf/xxx-jbossws-cxf.xml</config-property>
      <config-property name="ConfigName">port_x</config-property>
      <config-property name="SecurityType">WSSecurity</config-property>
   </no-tx-connection-factory>
</connection-factories>

Corresponding xxx-jbossws-cxf.xml file that adds a timestamp to the SOAP header

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:jaxws="http://cxf.apache.org/jaxws"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
          http://www.springframework.org/schema/beans/spring-beans.xsd
          http://cxf.apache.org/jaxws
          http://cxf.apache.org/schemas/jaxws.xsd">

    <jaxws:client name="{http://teiid.org}port_x" 
        createdFromAPI="true">
        <jaxws:outInterceptors>
            <bean class="org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor"/>
            <ref bean="Timestamp_Request"/>
        </jaxws:outInterceptors>
    </jaxws:client>

    <bean 
        class="org.apache.cxf.ws.security.wss4j.WSS4JOutInterceptor"
        id="Timestamp_Request">
        <constructor-arg>
            <map>
                <entry key="action" value="Timestamp"/>
            <map>
        </constructor-arg>
    </bean>
    
</beans>

Note that the client port configuration is matched to the data source instance by the QName {http://teiid.org}port_x. The configuration may contain other port confiruations with different local names.


For more information on configuring CXF interceptors, please consult the CXF documentation or the JBossWS-CXF documentation.

Salesforce data sources use a Teiid specific JCA connector. You need to create "-ds.xml" file and copy it to the "<jboss-install>/server/<profile>/deploy" directory.


VDB Versioning is a feature that allows multiple versions of a VDB to be deployed at the same time with additional support to determine which version will be used. When a user connects to Teiid the desired VDB version can be set as a connection property (See the Client Developers Guide). If a specific version is set, then only that VDB may be connected to. If no version is set, then the deployed VDBs are searched for the appropriate version. This feature helps support more fluid migration scenarios.

Setting the version can either be done in the vdb.xml, which is useful for dynamic vdbs, or through a naming convention of the deployment file - vdbname.version.vdb, e.g. marketdata.2.vdb. The deployer is responsible for choosing an appropriate version number. If there is alreay a VDB name/version that matches the current deployment, then connections to the previous VDB will be terminated and its cache entries will be flushed. Any new connections will then be made to the new VDB.

Once deployed a VDB has an updatable property called connection type, which is used to determine what connections can be made to the VDB. The connection type can be one of:

The connection type may be changed either through the AdminConsole or the AdminAPI.

VDBs from prior release contain an older configuration file version that is no longer supported. You can use the migration utility (bin/migrate.sh or bin/migrate.bat) supplied with the AdminShell download to update these VDBs for use with Teiid 7. Note - XML and File based sources from previous releases have changed, and require manual changes to the VDB.

The Teiid system provides a range of built-in and extensible security features to enable the secure access of data.

JDBC clients may use simple passwords to authenticate a user.

Typically a user name is required, however user names may be considered optional if the identity of the user can be discerned by the password credential alone.  In any case it is up to the configured security domain to determine whether a user can be authenticated. If you need authentication, the administrator must configure a LoginModule to be used with Teiid. See below for more information on how configure the Login module in JBoss AS.

LoginModules are an essential part of the JAAS security framework and provide Teiid customizable user authentication and the ability to reuse existing LoginModules defined for JBossAS. See JBossAS Security for general information on configuring security in JBossAS.

Teiid can be configured with multiple named application policies that group together relevant LoginModules. Each of these application policy (or domains) names can be used to fully qualify user names to authenticate only against that domain.  The format for a qualified name is username@domainname.

If a user name is not fully qualified, then the installed domains will be consulted in order until a domain successfully or unsuccessfully authenticates the user.

If no domain can authenticate the user, the login attempt will fail. Details of the failed attempt including invalid users, which domains were consulted, etc. will be in the server log with appropriate levels of severity.

Note

The security-domain defined for the JDBC connection and Admin connections are separate. The default name of JDBC connection's security-domain is "teiid-security". The default name for Admin connection is "jmx-console". For the Admin connection's security domain, the user is allowed to change which LoginModule that "jmx-console" pointing to, however should not change the name of the domain, as this name is shared between the "admin-console" application.

In some use cases, user might need to pass-in different credentials to their data sources based on the logged in user than using the shared credentials for all the logged users. To support this feature, JBoss AS and Teiid provide multiple different login modules to be used in conjunction with Teiid's main security domain. See this document for details on configuration. Note that the below directions need to be used in conjunction with this document.

If client wants to pass in simple text password or a certificate or a custom serialized object as token credential to the data source, user can configure "CallerIdentity" login module. Using this login module, user can pass-in same credential that user logged into Teiid security domain to the data source. Here is a sample configuration, this needs to be configured in "teiid-jboss-beans.xml" file.

            
    <application-policy xmlns="urn:jboss:security-beans:1.0" name="teiid-security">
        <authentication>
            
            <login-module code="org.jboss.security.auth.spi.UsersRolesLoginModule" flag="required">
                <module-option name = "password-stacking">useFirstPass</module-option>
                <module-option name="usersProperties">props/teiid-security-users.properties</module-option>
                <module-option name="rolesProperties">props/teiid-security-roles.properties</module-option>
            </login-module>
            
            <login-module code="org.jboss.resource.security.CallerIdentityLoginModule" flag="required">
                <module-option name = "password-stacking">useFirstPass</module-option>
                <module-option name = "managedConnectionFactoryName">jboss.jca:service=LocalTxCM,name=DefaultDS</module-option>
            </login-module>
                        
        </authentication>
    </application-policy>  
            

In the -ds.xml file that is defined as the "managedConnectionFactoryName" in the above configuration, you need to add the following element

                <security-domain>teiid-security</security-domain>
            

In the above configuration example, in the primary login module "UsersRolesLoginModule" is setup to hold the passwords in the file, and when user logs in with password, the same password will be also set on the logged in Subject after authentication. This credentials can be extracted by the data source by asking for Subject's private credentials.

To use a certificate or serialized object instead of plain password as the token, simply replace the simple text password with Base64 encoded contents of the serialized object. Please note that, encoding and decoding of this object is strictly up to the user as JBoss AS and Teiid will only act like carrier of the information from login module to connection factory. Using this CallerIdentity module, the connection pool for data source is segmented by Subject.

In some use cases, the users are divided by their functionality and they have varied level of security access to data sources. These types of users are identified by their roles as to what they have access to. In the above "CallerIdentity" login scenario, that may be too fine-grained security at data sources, that can lead resource exhaustion as every user has their own separate connection. Using Role based security gives a balance, where the users with same role are treated equally for authentication purposes at the data source. Teiid provides a login module called "RoleBasedCredentialMap" for this purposes, where administrator can define a role based authentication module, where given the role of the user from the primary login module, this module will hold credentail to that role. So, it is container of credentials that map to different roles. If a user has multiple roles, the first role that has the credential will be chosen. Below find the sample configuration.

            
    <application-policy xmlns="urn:jboss:security-beans:1.0" name="teiid-security">
        <authentication>
            
            <login-module code="org.jboss.security.auth.spi.UsersRolesLoginModule" flag="required">
                <module-option name = "password-stacking">useFirstPass</module-option>
                <module-option name="usersProperties">props/teiid-security-users.properties</module-option>
                <module-option name="rolesProperties">props/teiid-security-roles.properties</module-option>
            </login-module>
            
            <login-module code="org.teiid.jboss.RoleBasedCredentialMapIdentityLoginModule" flag="required">
                <module-option name = "password-stacking">useFirstPass</module-option>
                <module-option name="credentialMap">props/teiid-credentialmap.properties</module-option>
                <module-option name = "managedConnectionFactoryName">jboss.jca:service=LocalTxCM,name=DefaultDS</module-option>
            </login-module>            
                        
        </authentication>
    </application-policy>  
            

In the -ds.xml file that is defined as the "managedConnectionFactoryName" in the above configuration, you need to add the following element

                <security-domain>teiid-security</security-domain>
            

In the above configuration example, in the primary login module "UsersRolesLoginModule" is setup for logging in the primary user and assign some roles. The "RoleBasedCredentialMap" login module is configured to hold role to password information in the file defined by "credentialMap" property. When user logs in, the role information from the primary login module is taken, and extracts the role's passsword and attaches as a private credential to the Subject. If you want use this for role based trusted token, you can configure the Base64 based endcoding/decoded object as defined above.

You can also encrypt the password instead of plain text password using this module. Just include the encrypted password in the file defined by the "credentialMap" property, and define following properties in the "RoleBasedCredentialMap" login module.

            
            <login-module code="org.teiid.jboss.RoleBasedCredentialMapIdentityLoginModule" flag="required">
                <module-option name = "password-stacking">useFirstPass</module-option>
                <module-option name="credentialMap">props/teiid-credentialmap.properties</module-option>
                <module-option name = "managedConnectionFactoryName">jboss.jca:service=LocalTxCM,name=DefaultDS</module-option>
                
               <!-- below properties are only required when passwords are encrypted -->
               <module-option name = "pbealgo">PBEWithMD5AndDES</module-option>
               <module-option name = "pbepass">testPBEIdentityLoginModule</module-option>
               <module-option name = "salt">abcdefgh</module-option>
               <module-option name = "iterationCount">19</module-option>
            </login-module>            
            

For full details about encryption of the password, please follow this document's "A KeyStore based login module for encrypting a datasource password" section. Be sure to give the same configuration elements in the above configuration, as they are used to encrypt the password.

The Teiid's configuration file <jboss-install>/server/<profile>/deploy/teiid/teiid-jboss-beans.xml, contains the properties to configure SSL per socket transport.


Properties

The Teiid system provides a wealth of information via logging. To control logging level, contexts, and log locations, you should be familiar with log4j and the container's jboss-log4j.xml configuration file. Teiid also provides a <profile>/conf/jboss-teiid-log4j.xml containing much of information from chapter.

All the logs produced by Teiid are prefixed by "org.teiid". This makes it extremely easy to control of of Teiid logging from a single context. Note however that changes to the log configuration file require a restart to take affect

While all of Teiid's logs are prefixed with "org.teiid", there are more specific contexts depending on the functional area of the system. Note that logs originating from third-party code, including integrated org.jboss components, will be logged through their respective contexts and not through org.teiid. See the table below for information on contexts relevant to Teiid. See the container's jboss-log4j.xml for a more complete listing of logging contexts used in the container.

Context

Description

com.arjuna

Third-party transaction manager. This will include information about all transactions, not just those for Teiid.

org.teiid

Root context for all Teiid logs. Note: there are potentially other contexts used under org.teiid than are shown in this table.

org.teiid.PROCESSOR

Query processing logs. See also org.teiid.PLANNER for query planning logs.

org.teiid.PLANNER

Query planning logs.

org.teiid.SECURITY

Session/Authentication events - see also AUDIT logging

org.teiid.TRANSPORT

Events related to the socket transport.

org.teiid.RUNTIME

Events related to work management and system start/stop.

org.teiid.CONNECTOR

Connector logs.

org.teiid.BUFFER_MGR

Buffer and storage management logs.

org.teiid.TXN_LOG

Detail log of all transaction operations.

org.teiid.COMMAND_LOG

See command logging

org.teiid.AUDIT_LOG

See audit logging

org.teiid.ADMIN_API

Admin API logs.

org.teiid.ODBC

ODBC logs.

Since Teiid is installed in JBoss AS, there is no separate configuration needed on the part of the user to cluster the Teiid instances. To cluster JBoss AS instances use these instructions then Teiid instances are clustered as well. Just make sure that you installed Teiid in every JBoss AS node before starting the cluster. There is one specific configuration that needs to be done for enabling the replicated (distributed) cache in Teiid. To enable distributed caching, rename the "<jboss-as>/server/<profile>/deploy/teiid/teiid-cache-manager-jboss-beans-rename-me.xml" file to "<jboss-as>/server/<profile>/deploy/teiid/teiid-cache-manager-jboss-beans.xml".

Typically users create clusters to improve the performance of the system through:

  1. Load Balancing: Take look at the Client developers guide on how to use load balancing between multiple nodes.

  2. Fail Over: Take look at the Client developers guide on how to use fail over between multiple nodes.

  3. Distributed Caching: This is automatically done for you once you configure it as specified above.

  4. Event distribution: metadata and data modifications will be distributed to all cluster members.

If would like a clustered deployment of the VDB and data-source artifacts, i.e. deploy artifacts to a central location and let the system propagate deployments every where, then look into JBoss Farm Deployment. Note that this only supports hot deployments. Take look at some commonly asked questions here. If you need more fine grained control, you can use script based deployment, where you control the deployment of artifacts into each node, or JBoss AS "deploy" folder can be configured as a shared folder among all the clustered JBoss AS nodes to achieve farming.

The BufferManager is responsible for tracking both memory and disk usage by Teiid. Configuring the BufferManager properly is one of the most important parts of ensuring high performance. See the <jboss-install>/server/<profile>/deploy/teiid/teiid-jboss-beans.xml file for all BufferManager settings.

The Teiid engine uses batching to reduce the number of memory rows processed at a given time. The batch sizes may be adjusted to larger values if few clients will be accessing the Teiid server simultaneously.

The maxReserveBatchColumns setting determines the total size of batches that can be held by the BufferManager in memory. This number does not include persistent batches held by soft (such as index pages) or weak references. The value is treated internally as an approximation of bytes using the conversion maxReserveBatchColumns * processorBatchSize * (64bytes per column value). The default value of -1 will auto-calculate a typical max based upon the max heap available to the VM. The auto-calculated value assumes a 64bit architecture and will limit buffer usage to 50% of the first gigabyte of memory beyond the first 300 megabytes (which are assumed for use by the AS and other Teiid purposes) and 75% of the memory beyond that.

The BufferManager automatically triggers the use of a canonical value cache if enabled when more than 25% of the reserve is in use. This can dramatically cut the memory usage in situations where similar value sets are being read through Teiid, but does introduce a lookup cost. If you are processing small or highly similar datasets through Teiid, and wish to conserve memory, you should consider enabling value caching.

Note

Memory consumption can be significantly more or less than the nominal target depending upon actual column values and whether value caching is enabled. Large strings, bigintegers, bigdecimals, or values typed as object can exceed their default size estimate. If an out of memory errors occur, then set a lower the maxReserveBatchColumns value.

The maxProcessingBatchesColumns setting determines the total size of batches that can be used by active plans regardless of the memory held based on maxReserveBatchColumns. The value is treated internally as an approximation of bytes using the conversion maxProcessingBatchesColumns * processorBatchSize * (64bytes per column value). The default value of -1 will auto-calculate a typical max based upon the max heap available to the VM and max active plans. The auto-calculated value assumes a 64bit architecture and will limit processing batch usage to 10% of memory beyond the first 300 megabytes (which are assumed for use by the AS and other Teiid purposes).

In systems where large intermediate results are normal (scrolling cursors or sorting over millions of rows) you can consider increasing the maxProcessingBatchColumns and decreasing the maxReserveBatchColumns so that each request has access to an effectively smaller buffer space.

Each intermediate result buffer, temporary LOB, and temporary table is stored in its own set of buffer files, where an individual file is limited to maxFileSize megabytes. Consider increasing the storage space available to all such files maxBufferSpace if your installation makes use of internal materialization, makes heavy use of SQL/XML, or processes large row counts.

Socket threads are configured for each transport. They handle NIO non-blocking IO operations as well as directly servicing any operation that can run without blocking. For longer running operations, the socket threads queue with work the query engine.

The query engine has several settings that determine its thread utilization. maxThreads sets the total number of threads available for query engine work (processing plans, transaction control operations, processing source queries, etc.). You should consider increasing the maximum threads on systems with a large number of available processors and/or when it's common to issue non-transactional queries with that issue a large number of concurrent source requests. maxActivePlans, which should always be smaller than maxThreads, sets the number of the maxThreads that should be used for user query processing. Increasing the maxActivePlans should be considered for workloads with a high number of long running queries and/or systems with a large number of available processors. If memory issues arise from increasing the max threads and the max active plans, then consider decreasing the processor/connector batch sizes to limit the base number of memory rows consumed by each plan. userRequestSourceConcurrency, which should always be smaller than maxThreads, sets the number of concurrently executing source queries per user request. Setting this value to 1 forces serial execution of all source queries by the processing thread. The default value is computed based upon 2*maxThreads/maxActivePlans. Using the respective default values, this means that each user request would be allowed 6 concurrently executing source queries. If the default calculated value is not applicable to your workload, for example if you have queries that generate more concurrent long running source queries, you should adjust this value.

Caching can be tuned for cached result (including user query results and procedure results) and prepared plans (including user and stored procedure plans). Even though it is possible to disable or otherwise severely constrain these caches, this would probably never be done in practice as it would lead to poor performance.

Cache statistics can be obtained through the Admin Console or Adminshell. The statistics can be used to help tune cache parameters and ensure a hit ratio.

Plans are currently fully held in memory and may have a significant memory footprint. When making extensive use of prepared statements and/or virtual procedures, the size of the plan cache may be increased proportionally to number of gigabytes intended for use by Teiid.

While the result cache parameters control the cache result entries (max number, eviction, etc.), the result batches themselves are accessed through the BufferManager. If the size of the result cache is increased, you may need to tune the BufferManager configuration to ensure there is enough buffer space.

Result set and prepared plan caches have their entries invalidated by data and metadata events. By default these events are captured by running commands through Teiid. See the Developers Guide for further customization. Teiid stores compiled forms of update plans or trigger actions with the prepared plan, so it is recommended to leave the maxStaleness of the prepared plan cache set to 0 so that metadata changes, for example disabling a trigger, may take effect immediately. The default staleness for result set caching is 60 seconds to improve efficiency with rapidly changing sources. Consider decreasing this value to make the result set cache more consistent with the underlying data. Even with a setting of 0 full transactional consistency is not guaranteed.

The Teiid Admin Console is a web based administrative and monitoring tool for Teiid. Teiid's Admin Console is built using the Embedded JOPR library and adds a additional plugin into the Embeeded JOPR program already available in the JBoss AS.

Here are the steps to follow to install Teiid

The AdminShell tooling provides scripting based programming environments that enable user to access, monitor and control a Teiid Server. Both the command line and graphical console tools are built on functionality provide by the Groovy ( http://groovy.codehaus.org/ ) project. The AdminShell tools can be used in ad-hoc scripting mode or to run pre-defined scripts.

AdminShell features:

  1. fully functional programming environment with resource flow control and exception management. See Groovy docs for the full power of the language.

  2. quick administrative tool. The user can connect to a running Teiid Server and invoke any of the AdminAPI methods, such as "deployVDB" or "stopConnectionFactory", to control the Teiid System. Since this can be script driven, these tasks can be automated and re-run at a later time.

  3. simplified data access tool. The user can connect to a VDB, issue any SQL commands, and view the results of the query via Groovy Sql extensions.

  4. migration tool. This can be used to develop scripts like moving the Virtual Databases (VDB), Connection Factories, and Configuration from one development environment to another. This will enable users to test and automate their migration scripts before production deployments.

  5. testing tool. The JUnit ( http://junit.org ) test framework is built in, see Groovy Unit Tests. User can write regression tests for checking system health, or data integrity that can be used to validate a system functionality automatically instead of manual verification by QA personnel.

To learn the basics of Groovy take a look at their documents and tutorials on their website.

Basic knowledge of the Java programming language and types is required in order to effectively design and develop scripts using the AdminShell. To learn Java language find learning resources at http://java.sun.com.

You can learn about the Teiid AdminAPI either using “adminHelp()” function or by using the JavaDocs.

AdminShell is a specialized version of Groovy which works in several different modes: interactive shell, graphical console, or script run mode. In interactive shell mode (launched via adminshell), the user can invoke connect to a live Teiid system and issue any ad-hoc commands to control the system. The interactive buffer can be used to develop a script and the interactive session input and output can be captured into a log file, more on this later in the document.

In graphical mode (lanched via adminshell-console), the user can develop and run scripts using a text editor that supports syntax highlighting.

In the script run mode, the user can execute/play back previously developed scripts. This mode especially useful to automate any testing or to perform any repeated configurations/migrations changes to a Teiid system.

To use AdminShell successfully, there are some basic syntactical rules to keep in mind.

To execute the commands and arbitrary script in interactive mode you enter them first and press enter to execute, then enter the next line, so on.

To exit the tool in the interactive mode, first disconnect if you are connected to the Teiid system by executing “disconnect();” then type "exit". In the script mode, when execution of the script finishes the tool will exit automatically, however you still have to disconnect from Teiid system in the script.

Note: If SSL is turned on the Teiid server, you would need to adjust the connection URL and the client SSL settings as necessary (typically this will only be needed for 2-way SSL).

Using AdminShell, a user can actively manage more than one connection to a single or multiple Teiid systems. For example, two separate connections can be maintained, one to the development server and one to the integration server at the same time. This is possible because AdminShell supports a feature called named connections.

Every time a connection is made, the connection has an explicit or an implicitly assigned name.   If another connect command is executed then a new connection is made with a unique name and execution will be switched to use the new connection.   The previous connection will be held as it is in its current state, and will not be closed.

You can use the following command to find out the current connection's name

name = getConnectionName();    
      

Knowing the names of the connection that user is working with is important to switch the active connection to a previous connection. To switch the active connection, use the following command and supply the name of the connection to be used

useConnection("name");    
      

If user supplies the same name as the active connection as they are currently participating in, then this operation will simply return with out any modifications. There is no limitation the number of simultaneous connections.

The following shows an example of using and switching between two connections.

// creates a connection 
connectAsAdmin();	        	

//capture the connection name
conn1 = getConnectionName();

deployVDB("file.vdb")

// creates a second connection 
connectAsAdmin();		

conn2 = getConnectionName();

deployVDB("file.vdb")

// switch the connection to "conn1"
useConnection(conn1);	

// close the connection in the "conn1"
disconnectAll();

The interactive shell uses a special shell interpretter and therefore has different behavior than just writting a script in Groovy. See the Groovy Shell Documentation for more on its usage. Notable differences:

  • Def statements do not define a variable in the context of the Shell, e.g. do not use def x = 1, use x = 1

  • Shell commands (as seen through help) using the non-functional shell syntax are only available in the shell.

  • Groovy classes using annotations cannot be parsed in the Shell.

A.1. Why won't the adminhelp command work in the Console tool?
A.2. Are there any pre-built scripts available?
A.3. I have written a very useful script to do XYZ, I would like this to be part of the distribution?
A.4. What is different between "connectAsAdmin()" and "connect()"?
A.5. What does "getAdmin()" call do? Why do I need it?
A.6. Is IDE support available for writing the scripts?
A.7. Is debugging support available?

A.1.

Why won't the adminhelp command work in the Console tool?

The Console environment does not understand Shell commands (load, help, adminhelp, etc.), since they are not directly supported by Groovy. In the Console you should use the equivalent functional form / Groovy, e.g. instead of adminhelp, adminHelp()

A.2.

Are there any pre-built scripts available?

Currently no, but we will provide samples in subsequent releases.

A.3.

I have written a very useful script to do XYZ, I would like this to be part of the distribution?

Yes, we would love to hear from users. Please submit the script through the Teiid JIRA, and if this script popular, we will include the script in the scripts library in the following releases.

A.4.

What is different between "connectAsAdmin()" and "connect()"?

The connectAsAdmin methods create a contextual connection to the AdminAPI of the Teiid Server. The connect methods return an extension of the Groovy Sql object to be used for Sql calls to the Teiid Server.

A.5.

What does "getAdmin()" call do? Why do I need it?

"getAdmin()" returns this contextual connection object created when you executed "connectAsAdmin()" method. This object implements the interface "org.teiid.adminapi.Admin" and AdminShell commands provided are wrappers around this API. Advanced users can use this API directly if the provided wrapper commands do not meet their needs.

A.6.

Is IDE support available for writing the scripts?

The Admin Console tool is a light-weight IDE. Full IDE support is available for Groovy, but requires manual manipulation of the class path and script imports. See using AdminShell methods in other environments.

A.7.

Is debugging support available?

The interactive shell and console do have built-in support for inspection of the current state. Performing line based debugging is beyond the scope of this document.

The AdminShell methods (named contextual connections, AdminAPI wrapper, and help system) have no direct dependencies on Groovy and can be used in other scripting languages.

To use the AdminShell methods in another language, simply import the static methods and Admin classes into your script. You will also need to ensure that the <adminshell dist>/lib/teiid-7.4-client.jar and <adminshell dist>/lib/teiid-adminshell-7.4.jar are in your class path. The snippet below show import statements that would work in Java, BeanShell, Groovy, etc.

import static org.teiid.adminshell.AdminShell.*;
import static org.teiid.adminshell.GroovySqlExtensions.*;
import org.teiid.adminapi.*;

Note that the provided shell and console executables automatically have the proper class path set and inject the proper imports into running scripts. If you wish to use scripts in a non-Teiid Groovy environment, you will need to manually add these imports and add the admin/client jars to the class path.

Some of Teiid's low-level behavior can be configured via system properties, rather than through configuration files. A typical place to set system properties for JBoss AS launches is in the <jboss-install>/bin/run.conf. A property setting has the format -Dproperty=value