Teiid 7.4.0.Final Release Notes
Teiid 7.4.0.Final adds performance, integration, and runtime metadata update features.
Overview
Highlights
- Virtual procedure out params - virtual procedures can now have RETURN/OUT/INOUT parameters to return values.
- OLAP - OLAP translator is now part of Teiid kit using OLAP4J
- Multi-source procedures - multi-source handling was expanded to cover stored procedure execution. See the Reference for more.
- Function Support - additional system functions were made available.
- uuid was added to generate type 4 UUIDs and the Hibernate dialect was updated to support the GUIDGenerator.
- array_get was added to extract values from java.sql.Array or java array values.
- array_length was added to get the length of java.sql.Array or java array values.
- ARRAYTABLE - the ARRAYTABLE table function was added to simplify array value extraction into a tabular format.
- Ingres - Ingres database translator is now available to use as supported source under Teiid.
- Optional Join Enhancements - the optional join hint no longer requires the use of ANSI joins and can will not remove optional bridging tables that are used by two other tables that are required.
- InterSystems Cache - InterSystems Cache database translator is now available to use as supported source under Teiid.
- userRequestSourceConcurrency - was added to control the number of concurrent source queries allowed for each user request.
- Memory Management Improvements - maxReserveBatchColumns and maxProcessingBatchesColumns will be default be determined automatically and will more reliably prevent memory issues. See the admin guide for more.
- Subquery optimization control - added the MJ and NO_UNNEST hints and the org.teiid.subqueryUnnestDefault system property to control the optimization of subqueries to traditional joins or to a merge join implementation of a semijoin or antijoin.
- Local connection threads - local connection calling threads will be used to process work rather than using an engine thread. This helps decouple the configuration of maxThreads.
- Dependent Join Improvements - several major improvements were made to increase performance and develop better plans.
- Improved Planning - the decision to create a dependent join is now considered earlier in planning and is much more effective for dependent joins involving multiple unrelated independent tables.
- IN predicate splitting - the planner can now split large dependent IN predicates into multiple IN predicates, which is controlled by the translator property MaxDepdendentInPredicates. This allows for much larger dependent joins to be performed as a single query.
- Dependent query parallelization - when multiple dependent queries are still required, then they will be run in parallel (up to MaxUserSourceRequestConcurrency), rather than sequentially.
- Cost based back-off - for cost based dependent joins if the number of independent values is too large, then the join will be performed as normal.
- Enhanced Sort Join - the partitioned merge join was replaced with an enhanced sort join. The enhanced sort join will use the actual row counts from each side of the relation to perform a index based join if one side is small enough, a partial sort of the larger side and a repeated merge join if the tuples are unbalanced but one side is not small enough to form an index, or a standard sort merge join if the tuples are balanced.
- JDK1.5 JDBC Client JAR - A retro-translated Teiid client JDBC jar now available to use with JDK 1.5 VM. Note only the JDBC API is supported, not the Admin API, or retrieving query plans as XML.
- Security Improvements - UDF and pushdown functions can now be protected with data roles. Also the CommandContext can provide the Subject for custom security checks in UDFs.
- Cache Invalidation - Prepared plan and result set caches have will invalidate entries based upon metadata and data changes respectively. See the cache configuration maxStaleness setting and the Admin and Developer Guides for more.
- Runtime Updates of Metadata - ALTER statements have been added to change view/procedure/INSTEAD OF trigger (update procedure) definitions. A CREATE TRIGGER statement is also available to add an INSTEAD OF trigger (update procedures) to views.
System procedures were added to set extension metadata and stat values. By default all effects of metadata updates happen only on running vdbs across the cluster. To make the changes persistent see the Developers Guide Runtime Updates section.
- TRANSLATE/HAS CRITERIA has been deprecated. An alternative approach to update procedures will be introduced in a subsequent version.
- Support for named parameter syntax using param=value has been deprecated, since it is ambiguous with a comparison predicate boolean value expression. param=>value should be used instead.
from 7.3
- SYS.PROPERTIES has a new column, ClobValue, to get values exceeding the max string length
- SYS.COLUMNS has two new columns to get statistical information: DistinctCount and NullCount
- ARRAY_AGG is now a reserved word
- The use of an IN procedure parameter with the name "source_name" in a multi-source model, will now be treated the parameter that controls which source
the procedure will execute against.
- Dynamic VDB functions injected via ExecutionFactory.getPushdownFunctions are now scoped to the SYS schema and have a fully qualified name that includes their source type. For example, instead of oracle_model.relate - which was only valid against the oracle_model source, there is now the SYS.oracle_sdo.relate function that is valid for all Oracle sources. Any fully-qualified reference to these functions will need updated.
from 7.2
- The default JDBC credentials are user/user - not admin/teiid
- Unordered limits are no longer pushed through conditions, dup removal, or UNION not all. This prevents the possibility of getting less results than the logical intent of the limit.
from 7.1
- Subqueries are no longer allowed to be SELECT INTO.
- INSERT/UPDATE/DELETE cannot be used to create implicit return cursors in non-update virtual procedures. You can instead use "UPDATE ...; SELECT VARIABLES.ROWCOUNT;".
- The SYSADMIN schema was created to hold procedures and tables that should not be generally accessible. SYS and pg_catalog are now always accessible - permissions do not apply to these schemas. The SYS.getBinaryVDBResource, SYS.getCharacterVDBResource, and SYS.getVDBResourcePaths have been replaced with the
SYSADMIN.VDBResources table. The Matviews table and the refreshMatView/refreshMatViewRow procedures were also moved into SYSADMIN.
- Overwriting an existing VDB will cause old connections to be terminated. Production systems should rely on VDB versioning.
- The jdbc:metamatrix JDBC URL prefix is no longer accepted. Use jdbc:teiid instead.
- Model visibility no longer restricts access to tables and procedures. Setting visible to false will only hide entries from system tables. Data roles should be used to restrict data access.
- Admin API "getWorkManagerStats" methods renamed to "getWorkerPoolStats". Also, "setRuntimeProperty" and "getProcesses" methods were removed.
- By default the "ENV" system function is now turned off. To enable it, edit the teiid-jboss-beans.xml configuration file.
- The use of VARIABLES.ROWCOUNT is now reserved.
- Exec statements of the form "var = EXEC foo()" are only valid if the procedure foo has a return parameter.
- Cache hint now supports "scope" definition. For ex: /* cache(scope:user) */ select * from T1; This hint will override the computed scope based on the query.
from 7.0
- The term data policy was replaced with data role. The AdminAPI methods for adding role mappings have changed from addRoleToDataPolicy and removeRoleFromDataPolicy to addDataRoleMapping and removeDataRoleMapping respectively.
from 6.2
- The connector API has changed substantially. Custom connectors need to be retargeted to the new Translator API. See TEIID-1003 and the Developer's Guide for more information.
- Connector bindings are no longer valid. Use the Designer Tooling or see the AdminShell download for a migration utility.
- Calling out to web services and xml processing has been dramatically simplified and improved, but is a breaking change from prior releases.
XML/Relational processing has been replaced by XMLTABLE, retrieving XML from file has been replaced by the File Translator/Resource Adapter, and
web service calls have been replaced by the WS Translator/Resource Adapter. See TEIID-1118, TEIID-1114, and the Reference guide for more information.
- The direct usage of XQuery has been replaced with the XMLQuery function.
- The text connector file processing has been replaced with the TEXTTABLE function. See TEIID-1102 and the Reference guide for more information
- Double quoted values are now parsed as identifiers by default. See TEIID-145 and the Reference guide for more information.
- The system information schema has changed from System to SYS. The tables and table structures have also changed. See TEIID-871 and the Reference guide for more information.
- Client control over query plans has changed. OPTION PLANONLY, DEBUG, and SHOWPLAN are no longer allowed. This behavior should not be controlled by the SET statement, SET SHOWPLAN (ON|OFF|DEBUG) and SET NOEXEC (ON|OFF). Usage of the client PlanNode class will also need to be updated.
- Teiid specific JDBC client interfaces and methods have changed. In general these custom features have mostly been moved to other more standard locations or been removed. See TEIID-1020 and the Client Developers guide for more information.
- Teiid's reserved words have changed. Many of the Teiid specific reserved words have become non-reserved words. All of SQL Foundation and SQL/MED 2003, and SQL/XML from SQL 2006 reserved words have been added as reserved words. It is generally required to double quote all reserved words. It is advisable to double quote all non-reserved words as well. The use of the INPUT qualifier for procedure inputs has been deprecated in favor of INPUTS. The use of the USING qualifier for dynamic SQL inputs has been deprecated in favor of DVARS.
- The AdminApi has been redesigned for use with the new Translator/JCA split and for deployment in JBoss AS.
- The AdminShell is based on Groovy. Most BeanShell syntax is also valid in Groovy, but there are extensive changes in the AdminAPI that may necessitate rewriting scripts.
- OPTION NO CACHE <table list> will now only by-pass cache for the given tables at their immediate view layer. Previously all caches would have transitively been skipped. See TEIID-900 for more information.
- Nan and +-Infinity are no longer allowed as numerical results by default. This may be changed via the system property org.teiid.allowNanInfinity.
- The scale of BigDecimal values is now adjusted to allow for more intuitive results. Conversion from floating numeric types will preserve the approximate scale and division results will allow for additional scale. See TEIID-159 for more information.
- The 7.0 client is not compatible with 6.x or vice versa. It is expected that 7.0 clients will be compatible with 7.x releases for standard JDBC operations.
See the Admin Guide for more on configuration and installation.
from 7.3
- The default value for the JDBC dynamic vdb importer setting importer.useFullSchemaName is now true, which matches the expected behavior from the documentation.
- The prepared plan cache is now configured via the PreparedPlanCacheConfig bean, rather than through properties on the RuntimeEngineDeployer
- SocketConfiguration.maxSocketThreads will interpret a setting of 0 to mean use the system default of max available processors. Both the ODBC and JDBC transports now default to the 0 setting.
- maxReserveBatchColumns and maxProcessingBatchesColumns will interpret a setting of -1 to mean auto-calculate acceptable values given the max heap and other information. See the admin guide for more.
- The default for org.teiid.useValueCache has changed to false, since typical installations will not greatly benefit from the additional lookup cost.
- The property RuntimeEngineDeployer.allowFunctionCallsByDefault was added so that Teiid 7.4 behavior is compatible with Teiid 7.3. Set this property to false to require permissions for function calls when data roles are enabled.
from 7.2
- Temporary tables can now be restricted by data roles. Use the data-role attribute allow-create-temporary-tables to explicitly enable or disable the usage of temporary tables.
There is also a allowCreateTemporaryTablesByDefault property in the teiid-jboss-beans.xml to control whether usage is allowed by default. For compatibility with prior 7.x releases, the default is to allow
temporary table access.
from 7.1
- Teiid clients now allow the usage of anonymous SSL by default.
This changes allows the admin port (default 31443) to use anonymous SSL by default, rather than just securing login traffic.
Admin clients should therefore use the mms protocol instead of mm. This will encrypt all admin traffic and ensure that any passwords in configuration files will
be encrypted in transit. See the Admin Guide to upgrade from anonymous SSL to 1-way or 2-way authentication.
The config properties sslEnabled and clientEncryptionEnabled for SSLConfiguration beans have been combined to a single property mode, that can have the values disabled|login|enabled.
- Apache CXF is now expected to be used as the web services stack provider through JBossWS-CXF. See the Admin Guide for instructions on
installing CXF for use with Teiid's Salesforce and web service connectors. The WS Resource Adapter's -ds.xml files should no longer use
WSSecurityConfigURL and WSSecurityConfigName, rather they should be ConfigFile and ConfigName respectively. The property values should no longer
refer to jboss-wsse-client.xml, but instead they should reference a CXF Spring configuration file and particular port configuration. See the Admin Guide
for more on using CXF configuration files.
- The default for data role checking is now "true". However only VDBs with data roles will have roles enforced.
- The default prepared plan cache size was increased to 512, since it is targeted by internal plans as well.
from 7.0
- The property to enable data roles in teiid-jboss-beans.xml has changed from useEntitlements to useDataRoles.
- Rar file names no longer contain version numbers. -ds.xml files should be updated from connector-XXX-version.rar to teiid-connector-XXX.rar
- Code table relate configuration properties have been removed. Code tables are now implemented as materialized views.
from 6.2
- A JBoss AS instance is now required. Embedded mode is also currently not supported.
- Engine configuration is now handled through the admin console or deploy/teiid/teiid-jboss-beans.xml
- Logging is now done through the container's log4j instance. Any Teiid specific logging changes, see the conf/jboss-teiid-log4j.xml, need to be incorporated into the master jboss-log4j.xml.
- TEIID-1170 - correlated subqueries are not allowed in UPDATEs or DELETEs against internal Teiid tables (internal materialized views or temporary tables).
- TEIID-1281 - Negative start indexing is not supported by DB2 and Derby databases. Usage of the Teiid SUBSTRING against these sources should not use negative start values.
- TEIID-1008 - Most versions of Oracle and MySQL do not support deeply nested correlated references. There is currently no workaround for this issue.
- For compatibility with the 7.0 release if a stored procedure parameter list begins with identifier=, then it will be parsed as a named parameter invocation even if the intent was to use a comparison predicate
as the first parameter value. The workaround is to use nesting parens, e.g. call proc((identifier=value), ...), which clarifies that this is positional value. This workaround will not be needed in later releases.
- TEIID-1511 - When a VDB is deployed and undeployed a socket is left open, which may lead to "too manay open files". Check the defect comments for details.
from 7.0
- Fixed xsd type handling for SQL/XML and XML document models. xsd:date, xsd:dateTime, and xsd:time types will now all be displayed using the GMT timezone (Z). SQL types, such as timestamp, used for an XMLTABLE column will now expect their values to be in the form of the corresponding xsd type.
From 6.2
- Start time is now significantly longer due to the container deployment. However many operations can be performed on configruation files that will trigger a redeployment without the need for a restart.
- Inversion of parse, format, and convert system functions is not used if it can be narrowing. In situations where a predicate has the form: parseTime(column, 'format') = {t 'time value'} may lead to reduced performance, since
the parseTime function cannot be pushed down. This should be addressed in future releases.
The following components have been updated:
From 7.1
- json-simple 1.1 was added.
- Netty was upgraded to 3.2.1
From 7.0
- Direct integration of JBossCache jars was removed.
- Netty was upgraded to 3.2.0
- JDOM was removed.
From 6.2
- Netty was upgraded to 3.1.5
- Saxon was upgraded to 9.1.0.8
- A 5.1 container's version of Arjuna is newer than what was previously included in Teiid 6.x.
- Various other dependencies have been removed as they are no longer needed or now provided by the container.
Detailed Release Notes - Teiid - Version 7.4.0.Final
Bug
- [TEIID-99] - Stored procedures should work in multisource connector binding scenarios
- [TEIID-1163] - Provide LOB support in ODBC
- [TEIID-1164] - Getting the Procedure Parameter metadata is not supported in ODBC
- [TEIID-1432] - Getting SocketTimeout Exceptions on long running queries with CXF
- [TEIID-1458] - Wrong count(*) result when joining data from two models
- [TEIID-1466] - MySQL syntax error in cast from date to timestamp
- [TEIID-1467] - Procedures in Teiid are not Resolvable if Their Models are Marked as Not Visible
- [TEIID-1469] - EDS Teiid Plugin uses Invalid Datatype Mask for SessionID Parameter to Terminate Session Operation
- [TEIID-1470] - Deploying VDB XML file before data sources leads to failed deployment
- [TEIID-1476] - correct preventable exceptions
- [TEIID-1478] - various jdbc import issues
- [TEIID-1479] - Teiid Documentation does not include MS SQL Server as an available translator in section 10.2.2. JDBC Translator of the Reference Guide
- [TEIID-1480] - Teiid's ODBC interaction with PostgreSQL-ODBC fails when column lengths exceed 64K as PostgreSQL stores these values as a short
- [TEIID-1488] - EDS adminshell fails to connect to server with IBM JDK
- [TEIID-1489] - Teiid assumes fqns in transformations, but they are no longer required to be there
- [TEIID-1492] - StackOverflowError in DefaultCache
- [TEIID-1498] - Incorrect rewrite of not disjunctive criteria
- [TEIID-1502] - ODBC (and admin) scalability limited by maxWorker cap
- [TEIID-1503] - Exception during Optimization
- [TEIID-1504] - Internal Materialization does not honor the cache hint
- [TEIID-1505] - MaxActivePlans is not being read from the configuration
- [TEIID-1506] - ODBC Driver reporting BigInteger, BigDecimal columns as Strings
- [TEIID-1511] - VDB deployment leaves with too many open files
- [TEIID-1512] - show plan statement clob isn't readable
- [TEIID-1515] - ODBC does not report primarykeys on the tables
- [TEIID-1516] - IS_NULLABLE is reported incorrectly for columns
- [TEIID-1521] - Dynamic vdb import logic error with procedures
- [TEIID-1523] - Dynamic sql failure with null procedure parameters
- [TEIID-1528] - ruleimplementjoin strategy may generate an order by that does not use a relevant index
- [TEIID-1529] - LIMIT not pushed down for queries with subselects and ORDER BY
- [TEIID-1530] - ODBC identifer matching issues
- [TEIID-1531] - ODBC charset issues
- [TEIID-1532] - ODBC prepared statement handling can hang
- [TEIID-1534] - The driver incorrectly reports column labels as column names
- [TEIID-1537] - Error loading older cached metadata
- [TEIID-1538] - Invalid conversion from type class java.lang.Object with value 'oracle.sql.ROWID@1379cbd' to type class java.lang.String
- [TEIID-1545] - Order By not pushed down
- [TEIID-1548] - Creating XA data sources from AdminShell: prop name "addtional-ds-properties" has misspelling - could cause confusion
- [TEIID-1549] - aggregate unrelated order by error
- [TEIID-1553] - UDF Function model fails to load when it located inside a folder in the VDB
- [TEIID-1554] - NullPointerException in NodeEditor
- [TEIID-1555] - NPE on server when admin-console requests for the "vdb requests"
- [TEIID-1558] - Salesforce Connector gives NPE when the resultset is empty
- [TEIID-1561] - Database Calender is not correctly being set in JDBC Translator
- [TEIID-1562] - ArrayIndexOutOfBounds and ASSERTION FAILED in models with UNION
- [TEIID-1564] - Clustered Caching Not Working
- [TEIID-1566] - More complex join: ORDER BY and LIMIT not pushed down
- [TEIID-1567] - Sybase translator needs to handle CROSS JOIN syntax.
- [TEIID-1570] - pg catalog transformation issues
- [TEIID-1571] - Error due to merge virtual with join and group by
- [TEIID-1572] - Teiid ODBC does not handle SET and Show operations correctly
- [TEIID-1576] - unexpected query result when joining with a virtual procedure
- [TEIID-1577] - ArrayIndexOutOfBoundsException in queries with ORDER BY
- [TEIID-1578] - Set query to string should account for branch limits/order bys
- [TEIID-1579] - LocalServer Connection is not working
- [TEIID-1580] - CME on VDB deploy
- [TEIID-1583] - importer.useFullSchemaName property default value is incorrectly documented
- [TEIID-1584] - Querying a virtual table backed by a SELECT x,y,z FROM EXECT ... TEXTTABLE() call fails to return data
Enhancement
- [TEIID-1473] - Don't batch more rows than is needed when query contains limit but translator does not support limit
- [TEIID-1482] - Prevent unnecessary rebuffering of sorted results
- [TEIID-1490] - In the process of loading the schema metadata and duplicate table is found, add a suggestion to the error message to use FQN option
- [TEIID-1519] - Changing dynamicportfolio example in the JBoss as kit to use HSQL instead of Derby
- [TEIID-1539] - Improve warning message when maximum thread count is encountered
- [TEIID-1556] - Improve xmlagg performance
- [TEIID-1569] - Documentation on creating UDF, extending ExecutionFactory and defining @Translator name should indicate what to do with the translator name
- [TEIID-1581] - Adding a JCA example for creating a Teiid Data Source to the kits
- [TEIID-1582] - Avoid buffering in enhanced sort merge join
Feature Request
- [TEIID-217] - LDAP Connector should provide a way to retrieve all values of an attribute that appears multiple times within a search result
- [TEIID-231] - Ability to support Connector Level UDF functions in the System
- [TEIID-239] - ENHANCEMENT: Costing beyond cardinality sould be propogated
- [TEIID-796] - Allow for the import of overloaded procedures
- [TEIID-946] - add support for in/out out return params
- [TEIID-1011] - Add support for connecting to Mondrian
- [TEIID-1059] - Add a specific translator for Ingres
- [TEIID-1060] - Add a specific translator for Intersystems Cache
- [TEIID-1061] - Add a specific translator for SAP Business Warehouse
- [TEIID-1176] - Provide a non-blocking JDBC api for ODBC support
- [TEIID-1217] - Derive VDB Name Using Version During Deployment in Jopr
- [TEIID-1326] - API request: rewrite query/command before it reaches the teiid planner
- [TEIID-1327] - API request: Fine grained security
- [TEIID-1329] - API to force metadata read
- [TEIID-1437] - It's not possible to deploy single file (*.xml) virtual database through web admin console
- [TEIID-1463] - limit buffering for large results
- [TEIID-1464] - Add uuid function
- [TEIID-1471] - Projection should be minimized prior to view removal
- [TEIID-1474] - Add a limit to the number of concurrent source queries issued.
- [TEIID-1487] - CLONE - Add implicit conversion of NULL to correct datatype in transformation validator
- [TEIID-1491] - Provide JDBC 3.0 JRE 1.5 backwards compatibility
- [TEIID-1494] - Allow the optional join hint to work with non-ansi joins
- [TEIID-1496] - disjunctive rewrite
- [TEIID-1497] - misc subquery enhancements
- [TEIID-1507] - Cache invalidation
- [TEIID-1510] - RulePushAggregates should consider cost in addition to key information
- [TEIID-1533] - additional controls for dependent joins
- [TEIID-1536] - Local connections should use calling thread
- [TEIID-1546] - Be more selective as to what are created as bind variables
- [TEIID-1550] - Row level security enhancements
Quality Risk
- [TEIID-1008] - Correlated subquery handling for Oracle/MySQL
- [TEIID-1281] - DB2 substring index bounds are different than teiid
- [TEIID-1493] - In a cluster the distributed cache is cleared by a node rejoining
- [TEIID-1508] - Dependent join planning problem
- [TEIID-1509] - Change the default for value caching
- [TEIID-1513] - Setting of translator properties should not be case sensitive
- [TEIID-1520] - Reduce the memory footprint of metadata
- [TEIID-1525] - update the hibernate usage with a note about id generation
- [TEIID-1527] - local connection doc provides an invalid example
- [TEIID-1535] - pg/odbc support should report rel oid with column metadata
- [TEIID-1574] - Add new permission types for alter/execute
Task
- [TEIID-1431] - provide a working example of data roles
- [TEIID-1481] - client url cleanups
- [TEIID-1483] - Resolving changes to consolidate teiid/teiid designer handling
- [TEIID-1495] - Validate the TeraData Translator
- [TEIID-1514] - testZipFile.zip contains absolute path which impacted the MEAD sign
- [TEIID-1551] - ODBC logging context is not documented
Sub-task
- [TEIID-1524] - Index metadata strings container their entire record line.
The Teiid community project is hosted on jboss.org.
Documentation and help may be obtained from the local distribution under teiid-docs or the following locations.
Teiid is licensed under the LGPL. The
license texts for Teiid and the thirdparty components it uses may be found in the teiid-docs/licenses
directory of the distribution.
JBoss, a division of Red
Hat, is in the business of
providing
superior technical support to our customers. Our goal is to make
Professional Open Source™ the SAFE
CHOICE
for you. We accomplish this by backing up our open source Java products
with technical support services that are delivered by the core
developers themselves. We can help you to train your staff and provide
you with support at every stage of the application lifecycle - from
development and integration through deployment and maintenance. Visit
the JBoss
Services page for more
information.