Release Notes

Current versions can always be downloaded at http://www.heimdalldata.com/download-self/.

Release version 1.0.19 (Aug 17, 2017)

Minor revision including the following changes:

  • Further improvement with SQL Server, MySQL proxy compatibility, as well as improved Oracle JDBC compability.
  • Optimized Cache behavior, including a filter to prevent caching in the local memory cache any resultset larger than 1/1000th of the configured memory cache size, to prevent pushing other useful content out.  The grid-cache interface is used for the retrieval of such objects however.
  • Resolved an issue that could prevent the test-source button from functioning in certain situations
  • Updated cluster detection, configuration, and monitoring logic
  • Initial testing for AWS Aurora for Postgres to verify it functions properly with the PostgreSQL proxy.

Release version 1.0.18 (Aug 6, 2017)

Minor revision including the following changes:

  • Improved compatibility with SQL Server, including a fix to an issue that could hang connections or trigger exceptions with cached content when using the proxy
  • Prevent errors if a database server isn’t ready in AWS when using the wizard
  • Added support links on the GUI to make it easier to access support
  • Honor the VDB disable flag when using the proxy, to terminate requests to an offline VDB, generally while the database is in maintenance
  • Upgrade the Windows installer to use Java 8, necessary for SQL server proxy support
  • In MS SQL proxy, add additional logs to help debug why some queries are not cached
  • Detect several clusters types when configuring a data source, and automatically configure the cluster peers including SQL Server Always On, Galera and stock MySQL clusters.  Postgresql clusters are pending for the next release.
  • Adds throttling of repeated log messages, to prevent flooding of the log file

Release version 1.0.17a (July 20, 2017)

Single fix patch, which resolves a timing related deadlock that could occur on driver/proxy initialization, most often with the Sql Server proxy.

Release version 1.0.17 (July 19, 2017)

In this release:

This is a minor bug-fix release, including:

  • Improvements to the SQL Server proxy for compatibility with various drivers and use-cases
  • Improved transaction state tracking for the MySQL proxy
  • Resolve class loading issues with optional extensions for PostgreSQL
  • Resolved a few table name extraction issues for various SQL dialects

One new feature was added to this new build however–when using the MySQL and Postgres proxies, on non-Windows systems, they will automatically attempt to setup and listen to Unix sockets that match the default sockets for MySQL and Postgres at the following locations:

  • MySQL:  /tmp/mysqld.sock and /var/run/mysqld/mysqld.sock
  • Postgres: /tmp/.s.PGSQL.$port and /var/run/postgresql/.s.PGSQL.$port

This should improve ease of use with applications that connect to “localhost” as this defaults to using Unix sockets.  The exact sockets attached to will be listed in the proxy startup, i.e.:

[2017-07-20 13:35:03,506] [2017-07-20 13:35:03,506] localhost jira: Listening on 5050
[2017-07-20 13:35:04,116] [2017-07-20 13:35:04,116] localhost jira: Proxy: Listening on socket: /var/run/postgresql/.s.PGSQL.5050
[2017-07-20 13:35:04,122] [2017-07-20 13:35:04,122] localhost jira: Proxy: Listening on socket: /tmp/.s.PGSQL.5050

Release version 1.0.16 (July 11, 2017)

In this release:

This is a minor bug-fix release, including:

  • Improvements to the SQL Server proxy for compatibility with various drivers and use-cases, as well as some lesser-used field types
  • Support configuration of a SQL Server proxy via the wizard
  • Improve the error messages when an old version of TDS is used with the SQL server proxy
  • Allow connection closes to be handled properly for the MySQL proxy, in particular when canceling a query
  • Support larger than 16MB responses that use multiple MySQL packets in the proxy
  • Include more information about the client on the status screen
  • prevent spaces in configuration names, now they will be replaced with underscores automatically
  • Adjust the health-check behavior on an exception from the underlying JDBC driver
  • Resolve an NPE when using a heimdall configuration file to set the default hduser and hdpassword
  • Add an option to the heimdall configuration file to disable testing for AWS.  Use “aws=false” in the file to avoid the error that AWS was not detected

Release version 1.0.15 (June 26, 2017)

In this release:

Most of the changes in this build are bug-fixes, but it does contain a few new features:

  1. Improved MS-SQL proxy support, approaching release quality.  We are testing with a wide variety of applications, and most appear to work correctly.  Only SQL Server 2008+ is supported at this time, and this requires the proxy code to run on Java 7+, while other functions only require Java 6;
  2. Added the ability to pause on the dashboard, and other cosmetic changes to the GUI to improve usability;
  3. Added support to control the initial password for the server based on either the environment or a /etc/heimdall.conf file.  Set the options as “hduser=” and “hdpassword=” to override any defaults on startup;
  4. Improved invalidation logic to resolve some corner cases where it would not be honored;
  5. Improved logging to help find problems;
  6. Other small issues corrected that in general didn’t impact behavior but resulted in errors in the logs.

Release version 1.0.14 (June 11, 2017)

In this release:

  1. A new configuration wizard is now available.  This combines the capability of the original AWS wizard with generic configuration support, to help bring online an application with a minimum amount of effort.  The wizard allows for simple configuration of baseline cache and read/write split along with many other features on a vdb.
  2. Grid-clustering capability to allow redundancy of a management server–to enable, select the “Cluster manager via cache” option in the cache settings.  This feature operates on a per-vdb basis, and uses the vdb name as a unique key to synchronize the configurations across nodes.  This feature replaces the “peercache” configuration option previously supported to allow nodes to share cache information.
  3. A new status page has been added, to allow the health status of databases, as well as what clients are connecting to be viewed;
  4. A new log tab has been added, to allow inspection of up-to the last 1000 log entries on the server;
  5. AWS CloudWatch support on a per-vdb level, providing per-instance and vdb statistics;
  6. Configuration caching support on the driver.  To use, specify a Heimdall JDBC URL parameter of “localconfig=<pathToDirectory>”.  When used, new configurations from the management servers will be written to this directory, allowing the driver to come online when a central manager is not available;
  7. An option to ship the proxy or driver logs directly to the management server, avoiding the need to review application logs to find Heimdall messages;
  8. Exposing the VDB level debug option as a checkbox to simplify debugging–per client Heimdall JDBC URL use of the debug option will continue to be supported to allow individual nodes to enable debug mode;
  9. Adjusted the “delay” rule property to operate only after cache logic is complete.  This effectively allows before and after cache performance to be evaluated in conjunction with WAN latency simulation;
  10. On the MySQL proxy, allowing the proxy to honor the database catalog “use” command, as well as the initial catalog desired on connect.  The proxy now supports multiple catalogs through a single proxy instance–PostgreSQL does not support the “setCatalog()” JDBC api to support this.
  11. On all proxies, the support for multiple user logins, including adjusting users at runtime without a proxy reload.  If the user is not allowed to connect to the database, the connection will be refused.  This option, along with the catalog selection, are connection pooling compatible, but may reduce the effectiveness of the connection pool when used heavily.
  12. user:<user1>,<user2> matching support to allow any rule type to apply to a given user;
  13. Forwarding now has two new options:  setcatalog and readonly.  If readonly is true, then a read-only server in the target source will be considered eligible to serve forwarded requests.  If the setcatalog option is true, then the catalog on the target source will be forced to match the source connection’s catalog.  This provides the capability to redirect queries to alternate catalogs if set to false.  The combination of the two allows for effective read-write split, and is used when read-write splitting is selected via the wizard.
  14. Improved read-only account security, to prevent password leakage for restricted users;
  15. Removed Memcached, EHcache and Jcache interfaces, as they were insufficient to provide the best experience for customers.  Currently, only Hazelcast and Redis are supported.  The main limiting feature was the lack of pub/sub interfaces, which are now used for messaging between nodes for optimal efficiency;
  16. Simplification of configuration to remove unnecessary options, including those that could be auto-detected based on the environment (such as Redis cluster mode);
  17. Adjusted how driver dependencies are packaged, to make them invisible to host applications in JDBC mode.  Grid-cache libraries are now included inside the jar files, and don’t need an external lib directory (removed from the packaging).  Please report any other class loading conflicts or issues if observed;
  18. Many other minor bugs and improvements;
  19. Initial preview build of Microsoft SQL Server support as a proxy.  This is still a work in progress, and should not be used in production.  Currently, transaction tracking is not fully online, and the SQLVariant result type is not yet supported, along with other corner cases.

Release version 1.0.13 (Apr 17, 2017)

Note:  Release 1.0.12 was provided to many customers in a variety of pre-release forms, so it was decided to skip 1.0.12 as a formal release

In this release:

  1. Improved compatibility for both the PostgreSQL and MySQL proxies, including full prepared statement support for both;
  2. Improved performance for proxy mode operations, both in memory usage as well as CPU overhead.
  3. The MySQL proxy automatically sets “best practice” properties for the JDBC driver, to provide compatibility to non-Java applications.  These properties are displayed on startup, and can be overridden in the data source properties configuration.  Connection pool properties are also set this way as well and also able to be overridden.
  4. tables:<tablename1>,<tablename2> matching support to allow any rule type to apply to a given table;
  5. ${catalog} or ${database}, ${user}, and ${vdb} expansion in rule properties and table selection to improve dynamic configuration for simplified rules;
  6. Catalog selection support in the MySQL proxy, allowing “use <database>” commands to be honored.  To best use this feature, specify a JDBC URL without the database/catalog specified;
  7. Forwarding support to specify the use of read-only servers in the target data source, as well as to either inherit the parent connection’s database/catalog, or allow it to be overridden;
  8. The ability to ignore matching queries from statistics, to allow the dashboard to reflect only user-impacting query patterns, i.e. to exclude background batch operations;
  9. Improved logging in debug mode (specify a VDB property of “debug” with a value of “true” to activate), including a prefix specifying which module the log entry applies to;
  10. Enhanced diagnostics on non-Windows systems, by sending the proxy/driver the HUP signal.  This data is also generated every 24 hours automatically.  Windows does not support a HUP signal.  If the parent application in the JDBC case has registered the HUP signal, it will attempt to register the signal handler on SIGUSR2–this will be logged on startup.  This will include per-thread stack traces to debug deadlocks conditions, per-connection SQL activity, all SQL patterns observed in the last two hours, and per-table performance statistics;
  11. Streamlined the per-minute statistics generated by the system to only include relevent information, including process cpu usage, garbage collection stats, and overall performance stats;
  12. Huge number of bug-fixes and overall optimizations;
  13. Build times are now displayed on startup, so that the actual build version can be tracked independently of the release version;
  14. Additional environment information, such as jar files in the classpath, are displayed on startup, to help debug class load issues;
  15. When using the MySQL proxy, if the server isn’t set to negotiate to UTF8MB4, warnings will be displayed on startup.  This is necessary to fully support all character sets.

Release Version 1.0.11 (Feb 28, 2016)

New in this release:

  1.  Improved auto-tune functionality
  2. MySQL JDBC defaults are now built-in when using the MySQL proxy.  These JDBC settings are displayed on proxy startup, and can be overridden in the data source property configuration.
  3. Improved PostgreSQL protocol handling.
  4. AWS Configuration Wizard–allows RDS and Elasticache settings to be read out of the AWS configuration, to populate the VDB, Data Source, and Rules tabs automatically.
  5. General bug fixes.

Release Version 1.0.10 (Jan 24, 2016)

This release is a smaller release, and provides the following:

  1.  Auto restart on update–before, the upgrade of the server would not restart the server.  As this can cause java class load issues, it will also restart the server after mailing any notification on a successful update.
  2. Autotune on the cache configuration–when enabled, the vdb will process in a learning mode before caching, and will disable caching for tables that are written to often, or that result in cache hit rates that are non-beneficial.  This tuning adjusts based on the cache configuration, and will account for overheads in putting objects into the cache that end up not providing a benefit.
  3. MySQL character set debugging information is now available on startup of the driver, to help insure that the character sets are correct.  For correct behavior, utf8mb4 should be used on the server, and should be inherited on the client without overrides.  Please see the sample configuration for a full list of recommended MySQL configuration options.
  4. Improved compatibility for XA and non-XA data source configurations.
  5. Improved PostgreSQL proxy compatibility.  Current, binary transfer formats are not supported.

Release Version 1.0.9 (Dec 15, 2016)

This release includes significant changes to the overall configuration of caching.  Instead of cache configuration parameters being in the cache rule, it is now configured in the VDB section, and provides GUI driven options to simplify the process.  In addition, proxy settings are now configured in the VDB tab as well, and the proxy can be configured to be started and managed by the central manager as desired.  Postgres proxy support is now added in addition to the MySQL proxy, although these proxy options should be considered beta for the moment.  The Linux & vendor neutral packaging has been updated to include Docker and Packer samples for creating docker containers for the central manager/proxy and AWS AMI builds.

A public AMI has been made available (please search for “Heimdall” in community AMIs in the US-East region) to simplify testing.  The default GUI user is “admin” and the password will be the instance ID of the system, and the AMI has been scanned and passed in preparation for AWS Marketplace submission.

Release Version 1.0.8 (Oct 24, 2016)

Release 1.0.8 includes the following

  1.  VDB level properties–these should be used to configure the cache instead of applying them to the cache rules.
  2. Debug mode–if the VDB property “debug=true” is set, then this will activate verbose logging of the driver.  Please use caution when enabling this, as it will impact performance.  Most actions will now account for their operation when this log is enabled, to help verify that the desired behavior is occurring.
  3. Support for Hazelcast-client instead of embedded behavior.  Please set the vdb property “cacheconfig” to point to the hazelcast-client.xml file to configure the upstream servers (this should be in the current working directory of the application).  The server/embedded XML configuration can be specified this way as well.
  4. Improved support for Redis and Redis cluster configurations.
  5. Added clear all logs button to Analytics dashboard to simplify the test process.
  6. Added the ability to specify ${vdb}, ${user} or ${catalog} in the forwarding rule source properties, to have these values filled in dynamically, improving the reusability of rules in certain situations.
  7. All grid-cache libraries are now merged into the heimdalldriver.jar to prevent classpath issues loading the drivers.  If class conflicts cause issues in your application,  please contact Heimdall support for a stripped driver that requires external dependency jar files.
  8. Parameter help from the driver–on startup, the Heimdall Driver will dump a list of all parameters and the context they can be used, along with default values.  These will be exposed directly from the server in the next revision of code, to guarantee help documentation matches the version of software being used.
  9. Per-driver table usage statistics–each driver will print a running documentation of tables observed, and statistics about their usage.  These can help tune table-level cache configuration.  Future revisions of code will expose this data on the analytics tab.
  10. Fixed a bug that could account for non-cached queries twice in the query per second statistics under certain situations.

Release Version 1.0.7 (Oct 4, 2016)

Release 1.0.7 includes all the fixed intended for 1.0.6, but as several customers had been using internal builds of 1.0.6, we have chosen to skip 1.0.6 as a formal release to insure that update notifications operate properly for those users.  Additionally, this release includes:

  1.  Fix for EHCache performance issues, but also reduces the ability to control the size of the cache.  It was found that the logic to control cache size by bytes was extremely expensive with EHCache, so we have tuned it to dynamically compute the number of cache objects based on size/1000.
  2. Provide an option for “Verify Cache” on the VDB–this will result in the object being retrieved from the cache AND from the back-end database, then compared.  Alerts will be written to stdout indicating if an object compared differently, and will provide a diff of the difference or other details on what was different.  This is to be used to validate that the cache is performing properly, and returning correct results before being used in production.
  3. Release packaging changed from heimdalldemo to just “heimdall” due to confusion over if it was a beta release only or not.

Release Version 1.0.6 (formal release skipped)

IMPORTANT:  Release 1.0.6 changes the password encoding format to exactly match RFC-2307 for SSHA.  Previously, it was incorrectly encoded in hex vs. base64.  As such, upon upgrade, any users will need to have their password fields reset.  If you need assistance with this, please contact Heimdall Support.

  1. Provide a Heimdall JDBC URL option of peerconf=true, enabling one JDBC instance to leverage Hazelcast to pull configuration information from another JDBC instance, in the event the management server is down.  Both instances must be configured with this option at startup for this feature to function.
  2. Access control changes:
    • Move password encoding to strictly follow RFC-2307 for SSHA.
    • Fixed password changes not changing the password.
    • Added a read-only flag for users.  This flag can be used with client/application server logons to better secure the configuration management.
  3. GUI Changes & Fixes
    • Improve PDF handling to prevent large pages from being cut short.
    • Improved graph behavior and memory utilization.  Stats API now includes streamlined data structures for use with the Charts.js package to simplify custom dashboard creation.
    • Added SQL Highlighting support to make the analytics tab easier to read.
    • Resolved the placement of one-click rules, so that they don’t always to go the top of the rule list.
    • Added a one-click Async configuration option.
    • Resolved a Null pointer exception that would be triggered by using the VDB cache reset button.
    • Added an API to force reloading of configurations from the disk via /api/config/reload.
  4. Numerous fixes to various compatibility issue in resultset Metadata, which in rare situations could impact an application.  Now, testing pulls through Heimdall and direct to the database server, and verifies that all fields and data match to the byte, including in Blob and Clob field cases.  These fixes also resulted in a measurable performance boost, in particular in high-cache hit rate situations.
  5. Improved simplifying of SQL, in the case where a field includes an escaped quote.
  6. Added “rediscluster” as a cache option, leveraging the Lettuce libraries (non-cluster Redis still uses Jedis).
  7. Default “gridcache=jcache” to use Hazelcast’s jcache class provider for easier testing of Jcache.
  8. Added a cache option of “uselocal” which if set to “false” will prevent the use of any local (non-grid) cache.  May cause performance slowdowns in particular on heavy cache writes.
  9. Added a cache option for “serialize” to force serialization of objects when stored in the local cache.  While this adds overhead on both store and retrieval, it allows much more accurate cache size numbers to be tracked.  Memory usage of the grid-cache interfaces will continue to be unaccounted for in most cases.
  10. When serialization is not in use, cached objects are now measured by the number of rows, and assumed to be 1kb per row for memory utilization purposes.  This may result in over or under-estimated cache usage.
  11. Added the ability to specify a Hazelcast configuration XML to override built-in configuration options.
  12. Enabled enforcement of the enable/disable flag on the data source configuration objects.
  13. Beta test support for a MySQL protocol level proxy.  This code allows applications such as WordPress, Magento and others to leverage Heimdall functionality for caching, security and reliability.  Please contact Heimdall support if you wish to test against this beta feature for more information.

Release Version 1.0.5 (May 23, 2016)

Release 1.0.5 is primarily a bug-fix release, resolving the following issues:

  1.  A packaging issue with the 1.0.4 release vendor neutral package is resolved, which if installed in a non-root account could cause problems with driver download on an application start;
  2. Resolved two issues preventing the cache button from properly identifying patterns that were covered by a cache rule already;
  3. If the Heimdall server component was restarted after the application was started, and the server had incremented the configuration counter, then the driver may not pull new configurations until the configuration version matches what had been set by the previous instance of the server;
  4. Improved database failure detection with multiple application servers;
  5. Improved thread behavior on the server to help increase the performance of logging.

Additionally, two new features have been introduced:

  1. A “stack trace” rule has been added, to help identify the source of queries, including an option to only generate a trace if a query is slow;
  2. A default option on tag rules, to allow the creation of a default rule template for a long rule list.

Release Version 1.0.4 (May 13th, 2016)

  1. Add rate limit policy parameter to specify the queries matching the policy to X queries per second.   Allows bursting for up to 10s worth of queries, then will back down.  Rate limiting follows the “leaky bucket” model.  To use, specify the parameter “ratelimit” and a value in queries per second allowed.  Traffic exceeding this will be throttled to the limit.  The maximum burst size is based on ten seconds worth of traffic by default, although this can be overridden using the “maxburst” parameter, and is the number of seconds worth of traffic that can be burst before pacing occurs.  Once throttling is enforced, pauses will be for no smaller of an interval than 1ms.
  2. Add a match rate limit parameter.  Similar to the rate limit, but restricts how often a particular policy will be matched.  This is particularly useful to limit the volume of logs to at most X log entries per second to prevent log overflow and reduce overhead on logging.  To use, specify “matchlimit” with an integer numeric value, for matches/second allowed.  The “maxburst” option is also used by this parameter.
  3. Add a “stop” parameter, to indicate rule processing should not continue after a match on this rule, accepts the value of “True” or “Yes”.
  4. Call action for a parameter–this allows nested rule-lists to be created, i.e. to allow one rule-list to call another rule-list for cache policies, another for firewall policies, etc.  Execution is continued on a rule-list after the called rule-list completes. 
  5. Add a “learn pattern” function to learn query parameters.  On a match to the learn pattern policy, a new policy will be created in the named rule-list, of the type specified.  If no rule-list is named, it will be added to the current rule-list, and if no type is specified, it will be an “allow” action.  The primary purpose of this is to allow learning of normal traffic, and automatic generation of whitelist rules, allowing a drop of unknown traffic to be usable.  It can be used to generate other rule types as well however, allowing fine-tuning of traffic on a per-query pattern basis.
  6. Allow action type–in conjunction with the learning mode, an allow action will force the bypass of any further “allow”, “drop” or “learn pattern” rules.  This bypasses all rule processing except for sample, exclude or matchlimit processing.
  7. In support of the call function, now multiple rule lists can be attached to a vdb.  If not attached, a called rule-list will not be executed, as it will not be provided as part of the configuration for the driver.
  8. Enhanced configuration server logging to report the status of the monitored servers in a more visually useful way.
  9. Added a summary tab, to provide an overview of the benefits that optimization is providing, if enabled.

Release Version 1.0.3 (May 1, 2016)

  1. Added PosgreSQL statechange script, with docker integration for full redundancy of PostgreSQL.
  2. Added full Postgres+Jira install script in the jira directory in the platform neutral install package.  Tested on Ubuntu 16.04 LTS, and allows fast, easy setup of redundant Postgres servers and the latest Jira via Docker.  When run, it will setup the Heimdall instance to manage the PostgreSQL instances and will configure Jira to use them.
  3. Improved logging for failovers.
  4. Added the ability to flag a rule with a “capture=true” property, allowing property values to be modified by regex capture groups, much like transformation does.  Example rule using this:
    regex: “^/\* (.*) \*/”, action: “forward”, parameter: target=”source-$1″, capture=true.  Target name would extract from the SQL comment and add to the data source name, for dynamic data sourcing.
  5. Improved logging of connections, to match what is logged in the query logs.
  6. Added the ability to monitor PostgreSQL pg_stat_statements on a data source by adding a connection property “pgmon=name”, where name is the vdb the data will be associated with.
  7. Resolved an issue with batch execution, where it would report an exception, even though the batch had been executed properly.
  8. Added logging of newly observed patterns in the logs, even with SQL logging disabled.  This can be useful for helping track security issues after the fact, yet in most configurations will result in very little logging.
  9. Add several new action types:
    • Drop:  Results in an SQL exception for a matching query
    • Evict:  Accepts a list of tables to evict, or the keyword “all”.  Failure to list tables will evict all.  Also accepts a parameter of “olderthan” which can specify an age an object can be to avoid expiry.  This value can be negative to prevent objects from being returned from the cache for a given period of time.  This value is in ms, so 10,000 would be 10 seconds.
    • Ignore:  Accepts the name of a ruletype following to ignore, will honor order of operations.  Table cache rules can’t be ignored this way.
    • Tag:  A no-op action that accepts global parameters, i.e. tables and delay.
  10. Added new global option “printtables” to print the tables extracted for queries to the app’s stdout, for debugging table extraction issues.

Release Version 1.0.2 (March 2, 2016)

A number of issues were resolved:

  1. If a rule was not bound to a vdb, then the “log sql” option was not logging SQL.  Even an empty rule would resolve this.
  2. In certain corner cases, SQL parsing was triggering a loop in isolating tables.
  3. When using table caching, and a whitelist configuration, grid caches were not being initialized.
  4. A memory leak was resolved when applications were not closing connections due to connection pooling, and were not reusing statements.

Many improvements were also made:

  1. Extensive optimization to lower overhead.
  2. Improvements were made in the performance logging to more accurately log the execution time of queries.
  3. The trace action has been removed, and rolled into logging, to simplify configurations, and to better support the analytics only setup.
  4. Added a generic local cache layer above grid caches.  This improves performance on ingestion as well as maintains a small number of cache entries in local heap for faster retrieval.  The size of this local cache is computed from the cache size, at 1/1000 of the configured cache size, but in entry count.  A 50MB cache setting will result in up to 50k objects being maintained locally.  To only use the local cache, the grid cache name of “none” can be specified.
    each page now has an export to pdf option to save the contents of the page.
  5. updated the default grid-cache to Hazelcast 3.6, and Hazelcast near-cache was removed, as a generic equivalent was added for all grid caches.
  6. removed any direct use of the java class “sun.misc.Unsafe” in advance of Java 9 being released, which may remove support for this.  Grid libraries may still use this however.
  7. Heartbeat packets to the central server now report memory usage by the JVM, to help track down heap utilization issues after the fact.  Future versions of code may allow this to be analyzed in the GUI, but currently it is logged for offline analysis.

Notes on behavior:
The memory reporting and control of cache size is very inexact, as with Java, it can actually be costly to compute the size of objects in memory.  The local cache simply uses an estimate that each object will take up 1000 bytes, but on the grid-cache level, it will query, when available, an appropriate api to ask the grid how much memory it is using.  This is supported on Hazelcast (when using the default binary format) and EHcache, but not on other caches.  In such a case, the memory usage of these caches will not be reported at all.

Version 1.0.1 Initial Release (Jan 18, 2016)

The significant change in this build is the release model.  As of this build, pooling, logging and analytics are free.  Additional features including any rule processing and load balancing are premium features.  Please contact us for a quote for your install if you are interested in these features.  Additional changes include:

1)  Resolved a set of issues involving JDBC batch execution, both with logging and actual response exceptions;
2)  Minor cosmetic changes

License behavior:  In order to ensure that customers don’t ever have an outage caused by a license issue, the license model is currently an advisory model–it will tell you if your use of the system is out of license, so you can correct it or contact sales to obtain a new license.  Otherwise, features are free!  Please check the “Files” tab for information on what is allowed.  For the first 30 days, all features will be enabled for up to 4 client systems/application servers and up to 4 load balanced database servers.  Clients not accessing VDBs with rules will not count to this limit, nor will data sources that are not load balanced.

As always, please contact support if you have any issues with this product.

Version 1.0 Beta 4 Release Notes (Dec 29, 2015) — Major Upgrade
IMPORTANT NOTES: 

1) For caching to operate properly, either the Hazelcast jar file needs to be in the classpath used by the driver, OR an alternate grid cache provider needs to be specified in the cache configuration.  As part of a partnership with Hazelcast, we are now defaulting to using Hazelcast as the primary cache provider.  This will automatically provide shared cache resources if multiple JVMs are used on the same server as an added benefit.

2) The interface between the client and server has been adjusted, and requires that both be upgraded together.

New Features:
Hazelcast is now default:
  The internal cache subsystem has been removed, and Hazelcast has been added as a default.  It is configured with a near-cache as well as a local shared cache setup, allowing processes to share their cache without additional configuration.  As a side-effect of this, the cache memory graph is only an estimate of the memory used, and is often higher than is actually being used.

Table level locking during Asynchronous Modifications.  When using the Async Execute method, the table the query is executed against is detected, and the table locked from reads until that execute has completed, if the execute is made on the same JDBC connection.  This should prevent most cases where a read expects a write to have been completed.  Further, in conjunction with our table based expiry, it will also insure that the data read will not be stale content if caching is also in play.

Cache clear button:  Each VDB now has a button to purge the cache for that VDB.  This button will not actually purge the cache memory, but sets a watermark that any object retrieved from cache older than the current time will be considered stale, and purged on retrieval.

Log management:  The files tab now has log management buttons.  Clearing the raw log data (or all logs) will clear both the on-disk logs and in-memory log data that has not yet been written, allowing a clean start when comparing behavior changes.

Automatic log analysis on log roll:  On a log roll, the data in the log will be summarized automatically, resulting in significantly faster “analysis” operations, in particular under high log volume.  Log analysis is also done in a parallel manner, with one thread per VDB, to speed up analysis.

Removal of the trace action:  The logic for trace has been rolled into the log action, and removed from the menu.  If a query returns a result-set, a trace record will be generated.  If it was a cached result-set, only partial trace data will be included, to support timing operations.

Behavior Changes:

Async batching removed:  When using Async Modifications, no batching will occur.  This conflicted with the locking to prevent reads, and impacted performance during batch creation, so was removed for now.  This may be added back in as an option in the Async command if requested.

Catalogs need to be specified in table caching:  In order to guarantee consistent behavior across table caching, table purging, and table blocking for async operations, the table name has been normalized to include the catalog.tablename.

Autocommit caching:  To prevent accessing the database to find out the state of Autocommit, the value is now cached in the Heimdall driver.  If a command outside of the JDBC API causes this to change, it will not be tracked by the driver.

Analytics tab columns:  The “cache benefit” column was deemed confusing in intent, and has been broken into a server time % and duplicate query and response column.  The previous behavior was to multiply these together, but this ended up giving unreasonably low numbers on what can be expected.  These are now broken up for individual analysis.

VDB Cache Memory limit:  When used with EHCache, this is enforced, but as Hazelcast doesn’t have an explicit memory control, it is not enforced at this time.  Instead, this value is divided by 1000 to set the object limit in the cache.

Bug Fixes:
Properly escape single quotes when used as part of a prepare-execute, for logging and other purposes.
Improved response time reporting in several cases.  The logs should be more accurate than before.
Removed reference to some SSL functions in the driver code.  This may have impacted some environments using SSL depending initialization order.
General cleanup of code and consistency of behavior.

Version 1.0 Beta 3 Release Notes (Nov 30, 2015) — Major Update
Improved compatibility was added for trigger based expiry for databases other than MySQL.  The trigger expiry system now provides example configuration for Microsoft SQL Server, and allows customization of the stored procedure calls made by the Heimdall driver to the database.  The driver will now also attempt more than one connection try during a health-check based on the data source connection parameter “failcount”, which defaults to a value of 3.  The state change script now executes each output command as they are received from the script, allowing multi-state “commits” of the configuration, allowing all data sources to be disabled for a period while orchestration brings online new data sources.  Help pages have been updated to document the behaviors, including a new load balancing section in the data sources help.  Help for the Analytics tab has also been added, providing detailed explanation of the result columns.

Version 1.0 Beta 2.2 Release Notes (Nov 22, 2015) — Minor update
Further improvement in HA, to improve the behavior of a server with a weight of “0”.  This value can now be set to specify a “server of last resort” for a data source, which can be used immediately by a driver when other higher weight servers are not accessible.  This is best used when a server cluster is setup as active-active, but only one is intended to be used at a time, but both in theory could be used at once.  This allows nearly instant failure to be performed without synchronizing with the central server.  For best results, this should be used in conjunction with a fail-over script that will guarantee consistent behavior between all nodes in the event of a split-brain network issue in a short period of time.

Additionally, the script name that is used to initiate centralized failure configuration changes has a new naming scheme.  Now, instead of it being a static name of “statechange.(sh|ps1|bat|py|pl)” it will now append the name of the data source that failed when finding the script to “statechange-“, i.e. instead of statechange.bat, it will look for statechange-dbdemo-mysql.bat for the servers in the example configuration.  An issue in locating this script on Unix systems was also resolved.

Version 1.0 Beta 2.1 Release Notes (Nov 21, 2015) — Minor update
Minor update improving HA functionality, including adding a connection property of “failCount”, specifying the number of times a health check needs to fail before a server is flagged as down.  If not specified, the default value of “3” is used, corresponding to a three second outage..  Download link and documentation remains the same.  Failover should also be more reliable under all situations (pooled and non-pooled setups).

Version 1.0 Beta 2 Release notes (Nov 17, 2015) — Major update

New Features:
Asynchronous Database Modification

Insert, Updates and Deletes can now be flagged in the rules to operate in an asynchronous way.  Any such calls will be immediately returned with a success flag, and will be executed in batch mode against the database.  When performed against many inserts of the same type, this can result in a 10x improved insert response, depending on driver support for executeBatch.  It is suggested that when using this feature on MySQL, the connection parameter of “rewriteBatchedStatements=true” be used.  Note:  If the call has been flagged to provide auto-generated keys, then it will automatically bypass the async feature for proper operation.  Additionally, only queries flagged or detected as an “update” will be processed this way.

Database Trigger Synchronized Tables
This feature allows the Heimdall Driver to synchronize its cache with the database server, to insure that cache entries are expired within one second.  This feature requires the use of trigger configuration on the server, which is a feature supported by all major RDBS vendors.  An example script for the MySQL database is available at http://download.heimdalldata.com/mysql-triggers.sh and can easily be adopted to other environments.  Contact customer support if you wish to have an example script created for other database products.  This feature will autodetect if the proper configuration is in place on the server side, and will report in the application log if it was disabled after probing.

JSR107 aka JCache support
A generic JCache interface has been created, and now replaces the Hazelcast option, as Hazelcast was being used in JCache mode.  Please see rule documentation for details on how to specify this.  Note:  Apache Ignite has been tested, but appears to have an issue where memory garbage collection can impact performance even under under heavy read load.  The EHcache interface does not have this issue.

Explicit Tables and Update Flag
As stored procedures may or may not make modifications to tables, rule parameters of “tables” and “update” has been added to allow the internal logic to be overruled, and tables to be specified and update to be flagged.  This allows proper purging of tables on a stored procedure call.  This can also allow better support for purging of underlying tables when a view is used on the database.

Induced Delay on Rule
In order to help determine if a cache policy will impact performance, a simple, if counter-intuitive option is now available:  induced delay.  By adding a cache policy with a TTL of 0, then a second attribute of “delay=x” in milliseconds, you can add additional latency to queries, then measure the application performance change.  If by doubling the average response time, you notice only a 1% change in actual application performance behavior, then caching will at best have an inverse performance benefit.  This can provide a safer mechanism for testing, as the risk of stale data is not present when used this way.

In addition, many issues and minor improvements have been resolved:
1. Reduced overhead of resultset handling, improving performance by up to 66% on high-cache hit workloads, along with improved performance when using prepared statements.
2. Allowed sub-second cache TTL even on grid caches that don’t support this (ehcache, redis, memcached, jcache, etc)
3. Impoved TTL handling on Ehcache and other grid caches, including randomizing an object’s TTL value from 90% to 100% of the specified TTL, so that if a large number of cached items are inserted at once, they will not expire at the same time.
4. Corrected an issue with the analytics tab when running on Linux and similar environments, where only the most recent log data was processed.
5. Improved table extraction to account for more cases
6. Fixed an issue when Clob responses had the toString() method called on them, when served from cache, and if the clob getSubString() method had a length specified of zero.
7. Adjusted license enforcement to only count clients and database nodes, cpu cores, jvms and other license artifacts have been removed.
8. Always log rollback and commit methods, to allow transaction time to be computed on a connection.  Additionally, any method that takes over 100ms to execute will be logged as well.
9. Improve support for older ehcache versions, to as old as 2.1.0 (May, 2010)
10. Adjusted JSON processing to ensure that all messages are sent to the log server and processed correctly.
11. Adjust the log reporting of prepared statements to report the connection the resulting statement was prepared on instead of the prepare itself (for ease of debugging)

Known issues:
HEIM-45:  In Internet Explorer and Edge, dashboard graphs do not refresh autocratically.  They have been tested to work properly on Firefox and Chrome.
HEIM-124:  If you delete an active ruleset from the GUI, the change isn’t pushed out.  To remove a ruleset in use, first unbind it from the VDB, then delete the ruleset.
HEIM-135:  Due to compatibility with Java 6, many Java 7 and Java 8 JDBC methods are not yet supported.  Please let Heimdall support know if particular methods are required for your application.  Most applications and frameworks limit JDBC calls to those supported in Java 6, so this should not impact most applications.

Version 1.0 Beta Release Notes (Oct 30, 2015)
Initial public release made.  Installation guide can be found at http://download.heimdalldata.com/downloads/Heimdall_Data_Access_Platform_Installation_and_Setup_Guide.pdf, and the Windows 64 bit release image is at http://download.heimdalldata.com/downloads/heimdalldemo.msi.  The platform neutral install package is pending, and will be available at http://download.heimdalldata.com/downloads/heimdalldemo.tgz shortly.

Known issues:
HEIM-45:  In Internet Explorer, dashboard graphs do not refresh automatically.

HEIM-124:  If you delete an active ruleset from the GUI, the change isn’t pushed out.  To remove a ruleset in use, first unbind it from the VDB, then delete the ruleset.

HEIM-133:  When a query contains two or more table names in the from clause, if there isn’t a space after the comma dividing them, then it can result in the table name not being extracted properly.  This can impact table based caching and table based purge during writes. (resolved in Beta 2)

HEIM-135:  Due to compatibility with Java 6, many Java 7 and Java 8 JDBC methods are not yet supported.  Please let Heimdall support know if particular methods are required for your application.  Most applications and frameworks limit JDBC calls to those supported in Java 6, so this should not impact most applications.