HELP - VDB

Virtual Databases (VDB) are the basic container for configurations, and aggregate all the settings that will apply to any traffic passed through the VDB.  There are two ways to access a VDB, first via JDBC, which supports any JDBC compliant data source, or via a protocol level proxy.  The first step in configuring a VDB is to select the access mode, which will adjust what options are presented, to simplify the configuration.

When selecting JDBC, a JDBC URL is provided, which is used to configure the application.  The JDBC URL is of the format:

JDBC:heimdall://IP:port/{vdb name}?<parameters>

Optionally, several parameters can be used in conjunction with the Heimdall Connect string:

  • user=[username]: The username to use to authenticate against the Heimdall server with.
  • password=[password]: The password for the Heimdall server.
    In some cases, the application will need to control the actual username and password for accessing the database. In such a case, an alternative to the user and password should be used:
  • hduser=[username]: The username to authenticate against the Heimdall server with. Any “user” parameter will be passed directly to the database, overriding the user configured in the data source.
  • hdpassword=[password]: The password for the Heimdall server, again overriding the parameter in the data source.

Note: When using the hduser and hdpassword, the data source credentials will still be used for health checking the application.

Important:  In most cases, the hduser and hdpassword need to be specified.  In secure environments, the hduser, hdpassword, user and password can also be passed as environmental variables for proxy operation.

Proxy Configuration

When in proxy mode, several options will be available, with only the proxy port being required.  This port should be specified so as to not conflict with any other application on the system the proxy will run, i.e. if a MySQL server is on the same system, you can not use port 3306, if that is what the MySQL server is configured for.

Additionally, there are several other options:

  • Localhost only:  When enabled, the proxy will only accept traffic on the localhost IP of 127.0.0.1.  This is as safe as if a firewall were in place preventing external traffic to connect to the proxy.
  • Management Server Proxy:  This option allows the management server to start and manage restarts of a proxy instance on the same server as the management server itself.  This option is a great way to simplify the testing of Heimdall in limited environments.
  • Proxy-auth Enabled:  In most cases, connectivity to the proxy should be authenticated.  As JDBC requires a username and password behind the scenes to connect to the server, the user and password must be specified here, then when authenticated, will be used to connect to the back-end database.  The user and password specified in the data source will be used if proxy authentication is not allowed, but in such a case, it is STRONGLY recommended that the localhost only option be provided to prevent security issues.  The same username may be specified multiple times with different passwords, in order to facilitate the changing of the password in production environments with minimal coordination.

Caching

This set of options configures the base cache used by a given VDB.  There can be only one cache per VDB at any time, and if the cache settings for a given type are to be changed at runtime, it is suggested that the cache be disabled first, then enabled again with the new settings.

Options:

  • Enable query caching:  Check to enable caching–if not checked, no cache rules will trigger caching, although rule processing will be done as normal.
  • Auto-tune Cache:  Enabling this option will enable the cache logic to disable caching in situations where it doesn’t make sense from a performance perspective.  This includes if a query pattern isn’t providing any cache hits, or the benefit of the hits that do occur isn’t sufficient to justify caching.
  • Cache Type:  Select the specified cache type.  Options will adjust based on the cache type selected.
  • Server:  For external grid cache interfaces, specifies the server name or IP to connect to. 
  • Port:  When using an external grid cache, specifies the TCP port to connect to.
  • Cache Password:  For interfaces with a password authenticated interface, the password to connect to the cache with.
  • API Cache Name:  For interfaces that utilize a “named” cache interface, this is the name of the cache to use.
  • Cache Configuration File:  When supported, this is an external configuration file to configure the cache, i.e. a Hazelcast client or server XML configuration file.  The location of the file should be relative to the current working directory of the JDBC driver or proxy, which will be printed on startup.
  • Grid Cache Offload:  This option, if selected, enables a first-tier local cache, which allows hot content to be served out of local heap.  If the serialize option is enabled, requests will be stored in serialized form vs. object form however, and will still have a computational overhead to de-serialize.  When enabled, the local cache also observes the cache size, although the size is a simple low-overhead estimate when serialization is not used.
  • Customize query cache key:  This allows the key used to access cached objects to be customized in order to expand when the result is considered the same.  By default, all queries will include a hash of the complete SQL query, but also includes the VDB name the request was made through, the database user, and the database catalog.  These fields can be removed to allow, for example, multiple VDBs to share the same data in the cache.
  • Cluster manager via cache:  This option allows a VDB to be managed by multiple management nodes at once, via the cache interface.  For this to work, all management nodes must be populated with the vdb, and the cache settings configured to be identical.  Once done, configuration changes for that VDB will be synchronized between nodes, and the cache will be used to propagate metrics as well, allowing the dashboard to be used at the same time on both nodes.  Further, if one management node goes offline, the access layer nodes will remain unaffected.

Data Source

At least one data source needs to be configured, as a default for data to be retrieved from. If a forward policy is specified in the rules for a vdb, it also must be selected here to insure proper connectivity is established to that data source for the forward function to work properly. A connection to the data source will only be established when used if not the primary data source.

Rule List

Here, the rule list that is to be applied should be configured. If empty, no rule list will be used.

VDB Properties

Allows a list of name-value pairs to be used to configure various options, in general per direction from Heimdall support.

Logging

The following log options are available:

  • Log Connections: to log when a JDBC connection is established by the calling application
  • Log All SQL: equivalent to enabling a LOG policy with a .* wildcard
  • AWS Cloudwatch Metrics:  If in AWS, and selected, then up to eight metric points will be generated under the Heimdall category for the VDB and AWS instance to allow monitoring of the access layer performance.
  • Aggregate Console Logs:  In order to simplify debugging, this option allows console logs from the access layers to be sent to the central manager for logging.  This avoids issues where the logs may not be showing, due to Log4j configuration, for example.

Important: When logging, a large amount of data may be generated, and logging can impact the overall performance.

Debugging

  • Log Methods: log all JDBC method calls made by an application, excluding those relating to resultSets. Warning!
  • Log ResultSet Methods: Include ResultSet operations when logging other methods. Warning!
  • Verify Cache:  When in JDBC mode, this option allows the resultset returned by the cache and the database to be compared, and generate a log entry if they are different.  This is useful for validating that the correct behavior is being observed.
  • Verbose Debug Mode:  In order to diagnose issues with rule processing or other behavior anomalies, this option can be set in order to track the processing of a query through the access layer.  This option can cause significantly performance penalties and should be used with caution.

Important: When adding in method logging, for every query generated, it may result in a dozen records for JDBC methods, and with resultset methods, every row may have one or more log records.  Caution should be observed when using these two log options are used, and should in general only be used in low-volume lab conditions.

VDB Properties

Allows a list of name-value pairs to be used to configure various options, in general per direction from Heimdall support.