Help – Heimdall Logging

When Heimdall starts, either in proxy mode, or in JDBC driver mode, it will generate a variety of log messages that provide an indication on what is configured, and if there are errors, what the errors are.  Below is an example of Heimdall starting up in MySQL proxy mode, with a Redis cache configuration.  These logs can be found in any one of several locations:

  1. In the case of the JDBC driver, this will be in the stdout of the application, often in the catalina.out for Apache Tomcat based applications.
  2. In the case of the Heimdall Central Manager starting and managing the proxy, these logs will be included in the /var/log/heimdallserver.out.log file.
  3. In the case of a standalone proxy install, they will be located in /var/log/heimdallproxy.out.log files.

In the case of an error or warning, these should include the string “Error” or “Warning”.\

In order to trigger extremely verbose logging of the Heimdall processing, you can add a VDB parameter of “debug” and set the value to “true”.  This will impact performance, so should be used carefully, but will provide information about what queries are being processed, what rules are matching, and any errors or warnings during processing that should be noted.

[2017-03-06 15:43:42.690] Heimdall-proxy magento-old-vdb: Proxy initialization started
[2017-03-06 15:43:42.692] Heimdall-proxy magento-old-vdb: Log4j library is not found in classpath, so logging is switched to console mode
[2017-03-06 15:43:42.692] Heimdall-proxy magento-old-vdb: Classpath URL's (will add from ./libs/ automatically):
[2017-03-06 15:43:42.775] Heimdall-proxy magento-old-vdb: Initializing Heimdall driver version:1.0.11
[2017-03-06 15:43:42.776] Heimdall-proxy magento-old-vdb: Class jars in classpath:
[2017-03-06 15:43:42.776] Heimdall-proxy magento-old-vdb: /opt/heimdall/heimdalldriver.jar
[2017-03-06 15:43:42.776] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/netty-buffer-4.0.37.Final.jar
[2017-03-06 15:43:42.777] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/netty-codec-4.0.37.Final.jar
[2017-03-06 15:43:42.777] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/netty-transport-4.0.37.Final.jar
[2017-03-06 15:43:42.777] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/slf4j-api-1.7.7.jar
[2017-03-06 15:43:42.778] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/lettuce-3.5.0.Final.jar
[2017-03-06 15:43:42.778] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/ehcache-2.10.3.jar
[2017-03-06 15:43:42.778] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/rxjava-1.1.6.jar
[2017-03-06 15:43:42.779] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/hazelcast-all-3.7.4.jar
[2017-03-06 15:43:42.779] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/spymemcached-2.12.1.jar
[2017-03-06 15:43:42.779] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/guava-17.0.jar
[2017-03-06 15:43:42.779] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/netty-common-4.0.37.Final.jar
[2017-03-06 15:43:42.779] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/netty-handler-4.0.37.Final.jar
[2017-03-06 15:43:42.780] Heimdall-proxy magento-old-vdb: /opt/heimdall/libs/cache-api-1.0.0.jar
[2017-03-06 15:43:42.780] Heimdall-proxy magento-old-vdb: Heimdall driver loaded from: file:/opt/heimdall/heimdalldriver.jar
[2017-03-06 15:43:42.780] Heimdall-proxy magento-old-vdb: Heimdall current working directory: /opt/heimdall
[2017-03-06 15:43:42.793] Heimdall-proxy magento-old-vdb: Supported actions and parameters:
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: Context        Param Name    Default Value         Help
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            capture       false                 Specifies that capture groups should be used from the matched regex, to fill in values in the parameters, $1 for the first, $2, for second, etc
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            delay         null                  Adds X ms to each query, to help model the impact of latency between the db client and server, for hybrid cloud impact testing
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            exclude       null                  Exclude matching the rule 1/X times, to allow a small sample of queries to match
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            invalidate    true                  Can be specified to override invalidation behavior of writes for matching queries, default is to allow invalidations to occur.
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            log           false                 When specified, behave as if a log rule had matched
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            matchlimit    null                  Specifies the frequency (in X per second) that a rule should match
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            maxburst      10                    When using matchlimit or ratelimit, specify the number of seconds worth of bursting to allow before limiting the rate
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            notify        null                  If specified as true, flag that the notification configuration should be used to notify
[2017-03-06 15:43:42.794] Heimdall-proxy magento-old-vdb: all            printmatch    false                 When a rule matches, print the rule
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            printresults  false                 Prints to stdout the result of matching queries
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            printtables   false                 If specified, print the table attributes to be used for caching & invalidation
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            printtiming   false                 Prints to stdout the execution time of this rule, useful for profiling called rulelists, such as for the SQL Firewall
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            ratelimit     null                  Limits matching queries to X per second.  This is tracked on a per-rulelist basis, so multiple rules can impact each other--use with care
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            sample        null                  Match the rule only 1/X times, often to reduce frequency of logging, but remain proportionate to load
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            stop          false                 If specified on a matching rule, no further rule processing in the current rule list will occur
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            tables        null                  For any matching rule, specifies the tables that should be added to a query's table list.  Use "none" to allow caching without tables
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: all            update        false                 Specifies if the matching queries should be processed as updates by the cache engine
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: async execute  asyncsize     32768                 The maximum size of a query that can be made asynchronous
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: cache          ttl           0                     In table cache rules, specifies the name of the table to apply settings for
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: call           rule          null                  The name of the target rule of the action.  This rule must be in the vdb configuration for it to be called.
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: evict          evicttables   null                  Tables that should be evicted when a query matches the rule
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: evict          olderthan     0                     specify the time window in ms to allow content to not be evicted on eviction.  Negative prevents caching into the future.
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: forward        source        null                  The name of the data source to forward the query to.  This source must be specified in the VDB definition.
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: ignore         action        allow                 The action type to apply to the current rule.  For leaning, specifies the default rule type, for ignore, the type of rule to ignore
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: learn          action        allow                 The action type to apply to the current rule.  For leaning, specifies the default rule type, for ignore, the type of rule to ignore
[2017-03-06 15:43:42.795] Heimdall-proxy magento-old-vdb: learn pattern  rule          null                  The name of the target rule of the action.  This rule must be in the vdb configuration for it to be called.
[2017-03-06 15:43:42.796] Heimdall-proxy magento-old-vdb: stack trace    threshold     0                     Response time threshold for stack traces to trigger
[2017-03-06 15:43:42.796] Heimdall-proxy magento-old-vdb: table cache    table         null                  In table cache rules, specifies the name of the table to apply settings for
[2017-03-06 15:43:42.796] Heimdall-proxy magento-old-vdb: table cache    ttl           0                     In table cache rules, specifies the name of the table to apply settings for
[2017-03-06 15:43:42.796] Heimdall-proxy magento-old-vdb: tag            defaults      false                 Specifies a list of default properties to apply to all following rules in rulelist
[2017-03-06 15:43:42.796] Heimdall-proxy magento-old-vdb: transform      target        null                  When translating a query, the target of the translation, with $1 representing the first capture group, $2 the second, etc.
[2017-03-06 15:43:42.796] Heimdall-proxy magento-old-vdb: vdb            debug         false                 Force driver to print debug messages as queries are processed and acted upon.
vdb            proxydriver   com.heimdalldata.HeimdallDriver The JDBC driver the proxy should use, defaults to the Heimdall JDBC driver, but can specify a vendor JDBC driver if desired
vdb            proxyurl      null                  The URL of target DB for the case if vendor driver is used instead of Heimdall
[2017-03-06 15:43:43.242] Heimdall-proxy magento-old-vdb: SIGHUP registered to trigger extra logging
[2017-03-06 15:43:43.243] Heimdall-proxy magento-old-vdb: Heimdall URL:jdbc:heimdall://localhost:8087/magento-old-vdb?hduser=magento&hdpassword=i-0d640e105fec3774a
[2017-03-06 15:43:43.246] Heimdall-proxy magento-old-vdb: Getting config from: http://localhost:8087/api/config/vdb/magento-old-vdb
[2017-03-06 15:43:43.530] Heimdall-proxy magento-old-vdb: Connection properties set to: {jdbcCompliantTruncation=false, url=jdbc:mysql://172.31.56.200/bitnami_magento, relaxAutoCommit=true, sessionVariables=sql_mode='', elideSetAutoCommits=true, useDynamicCharsetInfo=false, useInformationSchema=false, user=root, useSSL=false, useLocalSessionState=true, testQuery=SELECT 1, tinyInt1isBit=false, useUnicode=true, password=i-0d640e105fec3774a, zeroDateTimeBehavior=convertToNull}
[2017-03-06 15:43:43.530] Heimdall-proxy magento-old-vdb: Connection properties set to: {jdbcCompliantTruncation=false, url=jdbc:mysql://172.31.56.200/bitnami_magento, relaxAutoCommit=true, sessionVariables=sql_mode='', elideSetAutoCommits=true, useDynamicCharsetInfo=false, useInformationSchema=false, user=root, useSSL=false, useLocalSessionState=true, testQuery=SELECT 1, tinyInt1isBit=false, useUnicode=true, password=i-0d640e105fec3774a, zeroDateTimeBehavior=convertToNull}
[2017-03-06 15:43:43.531] Heimdall-proxy magento-old-vdb: Grid cache for magento-old-vdb was null, initializing grid cache.
[2017-03-06 15:43:43.533] Heimdall-proxy magento-old-vdb: Setting up grid cache interface for vdb: magento-old-vdb
[2017-03-06 15:43:43.544] Heimdall-proxy magento-old-vdb: Grid cache option serialize: true useLocal: true
[2017-03-06 15:43:43.545] Heimdall-proxy magento-old-vdb: Grid cache selected: redis
[2017-03-06 15:43:43.550] Heimdall-proxy magento-old-vdb: Connecting to redis server: wp-test.gpfoym.0001.use1.cache.amazonaws.com port:6379 password: xxxxxxxx mode: standalone
[2017-03-06 15:43:43.646] Heimdall-proxy magento-old-vdb: Mar 06, 2017 3:43:43 PM com.lambdaworks.redis.resource.DefaultClientResources <init>
[2017-03-06 15:43:43.646] Heimdall-proxy magento-old-vdb: INFO: ioThreadPoolSize is less than 3 (2), setting to: 3
[2017-03-06 15:43:43.649] Heimdall-proxy magento-old-vdb: Mar 06, 2017 3:43:43 PM com.lambdaworks.redis.resource.DefaultClientResources <init>
[2017-03-06 15:43:43.649] Heimdall-proxy magento-old-vdb: INFO: computationThreadPoolSize is less than 3 (2), setting to: 3
[2017-03-06 15:43:43.892] Heimdall-proxy magento-old-vdb: Mar 06, 2017 3:43:43 PM com.lambdaworks.redis.resource.DefaultClientResources <init>
[2017-03-06 15:43:43.892] Heimdall-proxy magento-old-vdb: INFO: ioThreadPoolSize is less than 3 (2), setting to: 3
[2017-03-06 15:43:43.893] Heimdall-proxy magento-old-vdb: Mar 06, 2017 3:43:43 PM com.lambdaworks.redis.resource.DefaultClientResources <init>
[2017-03-06 15:43:43.893] Heimdall-proxy magento-old-vdb: INFO: computationThreadPoolSize is less than 3 (2), setting to: 3
[2017-03-06 15:43:43.911] Heimdall-proxy magento-old-vdb: Mar 06, 2017 3:43:43 PM com.lambdaworks.redis.resource.DefaultClientResources <init>
[2017-03-06 15:43:43.911] Heimdall-proxy magento-old-vdb: INFO: ioThreadPoolSize is less than 3 (2), setting to: 3
[2017-03-06 15:43:43.917] Heimdall-proxy magento-old-vdb: Mar 06, 2017 3:43:43 PM com.lambdaworks.redis.resource.DefaultClientResources <init>
[2017-03-06 15:43:43.917] Heimdall-proxy magento-old-vdb: INFO: computationThreadPoolSize is less than 3 (2), setting to: 3
[2017-03-06 15:43:43.952] Heimdall-proxy magento-old-vdb: Client startup message received on channel:hd.172.31.52.203 Sync delta: 2
[2017-03-06 15:43:43.953] Heimdall-proxy magento-old-vdb: Redis pubsub channels:1
[2017-03-06 15:43:43.953] Heimdall-proxy magento-old-vdb: Cache client is ready
[2017-03-06 15:43:43.962] Heimdall-proxy magento-old-vdb: heimdall: Startup performed, instance ID: 72f2f8c5-ab2c-4626-869a-5bda9361f563
[2017-03-06 15:43:44.005] Heimdall-proxy magento-old-vdb: Creating new ConnectionPool for source: magento-1.9-ds URL: jdbc:mysql://172.31.56.200/bitnami_magento
[2017-03-06 15:43:44.364] Heimdall-proxy magento-old-vdb: Connection character set information--should start with utf8mb4 for proper chacter handling behaviors:
[2017-03-06 15:43:44.367] Heimdall-proxy magento-old-vdb: Warning: Variable Name=character_set_client Value:utf8
[2017-03-06 15:43:44.367] Heimdall-proxy magento-old-vdb: Warning: Variable Name=character_set_connection Value:utf8
[2017-03-06 15:43:44.368] Heimdall-proxy magento-old-vdb: Warning: Variable Name=character_set_database Value:utf8
[2017-03-06 15:43:44.368] Heimdall-proxy magento-old-vdb: Warning: Variable Name=character_set_server Value:utf8
[2017-03-06 15:43:44.368] Heimdall-proxy magento-old-vdb: Warning: Variable Name=collation_connection Value:utf8_general_ci
[2017-03-06 15:43:44.368] Heimdall-proxy magento-old-vdb: Warning: Variable Name=collation_database Value:utf8_general_ci
[2017-03-06 15:43:44.369] Heimdall-proxy magento-old-vdb: Warning: Variable Name=collation_server Value:utf8_general_ci
[2017-03-06 15:43:44.371] Heimdall-proxy magento-old-vdb: Got proxy port:5052
[2017-03-06 15:43:44.397] Heimdall-proxy magento-old-vdb: Listening on 5052