Heimdall Data for Amazon

Backend network latency often slows down application performance across your AWS and hybrid environment. Heimdall Data is a database proxy (a.k.a. SQL traffic manager) providing:

  • Intelligent routing to the best data source (OLTP vs. OLAP)
  • Application-tier SQL caching and invalidation without code changes
  • SQL optimization insights for your application, network, and database

Heimdall Use Cases on AWS

Amazon ElastiCache

Amazon ElastiCache

  • Heimdall provides cache logic
  • Automatic Invalidation
  • ElastiCache = Look-aside cache
  • No code changes.
  • AWS Blog or video below
Amazon Redshift

Amazon Redshift

  • Routes OLTP queries to Postgres
  • Materialized view management
  • Improve performance up to 100x
  • See below section
Amazon RDS / Aurora

Amazon RDS/Aurora

  • Auto-routes to  write /read replicas
  • Reads are ensured fresh data by lag detection logic.

Smart Routing with Heimdall Traffic Manager

Companies deploy Amazon Redshift for large-scale MPP (Massively Parallel Processing) data analytics. These systems apply parallel compute resources to answer queries quickly. However, there are still performance challenges:

     –  High latency: Distributed queries require coordination of multiple nodes to generate an answer

     – No materialized views: If a query result is used for future queries, additional work is required to preserve the result or repeated calls are needed to the same base query

     – Frequent queries slowing Redshift down processing

The ideal solution should require no code changes. Heimdall Data is that solution providing Amazon Redshift users the following features:

     –  Intelligently routes Analytics and OLTP traffic

     –  Materialized views management via Postgres

     –  Batches DML operations (e.g. singleton transactions) via Postgres

     –  Automatically caches frequent queries via ElastiCache

Fast materialized views are very important in analytics environments.  When reports are generated, a subset of data is pulled from the back-end data store. Heimdall provides the following performance enhancement:

     –  Queries against materialized views are routed to an alternate database (e.g. Amazon Aurora Postgres), acting on behalf of Amazon Redshift. Heimdall intelligently routes Analytics and OLTP traffic to the appropriate data source for optimal performance.

     –  Heimdall automatically triggers a refresh of the view and is aware of modifications from Amazon Redshift. The net result is faster reports and increased Redshift scale.

     –  DML statements are aggregated to be efficiently sent from Postgres to Greenplum.

SQL Results Caching: Heimdall’s intelligent auto-caching is executed in the EC2 application tier, removing network latency. It works together with Amazon Redshift’s query caching. The Heimdall distributed architecture allows caching to be scalable, while acting as one cache cluster. Result sets are cached in tandem from 1) Local memory and 2) Amazon ElastiCache and are invalidated upon writes to the table. Best of all, Heimdall deployment requires zero code changes.

Heimdall for Amazon Redshift Whitepaper


Heimdall for Amazon Redshift Whitepaper