Continuous Integration

Introduction

This guide explains how to set up the Snowflake emulator in a continuous integration (CI) environment. You can use the emulator to test your Snowflake integration without connecting to the real Snowflake instance.

To install the Snowflake emulator, you need to set up the lstk CLI, which authenticates, pulls the Docker image, and starts the container for you. After starting the Docker container, an endpoint (snowflake.localhost.localstack.cloud) is made available to connect to the emulated Snowflake instance, which you can use to test your data integrations in CI environments.

Configure the LocalStack CI Key

Before you begin, set up the LocalStack CI key to activate the Snowflake emulator in your CI environment. LocalStack requires a CI Key for use in continuous integration (CI) or similar machine environments. Each instance startup in a CI or comparable environment consumes one CI token.

To create a CI key, follow these steps:

1. Open the LocalStack Web Application.
2. Click the CI Keys on the left navigation pane.
3. Navigate to the Generate CI Key tab, enter a description, and click Generate CI Key.
4. Copy the generated CI key and set it as the LOCALSTACK_AUTH_TOKEN environment variable in your CI environment.

Setup the CI configuration

The following examples demonstrate how to set up the emulator in GitHub Actions, CircleCI, and GitLab CI.

GitHub Actions

name: LocalStack Test
on: [ push, pull_request ]

jobs:
  localstack-action-test:
    name: 'Test LocalStack GitHub Action'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Start Snowflake emulator
        run: |
          npm install -g @localstack/lstk
          lstk start --type snowflake --timeout 90s
          echo "Startup complete"
        env:
          LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }}

CircleCI

version: 2.1

jobs:
  example-job:
    machine:
      image: ubuntu-2004:2022.04.1

    steps:
      - checkout

      - run:
          name: Start Snowflake emulator
          command: |
            npm install -g @localstack/lstk
            lstk start --type snowflake --timeout 90s
            echo "Startup complete"

workflows:
  version: 2
  build:
    jobs:
      - example-job

GitLab CI

image: docker:20.10.16

stages:
  - test

test:
  stage: test
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    LOCALSTACK_AUTH_TOKEN: $LOCALSTACK_AUTH_TOKEN

  services:
    - name: docker:20.10.16-dind
      alias: docker
      command: ["--tls=false"]

  before_script:
    - apk update
    - apk add --no-cache nodejs npm
    - npm install -g @localstack/lstk
  script:
    - dind_ip="$(getent hosts docker | cut -d' ' -f1)"
    - echo "${dind_ip} localhost.localstack.cloud " >> /etc/hosts
    - DOCKER_HOST="tcp://${dind_ip}:2375" lstk start --type snowflake --timeout 90s