aws

S3 and Typescript

Max Rohde

Apr 27, 2022 — 3 min read

AWS S3 is a cloud service for storing data using a simple API. In contrast to a traditional database, S3 is more akin to a file system. Data is stored under a certain path. The strengths of S3 are its easy to use API as well as its very low cost for storing data.

While S3 has traditionally been used for storing larger files, it can also be used as a lightweight key-value store. Using Amazon Athena we can even run rather complex queries over our data if we store it in a structured data format such as JSON.

This article describes how we can use AWS S3 in a TypeScript application. This article uses the S3 Terraform TypeScript boilerplate as reference. You can clone the project on GitHub or create a project using the S3 Template.

Infrastructure Setup

Before we can interact with S3 from our TypeScript application, we need to create the S3 bucket on AWS. This can be achieved by using the AWS console. We could also use the SDK in TypeScript to create a bucket. However, creating infrastructure such as a bucket programmatically has a number of drawbacks (such as we need to keep track of what resources we have already created). Thus it is better to use Infrastructure as Code such as using Terraform.

While there are alternatives to Terraform such as Pulumi, Terraform is very widely used and thus we will use it in this article. Thankfully, it is very easy to set up an S3 bucket in Terraform. We need can define it with aws_s3_bucket resource in a file as follows (see main.tf):

resource "aws_s3_bucket" "main" {
  bucket = "${var.bucket_name}"

  # Remove this line if you want to prevent accidental deletion of bucket
  force_destroy = true

  tags = {
    ManagedBy = "terraform"
    Changed   = formatdate("YYYY-MM-DD hh:mm ZZZ", timestamp())
  }

  lifecycle {
    ignore_changes = [tags]
  }
}

In additional we need to configure Terraform to use the AWS provider (see providers.tf):

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.72.0"
    }
  }
  required_version = ">= 0.13"
}

provider "aws" {
  region                  = var.aws_region

   # Skipping various checks to speed up AWS provider
  skip_region_validation      = true
  skip_get_ec2_platforms      = true
  skip_metadata_api_check     = true
  skip_credentials_validation = true
}

terraform {
  backend "s3" {
    # config provided dynamically using `--backend-config` CLI parameters

    # Skipping various checks to speed up backend initialisation
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    skip_region_validation      = true
  }
}

Note this contains a number of additional configuration options that can help running Terraform faster.

Working with Buckets in TypeScript

In order to work with the bucket we have created in TypeScript, we first need to authenticate our application with AWS. For this, we first need to create a user on AWS. Once we have done that, we can supply our credentials to the AWS SDK in various ways.

In this sample project, we use the @goldstack/template-s3 library to make it easier for us to integrate with AWS. Please see the Goldstack documentation for more information of how the configuration works. If you simply want to clone the boilerplate you can follow the Getting Started documentation there.

Once we have our credentials, we simply instantiate the AWS.S3 class (see templateS3Bucket.ts):

const s3 = new S3({
  apiVersion: '2006-03-01',
  credentials: credentials,
  region: region,
});

We can then use the resulting object to work with our bucket. The most common operations will be putObject to add data to our bucket and getObject to read from our bucket.

await s3.putObject({
  BucketName: 'my-bucket',
  Key: 'my-doc',
  Body: 'content',
});

Using the S3 TypeScript Boilerplate

If you want to use the boilerplate project, there are detailed instructions of how to set it up on GitHub.

Using the Goldstack Project Builder

Usually when working with an S3 bucket in an application, we would want the application to do additional things as well, such as provide a backend server or a frontend. The Goldstack Project Builder enables us to combine the S3 templates with other templates such as a Serverless API template and configure our project using a web-based interface.