Notes on AWS's serverless framework

Wed Mar 24 2021

tags: draft programming self study notes public aws webdev imda

(written for work)

Introduction

On Hui Lim's recommmendation, I'm working through the following courses to familiarise myself with the AWS Serverless framework:

I'm learning a lot from these courses and I don't want to forget this stuff, so I'm writing it down as notes.

Maximilian's course covers the following:

  • API Gateway
  • AWS Lambda
  • DynamoDB
  • AWS Cognito
  • S3
  • AWS Route53

A lot of the course focuses on manual setup of the API Gateway and DynamoDB via the web portal which is of course automatable by AWS's CLI. But going through the UI and doing the manual setup actually helps a lot to understand the "big picture": the structure of the entire AWS ecosystem, the API gateway, and so on.

Ariel's course covers some of the same ground of Maximilian's. I'll be working through it very shortly and writing my notes here too.

API Gateway/AWS Lambda

Alex de Brie's "A Detailed Overview of AWS API Gateway is a great overview, and is the source of the following image:

There are six main steps in the AWS Gateway request-response cycle.

  1. Authentication
  2. Method Request: in this step we validate the incoming request to make sure it corresponds to a particular JSON schema. This simplifies the backend/error handling code.
  3. Integration Request: transforms the client's request into a shape that the server/Lambda function can read.
  4. Integration: the API gateway sends the request to the backend server/Lambda function, which receives the request and does stuff with it (sends an email, reads/writes a DB, performs a calculation...).
  5. Integration Response: this replaces the result of the lambda function with a function that the API Gateway can receive.
  6. Method Response: this returns a response with a proper HTTP response code.

The API Gateway's request-response cycle in detail

Six steps, five if you ignore the actual integration because it's not part of the API Gateway:

  1. Authentication
  2. Method Request
  3. Integration Request
  4. (actual integration -- lambda function or other backend service)
  5. Integration Response
  6. Method Response

The first gatekeeper is authentication: this can use a custom (lambda) authoriser, or use Amazon's Cognito authoriser. More on this later.

The Method Request step validates the incoming request to make sure it adheres to a particular model (has particular fields, etc).

If it does then we move to the Integration Request: transforming incoming data into a shape we want to use on the action we're about to trigger. A transformation can be done using the Velocity Template Language (VTL) to add headers, metadata, auth info, and so on. One good usecase for this is in retrofitting a new JSON API to an existing older backend. For example, your server might accept only application/xml while your requests are in application/json. This is where you would translate your incoming request inoto a format the server can read.

Then of course you move on to the actual integration step, where a backend (or Lambda function of your choice) is run. This Lambda function

Integration response: this is the return of the lambda function, resolved calculation/db look up etc.

Method Response shapes the response received by the client

What does it mean to "configure as proxy resource"?

Means catch-all: catch all requests.

If you catch all incoming requests, you can create your own router

What is "API Gateway CORS"?

CORS: cross origin resource sharing.

Imagine your SPA runs on example.com and the API runs on api.com. When the SPA tries to ask for a request from api.com, the browser will prevent this request from being handled, so api.com needs to send some "preflight headers" that will convince the browser that it's OK (but what if api.com is malicious?) before this cross-domain request will be allowed.

  • "Access-Control-Allow-Headers" allow
  • "Access-Control-Allow-Origin": all domains may send these requests and therefore we're good to go.

What is "Lambda Proxy Integration"?

use incoming metadata like headers and authentication etc to pass that unfiltered data over to the lambda

This also means that the Lambda function will be responsible for handling the authentication rather than the integration request

How do I see what the integration receives/logs?

Use AWS Cloudwatch.

What are "body mapping templates" and why would we use this?

We can use body mapping templates to transform the result of a lambda function into a more standardised form. Why would we do this? If you have a lambda function that returns a value but your client is expecting a particular response schema, you can use the body mapping template inside the integration response step to transform it.

Similarly in the integration request step we can also use this template language to edit the request body your integration receives. I've given an example above:

What are "variable path parameters"?

Variable path parameters allow you to run the same Lambda function with different input parameters (path parameters). For example, GET request to /all vs GET request to /single can be handled with the path parameter on resource /{type} and then in the Integration Request we can get this path parameter by doing $input.type.

You can use query parameters too ?someParam=someValue; use them for validation and extract them with templates.

How does AWS Custom authorisation work?

We can create a custom authoriser (also called Lambda authoriser) on the Method Request step.

The custom authoriser is a Lambda function that takes in some information from the incoming request (specifically, an authorisation token). That function will run some code to validate/identify that user, and returns an IAM policy to us + a user ID + optional "context".

If the auth token is missing or invalid the IAM policy will give a rejection via a Gateway Response. A Gateway Response is a "short-circuit" response returned by the API Gateway; it's a response that is returned before the request hits the backend.

If it is a success then this 3-tuple will then be passed it into the next step, the Method Request step. (? Is it the Method Request step ?) "This context will be added to the event object in your backing Lambda function"

API Gateway Lambda authorization workflow

Lambda authorizer Auth workflow

source: AWS docs

First, the client calls a method on an API Gateway API method, passing a bearer token or request parameters. The API Gateway checks whether a Lambda authorizer is configured for the method. If it is, API Gateway calls the Lambda function.

The Lambda function authenticates the caller by means such as the following:

  • Calling out to an OAuth provider to get an OAuth access token.
  • Calling out to a SAML provider to get a SAML assertion.
  • Generating an IAM policy based on the request parameter values.
  • Retrieving credentials from a database.

If the call succeeds, the Lambda function grants access by returning an output object containing at least an IAM policy and a principal identifier.

API Gateway evaluates the policy.

  • If access is denied, API Gateway returns a suitable HTTP status code, such as 403 ACCESS_DENIED.
  • If access is allowed, API Gateway executes the method. If caching is enabled in the authorizer settings, API Gateway also caches the policy so that the Lambda authorizer function doesn't need to be invoked again.

Gotchas

What do I do if I get an "Unsupported Media Type Error" on the client side?

Make sure to set content-type to application/json rather than text/plain

Things to learn

How do IAM policies work? What happens when the Lambda authoriser returns an IAM policy?

IAM policy docs

How do CORS headers work?

What's a Swagger definition?

What's the difference between the Integration Response step and the Method Response step?

Integration responses transform the response from the backend lambda function (which can be any value at all) into a response that API Gateway can handle.

Method responses validate the output to the client. You must first create your method repsonses before you can use a given status code in an integration response. (source: Alex de Brie)

You can use a VTL template to transform the response in both the integration response step and the method response step (? is this true?)

DynamoDB

What is a Partition Key?

A partition key must be unique.

DynamoDB allows you to use a partition key PLUS a sort key as a primary key, where the combination has to be unique.

Why is it called a partition key? Because these keys are assigned to different partitions. (Are they hashed?)

What is a Global Secondary Index? What's a Local Secondary Index?

What is read and write capacity?

You provision certain read and write capacity units: kilobytes read/written per second. You can also just use on demand.

What's the difference between scan() and getItem()?

scan() returns an array, getItem returns only one item.

Gotchas

Why do I get an Access Denied Exception?

You haven't set permissions right: you need to give your lambda function the correct DynamoDB policy.

AWS Cognito

Cognito flow

  • Initialise UserPoolId and ClientId.
  • Create a new CognitoUserPool. It is this object that will be used behind the scenes.
  • Create a new attribute object (standard attribute)
  • Create a CognitoUserAttribute array
  • Call the userPool.signUp method with username, password, attributeList, and callback function that gets executed once Cognito is done.
  • Cognito should actually send you an email and you should be able to verify.

After signup:

  • Initialise a CognitoUser object.
  • Call CognitoUser.authenticateUser() with AuthenticationDetails and a callback function.

What is a Cognito User Pool?

Amazon Cognito user pools documentation

A user pool is a user directory in Amazon Cognito.

What's an App Client?

App clients have access to the user pool. Neither user pool ID nor app client ID are secrets.

AuthenticationDetails object

Should contain username and password

CognitoUser object

should be initialised with a username and userPool.

CognitoUserSession object?

CognitoUserSession will give you JWTs: CognitoAccessToken, CognitoIdToken, and CognitoRefreshToken.

authIsLoading

What's the difference between CognitoAccessToken, CognitoIdToken, and CognitoRefreshToken?

TODO

this.getAuthenticatedUser().signOut()

will actually delete the localstorage tokens

this.authService.getAuthenticatedUser().getSession()

checks if tokens are still valid

CognitoIdentityServiceProvider

Why would you use query string parameters to pass tokens?

Rather than passing it in the body request/headers? It seems like that would be bad because this would allow MITMs to use your token

Gotchas

Why do I receive a empty userId when using Cognito authoriser?

When using Cognito user authoriser, our existing body mapping template will not get the userId with $context.authorizer.principalId. Instead, we need to use $context.authorizer.claims.sub to get the userID.

Security

Acknowledgements

Thanks to Hui Lim for sending me these excellent courses.

Bibliography