melgenek

Getting a development domain and TLS certificates for free

Wed, 28 Jul 2021 00:00:00 +0000

Recently I have been experimenting with Http/3 protocol which uses TLS by default. Having self-signed TLS certificates makes browsers unhappy about the traffic, that’s why I needed a way to get a trusted certificate. While it took me more time than expected, I decided to write down the taken steps.

Getting the domain and certificates

This section walks through the process of getting a domain name and TLS certificates.

To start with, we need to have access to a DNS provider and a domain. I chose DuckDNS because it is absolutely free and allows creating TXT DNS records, that are used to pass the ACME DNS challenge. At DuckDNS we create our own domain and copy the token that looks like UUIDv4 string. This token is needed to update the DNS records via the DuckDNS api.
Next we pass the ACME challenge and get the TLS certificates utilising Let’s Encrypt and CertBot. Let’s Encrypt is the certificate authority that issues certificates that are valid in all the browsers. CertBot is a tool that is usually used to pass ACME challenges on a wide variety of server platforms. Fortunately for us, there is a docker container dedicated to getting certificates at Let’s Encrypt for DuckDNS domains. The usage is as easy as running this command in the terminal:

docker run -it --rm \
-e DUCKDNS_TOKEN="396db3e9-5ecc-4d0c-a708-70a926b1389c" \
-e DUCKDNS_DOMAIN="melgenek.duckdns.org" \
-v `pwd`/melgenek_certs:/etc/letsencrypt \
maksimstojkovic/letsencrypt:latest

In the docker run we specified the token that DuckDNS provides for accessing their API and my domain. As a result of running this command the ACME challenge is succeeds, and the volume attached to the container contains the certificates. The private key and the public certificate can be found in the files ./melgenek_certs/live/melgenek.duckdns.org/privkey.pem and ./melgenek_certs/live/melgenek.duckdns.org/cert.pem respectively.

What has just happened

The docker container that we used in the previous section did the needed magic to validate the DNS record and get the certificates. In order to get the understanding of what the ACME flow looks like, let’s generate one more certificate using the plain CertBot. After passing the challenge, Let’s Encrypt is sure that the domain is ours and gives us a certificate.

The first step is to start the CertBot flow. The CertBot is installed as a standalone binary and can be run from terminal. The command specifies for which domain we want to pass the DNS challenge and where to store the certificates.

certbot certonly --manual \
    --preferred-challenges dns \
    --register-unsafely-without-email \
    -d "melgenek.duckdns.org" \
    --work-dir "./melgenek_certs/work_dir" \
    --logs-dir "./melgenek_certs/logs_dir" \
    --config-dir "./melgenek_certs/config_dir" \
    --agree-tos

As a result of the running the command CertBot generates a secret value and asks us to put it into the DNS record.

Please deploy a DNS TXT record under the name
_acme-challenge.melgenek.duckdns.org with the following value:

NaM0ODwHZfL6b1pjM_rrgfCSwVcy_CALKkxR2YzyE7A

Before continuing, verify the record is deployed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

CertBot proposes to set the TXT record at the _acme-challenge.melgenek.duckdns.org subdomain. The validation of such a record would result in issuing a wildcard certificate for *.melgenek.duckdns.org. We are setting the TXT record for the domain itself so that the certificate is valid only for the melgenek.duckdns.org domain. The next cUrl call creates the TXT record in DuckDns.

curl "https://www.duckdns.org/update?domains=melgenek.duckdns.org&token=396db3e9-5ecc-4d0c-a708-70a926b1389c&txt=NaM0ODwHZfL6b1pjM_rrgfCSwVcy_CALKkxR2YzyE7A"

Before proceeding with the ACME challenge, we can check that the DNS record is actually set using dig. In the lookup result we see that the TXT record is exactly what we expect it to be.

dig melgenek.duckdns.org TXT

;; ANSWER SECTION:
melgenek.duckdns.org.   59      IN      TXT     "NaM0ODwHZfL6b1pjM_rrgfCSwVcy_CALKkxR2YzyE7A"

Clicking “Enter” in the terminal where CertBot is waiting for input would trigger Let’s Encrypt validation of the TXT record. Passing this validation successfully produces certificates the specified folder ./melgenek_certs/config_dir/live/melgenek.duckdns.org

The challenge secret is a not reused in the future, so it makes sense to clean up the TXT record. This is done by adding the clear=true query param to the previous cUrl query.

curl "https://www.duckdns.org/update?domains=melgenek.duckdns.org&token=396db3e9-5ecc-4d0c-a708-70a926b1389c&txt=NaM0ODwHZfL6b1pjM_rrgfCSwVcy_CALKkxR2YzyE7A&clear=true"

Summary

In this article we described a simple way of getting perfectly valid TLS certificates as wells as a domain that can be used for development purposes. Hopefully, this information helps build exciting software and learn new technologies.

Enhancing DynamoDb client with Scala 3

Thu, 31 Dec 2020 00:00:00 +0000

After 8 years of development, Dotty is going to become Scala 3 soon. It’s the right time to try out Scala 3 and its new features. In this article, I am going to show a practical example of making DynamoDb more type-safe and convenient using macros, type class derivations, extensions methods, and a handful of implicits.

Introduction

To start with, let’s look at how to put and get an item from DynamoDb utilizing the plain aws-sdk client. The examples assume that there is a case class:

case class NewYear(year: Int, wish: String) {
  def gift: String = "A fairy pony"
}

val year = NewYear(2020, "I wish Scala 3 was released soon")

The case class is defined in the file NewYear.scala. The instance of the case class val year=... as well as the case class itself are at the top-level as well. This is the first notable feature of Scala 3: the package objects were removed from the language. The values and methods don’t need to be defined inside an object or class anymore.

The instances of NewYear are going to be stored in the dynamo table new-years. I am omitting the code of creating the dynamo table, the whole example can be found in the github repository.

Now we know the entities that are going to be stored in the database. We can take a look at the put/get code:

ddb.putItem(
  PutItemRequest.builder()
    .tableName(TableName)
    .item(Map(
      "year" -> AttributeValue.builder().n(year.year.toString).build(),
      "wish" -> AttributeValue.builder().s(year.wish).build()
    ).asJava)
    .build()
)

val item = ddb.getItem(
  GetItemRequest.builder()
    .tableName(TableName)
    .key(Map(
      "year" -> AttributeValue.builder().n("2020").build()
    ).asJava)
    .build()
)

What problems can we see here?

Firstly, the AttributeValues are being built explicitly, the year of type Int is transformed into a String in order to be set as the value of the attribute. Not only we can make a mistake of choosing the DynamoDb type by confusing the .n and .s value setters on the builder, but also the code looks ugly and verbose. It’s a great use case for type classes to provide a unified way of converting scala types to the ` AttributeValue`.

Secondly, the attribute names are passed as strings. The key of our table is year. A typo in the key of the get request can lead to no results being returned. You can argue that having constants for the attribute names is enough. However, keeping the same name of the key in the item map and the case class field is going to be crucial to write cases classes in the .item without an explicit construction of the map.

Here comes the last issue that we’ll try solving in this article. The .item ignores the instance of the class that we defined and expects a map. This case class has only 2 fields, so building a map is quite an easy task. Having 20 fields will definitely be error-prone because an additional field in the case class requires an additional line inside the .item map.

After making some improvements employing the features of Scala 3, we can get a neat code like this

ddb.putItem(
  PutItemRequest.builder()
    .tableName(TableName)
    .item(year)
    .build()
)

val item = ddb.getItem(
  GetItemRequest.builder()
    .tableName(TableName)
    .key[NewYear](_.year, 2021)
    .build()
)

In the following sections, I’ll explain how to create the utilities for the code above to work.

Encoding attributes. Implicits

I’m going to introduce the improvements step by step. To start with, we’ll make the attribute building more pleasant. In order to do this, we’ll have a type class for the conversion of a scala type to the AttributeValue and back. Every type that can be converted to the AttributeValue will have an implementation of the AttributeCodec trait:

trait AttributeCodec[A] {
  def encode(a: A): AttributeValue
  def decode(a: AttributeValue): A
}

given AttributeCodec[String] with {
  def encode(a: String): AttributeValue = AttributeValue.builder().s(a).build()
  def decode(a: AttributeValue): String = a.s()
}

implicit val intCodec: AttributeCodec[Int] = new AttributeCodec[Int] {
  def encode(a: Int): AttributeValue = AttributeValue.builder().n(a.toString).build()
  def decode(a: AttributeValue): Int = a.n().toInt
}

You can see that I defined two instances:

the instance for the String type is defined with Scala 3 syntax
the instance for the Int uses the old Scala 2 syntax

The Scala 3 syntax is slightly nicer, the instance names can be omitted. They are never used after all, because the instances are found in the implicit scope via summoning. In the new Scala, the method implicitly is renamed to be summon:

summon[AttributeCodec[Int]].encode(2021) // AttributeValue.builder().n(2021.toString).build()

We can make the encoding look even better by adding a so-called “summoner” method to the AttributeCodec. There is a new keyword using that marks the implicit function parameters. It replaces the old implicit:

object AttributeCodec {
  def apply[A](using codec: AttributeCodec[A]): AttributeCodec[A] = codec
  
  // The old syntax
  // def apply[A](implicit codec: AttributeCodec[A]): AttributeCodec[A] = codec
}

After this first round of enhancements our application is in the following state:

ddb.putItem(
  PutItemRequest.builder()
    .tableName(TableName)
    .item(Map(
      "year" -> AttributeCodec[Int].encode(year.year),
      "wish" -> AttributeCodec[String].encode(year.wish)
    ).asJava)
    .build()
)

val item = ddb.getItem(
  GetItemRequest.builder()
    .tableName(TableName)
    .key(Map(
      "year" -> AttributeCodec[Int].encode(2021)
    ).asJava)
    .build()
)

Obtaining class field names. Macros

Our next step is the derivation of the attribute names based on the case class field names. When we complete implementing this macro, the code will use fields instead of strings.

ddb.putItem(
  PutItemRequest.builder()
    .tableName(TableName)
    .item(Map(
      FieldName[NewYear](_.year) -> AttributeCodec[Int].encode(year.year),
      FieldName[NewYear](_.wish) -> AttributeCodec[String].encode(year.wish)
    ).asJava)
    .build()
)

val item = ddb.getItem(
  GetItemRequest.builder()
    .tableName(TableName)
    .key(Map(
      FieldName[NewYear](_.year) -> AttributeCodec[Int].encode(2021)
    ).asJava)
    .build()
)

As you can see the attribute name is defined via the accessor of the field FieldName[NewYear](_.year). The automatic acquisition of the field name is performed with a macro:

inline def apply[T](inline f: T => Any): String = ${getName('f)}

This is an apply method that is defined with the modifier inline and calls the macro implementation getName. The methods that are implemented via macros are always required to be defined with the inline modifier. The second inline modifier on the parameter is optional. Let’s look at a simple example to understand why it’s needed in this situation. The following code prints the parameter that is passed to the method:

import scala.quoted._

object InlineFunctions {
  inline def showExpr(expr: Any): String = ${showExprImpl('expr)}

  inline def showExprInlined(inline expr: Any): String = ${showExprImpl('expr)}

  private def showExprImpl(expr: Expr[Any])(using Quotes): Expr[String] =
    '{ ${Expr(expr.show)} + " = " + $expr }
}

The implementation of the macro is defined in the method showExprImpl. The first parameter has the type Expr. This type represents the abstract syntax tree for all the constructs that compose our code. For example, it’s subtype Literal represents a single value, and the subtype Block contains multiple statements. The ${expr} is the same as $expr. It’s called “splicing” and calculates the value of an expression. For example, the ${Expr("hello")} is just the string hello. The transformation can be reversed with the use of quotes '{expr} which is the same as 'expr. Thus '{"hello"} is equal to Expr("hello"). In essence the showExprImpl prints the string representation of the expression and its value using the splices and quotes. The Quotes context parameter contains some low-level operations and is used implicitly by these operations. I defined 2 different functions: one with the inline parameter and another one without it. Let’s call them and see the output.

import InlineFunctions._

object InlineMain extends App {
  val a = 1
  val b = 2

  println(showExprInlined(a + b))  //  demo.inline.InlineMain.a.+(demo.inline.InlineMain.b) = 3
  println(showExpr(a + b))  // expr$proxy2 = 3
}

The value of the sum operation is the same. The expressions that are passed to our macro are different though. The inline modifier preserves the original expression. That’s exactly what we need in order to get the field name from the function such as (w:NewYear) => w.year.

After we had a quick look at the inline modifier, splices, and quotes, it’s time to move on and implement the getName method.

private def getName[T](f: Expr[T => Any])(using Type[T], Quotes): Expr[String] = {
  import quotes.reflect._
  val acc = new TreeAccumulator[String] {
    def foldTree(names: String, tree: Tree)(owner: Symbol): String = tree match {
      case Select(_, name) => name
      case _ => foldOverTree(names, tree)(owner)
    }
  }
  val fieldName = acc.foldTree(null, f.asTerm)(Symbol.spliceOwner)
  Expr(fieldName)
}

In the implementation, we dived even deeper into the Scala magic by calling f.asTerm so as to get access to the AST that the compiler sees. This is so-called TASTy Reflect. It provides an even more comprehensive view of the structure of the code. The power comes with a cost. Using TASTy Reflect can break type correctness guarantees and may fail at macro expansion time. In this particular use case, we are safe because we are only interested in reading the syntax tree, not in its modification. The .asTerm call produces the Tree instance. Similarly to Expr, the Tree has multiple subclasses that together represent our code. For instance, the call FieldName[NewYear](_.year) is expanded to

Inlined(EmptyTree,List(),Block(List(DefDef($anonfun,List(),List(List(ValDef(_$1,TypeTree[TypeRef(ThisType(TypeRef(NoPrefix,module class demo)),class NewYear)],EmptyTree))),TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),module scala),Any)],Select(Ident(_$1),year))),Closure(List(),Ident($anonfun),EmptyTree)))

This AST has quite many nesting levels. That’s why we use the TreeAccumulator that traverses this tree for us. When the traversal reaches the desired Select instance, it returns the name of the field.

The NewYear has a method defined outside the constructor and in the current implementation the call FieldName[NewYear](_.gift) is perfectly valid. It returns the string gift even though the field is not defined in the primary constructor. In order to prevent any fields and methods to be passed into the getName method, we define a compile-time validation that issues a compilation error when the field is not a part of the primary constructor. Here is the final implementation of the macro including the validation:

import scala.quoted.Expr.{ofTuple, summon}
import scala.quoted._

object FieldName {
  inline def apply[T](inline f: T => Any): String = ${getName('f)}

  private def getName[T](f: Expr[T => Any])(using Type[T], Quotes): Expr[String] = {
    import quotes.reflect._
    val acc = new TreeAccumulator[String] {
      def foldTree(names: String, tree: Tree)(owner: Symbol): String = tree match {
        case Select(_, name) => name
        case _ => foldOverTree(names, tree)(owner)
      }
    }
    val fieldName = acc.foldTree(null, f.asTerm)(Symbol.spliceOwner)
    val primaryConstructorFields = TypeTree.of[T].symbol.caseFields.map(_.name)
    if(!primaryConstructorFields.contains(fieldName))
      report.error(s"The field '$fieldName' is not one of the primary constructor parameter.", f)
    Expr(fieldName)
  }
}

Avoiding map construction. Type class derivation

Earlier in this article, I mentioned that it is crucial to know how to derive the field name and omit to have the attribute names as strings. The reason is that we will be generating the map based on the case class.

trait ItemCodec[T] {
  def encode(t: T): Map[String, AttributeValue]
}

There will be an instance of the ItemCodec trait created for any case class. Unlike AttributeCodec, which had the explicitly defined instances, the type ItemCodec instances are derived automatically. In Scala 2 you would use libraries like Magnolia in order to construct the macros and generate these instances. Scala 3 introduces some convenience utilities in the language itself. One of them is the trait Mirror. The language provides an instance of Mirror.Product for every case class. For our NewYear the implementation of this trait looks like this:

class NewYearMirror extends Mirror {
  type MirroredMonoType    = NewYear
  type MirroredLabel       = "NewYear"
  type MirroredElemLabels  = ("year", "wish")
  type MirroredElemTypes   = (Int, String)
}

What we need to do is go field by field, and encode every field into the format that the DynamoDb client understands:

for every field get the AttributeCodec. For example, for the field year we need to summon an instance AttributeCodec[Int]
set the encoded value in the map with the key year

NewYearMirror provides enough information to write such a type class derivation, because we have both field names and field types.

private inline def getAttributeNamesAndCodecs[N <: Tuple, T <: Tuple]: List[(String, AttributeCodec[Any])] =
  inline (erasedValue[N], erasedValue[T]) match {
    case (_: EmptyTuple, _: EmptyTuple) => Nil
    case (_: (nameHead *: nameTail), _: (typeHead *: typeTail)) =>
      val attributeLabel = constValue[nameHead].toString
      val attributeCodec = summonInline[AttributeCodec[typeHead]].asInstanceOf[AttributeCodec[Any]]
      (attributeLabel, attributeCodec) :: getAttributeNamesAndCodecs[nameTail, typeTail]
  }

inline given derived[T <: Product](using m: Mirror.ProductOf[T]): ItemCodec[T] = {
  val namesAndCodecs = getAttributeNamesAndCodecs[m.MirroredElemLabels, m.MirroredElemTypes]
  new ItemCodec[T] {
    override def encode(t: T): Map[String, AttributeValue] = {
      namesAndCodecs.zip(t.productIterator)
        .map { case ((name, codec), value) =>
          name -> codec.encode(value)
        }
        .toMap
    }
  }
}

The most interesting parts of these two methods are how to go over fields one by one, and how to transform the field type to be a value.

Let’s understand the traversal first. We have a type type MirroredElemLabels = ("year", "wish") which is a tuple. In Scala 3 there is an extractor for the tuple type *:. It works the same way as for sequences so that there are a head and a tail element. In order to pattern match the tuple, we need to have its value. The erasedValue pretends to give us the value. In fact, it would always raise a NotImplementedError exception when called. That’s why we only pattern match the types and don’t use the values of erasedValue.

The second puzzle is how it’s possible to have the type of "year" instead of a type String, and how to transform the type "year" to the value year. In Scala 3 there are singleton types that’s why the type "year" is valid. These types have only one instance, in this case, year. In order to acquire this value we call the function constValue.

Convenience operators. Extension methods

All the planned functional improvements have been implemented. We can only add some operators to the GetItemRequest and PutItemRequest as if they were natively scala. Scala 2 approach is to define the implicit classes that wrap the objects and expose additional methods on them. Scala 3 has a dedicated keyword for adding such operators. It’s called extension methods.

extension[T] (b: GetItemRequest.Builder) {
  inline def key: GetItemRequestBuilderExtension[T] =
    new GetItemRequestBuilderExtension[T](b)
}

class GetItemRequestBuilderExtension[T](b: GetItemRequest.Builder) {
  inline def apply[A: AttributeCodec](inline k: T => A, v: A): GetItemRequest.Builder =
    b.key(Map(
      FieldName[T](k) -> AttributeCodec[A].encode(v)
    ).asJava)
}

extension[T: ItemCodec] (b: PutItemRequest.Builder) {
  def item(t: T): PutItemRequest.Builder =
    b.item(ItemCodec[T].encode(t).asJava)
}

Here is the resulting code that uses this syntactic sugar.

ddb.putItem(
  PutItemRequest.builder()
    .tableName(TableName)
    .item(year)
    .build()
)

val item = ddb.getItem(
  GetItemRequest.builder()
    .tableName(TableName)
    .key[NewYear](_.year, 2021)
    .build()
)

Conclusion

In this article, we got acquainted with some Scala 3 features based on a real world example of using the DynamoDb client. We also had a gentle introduction to the macros and type class derivations. If you need more information around this topic, then take a look at this arctile and the macro tutorial.

The result of the exercise that is described in this article is in github.

Serverless Tapir

Wed, 29 Jul 2020 00:00:00 +0000

This blog post could start with a drawing of a tapir that has a pair of tiny angel wings and sits on a cloud. Unfortunately, I cannot draw. But I can code. So in this article will tell how to run a Tapir program with AWS Lambda and API Gateway.

Introduction

Tapir is an amazing scala framework that allows defining http endpoints as scala values. Let’s look at the example that I borrowed from the official documentation:

import sttp.tapir._
import sttp.tapir.json.circe._
import io.circe.generic.auto._

type Limit = Int
type AuthToken = String
case class BooksFromYear(genre: String, year: Int)
case class Book(title: String)

val booksListing: Endpoint[(BooksFromYear, Limit, AuthToken), String, List[Book], Nothing] = 
  endpoint
    .get
    .in(("books" / path[String]("genre") / path[Int]("year")).mapTo(BooksFromYear))
    .in(query[Limit]("limit").description("Maximum number of books to retrieve"))
    .in(header[AuthToken]("X-Auth-Token"))
    .errorOut(stringBody)
    .out(jsonBody[List[Book]])

This is a representation of a single endpoint with a noticeable feature that it does not have any logic attached. The endpoint is solely a description of input parameters and outputs like response bodies.

In order to make this definition runnable, we have to use an interpreter. Here is an example implementation in akka-http:

import sttp.tapir.server.akkahttp._
import akka.http.scaladsl.server.Route
import scala.concurrent.Future

val booksListingRoute: Route =
    booksListing.toRoute { case (bfy: BooksFromYear, limit: Limit, at: AuthToken) =>
      Future.successful(Right(List(Book("The Sorrows of Young Werther"))))
    }

The logic of the route is generic, there is no explicit use of the akka-http response codes or marshallers. There are in fact many options of server interpreters such as play, http4s, finatra and vert.x. However, these frameworks help only with writing the code. The next steps are most likely the docker packaging, the definition of the kubernetes manifests and deployment scripts.

Deploying an application takes a signification amount of effort. I want to have the code up and running quickly and easily, preferably with a single bash command. Tapir, AWS Lambda and AWS CDK will help me with this venture. The complete code can be found in the github repo.

Implementing the interpreter

We start with an interpreter that transform Tapir code in an AWS Lambda. We will follow these steps:

define the logic to transform a Tapir request into the Lambda-specific request.
then Tapir parses this request and gives me the inputs like query and path params, an input body and headers.
the next step is to execute the user-defined logic using these inputs as arguments.
finally, the response from the user-defined logic is transformed back into the response that Lambda understands.

AWs Lambda itself does not know anything about http. It is just an execution environment. We utilize the API Gateway Http API. It parses the request and invokes our lambda as soon as the request is ready to be processed. Http API passes the parsed request in a predefined format. I use the "com.amazonaws" % "aws-lambda-java-events" to have the predefined java classes APIGatewayV2HTTPEvent and APIGatewayV2HTTPResponse that correspond to the Http API request and response models.

As mentioned above the implementation starts mapping the APIGatewayV2HTTPEvent into Tapir’s ServerRequest.

class HttpApiServerRequest(event: APIGatewayV2HTTPEvent) extends ServerRequest {
  def method: Method = Method(event.getRequestContext.getHttp.getMethod.toUpperCase)

  def protocol: String = event.getRequestContext.getHttp.getProtocol

  def uri: URI =
    new URI(s"https://${event.getRequestContext.getDomainName}${event.getRawPath}?${event.getRawQueryString}")

  def connectionInfo: ConnectionInfo = ConnectionInfo(
    local = None,
    remote = Some(InetSocketAddress.createUnresolved(event.getRequestContext.getHttp.getSourceIp, 0)),
    secure = Some(true)
  )

  lazy val headers: Seq[(String, String)] = event.getHeaders.asScala.toList
  
  def header(name: String): Option[String] = event.getHeaders.getIgnoreCase(name)
}

A couple of moments are worth clarification here.

First of all, Http API passes all the headers with lower cased names, so the header lookup is done ignoring the case.

Secondly, in the ConnectionInfo we have only the remote client, there is no port given to the lambda. The communication to Http API is always secure.

The next step after defining the server request is the DecodeInputsContext. As the name suggests, this class is used by Tapir during the inputs extraction.

class HttpApiDecodeInputsContext(event: APIGatewayV2HTTPEvent, pathConsumed: Int = 0) extends DecodeInputsContext {
  def method: Method = Method(event.getRequestContext.getHttp.getMethod.toUpperCase)

  def nextPathSegment: (Option[String], DecodeInputsContext) = {
    val path = event.getRawPath.drop(pathConsumed)
    val nextStart = path.dropWhile(_ == '/')
    val segment = nextStart.split("/", 2) match {
      case Array("") => None
      case Array(s) => Some(s)
      case Array(s, _) => Some(s)
    }
    val charactersConsumed = segment.map(_.length).getOrElse(0) + (path.length - nextStart.length)

    (segment, new HttpApiDecodeInputsContext(event, pathConsumed + charactersConsumed))
  }

  def header(name: String): List[String] = 
    event.getHeaders.getIgnoreCase(name).flatMap(_.split(",").toList).toList

  def headers: Seq[(String, String)] = event.getHeaders.asScala.toList

  def queryParameter(name: String): Seq[String] =
    event.getQueryStringParameters.getIgnoreCase(name).flatMap(_.split(",").toList).toList

  def queryParameters: QueryParams = QueryParams.fromMap(event.getQueryStringParameters.asScala.toMap)

  def bodyStream: Any =
    throw new UnsupportedOperationException("Trying to read streaming body from a non-streaming request")

  def serverRequest: ServerRequest = new HttpApiServerRequest(event)
}

Multivalued headers and query params are combined with commas in Http API. So we split them back into a List.

The body is always passed into the lambda as a string so the whole incoming request is consumed before the lambda invocation. That’s why the body cannot be streamed, and the bodyStream has no implementation.

Now that we know how to map the lambda input into the input that Tapir understands, let’s implement the routes. The same way akka http has akka.http.scaladsl.server.Route, or http4s has org.http4s.HttpRoutes, we will also have a Route type:

type Route = PartialFunction[APIGatewayV2HTTPEvent, APIGatewayV2HTTPResponse]

Tapir also requires a MonadError implementation. For akka http there is an instance of a MonadError based on Future. That’s why the akka http routes return Futures as results. A lambda function that runs on top of the Lambda Java runtime requires functions to be simple blocking functions with a signature def onEvent(event: APIGatewayV2HTTPEvent): APIGatewayV2HTTPResponse. I chose Try to be the result type of the Lambda logic. The example conversion of a function with some business logic into a Lambda route would be:

import scala.util.{Success, Try}
import sttp.tapir.server.httpapi._  

def logic(bfy: BooksFromYear, limit: Limit, at: AuthToken): Try[Either[String, List[Book]]] = {
  Success(Right(List(Book("The Sorrows of Young Werther"))))
}
val serverEndpoint = bookListing.serverLogic((logic _).tupled)
val booksListingRoute: Route = serverEndpoint.toRoute

The Route is a partial function. If the http request matches the Tapir endpoint definition then we execute the logic for this endpoint. If no then we try another route. This check is expressed via the isDefinedAt method of the partial function. We use the sttp.tapir.server.internal.DecodeInputs, pass the context and let Tapir parse the request for us.

def isDefinedAt(event: APIGatewayV2HTTPEvent): Boolean = {
  DecodeInputs(e.input, new HttpApiDecodeInputsContext(event)) match {
    case _: DecodeInputsResult.Values => true
    case _: DecodeInputsResult.Failure => false
  }
}

When the http request matches our endpoint, it’s time to run the actual application logic. The result of the DecodeInputs is just a sequence of values. Thus, we map these values into scala classes, so that we get a tuple of case classes, like (BooksFromYear, Limit, AuthToken) in the example above. This conversion is the responsibility of sttp.tapir.server.internal.InputValues.

def apply(event: APIGatewayV2HTTPEvent): APIGatewayV2HTTPResponse = {
  DecodeInputs(e.input, new HttpApiDecodeInputsContext(event)) match {
    case values: DecodeInputsResult.Values =>
      InputValues(e.input, values) match {
        case InputValuesResult.Value(params, _) => valueToResponse(params.asAny)
        case InputValuesResult.Failure(input, failure) => handleDecodeFailure(input, failure)
      }
    case DecodeInputsResult.Failure(input, failure) => handleDecodeFailure(input, failure)
  }
}

The next step, after the inputs are successfully parsed, is to pass these inputs into the user-defined logic.

def valueToResponse(value: Any): APIGatewayV2HTTPResponse = {
  endpoint.logic(TryMonadError)(value.asInstanceOf[I]) match {
    case Success(Right(result)) => OutputToHttpApiResponse(ServerDefaults.StatusCodes.success, endpoint.output, result)
    case Success(Left(err)) => OutputToHttpApiResponse(ServerDefaults.StatusCodes.error, endpoint.errorOutput, err)
    case Failure(e) => OutputToHttpApiResponse(StatusCode.InternalServerError, e.getMessage)
  }
}

The only thing left is to map the Tapir’s output to the response type that Http API understands. This is what OutputToHttpApiResponse does in the valueToResponse method. The implementation requires some lines of code and it can be found here.

We put the functionality that is described in the toRoute function. The Route is a partial function, so we can compose multiple routes into a single one via the orElse method. If none of the routes matches the request, we use the predefined EmptyRoute that answers with the 404 response code.

implicit class RichHttpApiServerEndpoint[I, E, O](endpoint: ServerEndpoint[I, E, O, Nothing, Try]) {
  def toRoute: Route = {
    // the partial function that is described above
  }
}

implicit class RichHttpApiServerEndpoints[I, E, O](serverEndpoints: List[ServerEndpoint[_, _, _, Nothing, Try]]) {
  def toRoutes: Route = {
    serverEndpoints
      .map(_.toRoute)
      .foldRight(EmptyRoute)(_ orElse _)
  }
}

Let’s also provide an interface that has to be implemented in order to have a complete lambda. This interface requires a list of ServerEndpoints and then uses these endpoints to process a request.

trait HttpApiFunction {
  val serverEndpoints: List[ServerEndpoint[_, _, _, Nothing, Try]]
  def onEvent(event: APIGatewayV2HTTPEvent): APIGatewayV2HTTPResponse = {
    serverEndpoints.toRoutes(event)
  }
}

object BookFunction extends HttpApiFunction {
  override val serverEndpoints = List(booksListingRoute)
}

Lambdas will have the handler that points to the your.package.BookFunction::onEvent. The class will be instantiated by the Java runtime and the onEvent function will be invoked for every http request.

Packaging the Lambda

I am not using lambda layers, so the Lambda function has to contain all of the dependencies. The fat jar is packaged using the sbt-assembly plugin. The only interesting part of this process is the merge conflict resolution. The reference configurations are concatenated, and some files with conflict names are discarded.

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", _@_*) => MergeStrategy.discard
  case PathList(ps@_*) if ps.last endsWith "reference-overrides.conf" => MergeStrategy.concat
  case PathList(ps@_*) if ps.last endsWith "module-info.class" => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

Deployment

The interpreter is ready, it can be used to compose Tapir endpoints together into a single Route. The Lambda function is also bundled into a fat jar. Everything is ready for uploading and running this jar. We can go into the AWS ui, click a couple of buttons, create the lambda and upload the code. However, this is a manual process. The whole idea of the experiment, that is described in this article, is to have an easy and automatic process of deploying applications.

In order to automate the lambda creation process, we will use AWS Cloud Development Kit. This is a tool that allows defining the AWS infrastructure in Java and transforming this definition into CloudFormation stacks. CloudFormation then makes sure that the infrastructure is created in AWS.

Let’s create a function that receives an HttpApiFunction and the name of the Lambda function. This function creates a so-called App, that contains all the resources. Then it synthesizes the Cloudformation template based on the code.

def deploy(httpApiFunction: HttpApiFunction, title: String): Unit = {
  val app = new awscdk.core.App()
  ...
  app.synth()
}

All the resources have scopes. The top-level scope is the App. Each app is a set of Stacks. Every Stack corresponds to a Cloudformation stack. For us, it is enough to create a single Stack.

val stack = new Stack(app, s"$title-stack")

The first meaningful resource that we create is the lambda function. It is a lambda that is based on the Java 8 runtime. It sets the handler method to be the def onEvent(event: APIGatewayV2HTTPEvent): APIGatewayV2HTTPResponse of our lambda class. Additionally, we set the local path of the fat jar file that we have bundled. This jar is automatically uploaded by CDK into an s3 bucket. Then this s3 object is used as the source for the lambda function.

val function = lambda.Function.Builder.create(stack, s"$title-lambda")
  .memorySize(192)
  .timeout(Duration.seconds(30))
  .functionName(title)
  .runtime(lambda.Runtime.JAVA_8)
  .handler(s"${httpApiFunction.getClass.getName.replace("$", "")}::onEvent")
  .code(Code.fromAsset("lambda/target/scala-2.13/assembly.jar"))
  .build()

After the lambda is present, we create the API Gateway Http API.

val api = HttpApi.Builder.create(stack, s"$title-api")
  .apiName(title)
  .build()

After both lambda and API are set up, we need to bind them together so that Http API passes the requests into lambda. For this to happen, we create a Lambda integration.

val integration = LambdaProxyIntegration.Builder.create()
  .handler(function)
  .payloadFormatVersion(PayloadFormatVersion.VERSION_2_0)
  .build()

The last step is to define all the http routes and point them to the same lambda function. Tapir’s ability to transform endpoints into Open API helps us construct the routes. For every path in the Open API specification, we create a route in the Http API.

import sttp.tapir.docs.openapi._
private def apiRoutes(httpApiFunction: HttpApiFunction, integration: LambdaProxyIntegration): List[AddRoutesOptions] = {
  val openAPI = httpApiFunction.serverEndpoints.map(_.endpoint).toOpenAPI("any name", "v1")
  openAPI.paths
    .map { case (path, pathItem) => 
    val methods =
      pathItem.get.map(_ => HttpMethod.GET) ++
        pathItem.post.map(_ => HttpMethod.POST) ++
        pathItem.delete.map(_ => HttpMethod.DELETE)
    AddRoutesOptions.builder()
      .methods(methods.toList.asJava)
      .path(path)
      .integration(integration)
      .build()
    }
    .toList
}

That’s it. Now we pass our lambda class into the deployment function, run the cdk deploy command in the terminal, and after a short time, our code is up and running in AWS.

object Main extends App {
  HttpApiCdkServer.deploy(BookFunction, "my-bookshop")
}

Cdk also allows adding outputs into stacks. One of such outputs can be the url of the Http API that we deployed.

CfnOutput.Builder.create(stack, "api-url").exportName("url").value(api.getUrl).build()

This output is shown after the cdk deploy succeeds.

✅  my-bookshop-stack

Outputs:
my-bookshop-stack.apiurl = https://2qpnchfg11.execute-api.us-east-1.amazonaws.com/

Calling this url results in the request being handled by our lambda that is described above.

curl -v -H "X-Auth-Token: Bearer token"  https://2qpnchfg11.execute-api.us-east-1.amazonaws.com/books/novel/2020?limit=10

Result:
[
  {
    "title": "The Sorrows of Young Werther"
  }
]

Summary

In this article, I showed how to run the Tapir application in Lambda and automatically deploy the code with CDK. I believe that the use of CDK brings a lot of benefit into scala applications because it keeps the infrastructure close to the code. The infrastructure has the same version as the code does. This helps us to be sure that the required infrastructure is created at the deployment time. In addition, CDK gives a way to deploy with a command.

The code for this article can be found in the github repo.

Building a video service using Spring Framework

Sun, 20 May 2018 00:00:00 +0000

Some time ago a brand new WebFlux by Spring was released introducing the non-blocking reactive programming paradigm. One of the domains to apply this new framework is data streaming, in particular video streaming. In this article I’m going to build a video streaming service and describe the threading model that is being used.

Byte-range requests

To begin with, let’s investigate how the <video> element works and what requests it sends to a server. For example, w3c has an overview of this html element with the one embedded. The very first request sent to service is as follows:

GET https://www.w3schools.com/html/mov_bbb.mp4
Range: bytes=0-
...

Generally speaking, it is a simple http GET request. The only additional piece of data that matters is the Range header. It comes from the Range Requests specification and asks the server for a specific part of the file. The header in the previous request means that the client expects data from position 0 in a byte representation of file till the end. A Range header can have a final point, e.g. Range: bytes=50-100.

There are two possible responses to such a kind of request:

Status: 200 OK. It indicates that the server can’t fulfil the request and is going to respond with a full resource in one request.
Status: 206 Partial Content. In this case server confirms its ability to respond to range requests. Here are the headers we are interested in:

Status: 206
Accept-ranges: bytes
Content-Length: 100
Content-Range: bytes 0-99/788493

Lets have a closer look at them:

Accept-ranges: bytes. This one says which range unit is being used. The most common is bytes but RFC allows any other, such as seconds. It can even be used for pagination.
Content-Length: 100. In contrast to status 200 response, which shows the full length of content, here the stated length refers to the size of the chunk received.
Content-Range: bytes 0-99/788493. This header shows that we have bytes from 0 to 99 and these are the 100 bytes mentioned in Content-Length header. The number 788493 after the slash tells us the full size of the file on the server. It is used by a video tag to figure out when the content is loaded entirely.

Using this simple protocol we can specify any point in a movie to watch. Thanks to getting only chunks of data, users don’t need to wait until an entire video is loaded, which is particularly useful for those with slow connection and per megabyte internet pricing, as well as for creation of a splendid user experience.

Spring MVC

The classical MVC was in use for a while and has lots and lots of different features and specification implementations. After version 4.2.4 of Spring, you can download dynamic files from a server with just a few lines of code. The byte-range support is included as well. The following code is based on the mvc branch of my sample project and you can find project dependencies in build.gradle file. Here is the only piece of code you need to implement your video server:

@GetMapping("/videos/{name}/full")
fun getFullVideo(@PathVariable name: String): ResponseEntity<UrlResource> {
	val video = UrlResource("file:$videoLocation/$name")
	return ResponseEntity.status(HttpStatus.PARTIAL_CONTENT)
			.contentType(MediaTypeFactory
					.getMediaType(video)
					.orElse(MediaType.APPLICATION_OCTET_STREAM))
			.body(video)
}

The purpose of the controller send files to client. Let’s explore it line by line:

@GetMapping obviously means that we’re handling GET http requests
The method declaration shows that it takes a name of the file as a path parameter and returns a UrlResource which is just any resource that can be reached by url.
Next line creates a resource from url. Resource is in essence an input stream with some additional methods. $videoLocation variable comes from environment and is the name of a folder that holds our videos.
Then we have HttpStatus. As mentioned earlier, we need a 206 status code response to specify that server accepts range request and answers properly.
The next three lines are setting media type. Browsers (except chrome) will refuse playing you video without a media type specified. MediaTypeFactory gives this type based on the file name. For example, if the file name is someVideoName.mp4, then the MIME type video/mp4 is returned. If the MIME cannot be determined, the content type is specified as byte stream.
finally, we set video resource as a body.

The only thing left to make it a full-blown video player is a bit of html :)

<video autoplay="autoplay" controls>
    <source src="/videos/fish.mp4/full" type="video/mp4">
</video>

This section shows how simple it is to make a file server leveraging Spring. While I’m using UrlResource, anybody can implement their own Resource to get data from anywhere, for example, database.

MVC drawbacks

It is helpful to know how the code above works from threading perspective, and what problems we could face because of that. The threading model of MVC is a classical one-per-user thread model. It can be easily observed with Java VisualVM.

Let’s perform two requests to our server using curl (in separate terminal windows):

curl -s -o /dev/null -H "Range: bytes=0-" -D - http://localhost:8080/videos/fish.mp4/full

Everything works as expected and both loads finish. The threads view shows that for the load period two threads were occupied (green part of line) reading data from resource and writing to output stream (see StreamUtils.copy). It is performed inside Spring’s ResourceHttpMessageConverter for you.

For the video streaming the main shortcoming is: server will not be able to process more users than number of threads it has. This means if there are 10 threads then there will be no more than 10 users. You can easily check this behaviour by setting property server.tomcat.max-threads=1 and calling curl twice. The second request will be processed only after the first one ends. Use a big file for this experiment, several gigabytes, because local download is really fast.

Imagine a situation when your website has exclusive rights for TV series. With an average length of episode of 1 hour, you will be able to pass the content to only 10 customers at the same time. A website with performance like this will never get a wide audience.

Responding with parts of data

One mechanism to address the issue described in the previous section is to respond only with parts of a file. It is suitable only for clients which support byte range responses and html video is one of them. This way, threads will be used for smaller amounts of time giving opportunity to other requests to be processed. Thanks to the ResourceRegion abstraction introduced in Spring version 4.3RC1 developer can now specify the range of bytes to be returned to the client.

@GetMapping("/videos/{name}")
fun getVideo(@PathVariable name: String, 
	     @RequestHeader headers: HttpHeaders): ResponseEntity<ResourceRegion> {
	val video = UrlResource("file:$videoLocation/$name")
	val region = resourceRegion(video, headers)
	return ResponseEntity.status(HttpStatus.PARTIAL_CONTENT)
			.contentType(MediaTypeFactory
					.getMediaType(video)
					.orElse(MediaType.APPLICATION_OCTET_STREAM))
			.body(region)
}

private fun resourceRegion(video: UrlResource, headers: HttpHeaders): ResourceRegion {
	val contentLength = video.contentLength()
	val range = headers.range.firstOrNull()
	return if (range != null) {
		val start = range.getRangeStart(contentLength)
		val end = range.getRangeEnd(contentLength)
		val rangeLength = min(1 * 1024 * 1024, end - start + 1)
		ResourceRegion(video, start, rangeLength)
	} else {
		val rangeLength = min(1 * 1024 * 1024, contentLength)
		ResourceRegion(video, 0, rangeLength)
	}
}

There are several additions worth to mention:

HttpHeaders are added to the controller to get the range header
the returned entity is now ResourceRegion. It specifies the position in the byte representations of the file along with the count of bytes to respond with.
resourceRegion method gets the range header to find the starting position. The size of chunk can be either 1 MB, the one specified in the range header or the length of contents of file. The smallest number is taken.

Now the server takes much less time to process requests and threads have opportunity to switch to other requests.

Spring WebFlux

Spring 5 has introduced a new paradigm of the reactive web applications, as well as the implementation of such an api called WebFlux. The main difference is an introduction of reactive streams as a core component.

The response body of resource in MVC was represented as java.io.InputStream. Consequently, to get data from a file and write it to a user connection we need to have a buffer, fill it with data from the file and only then write to the connection.

InputStream in = ??? // file stream
OutputStream out = ??? // socket connection stream

byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = in.read(buffer);
out.write(buffer, 0, bytesRead);

The main deficiency of this technique is read operations are blocking and hang until the buffer is fully read.

Thanks to creation of NIO in java it is now possible to operate on byte buffers and not streams of bytes. The response body in WebFlux is Publisher<DataBuffer>. DataBuffer is just an abstraction over byte buffer, can be considered as array of bytes. Publisher comes from the reactive streams specification and represents a potentially infinite stream of any entities, byte buffers in our case. The main idea of the Publisher is to work on demand. It does not produce elements eagerly but responds to requests. If the user of this streams requests 3 items then no more than 3 items will be returned. What’s more, there items are returned in an undefined point in future so you don’t need to block and wait them. When the items arrive, a callback provided is executed.

Fortunately, WebFlux api is mostly compatible with MVC and we only need to change underlying implementation to make things work. The webflux branch of my sample repo shows the dependencies needed. The code is left the same as in the MVC section of this article:

@GetMapping("/videos/{name}/full")
fun getFullVideo(@PathVariable name: String): ResponseEntity<UrlResource> {
	val video = UrlResource("file:$videoLocation/$name")
	return ResponseEntity.status(HttpStatus.PARTIAL_CONTENT)
			.contentType(MediaTypeFactory
					.getMediaType(video)
					.orElse(MediaType.APPLICATION_OCTET_STREAM))
			.body(video)
}

Now we’re ready to repeat the experiment with only one thread available. Set the property reactor.ipc.netty.workerCount=1 and call curl twice:

curl -s -o /dev/null -H "Range: bytes=0-" -D - http://localhost:8080/videos/fish.mp4/full

This time both files are loaded at the same time without the need to wait in a queue. I’d like to give you some intuition into how this works internally.

The server has two kinds of work done: selection and actual IO operations. By “selection” I mean the following:

the server has several connections, called channels
each channel can produce events
there is a special thread which tries to get these events. This thread is called selector. It performs an infinite loop inside itself (event loop). Each cycle of the loop asks the underlying OS about events available for each channel.
if the selector thread gets events, it does not read or write any data. It just fans out these events to free workers.

Our video server is mostly interested in write operations. Let’s imagine that write event pops up. Then worker thread takes over:

worker asks Publisher for the next byte buffer but not the whole file
at some point in the future this buffer arrives and callback comes into play. The buffer received is written to output stream.
only fraction of data is allowed to be written at once. Multiple write events appear during one client-server communication.

The pattern described is called Reactor. It allows to handle multiple requests at the same time using less threads by splitting the processing timeline into chunks. While it heavily relies on OS, our program will work only on Linux. The WebFlux server blocks on Windows like the MVC one.

The default implementation used by spring-boot-starter-webflux is Reactor Netty. If no selector threads are configured then workers do both selection and actual work functions. So they read events and handle them right away.

There is one additional bonus which comes with WebFlux: zero-copy data transfer. Byte buffers, copied from file to connection, are not loaded into application memory. All the work is being done on OS level. This feature is available using Reactor Netty (default backend) or Undertow.

Conclusion

In this article I tried to build simple yet efficient video streaming service and give intuition into how things work internally. If you want to read more about WebFlux, reactivity and NIO, here are some useful links:

Dave Syer’s notes on reactivity
Sébastien Deleuze tells about reactive types in spring
Threaded vs evented servers
Jakob Jenkov builds his own non-blocking web server
Awesome Doug Lea’s presentation about Scalable IO in Java
Step by step implementation of reactor using Java NIO: part1, part2, part3
And a couple more articles you can find useful