ETS FaaS Best Practises

This document covers best practices with regard to developing, implementing and reviewing FaaS functions. 

ETS has built many FaaS functions over the years, and we spend a lot of time redoing the same thing again and again. This document will provide templated code for repetitive tasks, such as:

• Retrying API calls
• Paginating API requests
• Error Handling
• Unit Testing templates

Any references to a Best Practise FaaS relate to THIS function stored in Gitlab.

FaaS Development

Development and Code Review Process

When starting a project where a FaaS is required, the below steps should be followed:

1. Create a repository in GitLab (HERE) called “ts-{{REGION}}-{{BRAND}}-faas-{{SOLUTION}}”
   1. Create a develop branch off of the main branch.
   2. Clone the repo to your machine.
   3. Checkout the Develop branch.
   4. From this point on, make frequent commits and pushes to the Develop branch.
2. Develop the FaaS function, documenting Environment Variables, Secrets and setup instructions as the development proceeds.
3. Deploy the code to a test account.
   1. The easy way is to simply copy and paste the code from VSCode into the FaaS UI.
   2. The hard way is to use the FaaS CLI to deploy directly from your machine to FaaS, skipping any need to login to the UI. See Further details HERE.
4. Test the FaaS function.
   1. Again, the easy way is to use the FaaS UI.
   2. The hard way is to use the CLI. See Further details HERE.
      Repeat steps 2 to 4 until the FaaS is perfect
5. Push the code to the develop branch and make a Merge Request from develop to main.
6. Agree who will be reviewing the code and then add them as a reviewer to the MR
   1. This reviewer should be another SE.
   2. The Project Manager should be aware that the review process can take some time, depending on the availability of other SEs and the issues that may be found. This should be included in the project plan.
7. Implement any fixes and improvements suggested by the reviewer.
8. Repeat steps 2 through 8 until the code is even more perfecter than in step 5.
9. Deploy to the brand’s production account.

Commonly Used Functionality

To assist with step 2, the next section of this document will link to example code that can be copied and pasted (or forked) into the new FaaS to speed things up.

API Exponential Back Off and Retry

TODO

Reading and Writing from CCS

TODO

Paginating Through the Messaging Interactions API

TODO

Calling the WebView API

Unit Testing

Unit Testing allows you to write code…that tests your code. This is done by faking all inputs, including those from any API calls and examining the outcome. The tests do not call any real APIs so no real credentials should be used.

For example, if your FaaS calls the secret storage client and then the Web View API, you can write the following test:

• Write a test where the secret client fails to find the secrets and throws an error
  • You then test the outcome: that the web view API is not called and that the FaaS calls the FaaS Callback with an error message
    
• Write a test where the secret client returns a secret, but then the Web View API fails
  • Again, the FaaS Callback should be called with an error message
    
• Finally, write a test where both the secret client and Webview API are called successfully.
  • The FaaS Callback should be called with a success message
    
• If there is any logic in the FaaS, this should also be tested as well, based on faking inputs from the FaaS invocation payload and responses from client/API requests.

HERE is an example of unit testing using the above methodology by testing “what happens if the first thing fails” followed by “what happens if the first thing succeeds, then the second thing fails” and so on.

Stubs

The example test file fakes the LpClient of the FaaS Toolbelt by “stubbing” it. In short, you’re replacing the actual LpClient with an interactive stub that you can specify what is returned on a test by test basis.

For a given test, if we wanted to run a test where the LpClient throws an error, we can simply write “stubs.tbLPClient.throws(‘AnUnhappyResponse’)”. If we wanted to resolve successfully, “stubs.tbLPClient.resolves(‘AHappyResponse’)”.

Tests

The “name” of the test should reflect what is being tested in plain text. In the test below, we have defined a fake “happy” response for the lpClient. Here, the lpClient is being used to call the Messaging Interactions API, so the response has been built to mimic that.

Next, the function is being called like any other function in javascript.

Finally, the outcomes are tested. In FaaS, in a happy flow, I expect that the first argument of the callback to be null and the second to be a string. You can access the call arguments and test the values of the types as demonstrated.

General Best Practises

Time

// TODO: 30 seconds in FaaS and 20 seconds in CB

The FaaS CLI

The FaaS CLI lets you do everything that the FaaS UI lets you do, however, it lets you do it all with one terminal command from your local machine. 

• You can run the FaaS debugger locally to test your FaaS without having to keep copying and pasting code from your machine to the UI. 
• You can deploy the local code to whatever account you want without worrying about copy and paste mistakes. 
• I also found that I would frequently end up where my local code had “desynced” from the code in the FaaS UI as I made small tweaks. Doing ALL development locally and then pushing to FaaS using the command line removed the chance of “desyncing”.

An example of working with the CLI is demonstrated HERE.

Automated Testing via Gitlab

With any kind of development we can repeatedly test and test on our local machines, but as soon as the code gets to a production environment, it stops working. There are many reasons why this can happen such as:

• Local environment is different from the production environment.
  • We develop on MacOS whereas the code is running on Linux.
  • Differing versions of NodeJs behave differently.
• An erroneous typo made it way into the last commit.
• Solar flare flipped a bit in RAM causing the code to work.

To remove some of these sources of pain, we can set up Gitlab to run the same tests that we would run locally, in a Docker container that more closely mirrors the real production environment.

You can configure these activities to run on every commit, or on creation of an MR.

Gitlab CLI File

To tell Gitlab to do something, you need a “.gitlab-ci.yml” file. This EXAMPLE shows us everything we might want to do when developing a FaaS. It shows us:

• How to pick what version of node/OS to run the below operations on.
• How to run unit tests.
• How to lint the code.
• How to run a code coverage report.

The “script blocks” simply lists the terminal commands that should be run inside the Docker container. Where there is a report generated, such as for code coverage, then it also specifies to Gitlab where to find that report so it can present the results in the UI.

Docker Image Version

When FaaS updates its NodeJS version, the node version in the Docker container should be updated as well. The Docker image in the template uses node 18.18 on the lightweight Alpine OS.

Instructions on how to update the image can be found HERE.

Running the Tests

The rest of the file in the example is pre-set to run any unit tests and run a coverage report.

Simply ensure that in the package.json file, there are three scripts:

• pipeline-unit-test 
• code-coverage
• lint

These are all demonstrated HERE.

Reviewing the Outcome of the Pipeline

There are two main audiences for the results of the pipeline tests; yourself and the reviewer. As a reviewer who has received an MR, you would be able to see the outcome of all completed jobs in the MR itself.

If I saw that the unit tests were failing, I would immediately ask the developer to review and fix the test. If the tests weren’t being run in Gitlab, the reviewer would likely miss the failing test.

The Code Coverage report highlights what the unit tests cover, as shown in the screenshot below. In the screenshot, you can see that there is one test that hits the “Missing Env Value” flow in the code.

As a reviewer, if the code coverage is low, or there are important flows that are not tested, that would be a red flag.

Documenting a FaaS Function

There are many types of documentation that one could produce:

• Code Level Documentation
• Technical Implementation Guide
• External End User Guide
• Internal Support Manual

Technical Implementation Guide - Readme.md

Only SEs and SAs have access to Gitlab, which means the readme can contain very low level, technical details.

They should take the form:

• Introduction explaining what the FaaS function does and what problem it solves
  
• A step by step guide of how to implement the FaaS:
  • What secrets are required, where you find them and what format they should be saved in
  • What environment variables are required, where you find them and what format they should be saved in
  • If users and/or skills are required
  • What domains need whitelisting
  • Steps to create CCS Namespaces if required
  • If there are other components such as an iHub project or CB Bot, these need documenting as well.
    
• Gitlab natively supports Sequence Diagrams written in the mermaid format in readme.md files. There is no need to paste a picture into the readme. The diagram should include:
  • What API calls are going where and a high level idea of what data is returned
  • What would happen if an API call fails
  • High level details of what logic is happening between requests
    
• The readme should include links to any other useful resources. For example, any BRDs, User Guides or other useful Google Docs.

See HERE for an example Technical Implementation Guide.

Code Level Documentation

Comments

When writing any code, it is important to comment throughout to explain what is actually going on. There are plenty of guides online to help give some examples of what good commenting looks like. To give some general guidance:

Don’t:

• Comment every line of code
• State the obvious

Do:

• Comment blocks of complex logic
• Explain things in plain English

JSDoc

An additional level of documentation is JSDoc. JS is not a strongly typed language, which means it’s impossible to tell whether a function is returning a complex object of objects, or is simply returning a string. VSCode can infer certain things, such as an async function will return a promise, but it can’t tell you much more. Luckily, we can manually add this information to our code. 

A full list of JSDoc functionality can be found HERE.

The below screenshot shows an example of the JSDoc syntax. You can see that you can add a description as to what the function does, what the arguments are and what the function returns.

In VSCode, hovering over the function name will show you the JSDoc details. This will also mean that if VSCode knows what type a variable is, it can suggest what methods you are allowed to use, i.e, it will let you loop through an array, but not an object.