Statelint: Rise of the State Machines

 
At the heart of Tymly are state machine definitions – they’re the recipes that describe how workflows in Tymly will behave.

State machines undertake all the heavy-lifting in Tymly: from overseeing a simple form-filling exercise through to orchestrating a complex data-mangling process. We define our state machines using Amazon States Language and we use our Statebox package to run them.

  • To support Statebox, we’re happy to announce the availability of Statelint, a JavaScript package for validating Amazon States Language JSON, and of Tymly-statelint, a validator for Amazon States Language JSON with Tymly extensions.

Why validate?

In Tymly, a simple state-machine definition looks something like this:
{
  "Comment": "A simple minimal example of the States language",
  "StartAt": "Hello World",
  "States": {
    "Hello World": {
      "Type": "Task",
      "Resource": "module:HelloWorld",
      "End": true
    }
  }
}

Nice, neat, straightforward. Typically, though, our state-machine definitions are a little longer:

{
  "Comment": "Gets the incidents in progress.",
  "name": "Incidents in Progress",
  "version": "1.0",
  "categories": [
    "incidents"
  ],
  "StartAt": "RemoveDocs",
  "States": {
    "RemoveDocs": {
      "Type": "Task",
      "Resource": "module:removeDocs",
      "InputPath": "$",
      "ResourceConfig": {
        "query": {
        "category": "iip",
        "activeEvent": true
      }
    },
    "Next": "GetIncidentsInProgress"
    },
    "GetIncidentsInProgress": {
      "Type": "Task",
      "Resource": "module:getDataFromRestApi",
      "ResourceConfig": {
        "templateUrlRegistryKey": "incidentsInProgressUrl",
        "authTokenRegistryKey": "incidentsToken",
        "resultPath": "incidentsInProgress",
        "namespace": "wmfs"
      },
      "ResultPath": "$.incidents",
      "Next": "AddDocs"
    },
    "AddDocs": {
      "Type": "Task",
      "Resource": "module:addDocs",
      "InputPath": "$.incidents.incidentsInProgress",
      "ResourceConfig": {
        "mapping": {
          "id": "incident#||incidentNumber",
          "docId": "incidentNumber",
          "domain": "search",
          "docType": "incident",
          "title": "Incident ||incidentNumber||/||callTimeYear",
          "description": "incidentClassificationLabel",
          "category": "iip",
          "point": "locationLatitude||,||locationLongitude",
          "activeEvent": true,
          "author": "incident",
          "roles": "$authenticated::text[]",
          "language": "ENG",
          "sortString": "incidentNumber",
          "created": "$NOW",
          "modified": "$NOW"
        }
      },
      "ResultPath": "$.incidents",
      "Next": "AwaitingHumanInput"
    },
    "AwaitingHumanInput": {
      "Type": "Task",
      "Resource": "module:awaitingHumanInput",
      "ResourceConfig": {
        "uiType": "board",
        "uiName": "wmfs_incidentsInProgress",
        "dataPath": "$.incidents"
      },
      "End": true
    }
  },
  "restrictions": [
    {
      "roleId": "$authenticated",
      "allows": [
        "*"
      ]
    }
  ]
}

Makes your eyes go funny just looking at it, doesn’t it?

  • As state-machine definitions get longer, they get increasingly difficult to write correctly.
  • You might add a new state, but forget to change an End to a Next so it never gets called.
  • You might misspell the name of a module, causing your state machine to mysteriously fail.
  • If a module’s ResourceConfig isn’t properly filled out, the state machine might run but not as you intended.
  • Tooling to help define state machines (especially graphical tools for use by “non traditional developers”) will benefit from clear, easily understood validation messages too.

Validation allows you to be sure your state machine definitions are syntactically correct – that they have no obvious errors. It’s a reassuring safety net.


But how?

We’ve talked before about  why we are open with Tymly and this work arose because of the openness of others, specifically AWSLabs, the originators of Amazon States Language (ASL).

Tim Bray, one of the members of the AWSLabs team, describes developing an ASL validator in this blog post. Actually, he describes writing two validators – one used internally by Amazon and one that’s publicly available.

So, we’ve we written one as well?

  • Tim’s validator is intended as a command line tool and is written in Ruby. We needed a library we could use within Tymly, which means it needs to be written in JavaScript.

Happily, Tim and AWSLabs made the source code of their validator available, and so we were able to port that code to JavaScript. By reusing his code, and specifically his tests, we were able to ensure that our validator worked identically to his.

As a pleasing side effect, our implementation revealed resolutions for a couple of issues raised against the original code, which we were able to submit back to Amazon.

It’s a nice circle to close.


Extending into Tymly

While Statebox and Statelint are available for people to use as standalone pieces of software, we of course use them as core components of Tymly.

Our state machines have some extensions to ASL, primarily additions to help with service discovery and access control. Rather than clutter-up Statelint with Tymly bits-and-bobs, we’ve developed a separate Tymly-specific validator on the top of it.

When hooked-up inside Tymly, Tymly-statelint has access to all the available “State Resources”:

  • State resources are chunks of executable code called by state machines – in the examples above they are the things referenced by the Resource elements.
  • The accompanying ResourceConfig elements are a Tymly extension… they define the startup inputs to a state resource.
  • Each state resource has a different ResourceConfig.
  • By leaning on the code developed in the Statelint port, Tymly-statelint is able to validate the ResourceConfig for the specific state resource named by a Resource.

It’s proving to be an extremely powerful tool to have available!

 

Photo by Isis França on Unsplash