Previous: Structured Builtins

Unstructured Builtins

In the last section we saw structured builtins, which work in a very prescribed way on a list of arguments. But now we look at unstructured builtins, which are less prescribed, but are more powerful.

Rather than working on an "args" list, unstructured builtins draw their input from the scope. To use them effectively we need to be able to modify the scope, which is done with the dictionary form of the builtin transform.

It looks like this:

transform = {
    "&": <builtin>,
    "arg1": ...,
    "arg2": ...,
    ...
    "argN": ...
}

To evaluate this form of builtin, we need to update the scope as follows:

new-scope = scope-update ( scope, transform )

then pass new-scope to the builtin as its scope.

scope-update

scope-update is more or less like a dictionary update operation. It takes an existing scope (can be any data), and a scope-delta (this is a dictionary).

scope-update ( scope, scope-delta )

It then does the following:

  • If scope-delta includes any special attributes ("!", "!!", "&"), then remove them.
  • If scope is not a dictionary, replace it with the empty dictionary {}.
  • Add all attributes from scope-delta to scope (replacing any conflicts), and return that.

(note: all these operations use immutable data, so where I say "remove", or "replace" read this as "make a copy that acts as though it is the original with something removed or replaced")

So what's the point?

This means you can gather up the arguments to the builtin in a natural way inside the builtin evaluation transform.

For example, say we want to know what type some data is. We can use the unstructured type builtin.

type takes the argument value from the scope (ie: "^@.value"), and returns the name of the type of this value as a string. But this value can come from anywhere, by means of a path.

Say we have this source:

source = {
    "thing": [
        {
          "goop": 4
        }
    ]
}

We know we're going to get source data where there is one element in a list called "thing", but we don't know what data types to expect. So, we can use this transform to find out:

transform = {
  "result": {
    "&": "type",
    "value": "^@.thing.0"
  }
}

Let's evaluate this. First, we need a new function, evaluate-builtin. It takes a builtin name (like type), and a scope, and evaluates to something. Like this:

result = evaluate-builtin ( builtin, scope )

So using this, let's evaluate the example.

evaluate ( source, transform, scope, library )
= {
    "result": evaluate ( source, transform["result"], scope, library )
}
= {
    "result": evaluate ( 
        {
          "thing": [
            {
              "goop": 4
            }
          ]
        }, 
        {
          "&": "type",
          "value": "^@.thing.0"
        },
        {
          "thing": [
            {
              "goop": 4
            }
          ]
        },
        library
    )
}
= {
    "result": evaluate-builtin (
        "type",
        scope-update ( 
            {
              "thing": [
                {
                  "goop": 4
                }
              ]
            },
            {
              "value": {
                "goop": 4
              }
            }
        )
    )
}
= {
    "result": evaluate-builtin (
        "type",
        {
          "thing": [
            {
              "goop": 4
            }
          ],
          "value": {
              "goop": 4
          }
        }
    )
}
= {
    "result": "map"
}

Here you can see scope-update adding value to the scope, and then passing the result as the new scope to evaluate-builtin, which invokes type and returns the string name for a dictionary, which is "map" (it should probably be "dict"; it's an historical thing that might be fixed in the future).

Try it out here:

Required Builtins

The builtins listed here are required to be provided by the interpreter. Arguments are listed in each one; they are to be drawn from the scope.

if (the conditional builtin)

This is sUTL's basic conditional. 

{
  "&": "if",
  "cond": {":": <conditional transform>},
  "true": {":": <true transform>},
  "false": {":": <false transform>}
}

Notice that the arguments to if are wrapped in quote transforms. This is because they will be evaluated twice; once because all entries in a dictionary are evaluated before it is treated as a builtin transform, and a second time when the if builtin is evaluated by evaluate-builtin. You will generally want to protect them from the first evaluation by wrapping them in the quote transform, leaving them intact for the real evaluation as part of the if.

Also note that when they are evaluated inside evaluate-builtin, they take the scope passed in to evaluate-builtin with them.

if first evaluates cond (ie: <conditional transform>). If the result is truthy, it evaluates and returns true (ie: <true transform>). Otherwise it evaluates and returns false (ie: <false transform>).

One of the benefits of if is the conditional evaluation. Only one path is evaluated, both paths are not. This can lead to significant efficiencies in a complex transform.

Here's an example. Have a play with it.

len (length of a list)

len works as follows:

{
    "&": "len",
    "list": <a list>
}

It evaluates to the length of the list, or 0 if this is not a list.

keys, values (get the keys or values from a dict)

Give keys a dictionary, it'll give you back a list of the keys, or null if it's not a dictionary. Give values the same dictionary, it'll give you back a list of the values.

{
    "&": "keys",
    "map": <dict>
}

{
    "&": "values",
    "map": <dict>
}

eg:

head, tail (get the head or tail of a list)

Get the head of a list as a single element (or null), or the tail of a list as a list, possibly empty.

{
    "&": "head",
    "b": <list>
}

{
    "&": "tail",
    "b": <list>
}

[ "&head", <list> ]

[ "&tail", <list> ]

Notice you can use array forms for these.

type (get the type of some data)

type examines "value" in the scope, and returns a string describing the type.

{
    "&": "type",
    "value": <data>
}

Return values:

  • Dictionary: "map"
  • List: "list"
  • String: "string"
  • Number: "number"
  • Boolean: "boolean"
  • Null: "null"

See the top of the article for an example of type.

makemap (makes a dictionary out of a list of pairs)

makemap takes a list of pairs (2 item lists) and returns a dictionary where every first element is a key, and every corresponding second element is that key's value. The keys must be strings (if they are not they will be skipped). Bad data otherwise will lead makemap to evaluate to null.

{
    "&": "makemap",
    "value": [
        [<key>, <value>],
        ...
    ]
}

reduce (transform a list, item by item)

sUTL doesn't have imperative loops. So algorithms must either use recursion (which is very possible, but expensive and limited in sUTL), or use reduce. reduce iterates over a given list, transforming each item into an accumulator, and returning that accumulator as the result at the end.

{
    "&": "reduce",
    "list": <list to iterate over>,
    "accum": <initial value of the accumulator>,
    "t": {":": <item to accumulator transform>}
}

Note that the <item to accumulator transform> is wrapped in a quote transform. This is to negate the initial evaluation of t, before it is passed through to evaluate-builtin. This is just like what is done for the if builtin; have a look at if for more discussion.

What reduce does is this:

  • Initialise the accumulator to accum
  • For each item in list,
    • add the item to the scope as item
    • add the accumulator to the scope as accum
    • result = evaluate ( source, transform["t"], scope, library )
    • set the accumulator to result.
  • the result is the final value of the accumulator

Here's an example of using reduce to calculate the largest number in a list:

You'll notice a few interesting things:

1 - the item transform t is wrapped in a quote transform as above. Then, you can see it is a big if builtin. Inside that, in the cond, you can see the use of > to compare the current item ("^@.item"), with the accumulator ("^@.accum). 

2 - I haven't provided the list in transform. This is because it appears in the source. The source becomes the initial value of the scope when the transform is evaluated on it, and because list is already in the scope, there's no need to provide it in the transform. All providing it in the transform does is to add it to the scope, ready for the builtin to use, which isn't necessary in this case.

string (convert value to string)

Cast another type to a string. 

  • If the value is a string, just return that.
  • If the value is a number, convert it to the string representation of that number
  • If the value is a boolean, convert it to "true" or "false"
  • If the value is null, convert it to "null"
  • If the value is a dictionary, evaluate to the string "map"
  • If the value is a list, evaluate to the string "list"
{
    "&": "string",
    "value": <any value>
}

number (convert value to number)

Cast another type to a number. 

  • If the value is a number, just return that.
  • If the value is a string, convert it to a numeric representation:
    • Can convert it? Return the number
    • Can't convert it? Return 0
  • If the value is a boolean, convert it to 1 if true else 0
  • If the value is null, return 0
  • If the value is a dictionary, return 0
  • If the value is a list, return 0
{
    "&": "number",
    "value": <any value>
}

boolean (convert value to boolean)

Cast another type to a boolean. This just uses truthiness: if the value is truthy, it converts to true, otherwise false.

{
    "&": "boolean",
    "value": <any value>
}