Introduction to Sircle

What is Sircle?

Sircle is a powerful DSL for defining tasks. It is in functional style, supporting first-class functions and currying.

Example

mkTask "main.py" { "seed" -> 123, "base_model" -> "GCN" } { "base_model" -> "GCN" }

This will create a task with executable main.py and arguments. It may be executed by the runner with something like

python main.py --seed 123 --base_model GCN  # ... and some other flags

The last argument, { "base_model" -> "GCN" } here, is the tag. It is used to identify the task, and is needed to aggregate and display task results.

The above Task datatype consists of three elements,

• Executable, which is the actual computational task. In most cases it is a Python script.
• Arguments, which specify the argument passed to the executable.
• Tag, which identifies the task and helps experiment results handling.

Apart from the above constructor, we can also construct a Task with two operators:

• task1 >> task2, which defines a task where tasks1 is executed first, then task2.
• task1 || task2, which defines a task where task1 and task2 can be executed in paralell.

Sircle DSL is highly expressive for defining computational tasks. For a real-world example, considering the conifig

def config = {
    "seed" -> [1, 2, 3],
    "dataset" -> ["cora", "citeseer", "pubmed"],
    "model" -> ["GCN", "GAT"]
}

and we want to run experiments with combination of different argments. We would like to run experiments with different seeds in parallel, and for any specific seeds, we run the experiments sequentially. To express the above task structures, we can write Sircle code like

def resolve = config => mkPar {
    def c = restrict ["dataset", "model"] config;
    for seed <- config."seed" do
      mkSeq $ map (x => mkTask "main.py" x $ x + { "seed" -> seed }) $ namedProd c
}

On the other hand, from the short example above, we can see some basic properties of Sircle.

A first glance at Sircle

Function application

The function application in Sircle follows the ML style, where brackets are omitted. In other words, instead of writing

f(x, y)

we write

f x y

in Sircle.

On of the greatest benefits of such style is that it can integrate well with currying. For example, we can write

def add = x => y => x + y
map (add 1) [1, 2, 3]

in Sircle, which will produce [2, 3, 4].

Name Binding

In sircle, we can use the def keyword to bind a name. For example,

def x = 1

And type annotation is also possible, we can write

def x: Int = 1

The evaluator will check the value type of the right hand and verify that it is consistent with the type annotation. In other words, the following binding

def x: Int = "1"

will result in a runtime error.

Actually, when we omit the type annotation in the binding, the type is assumed Any, which will be consistent with any value types.

In Sircle source, name bindings can happen both at global level and in block expressions. For instance,

def x = 1

def main = unused: Unit => {
    def y = x + 1;
    y = 2 * y;
    y
}

A block expression contains a sequence of effective statements separated by commas, enclosed in curly braces. There are three types of effects:

• Binding. Example: def y = x + 1. This will bind value x + 1 to name y and evaluate to the binded value.
• Reassignment. Example: y = 2 * y. This will assign a new value 2 * y to the name y and evaluate to the new value.
• Eval. Example: y. This is simply a pure expression and will evaluate to a value.

The block expression will evaluate the effects one by one, with possibly modification to variable environments, and return the value of the last effect. This style is similar to Scala.

The global bindings, unlike bindings in the block expressions, are immutable.

The Mapping Datatype

The expression

{
    "seed" -> [1, 2, 3],
    "dataset" -> ["cora", "citeseer", "pubmed"],
    "model" -> ["GCN", "GAT"]
}

will result in a Mapping value, which is similar to dict in Python and something called map in other languages. It is a key-value mapping, with keys must being String types. Mappings are used in Sircle to describe task arguments and tags.

It is possible to access the Mapping values by

config."seed" 
// => [1, 2, 3]

and update it with

config + { "seed" -> [3, 2, 1] } 
// => { "seed" -> [3, 2, 1], ... }

More details on datatypes will be discussed later.

Lambdas

The expression

x => x + 1

will evaluate to a lambda function. Function definition in Sircle, for instance,

def add = x: Int => y: Int => x + y

is simply binding a lambda to a name.

Note that we can use annotations to specify the argument type. The above definition is mostly equivalent to the code in other programming languages:

Example

Python

def add(x: int, y: int):
    return x + y

Scala

val add = (x: Int, y: Int) => x + y

Haskell

add :: Int -> Int -> Int
add x y = x + y

Generally, we can define a \(n\)-arg function with

def funcName = arg1: Type1 => arg2: Type2 => ... => argN: typeN => bodyExpr