Understanding operations

Qwikstart operations (a.k.a. the steps of a qwikstart task) comprise the core functionality of qwikstart. Typically, a single operation does something quite simple, but they are combined to perform more complex tasks.

While operations are mostly independent, an operation can affect other operations by:

1. Modifying the filesystem

2. Modifying the global context

The global context is really just a Python dictionary with variables passed from one operation to the next. Operations can choose which variables they want to use from the global context and add or update variables in the global context, as desired.

Tasks and operations

A qwikstart task definition is simply a yaml file that defines a series of “steps”, a.k.a. “operations”. Take the following hello-world example:

steps:
    "Ask for name":     # 1. Description of operation
        name: prompt    # 2. Name of operation
        opconfig:       # 3. Common operation configuration
            output_namespace: "template_variables"
        inputs:         # 4. Context data specific to the `prompt` operation
            name: "name"
    "Display message":
        name: echo
        message: |
            Hello, {{ qwikstart.name }}!

This task composes two different operations: prompt and echo. The prompt operation is used to prompt the user for a name, and then the echo operation displays a greeting.

The prompt operation adds all user inputs into a template_variables dictionary, which is added to the global context. Note that the output_namespace config isn’t actually needed since that’s the default for the prompt operation. It’s added here just for clarity.

The echo operation expects a template_variables dictionary as input, which is used to render the message. The echo operation defaults to template_variable_prefix = "qwikstart", which is why the template variable is rendered using qwikstart.name instead of just name as specified in the inputs definition.

Anatomy of an operation

As outlined in the inline comments in the example, there are four different parts to a given operation:

1. The operation description. This is a descriptive string for the operation that is used for display purposes and must be unique.

2. name, which is the name of the operation being configured. This should be a name matching any of the Available operations for qwikstart tasks.

3. opconfig, which contains common configuration for qwikstart operations. This is described in more detail in the Common operation configuration section.

4. Context data specific to each of the Available operations. For example, the prompt operation expects a list of inputs and the echo operation expects a message string. The docs for prompt and echo will have more detail about the optional variables for these operations.

Context data

The context data specified in the operation definition above (comment 4), is just one way to define context data. That definition is known as “local” context data, since those definitions only affect the operation where they’re defined. To make the concept of local context clear, we can define template_variables for the echo operation as local context:

examples/opconfig/echo_with_local_context.yml

steps:
   "Display message":
       name: echo
       template_variables:
           name: "World"
       message: |
           Hello, {{ qwikstart.name }}!

Operations can have outputs, which are added to the global context for use by subsequent operations. The following example uses the define_context operation to define template_variables:

examples/opconfig/define_and_echo.yml

steps:
    "Define name to greet":
        name: define_context
        context_defs:
            template_variables:
                name: "World"
    "Display greeting":
        name: echo
        message: |
            Hello, {{ qwikstart.name }}!

Using define_context to add static values isn’t that useful, but operations such as context_from_regex, find_files, and prompt allow you to write tasks that define context based on the execution environment and the user.

Common operation configuration

The opconfig variable is a dictionary containing optional configuration common to all qwikstart operations.

input_mapping (default: {}):

Dictionary mapping new context variable names, which will be used by the operation, to variable names in the global context.

output_mapping (default: {}):

Dictionary mapping new context variable names, which will be stored in the global context, to variable names returned by the operation.

input_namespace (default: None):

String specifying a dictionary in the global context that will be used as input variables by the operation instead of the variables in the global context.

output_namespace (default: None):

String specifying the name of the dictionary in the global context where output variables from the operation are stored. By default, this is None, which means output variables are added directly to the global context. Some operations, notably prompt, specify the default of "template_variables", which is a special namespace used by many operations when rendering templates.

display_description (default: True):

Boolean value controlling whether to display the description of an operation on the command line during exection. This defaults to True but some operations override this default (though it’s possible to override that by when configuring an operation).

Operation execution sequence

The basic execution sequence is outlined below:

1. Inject global context

2. Remap variables based on opconfig.input_mapping

3. Isolate context based on opconfig.input_namespace

4. Add variables based on local context

5. Run operation

6. Nest output under namepace in opconfig.output_namespace

7. Remap output based on opconfig.output_mapping

8. Merge output with global context

Other than the run-operation step in step 5, all the other steps are to control either of the following: - The data (i.e. context data) passed to the operation (steps 1-4) - The data saved to the global context (steps 6-8)

These steps are explained in depth below.

Inject global context

The execution sequence starts with an operation’s execute method, which is passed the global context. This global context is just a dictionary containing output from prior operations.

The following example uses the define_context operation to add a template_variables dictionary (containing name) to the global context. That context is injected into the echo operation, which expects a template_variables dictionary used for rendering a message:

examples/opconfig/define_and_echo.yml

steps:
    "Define name to greet":
        name: define_context
        context_defs:
            template_variables:
                name: "World"
    "Display greeting":
        name: echo
        message: |
            Hello, {{ qwikstart.name }}!

Remap variables based on opconfig.input_mapping

Next, variables in the global context can be remapped to new variable names. Operations expect variables with specific names, so this can be used to combine operations that weren’t initially meant to be combined.

Instead of defining a template_variables dictionary directly, the following example just adds a name to the global context. Since the echo operation expects a template_variables dictionary, we can use the input_mapping to remap the name to template_variables.name, below), for use when rendering the template:

examples/opconfig/define_and_echo_with_input_mapping.yml

steps:
    "Define name to greet":
        name: define_context
        context_defs:
            name: "World"
    "Display greeting":
        name: echo
        opconfig:
            input_mapping:
                "name": "template_variables.name"
        message: |
            Hello, {{ qwikstart.name }}!

Note that the period in template_variables.name is a namespace separator.

Isolate context based on opconfig.input_namespace

Here, a “namespace” is really just a dictionary nested within the global context dictionary. If an input_namespace is specified, then only the data within the sub-dictionary will continue on this journey. Otherwise, the entire global context is passed along.

The following example uses the define_context operation to define a dictionary with the key context_for_echo containing a template_variables dictionary with the name that will be rendered by the echo operation. To use template_variables for rendering, the echo operation selects the context_for_echo dictionary from the global context using input_namespace:

examples/opconfig/define_and_echo_with_input_namespace.yml

steps:
    "Define name to greet":
        name: define_context
        context_defs:
            context_for_echo:
                template_variables:
                    name: "World"
    "Display greeting":
        name: echo
        opconfig:
            input_namespace: "context_for_echo"
        message: |
            Hello, {{ qwikstart.name }}!

Note that the mapping is defined using the source name as the key, and the target name as the value.

Add variables based on local context

The final step before running the operation is to add in the “local” context, which is just data defined as part of the operation. For example, the following defines the echo operation, with a local context variable, message:

steps:
    "Display message":
        name: echo
        message: "Hello"

This local context gets combined with the global context (after remapping and namespacing) to form the operation context.

Run operation

Finally, the actual work of the operation gets done. The operation context, which was created by the steps described above, is used to do whatever the operation wants using the operation’s run method. As part of this, the operation can return any data that it wants added to the global context.

After the operation is run, we basically rewind the steps from above.

Nest output under namepace in opconfig.output_namespace

The output from the operation (if there is any), can optionally be nested under a namespace. In other words, it can be placed in a subdictionary in the global context.

The following example uses the define_context operation to define a name, which is then put in a template_variables dictionary using the output_namespace operation config:

examples/opconfig/define_and_echo_with_output_namespace.yml

steps:
    "Define name to greet":
        name: define_context
        context_defs:
            name: "World"
        opconfig:
            output_namespace: "template_variables"
    "Display greeting":
        name: echo
        message: |
            Hello, {{ qwikstart.name }}!

Remap output based on opconfig.output_mapping

The output data from an operation can be renamed using an opconfig.output_mapping, just like inputs were renamed using opconfig.input_mapping.

The following example uses the define_context operation to define a name, which is then put in a template_variables dictionary using the output_mapping operation config:

examples/opconfig/define_and_echo_with_output_mapping.yml

steps:
    "Define name to greet":
        name: define_context
        context_defs:
            name: "World"
        opconfig:
            output_mapping:
                "name": "template_variables.name"
    "Display greeting":
        name: echo
        message: |
            Hello, {{ qwikstart.name }}!

Note that the mapping is defined using the source name as the key, and the target name as the value.

Merge output with global context

Finally, the output variables can be merged with the global context for subsequent commands to use.