Trial templates

Unlike stimuli and responses, which define only the content of a study, trial templates are primarily used to specify the runtime logic.

A trial template is defined by a property-value pair inside the top-level property trial_templates in a study runtime specification. The property name serves as the name of the trial template, which can be referenced in other parts of the specification (most likely, in procedure). Its value specifies the detailed properties of the trial template.

An intuitive way to understand the role of trial templates is to view them as definitions of how trials are realized from a hypothetical collection of all possible configurations. For example, consider the following definition of a trial template named CatLearnATS:

"CatLearnATS": {
  "type": "basic",
  "stimuli": ["singleShort1","singleShort2","singleShort3","singleShort4","singleShort5"],
  "stimulus_pattern": {"order": "descending", "attribute": "length"}
  "responses": ["CatLearnAResponse"],
},

This trial template CatLearnATS defines a set of trials that can be realized from this particular template. The stimuli to be displayed on those trials come from the set ["singleShort1","singleShort2","singleShort3","singleShort4","singleShort5"]. Implicitly, this also means that a total of five trials can be realized from this template. Across those five trials, these stimuli will be presented in an order that is descending based on the length attribute of those stimuli. Finally, on each of the five realizable trials, the same response CatLearnAResponse will be used.

Note

  • The attribute length must be defined as a property for the listed stimuli; otherwise, you will receive an error message complaining about unknown attributes.
  • There are a lot of quotation marks here. None of them can be left out. As a general rule of thumb, anything other than numbers and boolean values (true or false) should be in quotation marks.

Inheritance

Like stimuli and responses, trial templates also allow property inheritance. One similarly defines the property parent to enable this feature:

  • parent (optional) - the name of another trial template; if specified, then the properties of the other trial template will be inherited by this template, unless a property of the same name is defined in this template.

Built-in trial templates

basic

A trial template that does not fit any of the following types is a basic trial template.

Warning

Unlike trial templates, a type named basic does NOT exist for stimuli or responses.

Properties:
  • type - specified as basic
  • stimuli - a list of items specifying which stimuli to be used in a trial template and how they are used (such as the location of stimuli). See Specifying stimuli in a trial template for available options.
  • stimulus_pattern - a dictionary that specifies how the supplied stimuli are used on realized trials. See Stimulus patterns for details.
  • responses - a list of response names representing responses to be used on realized trials. On any given trial, only one response will be used (order is randomized). Similar to stimuli, nested lists can be used to present more than one response in a single realized trial (e.g., [["r1", "r2"]] will make r1 and r2 appear on the same trial).

Warning

When the responses list consists more than one top-level elements, such as ["r1", ["r2", "r3"]], the stimuli list MUST contain at most 1 top-level stimulus element (a single name or a list of names). This is to prevent simultaneously sampling from stimuli and responses, which can lead to unpredictable behavior.

  • response_confirm (optional, boolean) - if set to true, participants will need to explicitly confirm their responses before proceedinig to the next trial. Default (or omission) is false, in which case the study will proceed to the next trial immediately after all responses on the current trial have been recorded.
  • delay (optional, numeric) - the number of seconds (fractions are supported) to wait before starting a trial. This is particularly useful for implementing an inter-trial interval.
  • duration (optional, numeric) - the number of seconds (fractions are supported) to wait before automatically proceeding to the next trial. This is useful in cases where no responses are needed (such as during a training phase) or a speed-accuracy tradeoff is being evaluated (to turn off automatic advancement to the next trial, see “auto_advance”).
  • auto_advance (optional, boolean) - when duration is longer than 0, if set to false, a continue button will appear for participants to click on to advance to the next trial, instead of automatically advancing to the next trial. Default is true.
  • countdown_visible (optional) - if set to true and the duration property is greater than 0, then a timer will appear on each trial to remind participants explicitly that this is a timed trial. Default is false.
  • replayable (optional) - if set to true, a clickable banner saying “replay this trial” will appear on the top left corner of all trials generated from this trial template. Participants can refresh a trial by clicking that banner. Default is false.
instruction

An instruction trial template is specialized in displaying a single paragraph of text that usually serves as instructions for upcoming trials.

Properties:
  • type - must be specified as instruction
  • stimuli - same as in the basic trial template, but only the first stimulus is used
  • duration (optional, recommended) - the number of seconds that participants are forced to wait before being able to continue beyond this trial
AFC

An N alternative-forced-choice trial template is specialized in handling alternative-forced-choice trials.

All properties are the same as the basic trial template. Specifying AFC will trigger additional semantic checks for the convenience of the researcher.

Specifying stimuli in a trial template

The simple option

"stimuli" takes a list of stimulus names representing stimuli to be used on realized trials. On any given trial, only one stimulus will be used. To present multiple stimuli on a single realized trial, use a nested list, such as in "stimuli": ["s1", ["s2", "s3"], ... , "sn"], where s2 and s3 will appear on the same trial.

Note

Using the simple option accepts sensible defaults for the locations of stimuli on each realized trial. We recommend researchers use the simple option unless specific customizations are needed.

The custom option

"stimuli" takes a list of dictionaries, each of which represents one stimulus (or a set of stimuli) to be used on a single realized trial. Each dictionary accepts three properties:

  • "which" takes a single stimulus name, or a list of stimulus names, representing a stimulus (or a set of stimuli) to be used on a SINGLE trial realized from this template.

  • "location" takes a list of location indices of the same length as "which", specifying the locations where the stimuli (as specified in "which") should be displayed. The location indices should be between 1 and 9, referring to locations arranged from left to right and from top to bottom in a 3x3 grid.

    Note

    The 3x3 grid is organized as follows:

    1 2 3

    4 5 6

    7 8 9

    For example, to place two stimuli on a single trial so that one is at the top left corner and the other at the bottom right corner, one can specify "stimuli": [{"which": ["s1", "s2"], "location": [1, 9]}].

  • "randomize_location" - a boolean value of either true or false (default, optional), specifying whether the stimuli are randomly shuffled among the locations defined in "location" on each realized trial.

Stimulus patterns

In some studies, researchers may want to randomize the order in which stimuli are presented to participants to reduce order effects. In other scenarios, researchers may be interested in studying the effect of a particular stimulus order, where randomization is not needed. Manipulating the "stimulus_pattern" property of a trial template can help researchers achieve these goals, as well as many other variations.

The value of stimulus_pattern is a dictionary with three or more properties:

{"order": SOME_ORDER, "attribute": null or SOME_ATTRIBUTE, "pre_shuffle": true or false, ...}
“order”:
  • one of “fixed”, “random”, “pseudorandom”, “ascending”, “descending”, and “alternate”. Defaults to “fixed”.
  • “fixed” presents stimuli in the same order as they are listed in the "stimuli" property of a trial template
  • “random” reorders the stimuli randomly
  • “pseudorandom” reorders the stimuli randomly, subject to the constraint that stimuli with SOME_ATTRIBUTE will appear in a row at least "min_N" times (defaults to 1), and at most "max_N" times (defaults to and is always at least equal to "min_N"). See the example below.
  • “ascending” presents the stimuli in the order that the value of SOME_ATTRIBUTE of the stimuli increases
  • “descending” presents the stimuli in the order that the value of SOME_ATTRIBUTE of the stimuli decreases
  • “alternate” presents the stimuli so that they alternate in SOME_ATTRIBUTE
  • “every” presents one stimulus with SOME_ATTRIBUTE after presenting N-1 stimuli without this attribute. An additional property N must also be specified (see below).
“attribute”:
  • some property name defined for the stimuli listed in the "stimuli" property of a trial template.
  • Defaults to null.
“pre_shuffle”:
  • whether or not to randomize the stimulus order as given in the "stimuli" property of a trial template BEFORE applying the specified pattern.
  • ignored if “order” is “fixed” or “random”.
  • in cases where “order” implies strict ordering (such as “ascending”), “pre_shuffle” only affects cases where attribute values are the same. See more explanation in Ascending and descending in some attribute.
“N”:
  • required when the order is “every”, meaning that for every N trials, present one stimulus with the attribute specified in “every” (that is, present one trial with the attribute of interest after every N-1 trials without the attribute).

Examples

Fixed and simple or pseudo-randomization

"stimulus_pattern": {"order": "fixed", "attribute": null}

Stimuli are presented in realized trials in the order they are listed in the "stimuli" property of a trial template. This is the default and can be omitted.

"stimulus_pattern": {"order": "random"}

Stimuli are randomized per participant.

"stimulus_pattern": {"order": "pseudorandom", "attribute": "ungrammatical", "min_N": 2, "max_N": 4}

Stimuli are randomized per participant, so that ungrammatical stimuli (those with a custom attribute “ungrammatical” set to true) are presented at least twice in a row, but no more than 4 times consecutively. .

Ascending and descending in some attribute

Now, given the following definition of stimuli:

"stimuli": {
  "singleShort1": {"type": "image",
                   "content": "new-ambi1-a-1.jpg",
                   "length": 1
                  },
  "singleShort2": {"type": "image",
                   "content": "new-ambi1-a-2.jpg",
                   "length": 2
                  },
  "singleShort3": {"type": "image",
                   "content": "new-ambi1-a-3.jpg",
                   "length": 3
                  },
  "singleShort4": {"type": "image",
                   "content": "new-ambi1-a-4.jpg",
                   "length": 4
                  },
  "singleShort5": {"type": "image",
                   "content": "new-ambi1-a-5.jpg",
                   "length": 5
                  }
}

The following stimulus pattern

"stimulus_pattern": {"order": "ascending", "attribute": "length" }

presents the stimuli in the order that the value of "length" increases. Intuitively, changing "ascending" to "descending" has the exact opposite effect.

Note that if two or more stimuli have the same value for "length", then their relative order can be randomized setting "pre_shuffle" to true. If the values of the target attribute are all unique, then "pre_shuffle" has no effect.

Alternate in some attribute

Similarly, given the following definition of stimuli:

"stimuli": {
  "singleShort1": {"type": "image",
                   "content": "new-ambi1-a-1.jpg",
                   "difficulty": "easy"
                  },
  "singleShort2": {"type": "image",
                   "content": "new-ambi1-a-2.jpg",
                   "difficulty": "easy"
                  },
  "singleShort3": {"type": "image",
                   "content": "new-ambi1-a-3.jpg",
                   "difficulty": "easy"
                  },
  "singleShort4": {"type": "image",
                   "content": "new-ambi1-a-4.jpg",
                   "difficulty": "hard"
                  },
  "singleShort5": {"type": "image",
                   "content": "new-ambi1-a-5.jpg",
                   "difficulty": "hard"
                  }
}

The following stimulus pattern

"stimulus_pattern": {"order": "alternate", "attribute": "difficulty" }

presents the stimuli so that “easy” and “hard” alternate.

Effect of pre_shuffle

Using the stimulus definition in the above section, it is straightforward to understand the effect of "pre_shuffle" by comparing the following two stimulus patterns.

"stimulus_pattern": {"order": "alternate", "attribute": "difficulty", "pre_shuffle": false}
"stimulus_pattern": {"order": "alternate", "attribute": "difficulty", "pre_shuffle": true}

With pre_shuffle set to false (which can be omitted since it’s the default), the stimuli are always going to be presented in the order “singleShort1”, “singleShort4”, “singleShort2”, “singleShort5”, “singleShort3”.

However, when it is set to true, although “easy” and “hard” still alternate, it is possible to obtain a sequence such as “singleShort2”, “singleShort5”, “singleShort1”, “singleShort4”, “singleShort3”. That is, the order in which stimulus names are listed in the trial template has no effect on the generated alternating order anymore.