Crash Course

FindingFive allows you to quickly and easily design experiments for deployment on the web. In this crash course, we will take a detailed tour through some of FindingFive’s features by designing a simple memory study: a modification of Tulving (1975). We highly recommend you follow through this example carefully before creating your own experiment.

Getting Started

Let’s begin by clicking “RESEARCH” on the top right corner to go to the researcher’s view, where you’ll see your list of current and past studies. Clicking the “Create a new study” button at the top of the screen will take you to your new study’s profile:

_images/gettingStarted.png

Give your study a name and a short description, which will be used by participants to decide whether they want to participate in your experiment. Provide an estimate of the study duration and the compensation, then upload a copy of your IRB-approved consent form (you cannot publish a study unless a consent form is uploaded).

Building Your Study

The designing of your study takes place under the “Materials” tab. On the left-hand side, you can add stimuli and responses, which make up the basic building blocks of a study.

_images/stimuli-responses.png

On the right-hand side, you can use the code editor to create trial_templates, which combine stimuli and responses to form trials, and the procedure, which defines how trials are organized in a study.

_images/code-editor.png

Tulving (1975) in plain words

In our sample experiment, participants will be presented with word pairs, then asked to recognize and recall the word lists after a delay. The words and instructions are all stimuli which will be organized within trial_templates. These training-specific trial_templates will be organized into a training block in our procedure. There will also be a test block in our procedure. This block will contain responses, where the participant is asked if the presented pair of words is correct (a 2AFC recognition memory test), and other responses will require the participant to recall the target word and type it into a text box. These test trials will be organized into their own trial_templates.

Tulving (1975) in FindingFive Study Specification

Therefore, to build the study specification for our study, we must:

  • define stimuli which will include our training and test instructions to the participant as well the word-pairs the participant must learn during our training phase and the test questions
  • define responses for the test phase of the experiment, both 2AFC and free recall
  • define trial_templates for each set of trials in our study (instruction trials, training trials, 2AFC test trials, and free recall test trials)
  • define the procedure, which will consist of a training block then a testing block

Setting up the stimuli

We’ll build our experiment from the ground up, starting with the stimuli. Stimuli are items that are presented to the participant, including text instructions, audio clips, and images. For our experiment, we will need to define stimuli for our training and test instructions, each of our word-pairs and test questions, and a concluding statement to the participant at the end of the study.

Add the first two stimuli manually

To create our training instructions, let’s define a new stimulus called TrainingInst. This stimulus will only contain text, so we will set the type property to “text”. We will define the content property with our experiment instructions. To add this stimulus to our study, click on the + button under Stimulus:

_images/manual-add-stimulus-response.png

A new window will pop up, asking for the name of the stimulus, and the definition of the stimulus:

_images/add-stimulus-dialog.png

Type “TrainingInst” into the name field, and then paste the following code into the definition field, and click Save:

{
    "type": "text",
    "content": "In this first part of the experiment, you will see pairs of words presented on the screen. Your task is to <b>remember the capitalized words in each pair</b>. However, you should pay attention to both words and notice the relationship between them, since this relationship will help you remember the capitalized words."
}

Notice that we can use HTML tags to alter the appearance of our text stimulus.

As an exercise, add a second stimulus named “TestInst”, which has the following definition:

{
    "type": "text",
    "content": "In this portion of the study, you will be prompted with one of the two words in each word pair you learned. Your job is to indicate the correct match to the presented word. In some test trials, you will type the word into a text box. In others, you will indicate whether the presented match is the correct."
}

Congrats! You have successfully added the first two stimuli of your study. They should now be displayed under the Stimulus section.

Add more stimuli via CSV batch uploading

Next, we’ll create stimulus definitions for each of our words to be presented in the training phase of the experiment. Doing so individually can be quite tedious. Fortunately, FindingFive supports batch uploading stimuli via a CSV file, which makes life much easier when we are dealing with potentially hundreds of stimuli.

To get started, hover your mouse over the upload button under Stimuli, and a brief explanation on how to format stimulus definitions in CSV will pop up:

_images/batch-uploading-stimuli.png

For our current example, Tulving (1975), use your favorite spreadsheet software (Excel, Google Sheets, or other great programs) to create a CSV file that looks like this:

Warning

Please be sure to save the file as CSV - comma seperated values, instead of a proprietary format like xls.

name type content alignment
ConCue1 text player center
ConTarget1 text BALL center
ConCue2 text smooth center
ConTarget2 text HARD center
ConCue3 text furniture center
ConTarget3 text CHAIR center
ConCue4 text fire center
ConTarget4 text COLD center
InconCue1 text fruit center
InconTarget1 text FLOWER center
InconCue2 text moth center
InconTarget2 text FOOD center
InconCue3 text cheese center
InconTarget3 text GREEN center
InconCue4 text cave center
InconTarget4 text WET center
ConTest1y text Was <b>ball</b> the word paired with <b>player</b>? center
ConTest1n text Was <b>tennis</b> the word paired with <b>player</b>? center
ConTest2y text Was <b>hard</b> the word paired with <b>smooth</b>? center
ConTest2n text Was <b>soft</b> the word paired with <b>smooth</b>? center
InconTest1y text Was <b>flower</b> the word paired with <b>fruit</b>? center
InconTest1n text Was <b>bloom</b> the word paired with <b>fruit</b>? center
InconTest2y text Was <b>food</b> the word paired with <b>moth</b>? center
InconTest2n text Was <b>eat</b> the word paired with <b>moth</b>? center
ConTest3 text What was the word paired with <b>furniture</b>? center
ConTest4 text What was the word paired with <b>fire</b>? center
InconTest3 text What was the word paired with <b>cheese</b>? center
InconTest4 text What was the word paired with <b>cave</b>? center
BreakInst text Please take a short break center

You’ll notice that the first 16 stimuli are the training stimuli, followed by the stimuli to display for the test trials. For the test trials, the word stimuli is somewhat a misnomer - these are used to display the questions participants will answer. The last stimulus will be used to prompt the participant to take a break before the test portion of the experiment.

Note

For those of you who are hardcore programmers and want to create CSV files manually through a text editor, you are welcome to do so. Please be careful, however, about properly escaping commas and double quotes. FindingFive is capable of recognizing the standard CSV escaping methods (for example, using double double quotes to escape double quotes).

Let’s upload this CSV and create those stimuli all at once! Click the batch upload button, and select the CSV file on your computer. On a Linux computer, you may see something like this:

_images/batch-uploading-select-file.png

For this particular example, these stimuli should be imported in just a second. Depending on the speed of your computer and the Internet connection, it might take a couple of minutes if you are uploading hundreds of stimuli at once. Please be patient and do not refresh the page unless you are absolutely sure something unexpected has happened.

Setting up the responses

Responses define the actions that participants can take in response to stimuli. They also automatically specify what types of participant data will be recorded. In our sample experiment, we will conduct both a recognition (2AFC) memory test and a recall (fill-in-the-blank) memory test. We will set up the actions participants can take for these test trials with the responses property.

First, we will set up the recognition test for two of our semantically congruous word pairs and two of our incongruous word pairs. Since we are asking participants to make a choice between two options (correct pairing, or incorrect pairing), the type of response we will collect is a “choice” response. We can define the possible choices for the participant (YES or NO) using the choices property. Next, we will map these choices to keys on the participant’s keyboard using the key_mapping property and we will tell FindingFive the correct answer using the target property. Since the correct answer can either be YES or NO, we’ll make responses for each of these possibilities.

Adding responses to a study is very similar to adding a stimulus. Click the + button under Responses, and add the following two choice responses:

The first response is named “TestResponseTargetY”, with the definition:

{
    "type": "choice",
    "choices": ["YES", "NO"],
    "key_mapping": ["F", "J"],
    "target": "YES"
}

The second response is named “TestResponseTargetN”, with the definition:

{
    "type": "choice",
    "choices": ["YES", "NO"],
    "key_mapping": ["F", "J"],
    "target": "NO"
}

Next, we will set up our recall test. The type of response we will collect is a “text” response, so there’s no need to define choices or key_mapping, but we can set a character limit for participant’s responses using max_char. We will name the response “FreeRecall”, and use the following definition:

{
    "type": "text",
    "max_char": 10
}

We are all set with the responses.

At this point, your left-hand side should look likes this:

_images/finished-stimuli-responses.png

Setting up the trial templates

Now that we have the stimuli and responses defined, we can set up the trial_templates which define the organizational logic of our trials. We will define trial_templates for each set of trials in our study (instruction trials, randomized training trials, randomized 2AFC test trials, and randomized free recall test trials).

Instruction trials

First, let’s set up our instruction trials. We’ll need an instruction trial (“T1”) at the very beginning of the experiment to explain the training trials to our participants. Defining the type as “instruction” allows us to simply present text to the participant for a time in seconds specified by duration. Participants must wait until the specified duration has elapsed before continuing the experiment (by clicking a button labeled “continue” that appears after the specified duration).

"T1": {
    "type": "instruction",
    "stimuli": ["TrainingInst"],
    "duration": 10
}, // you should keep the comma here if more trial template definitions follow this one

We’ll define similar trial types for presenting the break instructions, test instructions, and a final goodbye message to the participants.

"PreTestBreak": {
    "type": "instruction",
    "stimuli": ["BreakInst"],
    "duration": 30
}, // you should keep the comma here if more trial template definitions follow this one

"T2": {
    "type": "instruction",
    "stimuli": ["TestInst"],
    "duration": 10
}, // you should keep the comma here if more trial template definitions follow this one

You can copy and paste the code above to the code editor - please make sure that “Trial Templates” is ticked before doing so:

_images/trial-templates-ticked.png

Training trials

For the training trials, we create a template named “TrainingTrials”. Since we need to present 8 word-pair stimuli in total, the trial template is set up to generate 8 unique trials, each of which presents two stimuli.

"TrainingTrials": {
   "type": "basic",
   "stimuli": [["ConCue1","ConTarget1"],["ConTarget2", "ConCue2"],["ConCue3", "ConTarget3"], ["ConTarget4", "ConCue4"], ["InconCue1","InconTarget1"],["InconTarget2", "InconCue2"],["InconCue3","InconTarget3"], ["InconTarget4", "InconCue4"]],
   "stimulus_pattern": {"order": "random"},
   "duration": 3,
   "delay":1,
   "responses": []
}, // you should keep the comma here if more trial template definitions follow this one

Note

To include two stimuli on each trial we create a list for each of our trials within the larger stimuli list.

Here, a simple per-participant randomization of the 8 trials is achieved by specifying the stimulus_pattern property. As in Tulving (1975), these stimuli will remain on the screen for 3 seconds each, achieved by the duration property. The delay property handles time between trials - here we set it to 1 to indicate a 1-second inter-trial-interval. We won’t collect any data from the participant in these training trials, so our responses property is set to an empty list [].

Test trials

Last, we create the trial templates for test trials. First, let’s handle the 2AFC test trials by making a trial template for trials where the correct response is YES and a second trial template for trials where the correct response is NO. FindingFive has a built-in trial template for alternative forced choice tasks (“AFC”) that performs additional sanity checks for this type of task, so we’ll use that here. Like the training trials, the test will present our test trials in a random order.

"TestTrials2AltY": {
  "type": "AFC",
  "stimuli": ["ConTest1y", "ConTest2y", "InconTest1y","InconTest2y"],
  "stimulus_pattern": {"order": "random"},
  "responses": ["TestResponseTargetY"]
}, // you should keep the comma here if more trial template definitions follow this one

"TestTrials2AltN": {
  "type": "AFC",
  "stimuli": ["ConTest1n", "ConTest2n", "InconTest1n","InconTest2n"],
  "stimulus_pattern": {"order": "random"},
  "responses": ["TestResponseTargetN"]
}, // you should keep the comma here if more trial template definitions follow this one

With our 2AFC test trials set, we just need to define the trial_template for the free recall test trials.

"TestTrialsFree": {
  "type": "basic",
  "stimuli": ["ConTest3", "ConTest4", "InconTest3","InconTest4"],
  "stimulus_pattern": {"order": "random"},
  "responses": ["FreeRecall"]
} // this will be last trial template definition, so there is no comma

At this point, your trial templates code should look like this:

_images/finished-trial-templates.png

Setting up the procedure

Now it’s time to put everything together into our procedure. We define the procedure to have two blocks: a training block and a testing block.

First things first, let’s tick the procedure option to switch the code editor to work on the procedure:

_images/procedure-ticked.png

In our training block, we will start by presenting our training instructions, housed in template T1. We’ll end the training block with our break instructions. These introductory and closing trials will be defined using the cover_trials and end_trials properties.

Then we need to tell the study specification which of the trial_templates houses the main trials for this block. In our study, the training trials live in the TrainingTrials template. Since we’re only presenting one main trial template in this block, we don’t need to randomize it, and we set the repeat property to 1. Our test block will look similar to our training block, except we want to randomize the 3 different trial_templates we specified for test trials. After defining both blocks, we set the order in which these blocks should appear using the block_sequence property: training first, then testing.

{
    "type": "blocking",
    "blocks": {
      "TrainingBlock": {
        "trial_templates": ["TrainingTrials"],
        "pattern": {"order": "fixed", "repeat": 1},
        "cover_trials": ["T1"],
        "end_trials": ["PreTestBreak"]
      },
      "TestingBlock": {
        "trial_templates": ["TestTrials2AltY", "TestTrials2AltN", "TestTrialsFree"],
        "pattern": {"order": "random", "repeat": 1},
        "cover_trials": ["T2"]
      }
    },
    "block_sequence": ["TrainingBlock","TestingBlock"]
}

Making your study ready to run

When you are working on the materials of a study, in the upper-right corner of the web page you’ll notice a button that switches between a red caution icon and an orange preview icon.

Syntax errors in study specification

As you type in the specification editor, some basic syntax errors, such as missing commas, mismatched parentheses, and unrecognized characters, will be checked in real time. If an error is found, the preview button turns into a red button with a caution icon inside: it means that there is some serious syntactic errors in your trial templates and/or procedure. Most commonly, you probably added a comma where you shouldn’t, or left out a comma where you shouldn’t have.

_images/basicError.png

These basic errors must be fixed before more complex errors, which often involve the logic of a study, are detected. These addition problems are investigated after you click on the orange preview icon, as they are checked by the more powerful study parser on our server. These more complex errors, if found, are reported to you in a language as plain as possible:

_images/target-missing.png

If no additional errors are found, you will be taken to preview the study directly. Be sure to make sure that your pop-up blocker won’t accidentally prevent the preview window from opening.

Upload necessary resource files for stimuli

Our study parser automatically infers from your stimulus definitions (such as image, audio, or video stimuli). If a corresponding resource file is missing, you will receive a prompt when your mouse hovers over the stimulus (you may need to refresh your browser to see this prompt):

_images/imageMissing.png

To upload a file, simply click on the stimulus and then the “upload file” button. If the upload was successful, the image will appear when your mouse hovers over the stimulus:

_images/allgood.png

Warning

There is currently a 2MB limit on the size of any stimulus file.

Note

In many cases, a researcher may create two studies are minor variations of each other. It would be a mind-numbing process to upload the same stimulus files all over again. Fortunately, stimulus files uploaded to one study can be directly accessed by filename in another within the same account. That is, two file-based stimuli sharing the same “content” property actually point to the same stimulus file uploaded to your account.

Starting a session

Once you’ve confirmed that your study is ready to run, it’s time to make it available to participants by creating a session. To do so, simply click the new session button under the sessions tab. Here you can specify how many participants you desire, how long you want your experiment to be available, and when you want it to be posted. Additionally, you can specify eligibility criteria (age, completion rate, previous participation) and access restriction (require codes to participate).

_images/newSession.png

If you have ongoing or scheduled sessions, these will appear under open and future sessions and will be visible to participants who meet your specified eligibility requirements. You’ll also be able to see how many participants have completed your study and how many are participating right now!

_images/openSession.png

Note

Specifically designed for the current beta, you can send emails to invite other participants to open sessions. All invitees will receive an invitation code for signing up as a participant.

Looking at the data

After a session has been completed, it will appear under completed sessions and you can download the data (.csv) by selecting the desired session and clicking download data. Note that you can only download the data once the session has ended (either by ending the session early or waiting until the requested number of participants have completed your study).

_images/downloadData.png

Wrap up

Congratulations on making it to the end of the crash course! Please do not hesitate to contact us at researcher.help@findingfive.com for any questions you may encounter while using FindingFive. We look forward to hearing from you.