FAQs for researchers

Purpose of FindingFive

What can I do at FindingFive?

FindingFive is a complete, standalone, one-stop platform for running online behavioral studies. You can quickly set up an online behavioral study without any prior experience in designing a web app. You have customized control over participant recruitment that makes sense. To make things even better, as soon as a study session is finished, participant data are organized in a sensible format for immediate download.

You can also use FindingFive with Amazon Mechanical Turk. You can still create and manage your studies entirely on FindingFive, but the studies are published on Amazon Mechanical Turk instead of FindingFive. Check out Launching a study on Mechanical Turk for details on how to set up your MTurk account on FindingFive.

Why should I use FindingFive instead of Amazon Mechanical Turk?

FindingFive streamlines the research process by allowing researchers to create professional-looking behavioral studies, control the recruitment of participants, run studies and collect data directly on FindingFive. These features are designed specifically for researchers and not available or not easily achievable with Mechanical Turk alone.

To make it easier for researchers to “make” online studies, FindingFive introduces a new grammar, named Study Specification Grammar (SSG), which does not involve much programming at all. Instead, it spells out both the content and logic of a study in intuitive terms that are already familiar to behavioral researchers. We have written a lot of documentation to help you get started using the SSG - check out the Crash Course and the Study Specification Grammar Reference.

Our ultimate goal is to build a high-quality participant community. FindingFive aims to build a community of participants who are genuinely interested in behavioral research. These participants will be given full privacy protection, and at the same time, are encouraged to interact with researchers regarding the studies that they have contributed to. We feel it is important to provide the feedback to study participants on how their data mattered, and led to scientific discoveries.

As we are working towards this goal, it is possible to use FindingFive with Amazon Mechanical Turk! You can reap the benefits of

  • Creating your studies using the FindingFive SSG
  • Using recruitment controls to perform tasks such as preventing Turk workers from participating in the same study twice
  • Hosting your study on FindingFive for free - All studies on FindingFive are also hosted on FindingFive, completely freeing researchers of the mundane tasks of setting up a server and obtaining SSL licenses that would be required to run an external study on Mechanical Turk.

Check out Launching a study on Mechanical Turk for details on how to set up your MTurk account on FindingFive.

I am a JavaScript expert and already know other platforms very well. Why should I invest time in using FindingFive?

Because coding up a complete experiment from scratch is a significant time commitment, and taking care of the nitty-gritty details can be annoying. For simple studies that do not require a lot of customizations, FindingFive can save you a lot of time! Check out the Crash Course to see how straightforward it is to create a study.

Logistics

I am a student running studies on behalf of my lab. How should I sign up?

Regardless of whether you are a graduate student dedicated to research or an undergraduate research assistant, we encourage you to sign up as a researcher and as yourself on FindingFive. To show that you are conducting research on behalf of your lab (or research group), go to your profile and add the corresponding affiliation information. When your study is published, participants will see it as coming from your lab rather than you personally.

How do I create a study?

Studies are created by specifying the structure of your experiment using our Study Specification Grammar. The grammar allows for various types of randomiziation, blocking control, presentation of different types of stimuli, and collection of many types of participant responses. In the very near future, the grammar will also support interactive presentation of stimuli, such as the mechanism of a self-paced reading study. Check out the Crash Course in research guide, and keep the Study Specification Grammar Reference open for reference.

How do I pay my participants?

For the current closed beta, you will need to pay participants in person.

In the future, we will be able to process common methods of payments and handle the payment to participants for you.

How much commission do you charge?

For the current closed beta, zero.

Wow. Is FindingFive entirely free to use?

For now, yes!

However, to make sure that FindingFive remains a sustainable effort that benefits the community of behavioral researchers in the long term, we may have to bug you for donations periodically. We may also charge some (reasonable) fees for extra-fancy capabilities or storage in the future.

Study design

What types of stimuli can I use in my study?

We currently allow you to use Stimuli in the form of text, images, audio, and video files. We make it simple for you to upload your stimulus files into FindingFive’s system (with a temporary restriction of 2MB or smaller). In the future, we plan to lift this size restriction as much as possible.

What types of participant data are recorded?

The types of participant data recorded during a study session are dependent on the types of Responses used in a study. For example, if a trial features a two-alternative forced choice response, it automatically records a participant’s choice, the reaction time, and whether it is a correct response or not (if the correct answer is given). In general, recording of participant data is automatically handled by our platform for researchers.

If my participants aren’t in the lab, how do I make sure they’re paying attention during my study?

The study specification grammar offers a built-in catch trial feature. This allows you to set up your own test to make sure that participants are paying attention and on-track with your study. For example, let’s say your study asks participants to listen to lists of words, presented as audio stimuli. You could set up a catch trial every 5 trials, where participants are asked to type the word that they just heard on the previous trial. This allows you to check whether the participant correctly heard your audio stimulus, and whether their attention wavered at any point during the study. Read more about it in the Procedures documentation.

The study specification grammar isn’t sophisticated enough for my study. What can I do?

We are continuously improving the grammar to include more features. In the future, we also plan to:

  • allow researchers with programming skills to write their own JavaScript code
  • offer the service of creating custom studies for researchers with specialized requirements

However, at the present moment, we’d encourage researchers to find creative ways to achieve their goals within the current limits of the grammar. We’d also appreciate it if you could email us at feedback@findingfive.com and tell us how our grammar falls short for your study, so that we can improve it.

Privacy concerns and IRB issues

To what extent does FindingFive remove identifiable personal information?

Researchers will not be able to see the FindingFive ID, the name, and any protected personal information of a participant.

Once a study session is finished, you will be able to download participant data. Each participant will be identified by a special ID, non-reversably generated from their real ID and your researcher ID. As a result, each participant will have a persistent ID in the studies of the same researcher, but that ID differs between researchers, and does not exist in our database at all.

What if my home institution requires the collection of demographic information?

Some non-identifiable demographic information can be requested at the time of data download. We encourage you to negoitate with your IRB on this issue, however. In our experience, most IRBs are fine with collecting data from anonymous participants online.

Will the IRB at my institution give approval for running studies on FindingFive?

Does your IRB approve studies conducted on online platforms in general? If so, there shouldn’t be any problems obtaining approvals for FindingFive.

In fact, we adopt more research-friendly policies than possibly all alternatives out there: we don’t own your data (the public does if your study is funded by a government agency), and we protect participant identies for you through both encryption and obscurification.

If there is any specific concern, we’d be happy to work with you and your IRB. Feel free to contact us at researcher.help@findingfive.com for this purpose.

Misc

Why do I have a participant page as well?

We encourage everyone, including researchers, to participate in behavioral studies. You can be the participant of other researchers’ studies and help contribute to our high-quality participant pool!