Frequently Asked Questions for Researchers
Part 1. The Basics
What can I do at FindingFive?
FindingFive is a complete, standalone, one-stop platform for running online behavioral studies. You can quickly set up an online behavioral study without any prior experience in designing a web app. You have customized control over participant recruitment that makes sense. To make things even better, as soon as a study session is finished, participant data are organized in a sensible format for immediate download.
You can also use FindingFive with Amazon Mechanical Turk. You can still create and manage your studies entirely on FindingFive, but the studies are launched on Amazon Mechanical Turk instead of FindingFive. Check out this Launching Your Studies on Mechanical Turk tutorial where we walk you step-by-step through setting up your MTurk account on FindingFive.
Why should I use FindingFive instead of Amazon Mechanical Turk?
FindingFive streamlines the research process by allowing researchers to create professional-looking web-based studies, control the recruitment of participants, run studies and collect data directly on FindingFive. These features are designed specifically for researchers and not available or not easily achievable with Mechanical Turk alone.
To make it easier for researchers to "make" online studies, FindingFive uses a new grammar, named FindingFive Study Grammar, which does not involve much programming at all. Instead, it spells out both the content and logic of a study in intuitive terms that are already familiar to you. We have written a lot of documentation to help you get started using the Study Grammar - check out our comprehensive Study Grammar Documentation.
I am a JavaScript expert and already know other platforms very well. Why should I invest time in using FindingFive?
Because coding up a complete experiment from scratch is a significant time commitment, and taking care of the nitty-gritty details such as data encryption can be burdensome. For simple studies that do not require a lot of customizations, FindingFive can save you a lot of time! Check out the Crash Course to see how straightforward it is to create a study.
I am a student running studies on behalf of my lab. How should I sign up?
Regardless of whether you are a graduate student dedicated to research or an undergraduate research assistant, we encourage you to sign up as a researcher and as **yourself** on FindingFive. To show that you are conducting research on behalf of your lab (or research group), follow this Create a "Lab Account" Tutorial to set up a separate account that centralizes study management. With a lab account, you and your lab mates can collaborate on experiments and run them in a way so that participants will see the studies as coming from your lab rather than you personally.
Will the IRB (or other equivalent subject protection committee) at my institution give approval for running studies on FindingFive?
Our service is used by many US-based and EU-based institutions. If there is any specific concern, we'd be happy to work with you and your institution. Feel free to contact us at [email protected] for this purpose.How do I pay my participants?
For studies launched directly on FindingFive, we support paying participants directly via PayPal or a digital gift card (including a digital debit card option). The choice between getting a PayPal transfer or a digital gift card is up to participants. See the Participation Reward and Direct Payment section for more details.
For studies launched on Mechanical Turk, payment is processed by Amazon. We will check your AWS account balance to make sure it has enough funds to cover the cost of your study session before launching it.
You are free to arrange payments on your own as well. Please provide sufficient instructions to your participants on how they will get paid if your study promises a cash reward.
Can I reward participants with course credits?
Of course! You can launch a study on FindingFive without any cash compensation. We are working on an exciting feature that streamlines the process of granting course credits to students, and have those records reported to your department's administrator for verification and management. In the meantime, you can specify what course credits will be granted to your participants in the description field of your study, and process completion information manually.
Part 2. Study design
How do I create a study?
Studies are created by specifying the structure of your experiment using our Study Grammar. The grammar allows for various types of randomiziation, blocking control, conditional branching, presentation of different types of stimuli, and collection of many types of participant responses such as choice, text, audio recording, self-paced reading, and mousetracking.
We also provide a small (but growing) collection of study templates so that you don't have to start from scratch when creating a new study.
What types of stimuli can I use in my study?
FindingFive support text, images, audio, and video stimuli. We have a dedicated "Files" page to make it simple to upload your stimulus files, either individually or as a ZIP file. Each upload must be less than 100MB in size, which should be sufficient for most cases. If you have particularly large files to use, please get in touch with us.What types of participant data are recorded?
The types of participant data recorded during a study session are dependent on the types of responses used in a study. For example, if a trial features a two-alternative forced choice response, it automatically records a participant's choice, the reaction time, and whether it is a correct response or not (if the correct answer is given). In general, recording of participant data is automatically handled by our platform for you.
If my participants aren't in the lab, how do I make sure they're paying attention during my study?
The study specification grammar offers a built-in "catch trials" feature. This allows you to set up your own test to make sure that participants are paying attention and on-track with your study. For example, let's say your study asks participants to listen to lists of words, presented as audio stimuli. You could set up a catch trial every 5 trials, where participants are asked to type the word that they just heard on the previous trial. This allows you to check whether the participant correctly heard your audio stimulus, and whether their attention wavered at any point during the study. You can click here to read more about it in the Study Grammar documentation.
The study specification grammar isn't sophisticated enough for my study. What can I do?
We are continuously improving the grammar to include more features. In the future, we also plan to:
- allow researchers with programming skills to write their own JavaScript code
- offer the service of creating custom studies for researchers with specialized requirements
However, at the present moment, we'd encourage researchers to find creative ways to achieve their goals within the current limits of the grammar. We'd also appreciate it if you could email us at [email protected] and tell us how our grammar falls short for your study, so that we can improve it.
Part 3. Researcher Subscriptions
What is a Researcher Subscription?
Researcher subscriptions are our main source of revenue as a nonprofit. There are two paid subscription plans: Standard and Pro. While it is possible to use FindingFive for free, you can support our operations by getting a subscription, starting at only $10 USD a month. A subscription gives you access to advanced features and also discounts on Session Fees. You can read more on our Pricing page for the US Server or the EU server.
How do I add a subscription?
Adding a subscription is now completely automated. Just sign into your FindingFive account and navigate to the "Expenses" page under "Research". There you can see the pricing for different plans, compare plans, and select a plan for purchase. You can also add "Plus 1s" to a subscription at a significant discount.
What are Plus 1s?
Plus 1s are significantly discounted "add-on subscriptions" that enable you to add colleagues or students to your subscription plan, so that they can enjoy the same plan benefits as you do. To make it even better, Plus 1s' usage quotas, including storage for files and participant entry exemptions (see below), are calculated separately so that you won't share those quotas with your Plus 1s. To add someone as a Plus 1, they must not already have an existing subscription. It is possible to add Plus 1s when you first purchase a subscription or to an existing subscription.
Please note that if you add a Plus 1 to an existing subscription late in the subscription cycle (e.g., near the end of your billing month), the cost will not be prorated. As a result, it is most cost-effective to add Plus 1s when you get a subscription or early in a billing cycle.
How do I pay for a subscription?
Once you've configured a plan, click "Check out" and add your debit or credit card information to pay. Your payment card information is securely processed by Square. Once the subscription is active, it'll renew automatically using the same card for each new billing cycle (monthly for monthly subscribers and annually for annual subscribers).How does upgrading or downgrading a subscription work?
If you upgrade your plan, the upgraded subscription will be effective immediately. If you are switching to a plan with fewer features (e.g., moving from Pro to Standard), your subscription will go into effect at the beginning of the next billing cycle.I am currently on a monthly subscription. How can I switch to an annual one? (or vice versa)
Just create a new annual subscription and activate it. The new annual subscription, if its plan is the same one as your current monthly one, will be scheduled to take effect at the moment your monthly subscription expires. The transition will be seamless. If it represents a plan upgrade, it will be effective immediately.What happens when I cancel my paid subscription?
When you cancel your paid subscription, your benefits, including those for your Plus 1s, will stay active until the end of the current billing cycle. After that, your subscription will expire, you and your Plus 1s will all lose the plan benefits, and your payment card will not be charged again.Can I get a refund on my subscription fee?
Unfortunately, we do not issue refunds for subscriptions under any circumstances.I want to set up subscriptions for my entire lab or research group, can I get a discount?
Yes! That's what Plus 1s are designed for - you can sign up for a primary subscription with as many Plus 1s as your lab's headcount! In addition, Standard and Pro plans offer new collaboration features that will make collaboration between lab members easier.Part 4. Participation Reward and Direct Payment
Where can I use the direct payment features?
The direct payment features are for sessions run on FindingFive only. Sessions launched on Mechanical Turk go through Amazon's payment system.How do I make a deposit?
You can make a deposit from the Expenses page under "Research". Depending the currency, you can either use PayPal, which accepts PayPal payments, credit and debit cards, or directly enter your debit or credit card info.How do I pay participants?
In short, we take care of most of the work. When you are ready to launch a study, use the calculator in the new session wizard to determine how much it will cost to run your study. Make an additional deposit if your account balance won't cover all the cost. Once your study successfully launches, we'll make sure that your participants get paid.Does FindingFive charge any fees?
Yes. FindingFive charges a flat fee of $0.75 per participant on the US server and €0.75 per participant on the EU server. These fees allow us to keep our payment system running, including by covering the fees that our payment processors charge us for accepting deposits from researchers and transferring payments to participants.When will my account balance be deducted for the cost of a session?
Upon launching the study, the funds will be immediately withheld from your account and held for participant payment. Participants who complete your study will automatically be approved for payment, which will be deposited in their reward balance accounts immediately.What if fewer participants complete my study than I requested?
If a study session ends or is terminated before the target participant count is reached, the unused portion of funds (including costs for participants and FindingFive fees) will be automatically returned to your account in 1-3 hours after the session terminates. Please note that compensation will be held for in-progress participants until they complete the study or abandon it. You may receive your returned funds in multiple installments as in-progress participants gradually abandon your study over time.Can I withdraw the leftover funds in my account balance?
Of course! If you wish to withdraw remaining funds from your FindingFive account, you may contact our researcher support team ([email protected]), and we will provide a refund to your original payment method within 3-5 business days. A refund fee of $0.50 (US server) or €0.50 (EU server) may be deducted from the refund.
In the case where the refund request is made more than 90 days after the original deposit, we can no longer issue a refund to the original payment method due to limitations imposed by our payment processors. In such cases, we will either make a direct bank transfer or mail a paper check (within the United States) to issue the refund. A refund fee of $1.00 (US server) or €1.00 (EU server) may be deducted from the refund.
Can I award bonuses to participants?
Yes. You can do that on the session management page.Can I *not* pay a participant by rejecting their work?
No, rejection of participant work on the basis of quality is not permitted on FindingFive. You may decide to exclude the participant's work from your data, but they still must be paid. Payment of participants will be automatic; you are not required to approve participant work for payment. You may terminate a research study at any time, but terminating a study does not mean that you will not have to pay participants who have submitted studies for payment.How do I verify that someone participated in my study if they do not complete it?
You should make sure that you request contact information from all participants in the new session wizard, as shown in the screenshot below:
What do I do if I'm having disputes with a participant?
If you are having disputes with a participant, efforts should first be made to resolve the issue through direct contact with the participant. If you are unable to resolve the issue, FindingFive may, but is not obligated to, assist in negotiating a mutually acceptable agreement. If FindingFive does intervene, and a mutually acceptable agreement cannot be reached, FindingFive reserves the right to make the final decision, which may require you to issue additional payment to participants.
Part 5. Session Fees
What is a session fee?
Starting October 1, 2021, we charge a session fee that is based on "participant entries", where a participant entry refers to a single completed participation event in a production session. This fee will apply to all studies, including course-credit studies, which were previously 100% free.Sessions fees are based on "participant entries", but what does this mean?
We are using the term "participant entries" mainly to highlight our policy that the same participant entering and completing in more than one session will be counted as multiple participant entries, instead of a single one.
Partial participation, including those who leave a study voluntarily, due to technical issues, or simply time out, does not count, so you will not be charged for participants whose data do not show up in the CSV file.
What is participant entry exemption?
For Standard or Pro subscribers, session fees will be waived for a certain number of participant entries per monthly subscription cycle (see details on our pricing page for the the US Server or the EU server). Those participant entries will not be counted when determining the balance due on a session fee bill. Participant entry exemptions are not available if you do not have a paid subscription.
How are participant entry exemptions tallied?
Participant entry exemptions will be applied per monthly subscription cycle. For monthly paid subscriptions, the billing cycle is the monthly subscription cycle. For annually paid subscriptions, the monthly subscription cycle is marked by the monthly anniversaries of your signup date..For example, if your sign up date is Jan 15, your participant entry exemptions will reset on Feb 15.How do monthly participant entry exemptions work in sessions that span multiple subscription cycles?
If your study session takes more than a month to finish recruiting, or it happens to span two monthly subscription cycles (even when it's only a few days), participant entry exemptions allocated across multiple subscription cycles will apply, based on the time when a participant starts the session.
For example, let's assume your monthly subscription cycle starts on the 15th of a month, and you have a subscription to the Standard Plan (which has 25 exemptions per cycle). A session is launched on the 10th and continues to be active till the 21th, spanning two monthly subscription cycles. During the 10th-14th, 13 participants started and completed the study; during the 15th-21th, 20 participants started and completed the study. Assuming you have not used any participant entry exemptions in these cycles, your final bill for this session is actually $0 - because the 13 participant entries are exempted under the first cycle's quota, and the other 20 are exempted under the second cycle's quota.
Will participant entry exemptions stay in effect even after I cancel my subscription?
The exemptions will still be in effect as long as your canceled subscription has a remaining term left (please see above for what happens when you cancel a subscription). In the case when a session remains active after your subscription expires, participant data collected after subscription expiration will be charged the session fee rate of the Lite plan.When are session fees charged?
Session fees will be charged only after researchers obtain the participant data. Specifically, a session fee bill will be generated upon the first download of data from a session. Subsequent downloads of the data from that session will not result in additional fees. You can choose to pay the session fee bill immediately or at a later date.How much will I be charged for session fees?
Session fees are calculated based on the subscription plan the researcher has at the time when participants join and start the session. If you have a Standard or Pro subscription, fees are only incurred once you've used up your participant entry exemptions for each billing month. If you don't have a paid subscription, you will always pay session fees. For detailed session fee rates, see the pricing page on the the US Server or the EU server.When do I need to pay my session fee bills?
Session fee bills are due within 15 days of downloading the data. You will receive a reminder to pay your bill before the 15 days are up. If you do not pay the bill in time, it will be considered overdue, and you will be unable to launch new sessions until you pay the overdue balance.
In addition, if your session fee balance reaches $100/€100, you must pay your balance before you can launch new sessions again. This is in parallel to the 15-day due date on individual bills.
How do FindingFive's rates compare to other platforms?
Even with the session fee, FindingFive is the most affordable platform that integrates study creation, recruitment, and payment all in the same place. As far as platform fees go, researchers running 100 participants a year can save almost $300 and large research labs can save over $3,000 by using FindingFive! For details on cost comparisons with other research platforms, see this blog post for details.
Part 6. Data Collection, Storage, and Identifiability
How secure is data collection on FindingFive?
FindingFive uses HTTPS, also known as HTTP over SSL, for *all* activities taking place on FindingFive's web platform. This means that all data, including participants' responses, are transmitted in encrypted forms over the Internet. The encryption methods we use are state-of-the-art and pose minimum risks in leaking participants' data.Where is participant data stored?
Participant data are stored on FindingFive's cloud servers, located in Newark, NJ, Chicago, IL, and Atlanta, GA (for all our US researchers), or Frankfurt, Germany and Paris, France (for all our EU researchers).How safe is the stored participant data?
Participant data are stored in password-protected databases, where the passwords provide a first layer of defense against data leak. Furthermore, these databases are set up behind a layer of Virtual Private Networks (VPN), meaning that only FindingFive's team can access such databases. As a result, participant data on FindingFive are very safe as they are completely closed off, as if non-existent, to the Internet at large.Is participant data stored in an encrypted form?
Optionally. For most studies, although the transmition of participant data *is* encrypted, once the data reach our servers, they are stored plainly on our servers. Researchers with a Premium or Laboratory membership can optionally create **encrypted studies**, where participant data not only are transmitted in an encrypted form, but also stay encrypted in our database. This practice, known as "encryption at rest", provides a strong layer of protection for participant data as they will be complete gibberish to an unauthorized party. Please consult your local laws and regulations if such encryption is necessary for your research, especially if your studies collect sensitive personally identifiable data.Is there any redundancy in the storage of participant data?
Of course! Data on FindingFive's US servers are backed up in three different locations, under the same level of security as described above. Data on the EU server are backed up in two different locations.To what extent does FindingFive remove identifiable personal information?
Once a study session is finished, you will be able to download participant data. Each participant will be identified by a special ID, non-reversably generated from their real FindingFive ID and your researcher FindingFive ID. As a result, each participant will have a persistent ID in the studies of the same researcher, but that ID differs between researchers, and does not exist in our database at all.
Under most circumstances, IRBs prohibit the collection of personally identifiable information in human subjects research. However, if a researcher has a unique situation that requires the collection of personally identifiable data, it is possible to collect this information via a response. In this case, researchers **must** use the Encrypted Study, rather than the Standard Study option. Researchers who collect personally identifiable information within their studies assume full responsibility for any losses or damages due to data security breaches. In the event of a data breach, researchers must promptly notify their IRBs (and other relevant authorities), as well as any affected parties. FindingFive reserves the right to require researchers to perform any action deemed necessary to remedy or mitigate the data breach or to prevent future data breaches.
Why can't a researcher download participant data from an active session?
Downloading participant data from an active session is a major privacy concern. As participants start working on your experiment, their names and emails may be made available to you (see the next question). If downloading participant data from an active session were allowed, then it would be theoretically possible to match up the identities of participants to their data by checking the data every time a new participant completes the study. Linking identities to data is explicitly prohibited by most IRBs, and is never appreciated by participants.
As a result, in order to minimize the chance of participant data being linked to their identities, we only allow participant data to be downloaded after a session is complete or canceled by a researcher.
Do I get participants' contact information?
It's up to you and what your IRB allows. You can choose to get contact information of all participants, only those who complete your study, or no one at all. Importantly, their contact information is provided to you separately from the participant data, so that you can never link the contact information to participants' data.