SweetPea With Firebase¶

Here we use autora to upload counterbalanced trial sequences to an experiment hosted with Firebase. You can find a tutorial on how to set up a firebase experiment, that is configured to work with AutoRA here: Firebase Tutorial

In [ ]:

!pip install sweetpea
!pip install "autora[experiment-runner-experimentation-manager-firebase]"

!pip install sweetpea !pip install "autora[experiment-runner-experimentation-manager-firebase]"

First we are going to create a counterbalanced stroop trial sequence (additional tutorials on how to use SweetPea can be found here: Tutorials)

Create The Trial Sequences¶

In [ ]:

# IMPORTS:
from sweetpea import (
    Factor, DerivedLevel, WithinTrial,
    CrossBlock, synthesize_trials,
    CMSGen, MinimumTrials
)

# DEFINE COLOR AND WORD FACTORS
color      = Factor("color",  ["red", "blue", "green", "brown"])
word       = Factor("word", ["red", "blue", "green", "brown"])

# DEFINE CONGRUENCY FACTOR

def congruent(color, word):
    return color == word

def incongruent(color, word):
    return not congruent(color, word)


conLevel = DerivedLevel("con", WithinTrial(congruent,   [color, word]))
incLevel = DerivedLevel("inc", WithinTrial(incongruent,   [color, word]))

congruency = Factor("congruency", [
    conLevel,
    incLevel
])

# DEFINE EXPERIMENT
design       = [color, word, congruency]
crossing     = [congruency]
block        = CrossBlock(design, crossing, [MinimumTrials(48)])

# SYNTHESIZE 5 TRIAL SEQUENCES

experiments  = synthesize_trials(block, 5, CMSGen)

# IMPORTS: from sweetpea import ( Factor, DerivedLevel, WithinTrial, CrossBlock, synthesize_trials, CMSGen, MinimumTrials ) # DEFINE COLOR AND WORD FACTORS color = Factor("color", ["red", "blue", "green", "brown"]) word = Factor("word", ["red", "blue", "green", "brown"]) # DEFINE CONGRUENCY FACTOR def congruent(color, word): return color == word def incongruent(color, word): return not congruent(color, word) conLevel = DerivedLevel("con", WithinTrial(congruent, [color, word])) incLevel = DerivedLevel("inc", WithinTrial(incongruent, [color, word])) congruency = Factor("congruency", [ conLevel, incLevel ]) # DEFINE EXPERIMENT design = [color, word, congruency] crossing = [congruency] block = CrossBlock(design, crossing, [MinimumTrials(48)]) # SYNTHESIZE 5 TRIAL SEQUENCES experiments = synthesize_trials(block, 5, CMSGen)

Let's see, what the experiment object is:

In [ ]:

print('All sequences:')
print(experiments)
print('One sequence:')
print(experiments[0])

print('All sequences:') print(experiments) print('One sequence:') print(experiments[0])

For the firebase experiments, it is more convenient to have lists of trials instead of feature list:

In [ ]:

firebase_formatted = []
for el in experiments:
    trials = []
    for color, word, congruency in zip(el["color"], el["word"], el["congruency"]):
        trial = {"color": color, "word": word, "congruency": congruency}
        trials.append(trial)
    firebase_formatted.append(trials)
print(firebase_formatted[0])

firebase_formatted = [] for el in experiments: trials = [] for color, word, congruency in zip(el["color"], el["word"], el["congruency"]): trial = {"color": color, "word": word, "congruency": congruency} trials.append(trial) firebase_formatted.append(trials) print(firebase_formatted[0])

Upload To Firebase¶

Let's import the send_condition from the experimentation_manager and see what arguments are expected:

In [ ]:

from autora.experiment_runner.experimentation_manager.firebase import send_conditions
help(send_conditions)

from autora.experiment_runner.experimentation_manager.firebase import send_conditions help(send_conditions)

collection_name: is a name that we choose (here we choose autora) conditions: the conditions (in this case the trial sequences) firebase_credentials: a dict with the credentials for firebase (found here: Firebase under project-settings -> service accounts -> generate key

In [ ]:

collection_name = 'autora'
conditions = firebase_formatted
firebase_credentials = {
  "type": "",
  "project_id": "",
  "private_key_id": "",
  "private_key": "",
  "client_email": "",
  "client_id": "",
  "auth_uri": "",
  "token_uri": "",
  "auth_provider_x509_cert_url": "",
  "client_x509_cert_url": ""
}

collection_name = 'autora' conditions = firebase_formatted firebase_credentials = { "type": "", "project_id": "", "private_key_id": "", "private_key": "", "client_email": "", "client_id": "", "auth_uri": "", "token_uri": "", "auth_provider_x509_cert_url": "", "client_x509_cert_url": "" }

In [ ]:

send_conditions(collection_name=collection_name, conditions=conditions, firebase_credentials=firebase_credentials)

send_conditions(collection_name=collection_name, conditions=conditions, firebase_credentials=firebase_credentials)

Check out your Firebase in the browser. The conditions were added.

Check Firebase Status¶

To check the status of the experiment you can use the function check_status (this is helpful to build a closed loop, where a autora-recruitment-manager starts and pauses the recruitment based on the status of the experiment, while the autora-experimentation-manager retrieves the observations and sends new conditions from the autora-experimentalist when data-collection is finished.

In [ ]:

from autora.experiment_runner.experimentation_manager.firebase import check_firebase_status

status = check_firebase_status(collection_name=collection_name, firebase_credentials=firebase_credentials, time_out=100)
print(status)

from autora.experiment_runner.experimentation_manager.firebase import check_firebase_status status = check_firebase_status(collection_name=collection_name, firebase_credentials=firebase_credentials, time_out=100) print(status)

This gives you a string (unavailable, available or finished):

• If all conditions are in use but not finished the function returns unavailable (the time_out governs how long a participant is allowed to take part in the experiment till the condition gets freed for the next user)
• If there are available spots (meaning there are conditions not started yet or there are conditions that have been timed out), the function returns available
• If all conditions are used and the data for these conditions has been collected, the function returns finished

Get Observations¶

To download the observations from the Firestore databse, we can use the get_observations function:

In [ ]:

from autora.experiment_runner.experimentation_manager.firebase import get_observations

observations = get_observations(collection_name=collection_name, firebase_credentials=firebase_credentials)
print(observations)

from autora.experiment_runner.experimentation_manager.firebase import get_observations observations = get_observations(collection_name=collection_name, firebase_credentials=firebase_credentials) print(observations)