Separating Code and Parameters using Configuration Files

Courses

Building Robust Neuroscience Experiments with Python and PsychoPy

Python Fundamentals for Creating Experiments

Author

Dr. Ole Bialas

Designing an experiment involves making lots of different choices, for example: What different conditions are being tested? What stimuli are presented? How many trials does the experiment contain? Often, as we pilot our studies or run additional control experiments, we want to change some aspect of our experiment. If all these parameters are directly stored in the code, we have to modify our program for every little change. Thus, it is convenient to store our parameters in a configuration file and load this file into our program. This way, we can run endless variations of our experiment without ever touching the code - this makes our program much more reusable!

import json
from psychopy.event import waitKeys
from psychopy.visual import Rect, Circle, TextStim, Window

Section 1: Reading and Writing Files in Python

To store and load data, we have to interact with files. Python’s builtin open() function allows to do exactly that: it returns a file object that we can use to read from and write to a specific file. The open() function is usually used within a context manager that is declared using the keyword with. This context manager makes our code more readable and also makes sure that our file is closed after we are done.

Reference Table

Code	Description
`f = open("novel.txt", "w")`	`open` the file `"novel.txt"` in `"w"` (writing) mode
`f = write("Once upon a time ...")`	Write to the opened file
`f.close()`	Close the file again
`f = open("novel.txt", "r")`	`open` the file `"novel.txt"` in “r” (reading) mode
`text = f.read()`	Read the opened file
`with open("novel.txt", "w") as f:` `f.write("...and they lived happily ever after.")`	Write to a file within a context manager that opens and automatically closes the file again.

Exercises

Example: Use open() the file "paper.txt" in "w" mode, write() the string "Introduction" to that file and close it again.

f = open("paper.txt", "w")
f.write("introduction")
f.close()

Example: Now, open "paper.txt" in "r" mode, read() the file’s content and print it. Then, close the file again.

f = open("paper.txt", "r")
text = f.read()
f.close()
text

'introduction'

Exercise: open() a file called "data.txt" in "w" mode, write() the string "1,2,3,4,5,6" to that file and close() it again.

Solution

f = open("data.txt", "w")
f.write("1,2,3,4,5,6")
f.close()

Exercise: Now, open "data.txt" in "r" mode, read() the file’s content and print it. Then, close the file again.

Solution

f = open("data.txt", "r")
text = f.read()
f.close()
text

'1,2,3,4,5,6'

Example: open() the file "conditions.txt" in "w" mode with a context manager write the string "a,b,c" to that file.

with open("conditions.txt", "w") as f:
    f.write("a,b,c")

Exercise: open() the file "conditions.txt" in "r" mode with a context manager, then read() and print its content.

Solution

with open("conditions.txt", "r") as f:
    text = f.read()
text

'a,b,c'

Exercise: open() the file "trials.txt" in "w" mode with a context manager write the string "[a,b,c,b,c,a]" to that file.

Solution

with open("trials.txt", "w") as f:
    f.write("abcbca")

Exercise: open() the file "trials.txt" in "r" mode with a context manager, then read() its content, convert it to a list() and print it.

Solution

with open("trials.txt", "r") as f:
    text = f.read()
list(text)

['a', 'b', 'c', 'b', 'c', 'a']

Section 2: Reading and Writing Configuration Files

To work with configuration files, we need a data structure to represent them in Python. We are going to use dictionaries, which store data as a combination of keys and values. This is useful because we can give informative names to our parameters and make our code easier to read. When we save these dictionaries to our computer we are going to use the JSON (JavaScript Object Notation) format. JSON has a syntax that is highly similar to Python dictionaries, so they can easily be converted to one another. Also, JSON files can be opened in any text editor, so we can edit them by hand which is very convenient!

Code	Description
`x = {"n": 100, "p": 0.5}`	Define a dictionary `x` with the keys `"n"` and `"p"` and the values `100` and `0.5`
`with open("p.json", "w") as f:` `json.dump(x, f)`	Write the dictionary `x` to the file `"p.json"`
`with open("p.json", "w") as f:` `json.dump(x, f, indent=3)`	Write the dictionary `x` to the file `"p.json"` and `indent` each level with `3` spaces
`with open("p.json", "w") as f:` `x = json.load(f)`	Load the file `"p.json"` and store its content in the dictionary `x`

Exercise: Write the dictionary params, defined below to a file called "parameters.json":

params = {"n_trials": 100, "training": True, "feedback": None}

Solution

with open("parameters.json", "w") as f:
    json.dump(params, f)

Exercise: Open "parameters.json" in your editor — how are the values True and None represented? Load the file’s contents into a dictionary called params and print it.

Solution

with open("parameters.json", "r") as f:
    params = json.load(f)
params

{'n_trials': 100, 'training': True, 'feedback': None}

Exercise: Write the params dictionary to the "parameters.json" file again and pass the optional argument indent=3. Then, open "parameters.json" in your editor — how does it look different from the previous version?

Solution

with open("parameters.json", "w") as f:
    json.dump(params, f, indent=3)

Exercise: Create a dictionary called config with "condition": ["a", "b", "c"]" and "probability": [0.2, 0.2, 0.6]" and print it.

Solution

config = {"condition": ["a", "b", "c"], "probability": [0.2, 0.2, 0.6]}

Exercise: Write config to a file called config.json with indent=4.

Solution

with open("config.json", "w") as f:
    json.dump(config, f, indent=4)

Exercise: Open "config.json" in your editor, add a field "n_blocks":3 and save. Then, load the file into a dictionary called config and print the value of config["n_blocks"].

Solution

with open("config.json", "r") as f:
    config = json.load(f)
config["n_blocks"]

Exercise: In your editor, create a new file called experiment.json and write a JSON file that contains "keys": ["left", "right"], "conditions": [1,2,3] and "training": null. Then, load this file into a dictionary called exp and print the value of exp["keys"].

Solution

with open("experiment.json", "r") as f:
    exp = json.load(f)
exp["keys"]

['left', 'right']

Section 3: Configuring Visual Stimuli

Now that we understand how to read and write config files, we can start to dive into PsychoPy which is the package we will use to present images and audio to and record responses from our subjects. In this section we will explore the presentation of visual stimuli with PsychoPy. A PsychoPy program usually starts by opening a Window, where images are shown to the subject. Just like our opened files, we can handle the Window using a context manager. This is very convenient because it prevents us from ending up with a frozen window - if our experiment crashes, the context manager will make sure that everything is cleaned up and the window is closed. PsychoPy used a dual-buffer system which means that there are two screens: one that we are seeing and one that is hidden. When we draw images, we are always drawing to the hidden screen. The images are only revealed once we call .flip().

Code	Description
`with Window() as win:`	Open a `Window` within a context manager
`win.flip()`	Clear the screen and display new image
`text = TextStim(win, text="Hi!")`	Create a text object for the given `Window`
`rect = Rect(win, pos=(0,0), width=1, height=1, lineColor="white")`	Create a rectangle for the given `Window`
`circle = Circle(win, radius=0.2, fillColor="blue")`	Create a circle for the given `Window`
`rect.draw(), circle.draw(), text.draw()`	Draw a visual stimulus (e.g. rectangle) to the window buffer
`waitKeys()`	Wait until a key was pressed
`waitKeys(keyList=["left", "right"])`	Wait until the left or right arrow key was pressed

Exercises

Example: Open a Window() within a context manager, then draw a "white" rectangle, flip() the window and wait for a key press.

with Window() as win:
    rect = Rect(win, lineColor="white")
    rect.draw()
    win.flip()
    waitKeys()

Exercise: Open a Window() within a context manager, then draw a "white" rectangle with width=1.0 and height=0.2, flip() the window and wait for a key press.

Solution

with Window() as win:
    rect = Rect(win, width=1.0, height=0.2, lineColor="white")
    rect.draw()
    win.flip()
    waitKeys()

Solution

Exercise: Create a file called "rect.json" to store the parameters "width": 1.0", "height":0.2 and "color":"white" (Hint: you can either write a Python dictionary and write it to a file or create the file manually in your editor).

Solution

params = {"width": 1.0, "height": 0.2, "color": "white"}
with open("rect.json", "w") as f:
    json.dump(params, f)

Exercise: Load "rect.json" into a dictionary called params. Then, open a Window() within a context manager and draw a rectangle using the values from params for width, height and lineColor (e.g. lineColor=params["color"]). Then, draw the rectangle, flip the window and wait for a key press.

Solution

with open("rect.json", "r") as f:
    params = json.load(f)

with Window() as win:
    rect = Rect(
        win, width=params["width"], height=params["height"], lineColor=params["color"]
    )
    rect.draw()
    win.flip()
    waitKeys()

Solution

Exercise: Open "rect.json" in your editor and change the "color" value to "pink" (don’t forget to save). Then, rerun the code from the previous exercise

Solution

with open("rect.json", "r") as f:
    params = json.load(f)

with Window() as win:
    rect = Rect(
        win, width=params["width"], height=params["height"], lineColor=params["color"]
    )
    rect.draw()
    win.flip()
    waitKeys()

Solution

Example: Open a Window() within a context manager, draw a "red" circle at the position (0, 0.5), flip the window and wait until a key was pressed.

with Window() as win:
    circle = Circle(win, fillColor="red", pos=(0, 0.5))
    circle.draw()
    win.flip()
    waitKeys()

Exercise: Re-run the code from above but change the circle’s color to "green" and it’s position to (-0.1, 0).

Solution

with Window() as win:
    circle = Circle(win, fillColor="green", pos=(-0.1, 0))
    circle.draw()
    win.flip()
    waitKeys()

Solution

Exercise: Create a file called "circle.json" to store the parameters "color": "green" and "pos":(-0.1, 0) (Hint: you can either write a Python dictionary and write it to a file or create the file manually in your editor).

Solution

params = {"color": "green", "pos": (-0.1, 0)}
with open("circle.json", "w") as f:
    json.dump(params, f)

Exercise: Load circle.json into a dictionary called params. Then, repeat the code from above that draws the circle but use the values from params for the circle’s parameters (e.g. fillColor=params["color"]

Solution

with open("circle.json", "r") as f:
    params = json.load(f)

with Window() as win:
    circle = Circle(win, fillColor=params["color"], pos=params["pos"])
    circle.draw()
    win.flip()
    waitKeys()

Solution

Exercise: Open circle.json in your editor and change the color to "yellow" and the position to (-0.5, 0.5). Then, save and re-run the code from above.

Solution

with open("circle.json", "r") as f:
    params = json.load(f)

with Window() as win:
    circle = Circle(win, fillColor=params["color"], pos=params["pos"])
    circle.draw()
    win.flip()
    waitKeys()

Solution

Example: Open a Window() within a context manager, draw the text "Welcome!", then, flip the window and wait until the "space" key is pressed.

with Window() as win:
    text = TextStim(win, text="Welcome!")
    text.draw()
    win.flip()
    waitKeys(keyList=["space"])

Exercise: Re-run the code from above but change the text to "Hello!" and wait until the "escape" key was pressed.

Solution

with Window() as win:
    text = TextStim(win, text="Hello!")
    text.draw()
    win.flip()
    waitKeys(keyList=["escape"])

Solution

Exercise: Create a file called "text.json" to store the parameters "text": "Hello!" and "keys":["escape"] (Hint: you can either write a Python dictionary and write it to a file or create the file manually in your editor).

Solution

params = {"text": "Hello!", "keys": ["escape"]}
with open("text.json", "w") as f:
    json.dump(params, f)

Exercise: Load "text.json" into a dictionary called params and re-run the code from @exr-text but use the values from params (e.g. keyList=params["keys"]).

Solution

with open("text.json", "r") as f:
    params = json.load(f)

with Window() as win:
    text = TextStim(win, text=params["text"])
    text.draw()
    win.flip()
    waitKeys(keyList=params["keys"])

Solution

Exercise: Open config.json in your text editor, change the text to "Hey!" and add "return" to the list of accepted keys. Then, re-run the code from above.

with open("text.json", "r") as f:
    params = json.load(f)

with Window() as win:
    text = TextStim(win, text=params["text"])
    text.draw()
    win.flip()
    waitKeys(keyList=params["keys"])

Section 4: Refactoring an Experiment to Make it Configurable

This script below contains an implementation of the Posner cueing task, a classic psychophysical experiment that tests how spatial attention is deployed in a reaction task. Execute the cell to run the experiment - the instructions will be displayed on screen. Unfortunately, all parameter values, like the number of trials or the positions of the visual objects are encoded directly in the script. This means, that whenever we want to re-configure the experiment, we have to modify the source code. In this section, you will apply what you have learned so far to refactor the script so that it reads parameter values from a JSON file. This allows us to re-configure the experiment without even touching the code. Execute the cell below to run the script!

Exercises

import json
from random import shuffle
from psychopy.core import Clock, wait
from psychopy.visual import Window, Rect, Circle, TextStim
from psychopy.event import waitKeys

#### Define parameters ####
N_TRIALS = 10  # number of trials
P_VALID = 0.8  # probability that a cue is valid
FIX_DUR = 0.5  # duration for which fixation is displayed
CUE_DUR = 0.5  # duration for which cue is displayed
INSTRUCTIONS = """
    Welcome! \n
    When the experiment starts, you'll see a white dot and two white boxes. \n
    Fixate the white dot whenever it appears!
    After a short while, a red dot will appear in one box.
    Use the arrow keys to say that the red dot is in the left or right box.
    Respond as soon as possible! \n
    Before the red dot appears, one of the boxes will be highlighted red.
    Most of the time, the red dot will appear in the highlighted box.\n
    Press space to start the experiment!
    """

##### Create the trial sequence ####
side, valid = [], []
if N_TRIALS / 2 * P_VALID % 1 != 0:
    raise ValueError("Trials can't be evenly divided between conditions!")
for s in ["left", "right"]:
    n = int(N_TRIALS / 2)
    side += [s] * n  # side the stimulus appears ("left" or "right")
    n_valid = int(n * P_VALID)
    valid += [True] * n_valid + [False] * (
        n - n_valid
    )  # whether the cue is valid (True or False)

idx = list(range(N_TRIALS))
shuffle(idx)  # randomize the order
trials = []
for i in idx:
    trials.append(
        [side[i], valid[i]]
    )  # list of trials where each element is a list of 2, e.g. ["left", True]

#### Run the Experiment ####
clock = Clock()
with Window() as win:

    #### Show instructions ####
    text = TextStim(win, text=INSTRUCTIONS, height=0.07)
    text.draw()
    win.flip()
    waitKeys(keyList=["space"])

    #### Run trials ####
    count = 0
    for t in trials:
        count += 1

        # show boxes and fixation
        box_left = Rect(win, lineColor="white", pos=(-0.5, 0))
        box_right = Rect(win, lineColor="white", pos=(0.5, 0))
        fixation = Circle(win, fillColor="white", radius=0.05)
        _, _, _ = box_left.draw(), box_right.draw(), fixation.draw()
        win.flip()

        wait(FIX_DUR)

        # if stim is on the left and cue is valid OR if stim is on the right and cue is invalid
        if (t[0] == "left" and t[1] == True) or (t[0] == "right" and t[1] == False):
            box_left = Rect(
                win, lineColor="red", pos=(-0.5, 0)
            )  # highlight the left box
            box_right = Rect(win, lineColor="white", pos=(0.5, 0))
        else:
            box_left = Rect(win, lineColor="white", pos=(-0.5, 0))
            box_right = Rect(
                win, lineColor="red", pos=(0.5, 0)
            )  # highlight the right box
        _, _, _ = box_left.draw(), box_right.draw(), fixation.draw()
        win.flip()

        wait(CUE_DUR)

        # show stimulus
        box_left = Rect(win, lineColor="white", pos=(-0.5, 0))
        box_right = Rect(win, lineColor="white", pos=(0.5, 0))
        if t[0] == "left":
            stim = Circle(win, fillColor="red", pos=(-0.5, 0), radius=0.05)
        else:
            stim = Circle(win, fillColor="red", pos=(0.5, 0), radius=0.05)
        _, _, _ = box_left.draw(), box_right.draw(), stim.draw()
        win.flip()

        #### Obtain Response ####
        clock.reset()
        keys = waitKeys(keyList=["left", "right"], timeStamped=clock)  # get response
        name = keys[0][0]  # key name
        rt = keys[0][1]  # reaction time
        rt = round(rt, 4)  # round to 4 decimals

        if name == t[0]:
            response = "correct"
        else:
            response = "wrong"
        print("Trial " + str(count) + ": " + response + " response with rt=" + str(rt))

Exercise: In the beginning of the script posner_task.py, multiple parameters are defined (e.g. N_TRIALS). Create a file called posner.json that stores these parameters. Then, modify the script so that it loads posner.json and replace the parameters with the values stored in the file. Try modifying the parameters and re-running the script!

Solution

import json
from random import shuffle
from psychopy.core import Clock, wait
from psychopy.visual import Window, Rect, Circle, TextStim
from psychopy.event import waitKeys

#### Load parameters ####
with open("posner.json", "r") as f:
    params = json.load(f)

N_TRIALS = params["n_trials"]  # number of trials
P_VALID = params["p_valid"]  # probability that a cue is valid
FIX_DUR = params["fix_dur"]  # duration for which fixation is displayed
CUE_DUR = params["cue_dur"]  # duration for which cue is displayed
INSTRUCTIONS = params["instructions"]
...

Exercise: Include the positions of the visual objects in posner.json and replace the values passed to the pos arguments with the values from that file (Hint: You’ll need four different positions: for the left and right box as well as the left and right circle). Then, try modifying the parameters and re-running the script!

...
    #### Run trials ####
    count = 0
    for t in trials:
        count += 1

        # show boxes and fixation
        box_left = Rect(win, lineColor="white", pos=params["pos_left_box"])
        box_right = Rect(win, lineColor="white", pos=params["pos_right_box"])
        fixation = Circle(win, fillColor="white", radius=0.05)
        _, _, _ = box_left.draw(), box_right.draw(), fixation.draw()
        win.flip()

        wait(FIX_DUR)
        
        ...

        # show stimulus
        box_left = Rect(win, lineColor="white", pos=params["pos_left_box"])
        box_right = Rect(win, lineColor="white", pos=params["pos_right_box"])
        if t[0] == "left":
            stim = Circle(win, fillColor="red", pos=params["pos_left_circle"], radius=0.05)
        else:
            stim = Circle(win, fillColor="red", pos=params["pos_right_circle"], radius=0.05)
        _, _, _ = box_left.draw(), box_right.draw(), stim.draw()
        win.flip()

        ...