Article comments

Description

I need 9 to 10 comments throughout the whole article. Simple words and comments. Thank you

Unformatted Attachment Preview

Acceptability and Feasibility of Virtual Behavior Analysis
Supervision
Christina A. Simmons, Kimberly R. Ford, Giovanna L. Salvatore, and Abigail E. Moretti
Author information Article notes Copyright and License information PMC Disclaimer
Associated Data
Data Availability Statement
Go to:
Abstract
The COVID-19 pandemic necessitated a rapid transition to virtual service
delivery and supervision. This preliminary study examined acceptability
and feasibility of virtual supervision for 94 BCBA/BCaBA trainees during
COVID-19, including variables that affected perceived satisfaction,
effectiveness, and supervision preference for this sample. Results
indicate a decrease in accrual of direct client hours during the pandemic,
with a third of participants reporting a decrease in individual
supervision. In general, participants were satisfied with virtual
individual and group supervision as indicated by high satisfaction
domain scores and individual item means, with minimal overall change
in satisfaction. Participants indicated preference for in-person or hybrid
supervision and considered in-person most effective. In general,
participants reported that virtual supervision was feasible and
supervisors used best-practice strategies. We discuss variables that
affected satisfaction (e.g., length of supervisory relationship), preference
(e.g., age, services provided), and perceived effectiveness (e.g., time
supervisor was a BCBA). We provide practical implications and
recommendations for virtual supervision.
Keywords: supervision, virtual, satisfaction, COVID-19
Supervision is a critical component of the training process toward
becoming a board certified behavior analyst (BCBA) or board certified
assistant behavior analyst (BCaBA). According to the Behavior Analyst
Certification Board (BACB), the goal of supervision is to build and
maintain the professional competence of the trainee and to ensure that
the trainee’s clients are receiving the highest quality care. Within the
supervisory experience, trainees are encouraged to gain practical
experience in the varying responsibilities of a BCBA. These
responsibilities include conducting behavioral assessments,
implementing and monitoring skill acquisition and behavior-reduction
programs, writing treatment plans, and overseeing others in the
implementation of behavior plans (BACB, 2020a).
Trainees in the field of applied behavior analysis must accrue hours
toward their BCBA or BCaBA, with the number of hours increasing
because the task list for the BACB, Fifth Edition, is set to go into effect in
January 2022 (BACB, 2020b). Trainees must accrue a minimum of 20 to a
maximum of 130 supervised hr per month. Supervised hours include
hours directly engaged with a client delivering specific “therapeutic or
instructional procedures” (i.e., restricted hours; BACB, 2020a, p. 12), and
hours related to behavior analytic duties such as conducting
assessments, analyzing data, and writing reports (i.e., unrestricted
hours), with a limit to the percentage of total hours that can be restricted
hours (BACB, 2020b). A minimum of 5% of hours must be supervised by
an approved BCBA supervisor. An integral part of supervision is the
supervisor directly observing the trainee with a client each month (i.e.,
client contact; BACB, 2020b). The literature has highlighted best-practice
recommendations for supervision of aspiring BCBAs (e.g., Garza et
al., 2018; Sellers et al., 2016; Valentino et al., 2016). For further
information, we recommend the reader reference a special issue
in Behavior Analysis in Practice by LeBlanc and Luiselli (2016) that
highlights essential elements of supervision (e.g., building and
maintaining supervisory relationships, ethical principles of supervision,
conflict resolution). Such guidelines have been extended to encompass
virtual supervision. In a recent publication, Britton and Cicoria (2019)
provide guidance to BCBAs regarding providing virtual supervision for
trainees. Authors describe contexts for growing consideration of virtual
supervision, including students in online programs seeking supervisors
that may not be available in their local communities and trainees in rural
or underrepresented areas. Further, authors describe challenges to
virtual supervision including ease of interaction, demonstration of
competency of key behavior analytic skills, and maintenance of
confidentiality. Further empirical studies are warranted on trainee’s
experiences with supervision, especially with regard to virtual
supervision.
The COVID-19 pandemic has greatly affected the lives of individuals
around the world. The nature of the virus has necessitated individuals,
communities, and institutions to assess the need for in-person
interactions and move to a virtual modality to reduce the spread of the
virus. This move to virtual work was initially intended to both conserve
personal protective equipment for healthcare providers and to reduce
the overall spread of the virus to “flatten the curve” of COVID-19 cases
(Caravella et al., 2020). However, the move to virtual modalities has had
a significant impact on how individuals, communities, and institutions
function.
Although the pandemic has affected the functionality of many
occupations, the effects are particularly significant for healthcare
professions that require in-person interactions with clients to deliver
treatment services. In addition to disrupting clinical services, the COVID19 pandemic also affected trainees’ opportunities for continued
education and supervision (Caravella et al., 2020). The COVID-19
pandemic in the United States led to the closure of institutions of higher
education, restriction of practicum and internship placements, and a
shift to virtual learning in March 2020 (Kim, 2020). This sudden move to
virtual modalities conceivably affected the nature of supervision for
trainees. To mitigate the potential disruption in services and supervision,
many supervisors had to rapidly adapt to virtual platforms in order to
maintain supervisory requirements.
The effects of COVID-19 likely affected trainees obtaining fieldwork
hours for the BCBA and BCaBA, across settings. Schools are one area of
practice for a subset of trainees in which the effect of the pandemic was
specifically documented. Due to the sudden closure of schools at the start
of the pandemic, some trainees working in school settings temporarily
halted accruing hours until adaptations could be made. Although such
adaptations were made (e.g., virtual platforms, in-home services),
trainees may have experienced a reduction in fieldwork hours, raising
concerns about a quality supervisory experience (Fronapfel &
Demchak, 2020). The difficulty of meeting the supervision requirements
during the COVID-19 pandemic resulted in the BACB temporarily
waiving the “observation with a client” requirement (BACB, 2020c). This
waiver was instated to ensure that trainees who did not have the ability
to work with their clients due to COVID-19 restrictions could still accrue
hours towards their BCBA and BCaBA.
Although virtual supervision has recently been adapted in the field of
behavior analysis, it has a long history of use across fields such as
psychotherapy (Caravella et al., 2020), counseling (Nadan et al., 2020),
education (Kim, 2020), speech therapy (Dudding & Justice, 2004), social
work (Panos et al., 2002), and medical services (Wearne et al., 2015).
Overall, studies have shown that virtual supervision has maintained
adequate quality of supervision and positively affected the development
of trainee competence in multiple domains of counseling and
psychotherapy (Manring et al., 2011; Nadan et al., 2020). In a particularly
noteworthy study, Caravella et al. (2020) explored the use of virtual
supervision in response to the COVID-19 pandemic in the New York
University Langone Health Consultant Liaison Psychiatry Service. The
hospital service created a virtual rotation for psychiatric trainees that
maintained supervision and provided exposure to all requisite aspects of
the trainee learning experience. Virtual supervision allowed supervisors
to sustain supervision of psychiatric trainees and to provide
opportunities for training when in-person supervision was not feasible.
The overall shortage of BCBAs (Rios et al., 2018), the high percentage of
BCBAs overextended with supervision responsibilities (Sellers et
al., 2019), and the growth of trainees seeking supervision outside their
immediate area (Britton & Cicoria, 2019) may have influenced the
introduction of virtual supervision in the field of applied behavior
analysis. To understand the barriers to supervision within behavior
analysis, Sellers et al. (2019) surveyed current BCBA supervisors and
found that supervisors reported that their schedules did not allow them
to adequately prepare for supervisory sessions, discussions, or meetings
with their trainees. With a high level of burnout in the field of behavior
analysis (Plantiveau et al., 2018), COVID-19 conceivably intensified the
marked stress placed on BCBAs to provide ethical and effective
supervision. Fronapfel and Demchak (2020) indicated that virtual
supervision technologies allow trainees to practice and demonstrate
various behavior analytic skills with their supervisor through either an
individual or group videoconference. These findings highlight the
potential of virtual supervision for avoiding barriers to effective
supervision in behavior analysis, both during the COVID-19 pandemic
and beyond.
There are many potential advantages of supervisors incorporating
virtual supervision into their supervisory relationship. Virtual
supervision allows for increased flexibility in scheduling (Amodeo &
Taylor, 2004), decreased distance and travel costs for supervisors
(Panos et al., 2002), access to supervision in rural areas (Wood et
al., 2005), and decreased reactivity of clients (Israel et al., 2009).
Decreasing reactivity (i.e., behavior change due to an extraneous
variable) is especially pertinent in behavior analysis with children who
engage in challenging behaviors maintained by access to attention
(Farley, 2019). In addition, researchers have found that virtual
supervision positively affects the supervisory experience (Nadan et
al., 2020) and increases the learning experience of trainees (Manring et
al., 2011).
However, there are also potential disadvantages to using virtual
supervision that have been documented in other fields. For example,
when a supervisor is supervising a live session using virtual platforms,
the session flow may be disrupted due to lengthy communication
processes or missed feedback (Nadan et al., 2020) and technical
difficulties (Chen et al., 2020). Further, Wilczenski and Coomey (2006)
suggest that virtual modalities can create an impersonal environment
where trainees may engage in inappropriate behaviors without
recognizing the personal impact, and the use of virtual supervision could
make it difficult for the supervisor to identify some inappropriate or
unethical trainee behaviors that would otherwise be evident during inperson observation. For example, a trainee delivering virtual services
may be in a location that does not maintain client confidentiality, may
simultaneously engage in personal activities on their device that are not
visible to the supervisor, or may discuss client behaviors within earshot
of the client and family. A trainee delivering in-person services may
engage in inappropriate body language (e.g., eye rolling) toward the
client or in response to feedback that is not visible on camera, may use
incorrect physical positioning during prompting procedures that the
supervisor cannot detect, or may discuss client behaviors while the client
and family are present but not visible to the supervisor on camera.
Finally, virtual supervision raises concerns of maintaining confidentiality
and security of client information during virtual communication
(Wilczenski & Coomey, 2006).
The current study aimed to conduct a preliminary examination of the
acceptability and feasibility of virtual supervision for BCBA/BCaBA
trainees during the COVID-19 pandemic, including potential variables
that contribute to perceived satisfaction, effectiveness, and supervision
preference. The goal of this study was to obtain preliminary data on
trainees’ experiences with virtual supervision and to document virtual
behavior analytic supervisory practices with our sample that could lead
to further research evaluations. We also explored the practical
implications of virtual supervision and recommendations for virtual
supervisory practices to maintain quality supervision in the field of
applied behavior analysis.
Go to:
Method
Recruitment
We recruited individuals in the United States who received virtual
supervision toward their BCBA or BCaBA certification between March 1,
2020 and August 31, 2020. Participants were recruited through online
platforms (e.g., Facebook groups, Reddit), listservs, and email invitations
sent to the coordinators of all programs in the United States with an
Association for Behavior Analysis International verified course sequence.
Although we do not have data on the number of trainees who received
the invitation to participate, our recruitment efforts aimed to capture a
broad range of trainees across the United States. A total of 136 responses
were collected: 6 (4.41%) reported that they did not receive virtual
supervision, 1 (0.74%) did not reside in the United States, 28 (20.59%)
did not complete at least 60% of the online survey, and 7 (5.15%) did not
complete all survey questions in at least one of the satisfaction and
change domains, yielding a final sample size of 94 participants (69.12%;
see Fig. Fig.1).1). We excluded participants who did not complete
satisfaction and/or change domains as these were our primary
dependent variables.
Fig. 1
Consort Chart for Recruited Participants
Response Definitions
We defined the period from March 1, 2020 through the end of data
collection on August 31, 2020 as “during COVID-19” in our survey
measures. March 2020 was selected because (1) a national emergency
was declared in the United States on March 13, (2) individual states
instated lockdown measures and other restrictions, and (3) confirmed
cases reached all 50 states. For the remainder of the manuscript, we will
use the terminology “during COVID-19” for ease of discussion. We refer
to any activities deemed supervision by the BACB and delivered through
virtual means as virtual supervision. We defined virtual as any
communication or observation between supervisor and trainee in which
both parties were not physically present in the same location (e.g., video
conference, phone call, instant message, text message, email). Virtual
supervision may include any of the following activities where the
supervisor is virtual: (1) client contacts (i.e., telehealth or in-person
service delivery); (2) individual supervision meetings; (3) group
supervision meetings; and (4) asynchronous communication (e.g.,
written feedback via email).
Measures
Survey Development
Due to the widespread necessity of virtual supervision during the COVID19 pandemic, this online survey was developed by the authors prior to
conducting this study to measure the feasibility and acceptability of
virtual supervision, with input from both BCBA/BCBA-D supervisors and
aspiring BCBA/BCaBA trainees. Prior to this study, the survey was
piloted by trainees and additional revisions were made based on their
feedback. The online survey was administered using the Qualtrics®
survey platform (2021). Questions included yes/no responses, multiple
choice responses, Likert scale ratings, and open-ended responses. The
survey took a median of 11.10 min to complete (range: 4.47–230.43
min).
Questions were grouped by conceptual similarity and yielded total
scores in four domains. Cronbach’s alpha was used as a general measure
of the item interrelatedness (Tavakol & Dennick, 2011). Domains
included: (1) satisfaction with virtual individual supervision (10 items, α =
.93, interitem correlations ranged from .39 to .86); (2) change in
individual supervision (11 items, α = .90, interitem correlations ranged
from .18 to .82); (3) satisfaction with virtual group supervision (7 items, α
= .94, interitem correlations ranged from .67 to .92); and (4) change in
group supervision (7 items, α = .94, interitem correlations ranged from
.60 to .88). Skewness and kurtosis values (i.e., to characterize the shape
and distribution of the data and identify outliers) were within normal
limits (+/-2) of a univariate distribution (George & Mallery, 2016) for
satisfaction scores. In particular, we analyzed these variables to
determine whether our data were normally distributed (i.e., symmetrical
with a similar number of low and high values and most values around
the middle) to inform the statistical analyses conducted. Within the
domains of change in individual supervision and change in group
supervision, kurtosis values were outside of normal limits of a univariate
distribution. When analyzing individual participant data of the 94
included participants for change in individual supervision, we detected
four outliers in participants’ data with overly high change scores (>
2 SD above M; i.e., > 44.85). As such, we removed these participants’ data
(4.26%) from all analyses involving change with individual supervision.
Likewise, within the domain of change in group supervision, we detected
eight outliers with overly high change scores (> 1.5 SD above M; i.e., >
29.97) and excluded these participants’ data (8.51%) from all analyses
involving change with group supervision. We chose to remove these
outliers as the few overly high reports of positive change that were
greater than 1.5 standard deviations above the mean may have been
influenced by participants’ overall positive perceptions of their
supervisor or supervision placement itself leading to selecting “greatly
improved” for all variable options or selecting down the row of response
options without attending to individual items. Further, we intended for
our preliminary results to generalize to the typical experiences of
behavior analytic trainees.
Procedure
Participant consent was obtained in Qualtrics prior to accessing the
survey and personal identifiers were not collected as part of the survey
to provide anonymity. The survey began on July 31, 2020 and ended on
August 31, 2020. Participants who provided their contact information in
a separate survey upon completion were entered into a random drawing
for a $100 digital gift card.
Data Analysis
We calculated individual supervision and group supervision satisfaction
scores by applying numerical values to each Likert scale question in the
corresponding domain and summing the item values for each
participant. For example, each participant responded to 10 questions
about satisfaction with individual supervision on a 5-point Likert scale
ranging from greatly dissatisfied (1) to greatly satisfied (5). If a
participant responded to each of the 10 questions with a rating of 5, we
summed together each of the item scores (5 * 10) to obtain a domain
score of 50. Likewise, we calculated individual supervision and group
supervision change scores by applying numerical values to each Likert
scale question in the corresponding domain and creating a total sum
score for each participant.
We conducted statistical analyses to determine the statistical
significance of selected variables on satisfaction, change, preference, and
perceived effectiveness. A resulting p-value less than .05 was deemed
statistically significant. Whereas our sample is representative of BCBAs
across the United States (e.g., geographic location, gender, race/ethnicity,
area of professional emphasis; BACB, 2020d), the small number of
participants selecting specific survey options limit the conclusions that
can be drawn from statistical analyses. We propose these results as a
preliminary investigation into the status of virtual supervision during
COVID-19 and as a methodological framework for future investigation.
Simple linear regression was conducted for two continuous variables
(e.g., age and satisfaction). Chi-square tests were conducted for
categorical independent variables (e.g., type of service provided) and
categorical dependent variables (e.g., preferred method of supervision).
Post-hoc chi-square tests were conducted if main effects were
significant. Multinomial logistic regression was conducted for a
continuous independent variable (i.e., age) on categorical dependent
variables (e.g., effectiveness of supervision). One-way analysis of
variance (ANOVA) was conducted to analyze differences between
polytomous (i.e., more than two options) categorical independent
variables (e.g., amount of time BCBA was supervisor) on continuous
dependent variables (e.g., satisfaction). Post-hoc Tukey tests were
conducted if a significant main effect was detected. Linear regression
was conducted for a dichotomous (i.e., two options) categorical
independent variable (i.e., type of service) and continuous dependent
variables (e.g., satisfaction score). Categories with two or fewer
responses were removed from analyses.
Open-ended responses to preferred method of supervision and method
of supervision that was considered most effective were thematically
analyzed to determine common themes identified across participants
using the constant comparative method (CCM) of qualitative data
analysis, first described by Glaser and Strauss (1967) and refined by
Strauss (1987). See Olson et al. (2016) for a discussion of the evolution of
CCM methodology and a description of the 10-step processes adapted for
the purposes of this analysis. In particular, the researchers allowed
themes to emerge from participant responses. Researchers extracted the
written responses and sorted them into categories based on participants’
selections for modality preference and modality perceived as most
effective (e.g., in-person, virtual, hybrid). In chronological order of
responses, two research assistants trained in qualitative data analysis
independently coded the extracted written responses (i.e., not associated
with any other participant data) into as many categories of analysis as
possible. Categories were added as they emerged, and data were fit to
existing categories. As subsequent text was coded, each incident of a
category was compared with previous incidents of that category across
participants. Subthemes were listed under categories to create an
accurate operational definition of each category. When incidences could
potentially be coded into multiple categories, specific rules and
exclusionary criteria were developed. Categories were informed by the
language of the participants. Researchers collaborated by comparing
categories across data collectors to arrive at a final categorization of the
data. Both researchers recoded data using unified categorizations. A
point of saturation was reached when no new themes emerged from
additional participants. We determined salience of themes by summing
the frequency of mention of that theme across all participants. To
determine interobserver agreement (IOA), referred to as intercoder
reliability by Olson et al., we randomly selected 34.49% of participant
text responses, across preference categories, and an independent data
collector trained in qualitative data analysis scored these data with the
categories we provided. We defined an agreement as both observers
scoring a text response with the same category or categories. We
calculated IOA using the formula: [agreements / (agreements +
disagreements)] *100. Interobserver agreement was 92.5%.
Disagreements included not scoring individual responses into more than
one relevant category.
Go to:
Results
Participant Demographics
Responses were collected from 136 participants. Ninety-four
participants who met all inclusion criteria were included in the final
sample (see Fig. Fig.11 for each exclusion criteria met), with outliers
removed for individual analyses, where relevant. The sample included
participants from 24 states with the largest representation from New
Jersey (17.02%), Pennsylvania (15.96%), and California (7.45%).
Participants were between 21 and 55 years of age (M = 31 years) and
predominantly female (93.62%). A majority of participants identified as
white (75.33%), followed by Asian (9.57%), Hispanic/Latinx (8.51%),
Black/African American (4.26%), and other (2.13%). Most participants
reported being enrolled in a master’s program (82.98%) and accruing
supervised hours from 1–3 months through 4 or more years (Mode = 7–
12 months [39.36%]). Participants provided multiple types of services
across a variety of settings during the 3 months before COVID-19. See
Table Table11 for a breakdown of location of services, type of services,
and ages of clients served, across participants. Most participants
reported that their supervisor was female (91.49%) and had been a
BCBA for 1–3 years (22.34%) or 3–5 years (27.66%).
Table 1
Services Provided Prior to COVID-19
n
%
Location of Services
Home
42 44.68
School
42 44.68
Clinic
39 41.49
n
%
Residential
15 15.96
Community
12 12.77
Other
6
6.38
Services Provided
Behavior Reduction
87 92.55
Daily Living Skills
74 78.72
Academic Skills
51 54.26
Early Intervention
46 48.94
Staff Training
42 44.68
Caregiver Training
37 39.36
Vocational Skills
28 29.79
Feeding
18 19.15
Promoting Health-Related Behaviors
13 13.83
Organizational Behavior Management 5
5.32
Treating Addiction
0
0
Other
0
0
Age Range of Clients
0–5 years
60 63.83
6–10 years
56 59.57
11–14 years
35 37.23
15–17 years
25 26.60
18–24 years
23 24.47
25–34 years
11 11.70
35–44 years
9
9.57
45–55 years
7
7.45
Over 55 years
4
4.26
Open in a separate window
State of Affairs
During the 3 months prior to COVID-19, the majority of our participants
(91.49%) reported providing exclusively in-person services, with small
percentages providing virtual services (5.32%) and a hybrid of in-person
and virtual services (6.38%). During COVID-19, the majority of
participants in this sample reported that they, for some period, provided
virtual services (70.21%), followed by in-person services (37.23%), and
a hybrid of in-person and virtual services (25.53%). Two participants
reported that they did not provide any services due to COVID-19, but still
received virtual supervision.
The majority of our participants reported that the amount of direct client
hours changed while they provided virtual services (83.33%) and inperson services (85.71%). See Table Table22 for information on length
of service delivery, number of clients served, and how client hours
changed during COVID-19.
Table 2
Services Provided during COVID-19
Virtual Services (n = 66)
In-Person Services (n = 35)
n
%
n
%
Length Services Provided < 2 weeks 3 4.55 10 28.57 2–4 weeks 9 13.64 7 20.00 4–6 weeks 10 15.15 5 14.29 6–8 weeks 3 4.55 2 5.71 8–10 weeks 5 7.58 3 8.57 > 10 weeks
36 54.55 8
22.86
Number of Clients Served 1
15 22.73 10
28.57
2
13 19.70 7
20.00
3
2
5
14.29
4
11 16.67 2
5.71
5
2
3
8.57
6+
23 34.85 8
22.86
Greatly decreased 28 42.42 11
31.43
Decreased
21 31.82 14
40.00
Stayed the Same
11 16.67 5
14.29
Increased
5
7.60
3
8.57
Greatly Increased 1
1.5
2
How Hours Changed
3.03
3.03
5.71
Open in a separate window
During COVID-19, over half of participants (59.57%) in this sample
reported that there was a period that they did not meet the BACB’s
“observation with a client” requirement due to the pandemic. Of these
individuals, only 60.53% reported that they completed the
BACB’s Compassionate Exception Attestation for
Experience/Fieldwork (BACB, 2020c) to waive the “observation with a
client” requirement with the majority doing so for only 1 month
(43.48%) or 2 months (34.78%), with two participants doing so for 3
months and one each for 4, 5, and 6 months.
By the end of August 2020, only 30.85% of participants in this sample
reported that they had resumed in-person supervision. Of the 29
participants who had resumed in-person supervision, about half were
providing in-person services (51.72%), followed by a hybrid of in-person
and virtual services (34.48%). A smaller percentage of these participants
(13.79%) were providing virtual services while receiving in-person
supervision. The majority of our participants (62.07%) who had
resumed in-person supervision indicated that the format had changed
from the supervision they received prior to COVID-19.
Description of Virtual Supervision
Prior to COVID-19, 57.45% of participants in this sample reported that
they received exclusively in-person supervision, 20.21% virtual, and
20.21% hybrid, with one reporting other. See Table Table33 for amount
of individual supervision hours per week and month during the 3
months prior to COVID-19. Approximately half of participants (55.32%)
in this sample indicated that the amount of individual supervision they
received changed during the virtual model, with the majority reporting
that supervision decreased (42.31%) or greatly deceased (23.08%). It is
interesting that of the participants who reported a change in amount of
individual supervision, 17.31% indicated that individual supervision
increased and 17.31% indicated that it greatly increased.
Table 3
Amount of Individual and Group Supervision Prior to COVID-19
Time per Week n
Individual < 30 min Group 1 % Time per Month n % 1.06 6 hr 18 19.15 < 30 min 19 27.54 6 hr 4 5.80 Open in a separate window A majority of our participants (73.40%) reported receiving group supervision prior to COVID-19, with 74.47% receiving group supervision during COVID-19. See Table Table33 for amount of group supervision hours per week and month during the 3 months prior to COVID-19. During COVID-19, participants in this sample reported receiving group supervision for a mode of less than 30 min per week (35.71%) and a mode of 1–2 hr per month (25.71%). Trainees reported the strategies used by their supervisor during virtual supervision. The most frequently endorsed strategies in our sample included answering questions (91.49%), discussion (85.11%), direct observation (77.66%), reviewing protocols and procedures (74.47%), and in-the-moment feedback (67.02%). See Table Table44 for percentages of all strategies reported. The majority of our participants (64.89%) reported that their supervisor’s supervision strategies remained the same during the virtual model, 27.66% reported that they changed, and 6.38% reported that they greatly changed. Table 4 Strategies Used during Virtual Supervision Technique n % Answering Questions 86 91.49 Discussion 80 85.11 Direct Observations 73 77.66 Reviewing Protocols and Procedures 70 74.47 In the Moment Feedback 63 67.02 Delayed Feedback 55 58.51 Modeling 43 45.74 Review Recorded Session 31 32.98 Role Playing 25 26.60 Others 2 2.13 Open in a separate window Note. Other was reported as reviewing research articles, reviewing written reports, and providing feedback on feedback to others. The majority of our participants reported using Zoom as a virtual platform for supervision (78.72%), followed by Google Hangouts (27.66%), phone call (23.40%), FaceTime (12.77%), and other options (11.70%), which included Microsoft Teams, WebEx, Skype, and GoToMeeting. Responses were variable with regard to frequency of technological difficulties during virtual supervision: (1) never (13.83%), (2) rarely (43.62%), (3) sometimes (34.04%), and (4) often (8.51%). Satisfaction with Virtual Supervision When reporting on overall satisfaction with their supervision experience before COVID-19, the majority of participants in this sample indicated that they were satisfied (43.62%) or greatly satisfied (42.55%). During the virtual model, although most participants still reported that they were satisfied (35.11%) or greatly satisfied (35.11%), these numbers decreased. See Figure Figure22. Fig. 2 Percentage of Self-Reported Overall Satisfaction with Supervision before and during COVID-19 Prior to COVID-19, participants in this sample largely reported that they were satisfied (29.79%) or greatly satisfied (54.26%) with individual supervision. During virtual individual supervision, most of our participants still reported that they were either satisfied (34.04%) or greatly satisfied (36.17%; see Fig. Fig.3).3). In general, these ratings corresponded with high domain scores in satisfaction with individual virtual supervision. The mean satisfaction score across our participants was 42.23 out of 50 (range: 22–50), with a median score of 44. See Table Table55 for means for each individual satisfaction item and Fig. Fig.44 for individual supervision satisfaction score distribution. Fig. 3 Per