Abstract
Midtown Educational Foundation (MEF) is a non-profit with the mission to guide low-income urban youth in Chicago along pathways of success. MEF has two centers; Midtown for Boys and Metro for Girls. The focus of this study is on Metro specifically.
Volunteers play a key role in the organization’s mission. MEF enlists hundreds of volunteers to provide individual attention to students across 7-grade levels. The educational impact of Metro plays out on two levels. On one level, marginalized students benefit from one-on-one attention from college students and professionals who volunteer their time as tutors and mentors. On another level, these volunteers are exposed, many for the first time, to the trials of underserved youth, especially the disparity in educational resources and systems, a chasm that has increased during the Covid pandemic.
There is an opportunity to build on this symbiotic educational relationship between volunteer and student such that volunteers are inspired to draw from their experience working with students at Metro to seek ways to promote social impact in their professional sphere of influence. Since volunteer onboarding, training and engagement are activities through which the desired educational goals will be learned and shared, it is essential to understand how volunteers currently experience these activities.
Executive Summary
Background
Midtown Educational Foundation (MEF) is a non-profit with the mission to guide low-income urban youth in Chicago along pathways of success. MEF has two centers; Midtown for Boys and Metro for Girls. The focus of this study is on Metro specifically.
Volunteers play a key role in the organization’s mission. MEF enlists hundreds of volunteers to provide individual attention to students across 7-grade levels. The educational impact of Metro plays out on two levels. On one level, marginalized students benefit from one-on-one attention from college students and professionals who volunteer their time as tutors and mentors. On another level, these volunteers are exposed, many for the first time, to the trials of underserved youth, especially the disparity in educational resources and systems, a chasm that has increased during the Covid pandemic.
There is an opportunity to build on this symbiotic educational relationship between volunteer and student such that volunteers are inspired to draw from their experience working with students at Metro to seek ways to promote social impact in their professional sphere of influence. Since volunteer onboarding, training and engagement are activities through which the desired educational goals will be learned and shared, it is essential to understand how volunteers currently experience these activities.
Project Scope
Created and executed a research protocol to understand the experiences of volunteers and provide preliminary insights to Metro Achievement Program.
Framing the problem
The staff at Metro wear many hats as they coordinate multiple youth programs across 7-grade levels. At the same time, they onboard and train a steady stream of professional and college volunteers who come to Metro from several different referral channels. This dynamic results in two possible challenges:
Alignment on Metro’s Mission
Metro has a loosely defined process for onboarding and training volunteers. The staff does not have clarity on whether the process is effective in helping the volunteers understand and support Metro’s mission.
Variation in Tutor Experience
Metro does not have one set method of running the orientations and training and there is the possibility of variation in tutor satisfaction and engagement across programs.
Purpose Statement
How could we make the onboarding, training, and engagement experience better for tutors and as a result improve the experience for the students?
Method Overview
The study was completed using qualitative methods of design ethnography which center on exploring people, their context, and behaviors with the goal of producing insight for solutions rather than verifiable theory.
Design ethnography is a way of understanding the particulars of daily life in such a way as to increase the success probability of a new product or service or, more appropriately, to reduce the probability of failure specifically due to a lack of understanding of the basic behaviors and frameworks of consumers (Salvador and Anderson 1999).
The following design ethnographic methods were used during the interviews:
Photo Elicitation: a dialogic technique that prompts conversation between researchers and participants. The goal of photo elicitation is not to substitute images for words, but to use pictures to stimulate vivid, concrete, meaningful words (Moed 2012).
Participatory Design: A method in which the people destined to use the system play a critical role in designing it ((Schuler and Namioka 2017), original emphasis).
In addition, a survey was completed to check if the insights derived from the qualitative research aligned with satisfaction and engagement levels across the volunteer base.
Conclusion
Interviews with staff were conducted to outline the expected tutor experience. The four main stages in which tutors engage and the accompanying sub-activities for each, shown in parentheses, include:
- Deciding (talking to friends, doing online research, visiting Metro, promoting or inviting friends)
- Onboarding (applying, interviewing, tour, statement of values, attending in-person orientation)
- Weekly Training (Socializing, snacking, waiting, listening, sharing)
- Weekly Tutoring (Goal setting, meeting parents)
Throughout the stages of engagement, the participants demonstrated several patterns in how they determined the value of their experience. The five “modes of meaning” that seem to be connected to participants’ satisfaction and sense of fulfillment are:
- Taking the initiative
- Resonating with Metro’s mission
- Teaming to better serve the students
- Social bonding with students and other volunteers
- Reflecting
The extent to which the staff at Metro design concrete activities that foster these five modes of meaning, the more satisfied the volunteers will be with the experience. Reflection is especially important because it is the mental feedback loop that allows volunteers to be more intentional in their work with students.
Future Work
The work to date has addressed the social issue targetting the experience of volunteers who mentor students, using Chicago as a case study. The outcome of the project has merits based on the surveys’ responses; these include:
- Improved volunteering approach which helps to balance the intention and expectation of tutors.
- Having an understanding of a volunteer’s stages of engagement in Metro. This is important in order to optimize the amount of effort they put into their tasks.
In the future, more data can be collected, i.e, the sample space is increased beyond 67 so that the response rate can exceed 50%.
Additionally, in the next phase of research, the scope of research can extend to students and parents (not only tutors), in order to balance the perspective between training/teaching and learning in the Metro Achievement Program.
Technical Report
Project Timeline
| Timeframe | Task |
|---|---|
| August 2022 | Researched and trialed ethnographic design methods |
| September 2022 | Completed staff interviews and observations of onboarding. Mapped tutor journey. |
| October 2022 | Designed a research plan. Created coding structure. |
| November 2022 | Completed tutor interview. Coded data. Ongoing analysis. |
| December 2022 | Synthesized data into framework. Prepared presentation and report. |
Research Questions
| What do we need to know? | Why do we need to know this? | What kind of data will answer the question? |
|---|---|---|
| When Metro was first explained, what about the explanation was attractive? | To assess what draws people in. Any connection to mission AVIP? | Interview; Emotion Projective |
| Do tutors have a stopping point in mind when they start? | To assess the initial level of commitment | Interview |
| What are positive and negative experiences related to the onboarding process? | To assess onboarding effectiveness. | Interview; Emotion Projective |
| How do tutors understand Metro’s mission? | To discover alignment or discontinuity between what the organization states is the mission and how volunteers understand it | Interview; Survey to assess alignment of specific aspects of the mission |
| Does the actual experience of tutors meet the expected experiences? Were workarounds needed? | To assess onboarding effectiveness | Interview; Survey |
| What are positive and negative experiences related to weekly training? | To assess training effectiveness | Interview; Emotion Projective |
| What are positive and negative experiences related to time with students? | To assess engagement effectiveness | Interview; Emotion Projective |
| What could Metro do to improve the volunteer experience? | To identify opportunities for improvement | Co-design within the context of the interview |
Method Overview
The following research methods were completed in parallel:
- Observation of onboarding and training activities.
- Three interviews with staff to map the volunteer journey.
- Survey to profile the volunteer base and gauge satisfaction and level of engagement.
- Interviews
- Utilizing photo-elicitation to understand negative and positive emotions associated with activity blocks
- Participatory questions to drive an optimized experience.
Participant selection, sample size, recruitment strategy, and eligibility inclusion/exclusion criteria are outlined below:
- Part 1 - Survey tutors to assess satisfaction at the three phases of
onboarding, training, and engagement
- Purpose: Assess the degree of satisfaction and engagement to guide and check qualitative research|
- Participants: Representative approach, volunteers across three programs
- Sample size: 34 (There are 67 total volunteers. The Survey met the goal of a 50% response rate. )
- Recruitment strategy: Ask Metro to fill in the form as a mandatory activity at weekly training. QR code.
- Eligibility criteria: Include all volunteers across three programs.
- Part 2 - Interviews to identify tutors’ activity blocks and
associated mindsets and emotions connected to the activities.
- Purpose: Understand tutors’ emotions, behavior, experiences, and phenomena at each stage of the volunteering experience
- Participants: Representative sample to study new tutors who are professionals working in Chicago
- **Sample size (Guest, Bunce, and Johnson 2006): 9 tutors, 3 each from ONE, MAP, COP. 1 extreme user (2+ years volunteering).
- Recruitment strategy: Metro staff will recruit participants purposefully, inviting participants from the representative group who differ in meaningful ways such as how they heard about Metro and their reason for volunteering, which is known from the initial interview.
- Eligibility criteria: Include: Tutors, new in fall 2022. Exclude: Advisors and returning volunteers
Survey Results
Sample size (responses): 36
Time: 10/21/2022 - 11/28/2022
Method Summary:
The survey was designed to assess the degree of satisfaction and engagement of the Metro Volunteer base and check if qualitative research results are representative of the volunteer population. Since the total number of Metro volunteers is relatively small (67), a free text form was added to each Likert scale. Verbal analysis was conducted on each question to connect to qualitative research insights.
Conclusions:
The survey showed that participants were satisfied with the volunteering experience and process. This conclusion aligns with qualitative research insights. There is no pattern with dissatisfaction factors through cross-aggregation analysis.
24 participants said the reason for volunteering is “to support equity in education”.
Only three participants selected “Strongly disagree (1)” and middle “(3)” at the question; I can see myself volunteering at Metro next year” because they would move out of Chicago.
At least 30 people out of 36 answered “Strongly agree (5)” at the question: see a clear link between my volunteer work and Metro’s mission at each mission: academic excellence, virtue development, individual attention, and parental engagement.
Complete survey results are included in Appendix A.
Analysis and Synthesis of Qualitative Data
A coding structure was set prior to interviews which consisted of tags for:
- Program (ONE, MAP, COP)
- Phase (Entice, Enter, Engage, Extend)
- Activities (Past Volunteering, Discovering, Deciding, corresponding, Applying, Reviewing Statement of Values, Touring, Interviewing, Orienting, Training, Socializing, Tutoring, Meeting Parents, Prepping, Commuting, Field Tripping, Promoting, Snacking, Transferring Knowledge).
Data snippets from transcribed interviews were captured and categorized under each research question. Data snippets were transferred to a digital whiteboard and initially clustered by activity. Each interviewee’s comments were captured on a different colored snippet. This allowed for a visual check to ensure emerging themes were relevant across programs and individuals.
Headlines were applied to clusters. It was at this stage that the “modes of meaning” framework began to emerge. Data was then re-clustered under modes rather than activities to ensure there was evidence for recommendations.
Through this research, a framework of meaning-making was synthesized. Three aspects of the human person are involved in meaning-making. The volunteer apprehends Metro’s mission in her intellect. The volunteer acts on this understanding using her talents. Finally, the volunteer has an emotional response to the experience. The modes of meaning are not chronological and discreet like the stages of engagement but happen throughout the volunteers’ time at metros and grow iteratively.
Meaning-making already starts in the volunteer’s decision process to
apply to Metro. Once in the door, the potential for meaning the
volunteer expects to find at Metro is realized and reinforced.
A summary of insights and recommendations by phase is included in the appendix.
In conclusion, the extent to which the staff at Metro design concrete activities that foster these five modes of meaning, the more satisfied the volunteers will be with the experience. Reflection is especially important because it is the mental feedback loop that allows volunteers to be more intentional in their work with students.
Appendix A: Survey Results
Table Ⅰ. Background Characteristics of Survey Participants
Table II. Five-Likert questions and results:
(1 = Strongly Disagree, 5 = Strongly Agree)
| Question | 1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
|---|---|---|---|---|---|
| I am satisfied with the training I have received so far | 0 | 0 | 5 | 11 | 20 |
| I understand my role and the responsibilities of my job | 0 | 0 | 1 | 7 | 28 |
| The role I am doing matches the role described to me in orientation and interviews | 0 | 0 | 1 | 3 | 32 |
| Volunteering at Metro is a good use of my time | 0 | 0 | 0 | 3 | 33 |
| Volunteer work at Metro is well-coordinated | 1 | 0 | 0 | 4 | 31 |
| I am proud to be volunteering at Metro | 0 | 0 | 0 | 1 | 35 |
| I would recommend Metro to people I know as a great place to volunteer | 0 | 0 | 0 | 3 | 33 |
| I can see myself volunteering at Metro next year | 1 | 0 | 2 | 6 | 27 |
| I can see a clear link between my volunteer work and Metro’s mission of academic excellence | 0 | 0 | 1 | 3 | 32 |
| I can see a clear link between my volunteer work and Metro’s mission of virtue development | 0 | 0 | 1 | 4 | 31 |
| I can see a clear link between my volunteer work and Metro’s mission of individual attention | 0 | 0 | 0 | 3 | 33 |
| I can see a clear link between my volunteer work and Metro’s mission of parental engagement | 0 | 0 | 1 | 5 | 30 |
Table III. Other questions and results:
| Question | to fulfill a school or work required | to build a social network | to support equity in education | to gain experience related to my profession | Other |
|---|---|---|---|---|---|
| My top reason for volunteering is: | 2 | 4 | 24 | 1 | 5 |
| Question | Training | Interaction with students | Interactions with staff/ volunteers | Interaction with parents | Other |
|---|---|---|---|---|---|
| What I enjoy most during my time at Metro is: | 3 | 30 | 0 | 0 | 2 |
Appendix B: Insights and recommendations by phase
Deciding
Volunteers appreciate convenience
- Lifestyle fit is important. They recognize they have time. “I don’t have anything stopping me” “I could do more important things with my time than just sitting on my phone after work”
- Five of the volunteers decided to volunteer after a significant life change (1 started a new job, 3 moved to the city, 1 experienced the death of a friend) “It’s been a nice way to get like a better feeling for the city”
- Concrete expectations are appealing. “It was just two hours a week”
- The potential to make personal connections to students over the course of the year is important.
- Several commented on Metro’s reputation in the city and among their friends. “It kept on showing up. Some describe it as the best-kept secret in Chicago.”
- Several volunteers were proud to share Metro at their place of work.
- Think about the right place for transmitting different aspects. Since most come through friends, how can Metro better equip existing volunteers to promote to friends? Are there social networks for people who just moved to the city? Promoting Metro there may lead to more volunteers applying.
Commitment is about mission fit
- Several spoke about volunteering as a way of life instilled in them from their parents. Common phrases when speaking about the motivation for volunteering are “giving back”, “being part of something bigger than themselves” and “supporting the community”
- 23 of 34 surveyed (2/3) say their main reason for volunteering is promoting equity in education.
- At the same time, volunteers are secondarily looking for a personal impact “I want to grow personally”
- The website does the job but doesn’t have the same heart as the tour and personal testimonies. Instagram is better. Several tutors were surprised by how professional Metro was. Perhaps a virtual tour helping potential volunteers see the personal impact before arriving would draw more in.
The tour is about education, not recruiting
- Several volunteers were already thinking of friends to invite as they applied.
- Some volunteers came to tour and interview already knowing they wanted to commit. “I walked into it understanding what the organization was, knowing that I wanted to commit to volunteering.”
- Stories are powerful.
- From one tour observation: [staff name removed] did a great job sharing stories on her tour. You could tell she knew the volunteers and the girls. If this storytelling is not across the board, encourage it.
- Consider leveraging the volunteer’s sense of initiative when deciding by encouraging her to invite friends and making it easy for her to spread the word.
Onboarding
Volunteers speak Metro’s language
- All participants incorporated Metro’s mission language into their description of the program using words like values, habits, character, and individual attention.
- Several volunteers called out the parent programs as a surprising and impactful aspect.
- Volunteers connect with the individual attention aspect of the mission.
- Stories and testimonies during the onboarding process were well received.
Tutoring in good faith
- The statement of values did not stand out as either positive or negative. Three participants didn’t remember it. Three recalled it vaguely. One referred to it as the 10 commandments.
- Participants did recognize Metro as a faith-based program and connected with this identity on some level. “Positive Environment” “Similar values”
- Common words used to describe the staff were “open” “friendly” and “professional”
- Those who share the same faith are reinvigorated. “At a place like Metro, you start to remember, this is my faith. I do have this to lean on.”
- Those who have different faiths feel accepted. “Metro talked a lot about the values and connection with the church, but they were very open about, like, I feel like they were it was just like very honest”.
Practice over theory
- Most participants felt underprepared on the first day.
- Overall, participants perceived the onboarding as heavy on the theory/mission and lacking in practicality.
- Some found the onboarding steps redundant, especially those who were already decided before going on the tour. Given the tour is mission-focused, the orientation can be heavier on the practical.
- Several suggested adding role play to the first orientation.
A renaissance for Renaissance
- Participants expressed an eagerness to use it more and recognize its potential but lack clarity on how to leverage it.
- Renaissance is the only true pain point.
Training
“Nothing sparks” for some and inspires change in others
- Some volunteers expressed feeling “talked at”
- A few commented in the survey that the training is redundant.
- The expectation is that it helps them in tutoring (because it’s called training) but some like that they learn too…some don’t care…but most see it as slightly different than expected.
- Stories: trying an activity with boyfriend, changing social media patterns, “makes me think”
Training as a time to ease in
- Two mentioned they are happy it’s first so they don’t miss the girls if they are late. They appreciate it as a transition time to change their mind from work to tutoring.
- Many appreciate snacks
- Professionalism is noticeable. Several surveys comment on emails as helpful in particular.
When in doubt, tell a story
- Trends in responses suggest many volunteers prefer stories and practice to theory
- Several suggest role-play
- Most we spoke to want more training on renaissance
Training happens throughout
- Sharing knowledge among volunteers is an effective and fulfilling form of training.
Tutoring
Individual attention is given within the context of a team
- Most bonding happens in the classrooms. Sense of team or camaraderie in the rooms.
- “We bond when we are trying to figure out what we are doing” (one tutor)
- Tutors turn to each other to play to strengths
- One tutor appreciated talking to the advisor about the girls
Taking Initiative increases fulfillment
- When asked to tell a story of when they felt accomplished, most tutors spoke to times they went beyond the normal. Teaching a new skill. Prepping outside of Metro. Bringing in flash cards.
Connectedness is a measure of the impact
- “The fact that she likes me means I’m doing a good job, right?”
Tutors grow alongside students
- “Seeing my like mentees’ progression and just like forming like a bond with her is helping me probably more than it’s helping her”
- Sense of gratitude
- “I want to communicate with people of all ages and learn how to treat every person like a person”
License
The author of this technical report, which was written as a deliverable for a SoReMo project, retains the copyright of the written material herein upon publication of this document in SoReMo Reports.