We are excited to announce the relaunch of the Southern California Evaluation Association (SCEA)! This year a group of evaluation professionals came together to envision how SCEA could address the interests and needs of those involved in evaluation in the Southern California area. We have recently updated the website and are currently planning events and other programming for evaluation professionals that provide professional development, collaboration, and networking opportunities. We are also going to be surveying our member list to get input on how SCEA can meet their needs. 

If you are interested in learning more about SCEA there are a few ways to connect with us:  

  • Review our website and learn about our mission statement, leadership team, and upcoming events
  • Follow us on LinkedIn and Twitter
  • Join our mailing list to hear updates about the group
  • Email us at scea.us.info@gmail.com

We look forward to connecting with you!

Advertisements

SCEA Event Recap: Data Visualization 101 for Evaluators (April 28, 2018)

On April 28, SCEA organized and facilitated an event focused on data visualization at WeWork in Hollywood. The topic came from our members, who indicated on a membership survey in 2017 that this was an area of professional interest. Two groups of presenters shared their expertise with us: Stephen Maack and Arlene Hopkins (GIS and Evaluation), and Alma Boutin-Martinez and Hilary Molina (Infogram and Tableau). It was a great time of learning and networking! Special thanks to Laura Keene for offering us a venue at WeWork to host our event.

 

Data visualization resources identified as useful by our presenters:

It was great seeing old and new faces. We hope to see you at our upcoming event in August! More information to come soon.

SCEA Member Spotlight – Michelle Molina

MicheleMolinaSCEA

Education:
University of California, Los Angeles – Bachelor of Arts in Psychology

University of California, Los Angeles – Master of Arts in Education, Social Research Methodology

What is your current position(s) and what do you do?

I’m an independent consultant who partners with non-profits to collect data so that they can show their impact, learn, and make better decisions. By helping the organizations consider their context, my aim is to help them refine their goals so that they can collect data that is meaningful.

A lot of my work comes from subcontracting agreements with other evaluators. I enjoy doing this because I get an inside look at how others work with clients, all while strengthening my skills. I often help others with qualitative and quantitative analysis (including social network analysis), data visualization, report writing, and facilitating discussions. 

The project I currently spend the most time on is a Kaiser Permanente initiative funding several coalitions implementing healthy eating and active living strategies in their communities. On this project, the evaluation team works really closely with the grant management team. Together, we regularly discuss the progress each community has made and consider how we can better support their work. Regularly our data collection is used to inform internal conversations within each coalition. For example, we recently collected survey data on coalition functioning. We took these results to coalition meetings and helped facilitate discussions around each coalitions strengths and weaknesses. These sessions will not only allowed the coalition time to reflect and identify opportunities to improve, but also gave us data on how coalitions function within the initiative.

What led you to the field of evaluation?

The programs I participated in had a huge impact on my life and helped guide me towards higher education. I was interested in giving back, but also curious about what made programs effective.

Finding evaluation itself was a bit of a stroke of luck. Early after transferring to UCLA, I had to take a 6 unit class that was notorious for being difficult. I decided to look for another class with a lot of units in order to better manage my course work. The class I chose was the first undergraduate class Dr. Marvin Alkin offered on program evaluation.

I fell in love with evaluation. At the time, I was just learning about research methods and how research is used to grow our body of knowledge. I was quickly able to see how evaluation would allow me to use those technical skills in a real world setting that could have an actual impact on the lives of others. I often think about how lucky I am to have found something that combines a lot of my interests. I will always be thankful to Dr. Alkin!

 What has been your favorite moment in your career so far?

Last year I helped a 4H group in Tucson, Arizona build their evaluation capacity by considering their context. We worked together to develop a theory of change, thought about their stakeholder’s perspectives, identified assumptions, thought about what lifecycle stage their program is in, and more. We reviewed the evaluation tools they are required to implement and identified outcomes they were still interested in exploring.

Recently, we planned an Evaluation-Skill-a-Ton where they can teach their youth about evaluation while collecting data. For example, the youth rotated between a few stations where they would respond to prompts aimed at answering the evaluation questions. Afterwards, the youth were asked to review the data provided by others and think about how to interpret it. The 4H coordinators were excited about this plan and thought it could provide more meaningful information than a survey could. I will be hearing back from them about the event soon.

What motivates you at work?

The organizations I work with are filled with dedicated and passionate people who are working to improve the lives of others. I help give them the tools they need so that they can tell a compelling story to others about the importance of their work. Plus, they are able to use the data to reflect upon what’s working and what’s not, allowing them to improve and make important decisions. So, when data collection and analysis starts to feel monotonous, I remind myself of how helpful the ultimate product might be.

If you could give advice to young professionals in evaluation, what would it be?

Continue learning. Stay on top of what’s going on in the field. Things change and so will evaluation. By keeping up with trends you’ll be able figure out what the best approach is faster. There will be situations where clients will ask you to use an approach. You’ll need to have a general understanding of it so that you can effectively advocate for or against it. Also, learn new skills that you’re interested in even if you’re not sure how they might be connected with evaluation. For example, I recently started dabbling in coding, I was able to use my very limited knowledge to help easily figure out how to use Kumu.io (a social network analysis software) when I was asked to use it on a project. Furthermore, you can learn from most situations (including missteps). For instance, you can reflect on conversations that didn’t go well to improve next time. As long as you have an eye out for lessons learned you can continue to strengthen your skills.  

Connect with other evaluators. Building a network is important in any field. Every project that I have enjoyed being a part of came about in some way because of the network I’ve built. Either they know of me through the evaluation book club. Or, received a recommendation from a friend to work with me. Luckily, most evaluators are kind people who care about the work they do. Evaluation is still a relatively small field and many people find their way to it indirectly. So, evaluators are usually more than happy to talk to other evaluators. If you reach out, you’re more than likely to receive a positive response.

What are your favorite resources for evaluators?

The Workbook for the Systems Evaluation Protocol – I’m really interested in helping organizations think about their context and develop a theory of change. This protocol offers guidance on how to take organizations through that process.  (https://core.human.cornell.edu/documents/Workbook_1.1_covered.pdf)

Netway – This helps you easily put a theory of change on paper. It is from the same people who developed the Systems Evaluation Protocol. (https://evaluationnetway.com/)

What do you like to do in your free time?

I spend a lot of my free time relaxing. I like to read books, listen to podcasts, watch YouTube videos, hang out, have a beer, and sometimes I also go on hikes.

RSVP for the Data Visualization 101 for Evaluators event on April 28!

Join us on April 28, 2018 from 1:30 – 4:00 pm in Hollywood for our event “Data Visualization 101 for Evaluators”! We have an exciting program that will feature two groups of presenters. They will discuss the use of data visualization techniques, such as GIS, Tableau, and Infogram, in their evaluation work.

Click here to RSVP!

1) Overview of GIS in Evaluation
Stephen C. Maack and Arlene Hopkins
DBA REAP Change Consultants

About the speakers:

Dr. Stephen C. Maack, Owner and Lead Consultant of REAP Change Consultants, has a Ph.D. in Anthropology with a specialty in Social Change.  He became interested in Geographic Information Systems (GIS) around 1978 when he was working as a City Planner at St. Paul (MN) City Planning.  He first had a colleague run a GIS process (geocoding) while liaison for the City of St. Paul to the U.S. Census Bureau during the 1980 census. He has watched the field grow from highly specialized programs run only on mainframe computers to a variety of specialized but accessible application available on multiple platforms, including laptops, with multiple uses.  He finally learned to program in ArcView by ESRI, the major commercially available GIS software, in 2002/2003 when he took GIS courses at Rio Hondo College and has offered computer mapping services through his firm since about 2004. He facilitated a workshop on GIS and Anthropology in 2005 and between 2009 and 2012 he and Arlene Hopkins offered a workshop at AEA Annual conferences titled “Introduction to GIS and Spatial Analysis in Evaluation”.  He, Ms. Hopkins and Aaron Wilson Kates are offering two half-day workshops titled “Computer Mapping (GIS) Applications in Evaluation” at the AEA 2018 Summer Institute in Atlanta on June 19, 2018.

Arlene Hopkins, Principal of Arlene Hopkins and Associates, is an architect, evaluator and educator.  Among her public service roles, she presently serves on the Board of the Santa Monica Public Library.   As a member of the Resilience Thematic Group of the International Union for the Conservation of Nature (IUCN), Graduate Women in Science, American Evaluation Association, American Library Association and the California Native Plant Society, among others, she maintains a trans-disciplinary approach to thinking boldly – and with an empirical basis, about how we need to adapt and innovate our cultural systems to both regenerate ecological systems and the equitable commons.  GIS is an essential tool for this work. She regularly attends ESRI conferences, and she has taken courses in GIS at Santa Monica Community College in Santa Monica, CA. She co-facilitated the AEA Annual conference workshops titled “Introduction to GIS and Spatial Analysis in Evaluation” between 2009 and 2012. She will be co-facilitating the half-day workshops titled “Computer Mapping (GIS) Applications in Evaluation” at the AEA 2018 Summer Institute in Atlanta on June 19, 2018.

About the presentation:

GIS is an underused technology in evaluation, with only two articles about it published in the American Journal of Evaluation.  It has most frequently been used in public health and environmental program evaluation. While many people are now familiar with Google Maps and the use of computer maps for navigating around cities and across the country with the help of the related Global Positioning System (GPS) technology, few understand how GIS can be used in evaluation.

During this brief overview presentation attendees will:

  • Hear a brief explanation of how GIS works;
  • Learn about several of the selection and analysis features of GIS software that make it a potentially powerful software tool in evaluation;
  • Hear how GIS can be used in program and policy evaluation for needs assessment, visualizing spatial relationship of disparate data, combining data from program and secondary sources, showing data correlations, mapping changes over time, tracing networks, displaying evaluation results, statistically testing evaluation results, and combining quantitative and qualitative data;
  • Gain a knowledge of specific examples of GIS uses in health, public safety, social services, fiscal/economic, environmental, education and other program and policy evaluations.   

2) Survey Data Visualizations Using Tableau and Infogram
Alma Boutin-Martinez and Hilary Molina
Fielding Graduate University

About the speakers:

Dr. Alma Boutin-Martinez is the Senior Institutional Research Analyst at Fielding Graduate University. In this role, she provides data and analyses to support program evaluation, institutional and program accreditation, learning assessment, and institutional decision-making to advance the university’s mission. She has created dashboards in Tableau that examine strategic enrollment management and alumni survey data. She has also used Tableau to present diversity and inclusion survey data. Before joining Fielding Graduate University, she conducted educational program evaluation research on the multi-campus UC online program. She also worked closely with Mosaic Network, Inc. to investigate how the Grant Evaluation and Management Solution (GEMS) system was used at the Partnership for Children of Cumberland County (PFC), and the degree to which participants perceived that it was improving the efficiency and outcomes for PFC. She earned a Ph.D. in Education from the Gevirtz Graduate School of Education, University of California, Santa Barbara. She has published research on parental engagement, literacy, academic intrinsic motivation, and resilience.

Hilary Molina, M.S currently works for Fielding Graduate University as the Director of Alumni Relations. She is responsible for the planning and implementation of university-wide strategic initiatives that engage alumni and provide tangible benefits to graduate and current students. Hilary’s experience in data analysis and evaluation began during her graduate work at California Polytechnic State University-San Luis Obispo. Her thesis research focused on the quantitative analysis of statewide exercise testing protocols and test results for firefighters, which was later used to propose California state standards. PowerPoint was the norm for static data visualization presentations at this time and when she transitioned into account management, she honed her skills in data analysis and professional presentations during her time in title insurance. It was not until Hilary began working in higher education with large-scale data that creating data visualizations became an integral part of her work. Her most recent alumni survey data continues to drive strategic planning for the university. Her office works closely with the institutional research department to develop, administer, and analyze alumni survey data. With the use of programs such as Infogram, Hilary has evolved her alumni survey data presentations into more dynamic and interactive demonstrations based on the audience and platform (e.g., print, PowerPoint, Prezi, website embedded, virtual)

About the presentation:

This presentation will examine data collected from the Fielding Graduate University Alumni Relations Survey. This presentation will include survey results disaggregated by program, alumni demographics, and describe how visualizations were created using Tableau and Infogram.

During this presentation, attendees will:

  • Obtain a brief overview of Infogram and Tableau;
  • Understand when it is best to use a bar graph, pie chart, and line graph;
  • Review the Data Visualization Checklist created by Stephanie Evergreen and Ann Emery;
  • Gain knowledge of best practices when creating dashboards; and
  • Understand how data visualizations created with Infogram and Tableau were used to inform decision making and evaluation.

SCEA April 28 2018 Event_Data Viz 101

Michael Quinn Patton on Evaluation Science: How Knowledge is Validated, How the World is Changed, and How Nobel Prizes are Awarded

Don’t miss Michael Quinn Patton’s webcast lecture, “Evaluation Science: How Knowledge is Validated, How the World is Changed, and How Nobel Prizes are Awarded.”

The event will be held at the Albrecht Auditorium at Claremont Graduate University on Thursday, April 12 at 4:00pm.

The lecture is also available as a webcast. Sign up here!

6ee86467-b670-4b31-bc97-372cbeda68e3-large

SCEA Event April 28: Data Visualization 101 for Evaluators!

SCEA April 28 2018 Event_Data Viz 101

Please mark your calendars and join us for our upcoming SCEA event on Saturday, April 28th from 1:30 – 4:00 pm in Hollywood. We have an exciting program that will feature two groups of presenters. They will discuss the use of data visualization techniques, such as GIS, Tableau, and Infogram, in their evaluation work.

GIS and Evaluation
Stephen C. Maack and Arlene Hopkins
DBA REAP Change Consultants

Survey Data Visualizations Using Tableau and Infogram
Alma Boutin-Martinez and Hilary Molina
Fielding Graduate University


Come mingle with other SCEA members, enjoy some light refreshments, and learn more about data visualization techniques.

Please RSVP to attend this event. Due to space constraints, attendance is capped at 25. Please click here to register. In addition to registration, we’re asking attendees to please contribute $5 that will go towards the purchase of snacks and beverages.

Have questions, comments, or concerns? Email us at events.scea@gmail.com.

More details on location, parking, and the agenda will be provided to those who register.

Thank you, and we look forward to receiving your RSVPs and seeing you at the end of April!

Contribute to SCEA by submitting a webpost!

SCEA Webpost Contribution Guidelines

The Southern California Evaluation Association (SCEA) is dedicated to advancing the field of evaluation by supporting and sharing topics of interest with evaluation practitioners and scholars within the Southern California region. SCEA is committed to disseminating content that provides perspective about the ever-evolving evaluation landscape and that will inform evaluation practice. We are a multi-disciplinary and forward-thinking organization that aims to address the ongoing interests and needs of those involved in evaluation in the Southern California area.

We are interested in timely and engaging topics that are informed by and provides open access to evaluative scholarship. If you have experience in the evaluation field — as a seasoned or novice evaluator — we invite you to share a topic of interest. Please consider submitting a draft post for our website. The post will also be disseminated via the SCEA Twitter and LinkedIn Group!  

Your post should include:

  • A title that is both descriptive and informative
  • A brief introduction of yourself (1-3 sentences). Don’t forget your professional affiliation!
  • Purpose of the post (e.g., “I want to share…”, “I’d like you to know more about…”)
  • Study summary: include the purpose, methods, main arguments, and scope of your topic of interest. The summary should convey the results and conclusions of your topic. You are encouraged to include any relevant images (provided you have use permission) that you feel would enhance the reader’s comprehension of your topic.
  • Lessons learned (e.g., what is the significance or implications of your study, what did you learn, what could you have done differently)
  • Optional: 1-2 links to resources so that readers can learn more about the topic (e.g., website links, journal articles).
  • Draft submissions should be limited to 300-500 words and written in first-person prose.

Need some ideas? Here are some topics our members stated they would like to hear more about:

  • Strategies for communicating evaluation findings (e.g., data visualizations)
  • Demonstrations of evaluator tools (e.g., Omigraffle, Dedoose)
  • Issues unique to independent consulting
  • Project management techniques
  • Strategies for local community engagement
  • Sector-specific issues and strategies (e.g., health, environmental, education)
  • Quantitative methods & tools (e.g., social network analysis, R)

For an example of a contribution, please see this post by SCEA member Dr. Katy Nilsen.

Please submit materials to scea.comm@gmail.com. We look forward to hearing from you!

 

SCEA Member Spotlight: Jacob Schreiber

Jacob_SchreiberName: Jacob Schreiber

Education:
California State University, Long Beach – Master of Arts in Applied Anthropology
University of California, Irvine – Bachelor of Arts in Psychology and Social Behavior

What is your current position(s) and what do you do?
I am currently a Research/Evaluation Analyst in the Department of Medical Education at the Keck School of Medicine of USC (KSOM). In this role, I support the Director of Research and Evaluation to shepherd student and faculty projects towards presentation and publication. I also track the completion of research/evaluation projects, coordinate research/evaluation workshops and meetings, analyze and write reports, and disseminate those reports to various stakeholders throughout KSOM.

What led you to the field of evaluation?
I took a program evaluation course as a component of my master’s program at CSULB. I enjoyed the applied aspect of the field. I knew that I wanted to do something that made a difference, and to me the most important way to do that is to ground decision-making with data and facts. Although I don’t have a formal background or degree in evaluation, I enjoy that it brings many different disciplines together to solve real-world problems.

What has been your favorite moment in your career so far?
I am conducting a longitudinal qualitative study of student feedback of the medical school curriculum over a four-year period. When I presented preliminary findings to our leadership, they were impressed by the methodology and surprised by the detail of the results. Working in a medical school with primarily MD’s and life scientists, it has been rewarding to show them how social research can be used to effect decision-making.

What motivates you at work?
Hearing how much the students appreciate what our office does, and that they can see changes based on the feedback they provide us is a big motivator. With my background in Anthropology, I also enjoy knowing that the research I do effects the way that training is conducted at medical schools to produce doctors that are treating the whole person: taking cultural, socio-economic, and personal experience into consideration when treating their patients in the future.

If you could give advice to young professionals in evaluation, what would it be?
As a young professional in evaluation, myself I’m somewhat hesitant to offer too much advice. But if you are still in school, I’d recommend building your skillset as a mixed-methods analyst. I have found that being open to taking on a variety of responsibilities that are sometimes outside of my training, and being able to interact with people from other disciplines to understand their theoretical perspectives and methods has been a key to growing in the field. If you are in the market for an evaluation job, I recommend being persistent and getting creative with your search strategies. There are great organizations hiring people with evaluative skillsets that aren’t necessarily posting a position for an “evaluator.” That was something I learned after graduating with a degree in Anthropology, which is a discipline with skills that fill many organizations’ needs, but you will rarely see in a job title. I think that can apply to many of the social sciences.

What are your favorite resources for evaluators?
In my opinion, AEA remains the best resource for evaluators, and their job boards were where I found my current position. I think that it’s also important to keep in touch with related disciplines, so I still follow trends at the American Anthropological Association and the American Psychological Association.

What do you like to do in your free time?
I’m a big nerd who loves to play Dungeons & Dragons and video/board games. I also enjoy listening to and collecting records, seeing live music any chance I get, and singing karaoke.

SCEA will be posting member spotlights every other month on our website. Stay tuned to learn more about SCEA members and check out our events page for ways to connect with other local evaluators.