Example of Outline for Quantitative Research

Proceedings of the 2013

Western Region Research Conference on the Education of Adults

October 3 – 5, 2013 Seattle University Seattle, Washington

Proceedings of the 2013 Western Region Research Conference on the Education of Adults October 3 – 5, 2013 Seattle University Seattle, Washington

Authors retain the copyright for all papers. No paper may be reproduced without the author’s permission. Contact individual authors for permission to reproduce materials contained in these proceedings.

Welcome to the 2013 Western Region Research Conference on the Education of Adults (WRRCEA). In addition to presentations of papers, roundtables, and symposia that discuss theory and practice for educating adults, WRRCEA provides an opportunity to learn, debate, explore ideas, collaborate, and dialogue with professors, graduate students, researchers, teachers, trainers, and administrators. We extend a special thank you to Dr. John Dirkx for his keynote address, “Reimagining Adult Education: The Role of Emotion-laden Experience.” Since its origins in 2007, WRRCEA has been a collaborative effort of institutions and organizations engaged in the study and practice of adult education in the western region of the US and Canada. This year’s conference is co-sponsored the Seattle University Adult Education and Training program, the British Columbia chapter of the Canadian Association for the Study of Adult Education (CASAE-BC) and the Puget Sound chapter of the American Society for Training & Development (ASTDps). Thank you to these organizations for their generous support and assistance in organizing panel discussions on workplace and community education. We also wish to thank the universities who have contributed to the success of WRRCEA. In particular, thank you to Western Washington University, whose faculty and graduate students first made this biannual conference a reality. We appreciate your continued support. Thank you also to San Francisco State University, Sacramento State University, the University of British Columbia, the University of Wyoming, and the WRRCEA Steering Committee without whom this fourth WRRCEA would not be possible. Finally, thank you to the faculty, staff, graduate student, and alumni volunteers from Seattle University who worked tirelessly to build a terrific conference. Special thanks to Amy D. Rose, who spearheaded the local planning committee during the 2012-13 school year as the William A. Allen Endowed Chair for the College of Education at Seattle University. We hope you enjoy your time at the conference and in Seattle. Sincerely,

Bob Hughes Associate Professor of Adult Education Seattle University October 2013

ii

WRRCEA 2013 Steering Committee Gretchen Bersch, University of Alaska, Anchorage Shauna Butterwick, University of British Columbia Sandra Daffron, Western Washington University Doris Flowers, San Francisco State University Stan Goto, Western Washington University Bob Hughes, Seattle University Vanessa Sheared, Sacramento State University Tom Sork, University of British Columbia

Local Planning Committee Amy D. Rose Amy Hitchcock Melinda Anderson Laura East Lindsey Kafer Sasha Lotas Megan Otis Ted Rice Anne Soerens

Sponsors The Adult Education and Training Program at Seattle University The Puget Sound chapter of the American Society for Training and Development (ASTDps) The British Columbia chapter of the Canadian Association for the Study of Adult Education (CASAE-BC) Special thanks to Rob McGowan, Nicole Forward, and Lisa Roy of ASTDps and Shauna Butterwick and Jude Walker of CASAE-BC

Participating Institutions Sacramento State University San Francisco State University Seattle University University of British Columbia University of Wyoming Western Washington University

iii

Table of Contents Papers Author

Title

Page

Judith A. Alamprese

State and Local Approaches for Bridging the Gap between Adult Basic Skills and Postsecondary Education

1

Nancy Chapko

Designing for Every Adult Learner

8

Larissa Chuprina Amy D. Rose

Standardized Tests in English Learning Assessment in Adult Education: Blessing or Curse?

15

Cynthia Clay

Crossing the Chasm from Classroom Training to HighPerformance Virtual Delivery

21

Joanne DeMark Christina Van Wingerden

What Makes A Socially Responsible College Student Leader? A Research Perspective

26

Aydin Yücesan Durgunoğlu Evaluation of a Technology-Enhanced Adult Literacy Hilal Gençay Program in Turkey: The Initial Findings

32

Alicia Emerson

Examining the Influence of Spirituality Among NonTraditional Female College Students: A Holistic Perspective

38

Gabrielle Gerhard Helen E. Burn

Non-Tenure-Track Community College Mathematics Faculty Learning Through Faculty Inquiry Groups

46

Stanford T. Goto Christina Van Wingerden Mikhail Burstein Bonnie Fallahi Glenda Langdon Sanja Babic

What an Experience! Identity and Community in a Graduate Course

53

Amy D. Hitchcock Mary Jane Oberhofer Anne Soerens Deborah Davolio

Behind the Classroom Door: A Process for Evaluating Teaching Practices in Volunteer Adult Literacy

57

Lindsey Kafer Christie Knighton

Professional Development in ABE Math Instruction: Seattle University’s Model of Best Practices Training for Washington State Instructors

64

Li Mao Donna Chovanec

An ESL International Student’s Service-Learning Experience: A Reflexive Dialogue Between Student and Instructor

70

Steven D. Monroe Dale A. Monroe

Exploring the Effect of Video Used to Enhance the Retrospective Verbal Protocol Analysis: A Multiple Case Study

77

iv

Author

Title

Page

Debora K. Nelli

Scholarship or “Just Art”? : An Assessment Rubric for Arts-based Inquiry

83

Laura Parson

Meeting the Adult Education Needs of Former Female Members of Polygamous Societies

87

Daniel D. Pratt John B. Collins

Mapping the Signature Pedagogies of Five Medical Specialties

94

Kevin M. Roessger

Using Gilbert’s Worthwhile Performance Model to Evaluate the Impact of Reflective Activities in a WorkRelated Skills-Based Learning Context

100

Carole Roy

Community-based Documentary Film Festivals: Their Role as Alternative Media

107

Jennifer Sorensen

Teaching Science Facilitation in a Community Organization

114

Max Sul

Students’ Perceptions of Problem Based Learning in New Intern Electronic Medical Record Training

120

Jennie Thede Jillian Douglas

Your Brain is Not a Bucket: Learning Through Experience

126

Christina Van Wingerden Kevin Majkut Sara Richards

Comprehensive Assessment of What College Students Gained Serving on Hiring Committees for Student Job Openings: Initial Survey to Transfer of Learning after Graduation

133

Christina Van Wingerden Ted Pratt

Ethics in Student Affairs: Research on Climate, Knowledge, Skills with a Proposed Training Intervention

141

Jude Walker

From Cheating to Learning: Academic Integrity via Adult Education

150

Author

Title

Page

Joanne DeMark Amy Appleton Olga Antonio

Character Strengths of College Freshmen Identified as Emerging Student Leaders

157

Mary DeWine Sabine Endicott Justine Francisco Johnny Gordon Kalvin Satiacum

Student Voice: An Ongoing Dialogue with Adult Development Learners

161

Roundtables

v

Author

Title

Page

Celia Farr

Surviving an Emotional Hijacking in the ABE/GED Classroom: An Exploration of Emotional Intelligence Through Experiential Learning Theory-to-Practice

164

Lindsey Kafer

Using LEA Instruction in Classroom and One-on-One Sessions with Beginning Literacy ABE Learners: Rationale, Results, and Suggestions for Practice

167

Katie Myers-Wiesen

The ABC’s of LGBTQ

170

Debora K. Nelli

“Minding the Gap”: Developing a Feminist Research Methodology Course

172

Debora K. Nelli Giovanni Milan Jared Ibarra

What do Multicultural Students “Know” about Graduate School: Focus Group Learning

175

Debora K. Nelli Jamie Olson Peterson

Arts-Based Needs Analysis: Transformative Potential for Minority Students & Programs

178

Leann Price

Inner Circle English and Attitudes Toward Varieties of English: An Analysis of Relationships of Power

181

Anne Soerens Alexa Wayman

Using Family Literacy to Reach Immigrant Mothers

184

Brandi Stevenson

Perceptions of Vocational Education

187

Ericka Turley

Being the Difference: Implications for People Who Want to Create Positive Social Change in the Workplace

190

Kellee R. Vess Valerie K. Ambrose Cecilia A. Teal

Deconstructing Online Collaboration: Connecting the Dots

192

Author

Title

Page

Mali Bain Shauna Butterwick Jude Walker

International Experiences and Colonialism: Points for Intervention

195

Ayesha Mian Donna Chovanec Misty Underwood Li Mao Evelyn Hamdon

Embracing the Spiral: Researcher Reflexivity in Diverse Critical Methodologies

203

Symposia

vi

State and Local Approaches for Bridging the Gap between Adult Basic Skills and Postsecondary Education Judith A. Alamprese, Abt Associates Abstract: Discussed are results from research on two state adult education initiatives and local adult education services in these states to prepare low-skilled adults to transition from basic skills programs to postsecondary education and employment. Introduction Postsecondary education is recognized as critical in preparing adults for gainful employment, career advancement, and economic well-being. (e.g., Butler, Beach, & Winfree, 2008; Carnevale, Rose, & Cheah, 2011). However, attainment of postsecondary credentials has been particularly challenging for adults who earn a General Educational Development (GED) or high school equivalency diploma, or who leave high school without the skills necessary to succeed in college (e.g., Goldrick-Rab, Harris, Mazzeo, & Kienzel, 2009). According to research conducted by Reder (2000), high school graduates attain associates’ degrees at almost twice the rate of GED recipients over five years: 54% versus 28%. Results from an American Council on Education longitudinal study of adults who obtained a GED in 2003 indicated that only 12 percent earned a postsecondary degree or credential within five years of their receipt of a GED (Patterson, Zhang, Song, & Guison-Dowdy, 2010). Among the factors affecting GED recipients’ success in postsecondary education is their ability to meet the academic demands of postsecondary courses (Guison-Dowdy & Patterson, 2011). The low postsecondary education completion rates of GED recipients have prompted adult basic education (ABE) state policymakers to implement state initiatives to improve local adult education services that can prepare low-skilled adults for success in postsecondary education. States have designed their own system-building initiatives in ABE transition to align policies, coordinate resources, and accelerate instructional services for low-skilled adults (e.g., CCWD, 2010; SBCTC, 2005), or have participated in foundation or Federally sponsored demonstration projects to test models of state systems for ABE transition (Alamprese, 2012; Pleasants, 2011; Roberts & Price, 2012). Although state approaches vary, most ABE transition service frameworks include the delivery of basic skills instruction that is accelerated in academic difficulty and in time required for completion; the use of information about high-demand occupations to contextualize instruction; and the provision of college and career awareness, academic advising, career advising, and support services (National Research Council, 2012; U.S. Department of Education, 2011; 2013). As more state adult education offices and ABE programs consider ABE transition services, there is a need for data about the effectiveness of ABE transition approaches and the types of services that can facilitate ABE learners’ enrollment in and completion of postsecondary education. This paper presents results from implementation research on ABE transition initiatives in Oregon and Washington State, two of the first states to implement transition initiatives. Discussed are key components of each of the state’s initiatives, and practices in ABE transition from ABE programs in these states. 1

Methods This paper is based on a subset of data collected as part of Abt Associates’ study of promising practices in ABE reading instruction to prepare GED-level learners to succeed in entry-level, postsecondary courses. In Abt’s study, candidate ABE programs were identified through: (a) a review of the literature on reading research, ABE transition, and postsecondary success, and (b) telephone interviews with adult education state directors in five states in the forefront of ABE transition to postsecondary education (Illinois, Minnesota, Oregon, Washington, and Wisconsin). Fifteen candidate ABE programs were identified, and five programs were selected as promising practices based on their: (1) engagement in reading instruction that was aligned with the reading demands of postsecondary education, (2) provision of advising, support, and college awareness for GED learners, and (3) preliminary results about learner outcomes. Two-day site visits were conducted to the programs during 2012, which involved individual interviews with ABE program directors, ABE reading and writing instructors, community college administrators, advising and counseling staff, and professional technical faculty. Topics addressed during the interviews were the implementation of the focal reading intervention, processes for addressing psychosocial skills, career and college readiness, and advising and supportive services. Instruction in reading and college readiness was observed in four of the five programs (as scheduling permitted), and materials used in instruction were reviewed in all programs. Programs’ learner outcome data were reviewed. Thematic and cross-case analyses of the interview and class observation data were conducted. This paper’s descriptions of the state initiatives in Oregon and Washington are based on reviews of literature about the initiatives and interviews with adult education state directors in the states during 2013. Information about the implementation of local ABE transition services is based on data collected during the site visits described above to four ABE programs, two in each state. State Initiatives and Local Practices: Oregon and Washington State Oregon’s OPABS Initiative In response to state leaders’ vision that all Oregonians have the skills and resources to achieve economic prosperity, Oregon’s Department of Community Colleges and Workforce Development (CCWD) developed the Oregon Pathways for Adult Basic Skills Transition to Education and Work Initiative (OPABS) in 2005 to: (a) increase the number of adult basic skills (ABS) learners who have a career goal and transition to postsecondary education, and (b) improve the quality of ABE programs so that there are systematic processes for ABS learners to transition to postsecondary courses and employment (Alamprese, 2007). OPABS has five components: instruction, college and career awareness, ABE program services, professional development, and data systems and evaluation. The instruction component involves six academically accelerated ABS courses in reading, mathematics, and writing that incorporate occupational applications from Oregon’s high-demand industries. The ABS courses, based on the state’s ABS Learning Standards and basic skills academic standards, integrate occupational information using a survey approach. While information about high-demand career areas is the focus, a range of occupational information is integrated into the courses. Each course has a criterion pre-test score and a standardized course format. The courses are designed at two skills 2

levels—Pre-Bridge (6th-8th grade equivalent) and Bridge (9th-12 grade equivalent)—and include 60 lessons per course. The course materials include a syllabus, course outline, lesson plans, instructor notes, and learner materials. A College and Career Awareness (CCA) course assists learners in identifying goals for further education and a career. This course provides information about the characteristics and educational requirements of jobs in local labor markets, the educational programs in the colleges related to these jobs, and learners’ skills, interests, and educational and employment background. CCA also has information about the college application process, admissions testing, and applying for financial aid. As part of OPABS, ABS programs have restructured their processes for learner recruitment, orientation, assessment, and course placement. OPABS professional development has required instructor training to prepare ABS faculty to teach OPABS courses (CCWD, 2010). OPABS data collection has included instructors’ completion of logs during their first year in teaching OPABS courses to assess their fidelity of implementation of thesecourses. OPABS learners complete customized course evaluation forms on which data are collected about their perceptions of the utility of course content and processes. Data on learners’ development of basic skills development and enrollment in postsecondary courses are reviewed as they are available. CCWD is part of a state project to develop a longitudinal data system that will facilitate analyses of ABS learners’ education and employment outcomes. Examples of OPABS Approaches Examples of two colleges’ implementation of OPABS instruction are provided to illustrate strategies used to facilitate learners’ academic success. In 2011-2012, Mt. Hood Community College implemented a cohort-based class for OPABS to promote learner retention and enrollment in postsecondary courses. Learners in OPABS Bridge courses participated in all of the same OPABS classes and signed a class contract regarding attendance. Two OPABS instructors managed the cohort. Each instructor taught two classes (Math and Writing; Reading and CCA) and each assisted with the other’s classes using a partial co-teaching model. Additional writing assignments were incorporated into the Reading and CCA classes to better prepare learners for the writing demands of college courses, and homework that extended the class instruction was assigned. During the CCA course, learners developed College and Career Plans in which they listed their college, career, and personal goals, timelines for achieving goals, and barriers to goal completion. The OPABS instructors met with learners as a class and individually to discuss specific steps that learners could take to achieve their college goals, including taking the college admissions test and preparing applications for financial aid. At the end of the cohort class, all of the learners who completed the class (83%) enrolled in credit-bearing college courses. During the term that followed learners’ OPABS participation, the OPABS instructors held a study group for former OPABS learners enrolled in college classes to discuss their challenges and successes, and to provide peer support. At Portland Community College, OPABS instructors have provided enhancements to OPABS courses to increase learners’ readiness for college. One OPABS instructor integrated the OPABS Bridge Reading and Writing courses into a unified course to increase the amount of academic writing experience for ABS learners. Learners were given writing assignments in conjunction with each reading exercise to develop learners’ comprehension and writing skills, and detailed feedback was provided on the writing assignments. CCA instructors arranged additional guest 3

speakers from the college’s professional and technical education departments to provide information about the programs of study that were of high interest to CCA class participants. The additional speakers enabled learners to establish points of contact with the professional and technical education faculty, which assisted learners in planning for postsecondary participation. Washington’s I-BEST To understand the needs of working-age adults with a high school education or less or who did not speak English to further their education and training, the Washington State Board for Career and Technical Colleges (SBCTC) conducted research which indicated that adults’ attainment of one year of college-level credits and a credential is the tipping point for them to obtain a substantial increase in their annual income (Prince & Jenkins, 2005). SBCTC used this information to develop a deliberate strategy for innovation and systemic change in the policies and educational programs for these working-age adults. In 2004, SBCTC began this strategy with a demonstration to implement the Integrated Basic Education and Skills Training (I-BEST) initiative (SBCTC, 2005). I-BEST was developed as a comprehensive approach to provide low-skilled adults with an education and career pathway that integrates learning outcomes and assessment and is matched to the local labor market. In the instructional component of I-BEST, English as a second language (ESL) or ABE instructors and professional-technical instructors work together in the classroom to concurrently provide learners with literacy education and workforce skills. This approach was designed to enable learners to develop their academic and workforce skills that can lead them to a living wage job that is on a career path in an occupational field. It also is intended to assist learners in attaining skills at a faster rate than in instructional programs in which basic skills instruction is provided prior to workforce training. The basic skills courses offered under IBEST are contextualized to skills that learners need to succeed in a particular job or career path (Wachen, Jenkins, Belfield, & Van Noy, 2012). Quasi-experimental analyses have shown that adults’ enrollment in I-BEST has had positive impacts on learners’ earning of any college credits and occupational college credit, the number of college credits and occupational college credits earned, whether learners earned a degree or certificate, and whether learners achieved point gains on basic skills tests (Zeidenberg, Cho, & Jenkins, 2010). Examples of I-BEST Approaches Illustrations of two colleges’ approaches to I-BEST’s signature practice, a co-teaching model, are described. In Renton Technical College’s (RTC) Allied Health I-BEST program, the ABE reading instructor collaborated with a faculty member from the Nursing Department to design and coteach a Certified Nursing Assistant (CNA) course. The ABE instructor adapted the Reading Apprenticeship® approach to address the reading requirements of the CNA course textbook. The CNA faculty member also was trained to use the Reading Apprenticeship® approach and supplemented the ABE teacher’s reading instruction. The ABE instructor monitored learners’ comprehension of the materials and assisted the nursing faculty member in pacing the class to accommodate the reading needs of learners. The nursing faculty member developed and 4

delivered the CNA content and was the authority on this content. She worked with the ABE instructor to ensure that the content of the course was accessible to learners and included additional reading strategies to assist learners in completing the course. Lower Columbia Community College (LCC) developed the academic version of I-BEST, the Integrated Transitional Studies (I-TRANS), which is a pathway program that enables individuals to earn an AA transfer degree or vocational degree and bridges the gap between ABE, English as a second language, and college-level courses. I-TRANS courses integrate the study of academic subjects. For example, LCC’s Humanities and English I-TRANS course was co-taught by a college humanities instructor with a background in reading instruction and a college writing instructor. Learners developed their reading, writing, and analytical skills through a series of integrated assignments. Prior to attending a seminar session on a topic in Humanities, leaners read background information about the topic to prepare them to participate in the seminar. They then completed writing assignments based on a prompt related to the material discussed during the seminar. Learners also developed a writing portfolio throughout the term, which included their writing assignments completed as part of the course. Learners were assigned nightly homework from the course text, and listed questions about the reading assignment as a strategy to improve their understanding of difficult concepts in the text. Findings OPABS and I-BEST illustrate two different but related approaches to assisting ABS learners prepare for postsecondary education. OPABS has focused on increasing the quality of ABS instruction and preparing ABS learners to set postsecondary goals and enter college. CCWD worked with expert ABS instructors to develop a set of standardized OPABS academic courses and a CCA course. These courses were to be implemented as designed, but could be enhanced within the context of the course design. Instructors who taught the OPABS courses have reported that having a standardized framework facilitated their teaching, and that the integration of the occupational information helped OPABS students learn about high-demand occupations in the state. The CCA course was considered instrumental in assisting learners in developing a specific career plan. OPABS instructors reported that many ABS learners enroll in courses without clear goals beyond the attainment of the GED. The OPABS courses enabled learners to prepare for postsecondary participation while studying for the GED. As OPABS instructors have gained experience in teaching the OPABS courses, they have expressed a desire to have flexibility in incorporating varied applications of occupational information in the OPABS basic skills courses. OPABS instructors have tracked OPABS learners as they have enrolled in postsecondary courses. Challenges to leaners’ successful completion of college course have included taking a full load of academic courses during their first term, rather than fewer courses or more professional-technical courses. These observations suggest that academic counseling for the first college term is a critical component of successful ABE transition. I-BEST is aimed at accelerating the time it takes for ABE or ESL learners to complete postsecondary certificates and credentials. I-BEST faculty have flexibility in designing their course materials and in defining approaches to co-teaching I-BEST courses. They also have developed strategies to assist I-BEST learners in addressing barriers to their success in occupational and academic courses. Some ABE programs have offered lab sessions to provide 5

extra academic assistance to I-BEST learners, and I-BEST instructors have provided additional assignments to support learners’ skill development. As colleges have worked to increase I-BEST enrollments, faculty have discussed strategies for assisting ABE learners in developing career goals that would motivate them to enroll in I-BEST classes. Both initiatives point to the need for high-quality basic skills instruction, deliberate efforts to work with ABE learners in setting education and career goals, and strong coordination between ABE and postsecondary professional-technical and advising. The initiatives illustrate the role of state leadership in facilitating instructional and organizational change at the local level in ABE. References Alamprese, J.A. (2007, October). Bridging the divide: Oregon’s Pathways for Adult Basic Skills Transition to Education and Work Initiative (OPABS). Paper prepared for the National Council for Workforce Education Conference. Bethesda, MD: Abt Associates. Alamprese, J.A. (2012). State ABE transition systems report. Arlington, VA: Kratos Learning. Butler, S.M., Beach, W.W., & Winfree, P.L. (2008, September). Pathways to economic mobility: Key indicators. Washington, DC: Pew Charitable Trusts. Carnevale, A.P., Rose, S.R., & Cheah, B. (2011, August). The college payoff: Education, occupations, lifetime earnings. Washington, DC: Georgetown University, Center on Education and the Workforce. CCWD [Oregon Department of Community Colleges and Workforce Development]. (2010, April). Oregon Pathways for Adult Basic Skills Transition to Education and Work Initiative (OPABS). Salem, OR: Author. Guison-Dowdy, A. & Patterson, M.B. (2011). Journey through college: Postsecondary transitions and outcomes of GED® test passers. Washington, DC: American Council on Education. National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century, Committee on Defining Deeper Learning and 21st Century Skills, J.W. Pellegrino and M.L. Hilton, Eds., Board on Testing and Assessment and Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Patterson, M.B., Zhang, J., Song, W. & Guison-Dowdy, A. (2010, April). Crossing the bridge: GED credentials and postsecondary educational outcomes. Washington, DC: GED Testing Service. Pleasants, R. (2011, August). Building integrated pathways to sustainable careers: An introduction to the Accelerating Opportunity initiative. Boston, MA: Jobs for the Future.

6

Prince, D. & Jenkins, D. (2005, April). Building pathways to success for low-skill adult students: Lessons for community college policy and practice from a statewide longitudinal tracking study. New York, NY: Community College Research Center, Columbia University. Reder, S. (2000). Adult literacy and postsecondary students: Overlapping populations and learning trajectories. In J. Comings, B. Garner, & C. Smith (Eds.), The annual review of adult learning and literacy (Vol. 1, pp. 111-157). San Francisco: Jossey-Bass. (ED 436 673). Roberts, B. & Price, D. (2012, December). Strengthening state systems for adult learners: An evaluation of the first five years of Shifting Gears. Chicago, IL: The Joyce Foundation. SBCTC [Washington State Board for Community and Technical Colleges]. (2005). I-BEST: A program integrating adult basic education and workforce training. Olympia, WA: Author. U.S. Department of Education, Office of Vocational and Adult Education (2011). Promoting college and career readiness: Bridge programs for low-skill adults. Washington, DC: Author. U.S. Department of Education, Office of Vocational and Adult Education (2013). College and career readiness standards for adult education. Washington, DC: Author. Wachen, J., Jenkins, D., Belfield, C., & Van Noy, M. (2012, December). Contextualized college transition strategies for adult basic skills students: Learning from Washington State’s IBEST program model. New York, NY: Community College Research Center, Teachers College, Columbia University. Zeidenberg, M., Cho, S-W., & Jenkins, D. (2010, September). Washington State’s Integrated Basic Educational Skills Training Program (I-BEST): New evidence of effectiveness. New York: Community college Research center, Teachers College, Columbia University. Back to Table of Contents

7

Designing for Every Adult Learner Nancy Chapko, Wisconsin Technical College System Abstract: Adults are increasingly choosing to go to college online. The demand for online learning by adult students has intensified the need for high quality online course content designed for the largest possible audience. Those who create online learning content must meet the needs of increasingly diverse online adult learners. Introduction In a scene repeated countless times a day in schools, homes, and the workplace, an adult learner logs into a computer-based Learning Management System (LMS) to access an online class. Valued by adult learners, flexibility is one of the top three attributes they consider when selecting a program or course (Eduventures, 2008). In 2011, approximately 32% of college students were taking at least one online course (Allen & Seaman, 2013). Almost half of adult learners considering non-traditional course delivery are likely or very likely to enroll in a course or program delivered online exclusively (Eduventures, 2008). Adult demand for online learning has increased, and so has the number of adult learners demanding it. This has implications for those who design, develop, and deliver adult learning. The study summarized in this paper describes how Wisconsin Technical College System faculty and staff who participated in Universal Design for Learning (UDL) training in 2010 acquired skills and enhanced their ability to apply UDL techniques in their online teaching practice. Adult Students and Online Learning A large body of research describes how and why adults learn. Plimmer and Schmidt (2007) portrayed adult learners as largely motivated by the desire to expand career opportunities and professional roles. The desire to change careers or broaden opportunities within a chosen career is a powerful motivator to engage in study. Hoare (2009) described adult learning as a developmental process which results in a change in behavior and reassessment of existing knowledge. For adults, learning is not merely the acquisition of knowledge; it is a developmental process that creates a change in behavior and perception of knowledge that has an effect on the adult learner’s personal environment. Adult students in the United States are increasingly choosing to go to college online rather than in a classroom, and though enrollment in online college courses has slowed in recent years, it is still three to four times that of enrollment in face-to-face courses (Aslanian & Clinefelter, 2013). Twothirds of adults who had completed a program of study within the past year described their learning experience as a worthy investment of time and money (Aslanian & Clinefelter, 2013). Nearly half the adult learners reported improved employment standing, and more than one-third reported salary increases and promotions within the 12 month period following graduation (Aslanian & Clinefelter, 2013).

8

Universal Design and Universal Design for Learning Buildings are made accessible when they are designed. They are equipped with power-assisted doors, ramps and railings, and alarms that can be seen and heard. These are accommodations experts consider when designing a building. They do not wait to see who will enter the building, and how those who do so will use the space. The building is designed to be accessible to most people. This is how instructional designers and instructors must approach online course development for adult learners. Innovations that grant physical access to all individuals are the result of universal design. Eliminating barriers to cognitive access is the result of Universal Design for Learning (UDL) pioneered by The Center for Applied Special Technology (CAST) in its exploration of the use of flexible instructional methods and materials (CAST, 2012). Universal Design for Learning focuses on learning support. It uses a template for online course design to eliminate – or at least reduce - potential barriers for all or most learners. CAST suggests three principles for ensuring cognitive access to the largest possible group of learners (CAST, 2012). Principle I requires that learners be provided with multiple means of representation. Principle II requires that learners be provided with multiple means of action and expression. Principle III requires that learners be provided with multiple means of engagement. Integrating these principles into learning design, development, and delivery reduces the need to provide individual learner adaptation to course content at a later time (Rose, Harbour, Johnston, Daley, & Abarbanell, 2006). Strategies and Techniques for Inclusive Adult Online Learning Increasing demand for both adult higher level learning and online content delivery poses challenges and opportunities. Providing adults with skills valued in the workplace is an opportunity. Providing online course content and delivery that meets the needs of diverse adult learners is a significant challenge. Applying UDL principles to adult online courses is likely to make course delivery more effective for learners (Edyburn, 2010; King-Sears, 2009; McGuire & Scott, 2006). While UDL is not intended only for online learning, it is easy to see how the techniques can be implemented with technology. Representing information in multiple ways helps learners to select among display, verbal, or auditory options for perceiving information. Learner comprehension is enhanced with opportunities to activate, highlight, and guide the application of the information (CAST, 2011). Learners who are provided with multiple means of action and expression can select methods of response and navigation and construction and performance that allow them to learn effectively (CAST, 2011). Providing multiple means of engagement allows learners to select options according to their preferences to indicate interest in learning activities, to persist, and to selfregulate (CAST, 2011). Online instructors should expect a wide variety of learning preferences in adult learners including some who may have a disability. In 2009, 46% of disabled young adults, who were within four years of leaving high school, reported their enrollment in a postsecondary school (Newman, 9

Wagner, Cameto, Knokey, & Shaver, 2010). The American with Disabilities Act (ADA) requires that online courses meet the needs of students with disabilities. Institutional LMS can be modified to ensure compliance. Individual course content developed by an online instructor, however, may not be subject to institutional review. This suggests that a proactive design approach and continual review is required to ensure that all learners can access course content with few barriers (Khajavinia, 2007). In higher education, the application of UDL techniques provides strategies that promote inclusion (Izzo, Murray, & Novak, 2008; Gradel & Edson, 2009; Wu, 2010). As the recipients of UDL, adult students should participate in its design. This invites discussion in pedagogical decisions and enhances access to course content. Faced with the complexity of UDL implementation (Edyburn, 2010), faculty in higher education may be confused by the relationship between individual learner accommodations and UDL. It may not be clear to faculty how to provide accessibility to learners who may not qualify for individual accommodations but who would benefit from enhanced access to content. Recent research indicated that nearly 25% of college students may identify an impairment to learning though not necessarily a learning disability requiring individual accommodations (Hirschman, Lemke, & Smith, 2010). Courses designed according to UDL principles benefit all learners. While struggling readers, English language learners, and those from disadvantaged socioeconomic backgrounds may not quality for individual accommodations, they would all benefit from the fewest barriers possible to cognitive content. Wisconsin Technical College System Study Summary In 2010, Wisconsin Technical College System (WTCS) faculty and staff were invited to participate in UDL training workshops held at various locations throughout the state. The workshops were intended to provide information about and suggestions for implementing UDL. Faculty and staff, representing 15 of the 16 Wisconsin Technical College districts, voluntarily attended training workshops. In 2012, those faculty and staff who had attended a 2010 UDL training workshop were invited to participate in a survey about their application of UDL techniques in their design, development, and delivery of online learning content. The survey asked training participants to provide responses to these questions. 1. Were UDL techniques being used by instructional designers and instructors? 2. In what areas (course design, development, and delivery) were UDL techniques being used? 3. Which UDL techniques were being applied? UDL workshops in which WTCS faculty and staff participated influenced their application of UDL techniques in the online learning environment. This is shown in Table 1. Table 1 Reported Influence of Training to Application of UDL Principle Reported Workshop Influence Principle I No influence 10% Applied some UDL techniques 50% Applied UDL techniques whenever possible 35% Used UDL techniques to guide all on-line course 5% design, development, and delivery 10

Principle II 25% 40% 35% 0%

Principle III 10% 50% 35% 5%

The application of UDL techniques occurred most frequently in online course delivery, less often in online course design, and least often in online course development. This is shown in Table 2. Table 2 Application of UDL Techniques UDL Principle Course Design Course Development Course Delivery Principle I 61% 50% 89% Principle II 60% 53% 93% Principle III 50% 39% 94% Note: Each respondent selected all areas in which UDL techniques were applied. A UDL technique associated with Principle III (multiple means of engagement), foster collaboration and community, was applied more frequently than any other UDL technique. More than 80% of survey respondents indicated they had implemented this technique, and they did so most often in course delivery. Several UDL techniques associated with Principle I (multiple means of representation) were applied frequently. This is shown in Table 3. Table 3 UDL Techniques Applied Most Frequently Rank UDL Technique 1 Foster collaboration and community 2 Illustrate through multiple media 3 Offer alternatives for visual information 4 Offer alternatives for auditory information

Associated UDL Principle Principle III Principle I Principle I Principle I

Conclusions and Recommendations There is evidence that the application of UDL principles enhances learning. In a study conducted by Rose et al. (2006), surveyed students who participated in a semester long university course indicated greater appreciation of flexibility of choice in assignments. Most online learners benefit from UDL instructional strategies that provide choice. Integrating UDL principles in online course design is an efficient and cost-effective method of developing curriculum (Cochran, Bowman, Madsen, King, & Shrilla, 2006). It is easier and more cost effective to remediate curriculum as it is designed rather than adapt it to individual learners at the time of their participation. Technology is essential to the effective implementation of UDL strategies (King-Sears, 2009; Rose et al., 2006). Preparing online instructors to incorporate UDL principles in their teaching practice is critical to meeting the diverse needs of adult online learners (Izzo, Murray, & Novak, 2008; McGuire & Scott, 2006; Rose et al., 2006). However, most college instructors have little preparation in creating inclusive classrooms (Ouellet, 2004). As online enrollments have grown, so has the need for an adequate supply of skilled online instructors. Evidence suggests that the existing supply of skilled online instructors is insufficient to meet the demand of online enrollments (Voytecki, Engleman & Jeffs, 2010).

11

If it is possible to extend what was learned from the Investigation into the Application of Universal Design for Learning Techniques by Wisconsin Technical College System Faculty and Staff (Chapko, 2012) to the larger higher education community of online course designers and instructors, the following recommendations should be considered. 1. Require comprehensive training specifically related to UDL strategy, techniques, and principles in the online learning environment for instructional designers and online instructors. 2. Provide a shared repository of easily accessible resources modeling the application of UDL techniques. As the application of UDL techniques among higher education professionals is largely an emerging practice, a shared repository could encourage greater use of UDL techniques in online course design, development, and delivery. 3. Begin with UDL; allow the development, design, and delivery of higher education online courses to be guided by UDL and its principles. 4. Assess online courses that are subject to institutional review in terms of their application of UDL techniques - among other criteria. 5. Compare and contrast how higher education faculty and staff apply UDL techniques. The research should consider all learning environments, all methods of instructional delivery, and all programs. The results of this research should be openly shared to encourage dialog and further study with a goal to identifying best practices. Acquiring skill in the application of UDL principles is a challenge for higher education faculty. Voytecki et al. (2010) described the most challenging aspects of university faculty teaching their first online course. They identified course design issues, lack of technology training, and lack of pedagogical training related to online teaching. These responses are not surprising given that some institutions provide instructors with training in UDL concepts and but fail to relate them adequately to the online learning environment (Izzo, Murray, & Novak, 2008; McGuire & Scott, 2006). Online course development that integrates UDL principles is best achieved by an institutional commitment to comprehensive support of online instructor development and practice (Roman, Kelsey, & Lin, 2010). Roberts (2004) reminded educators they are in a critical position to influence required changes in human perception and course design that supports UDL. It is likely that well developed and coordinated online instructor training courses result in greater faculty acceptance and sustained success (Roman et al., 2010). This supports technological and pedagogical skill development that forms the basis of effective UDL strategy implementation. References Aslanian, C. B., & Clinefelter, D. L. (2013) Online college students 2013: Comprehensive data on demands and preferences. Louisville, KY: The Learning House, Inc. Allen, I. E., & Seaman, J. (2013). Changing course: Ten years of tracking online education in the United States. Babson Park, MA: Babson Survey Research Group. Retrieved from www.onlinelearningsurvey.com/reports/changingcourse.pdf. CAST. (2011). Universal design for learning guidelines version 2.0. Retrieved from http://www.udlcenter.org/sites/udlcenter.org/files/updateguidelines2_0.pdf.

12

CAST. (2012). CAST timeline: One mission, many innovations, 1984-2010. In About CAST (history). Retrieved from http://www.cast.org/about/timeline/index.html. Chapko, N. A. (2012). An investigation into the application of Universal Design for Learning techniques by Wisconsin Technical College System faculty and staff. University of Wisconsin - Stout, Menomonie, WI. Cochran, D., Bowman, C., Madsen, C., King, J., & Shrilla, A. (2006). Universal design for learning (UDL): The hook to catch and hold diverse learners – lessons from designing graduate education instruction. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2006 (pp. 41874191). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/22762. Eduventures. (2008). The adult learner: An eduventures perspective who they are, what they want, and how to reach them. Retrieved from http://learn.eduventures.com/rs/eduventures/images/Eduventures_WP_AdultLearners.pdf Edyburn, D. L. (2010). Would you recognize universal design for learning if you saw it? Ten propositions for new directions for the second decade of UDL. Learning Disability Quarterly, 33(1), 33-41. Gradel, K., & Edson, A. J. (2009-2010). Putting universal design for learning on the higher ed agenda. Journal of Educational Technology Systems, 38(2), 111-121. doi:10.2190/ET.38.2.d. Hirschman, A. M., Lemke, M. R., & Smith, R. O. (2010, November 18). Real numbers and implications for interventions: The prevalence of disability on campus. Concurrent Workshop presented at the annual Accessing Higher Ground Conference, Assistive Technology in Higher Education Network, Westminster, CO. Hoare, C. (2009). Models of adult development in Bronfenbrenner's bioecological theory and Erickson's biopsychosocial life stage theory: Moving to a more complex three-model view. In M. C. Smith & N. DeFrates-Densch (Eds.), Handbook of research on adult learning and development (pp. 68-102). New York, NY: Routledge. Izzo, M., Murray, A., & Novak, J. (2008). The faculty perspective on universal design for learning. Journal of Postsecondary Education & Disability, 21(2), 60-72. Khajavinia, R. (2007). Online course designs: Anything to change? In T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007 (pp. 7159-7197). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/26919. King-Sears, M. (2009). Universal design for learning: Technology and pedagogy. Learning Disability Quarterly, 32(4), 199-201.

13

McGuire, J. M., & Scott, S. S. (2006). Universal design for instruction: Extending the universal design paradigm to college instruction. Journal of Postsecondary Education & Disability, 19(2), 124-134. Newman, L., Wagner, M., Cameto, R., Knokey, A. M., & Shaver, D. (2010). Comparisons across time of the outcomes of youth with disabilities up to 4 years after high school. A report of findings from the national longitudinal transition study-2 (NLTS2). Menlo Park, CA: SRI International. Retrieved from http://www.nlts2.org/reports/2010_09/nlts2_report_2010_09_complete.pdf. Ouellet, M. L. (2004). Faculty development and universal instructional design. Equity & Excellence in Education, 37(2), 135-144. Plimmer, G., & Schmidt, A. (2007). Possible selves and career transition: It's who you want to be, not what you want to do. New Directions for Adult and Continuing Education, (114), 61-74. Roberts, K. (2004). The need for universal design in online learning environments. AACE Journal, 12(2), 188-197. Roman, T., Kelsey, K., & Lin, H. (2010, Winter). Enhancing online education through instructor skill. Online Journal of Distance Learning Administration, 8(4). Retrieved from http://www.westga.edu/~distance/ojdla/winter134/roman_kelsey134.html. Rose, D. H., Harbour, W. S., Johnston, C., Daley, S. G., & Abarbanell, L. (2006). Universal design for learning in postsecondary education: Reflections on principles and their application. Journal of Postsecondary Education & Disability, 19(2), 135-151. Voytecki, K., Engleman, M., & Jeffs, T. (2010). A preliminary investigation into pedagogical and practical difficulties encountered by university faculty in online teaching. In D. Gibson & B. Dodge (Eds.), Proceedings of society for information technology & teacher education international conference 2010 (pp. 956-961). Chesapeake, VA: AACE. Wu, X. (2010, December 1). Universal design for learning: A collaborative framework for designing inclusive curriculum. i.e.: inquiry in education, 1(2), article 6. Retrieved from http://digitalcommons.nl.edu/ie/vol1/iss2/6/. Back to Table of Contents

14

Standardized Tests in English Learning Assessment in Adult Education: Blessing or Curse? Larissa Chuprina, Lake Washington Institute of Technology Amy D. Rose, Northern Illinois University Abstract: This paper examines some issues surrounding testing within ESL/ABE classes focusing on the interaction between the process of standardized testing and the teaching of classes. This review discusses questions about the validity and economic merits of testing within the framework of critique and the literature on cultural bias in testing. Although standardized testing has been a critical topic within education for the previous decade, only recently has the debate reached Adult Basic Education. In this paper, we begin with a brief overview of the issues connected to testing in the United States, we then discuss the basic premises connected to the issue of Standardized Testing in Adult Basic Education and English as a Second Language. We then begin an analysis of the unique aspects of this debate. Since standardized testing was first introduced in the 1920s, it has been hotly contested. The first question that must be asked is what exactly we mean by this term. There are certainly many different approaches. In general, the term refers to grading of examinations in a standard way. While the definition does include a requirement that these tests be multiple choice, machine graded examinations; the reality is that this is very often the case. Recently, under the impetus of federal legislation, particularly No Child Left Behind, states have been required to show student progress as demonstrated by testing. This has introduced a whole new era of high stakes testing into the educational world with both students and teachers facing the consequences of poor performance. In recent years, critics of testing have pointed to several issues. The first is the basic expense related to high stakes testing. In addition, to the expenses, critics of testing point to the problems of measuring true learning in discrete testable units; of teaching to what can be tested rather than deciding what should be taught and then testing for that; and finally changing the content to fit the test. Additional concern has focused on the cultural biases of high stakes testing. For example, looking at Texas, Helig (2011) examines the ways the relationship between high stakes testing and outcomes for English language learners. He found that there was a great cause for concern about the English language learner population mandated to participate in high stakes testing. In particular, he examines whether high stakes testing in the public schools undermines the trust or respect that immigrant families grant to educational institutions. His tentative findings are that they do. This national discussion has been in place since NCLB was passed, but more recently we have seen an extension of the discussion to various areas of adult education. While the stakes are not as large, the potential for damage could be even more severe. This is because, the adoption of these so-called high-stakes tests do not take into account some of the unique aspects of adult students in these programs.

15

Testing in ABE/GED/ESL Certainly testing within adult education is not new For example, the General Educational Development (GED) examination developed during World War II has been the mainstay of the adult education program since the passage of the Adult Education Act of 1966. (Rose, 1991). Linn (2000) identified five “waves of reform” that have pervaded education since World War II. The first wave occurred in the 1950s with the use of standardized tests to track students. The second occurred in the 1960s after the passage of the Elementary and Secondary Education Act where tests were used for accountability of funding. The third wave concerned “minimum competency testing” in the 1970s and 1980s. (Kruidenier, 2002, p. 93). In the fourth wave, high stakes testing introduced the idea of holding teachers and administrators accountable for student scores; and finally the then current reform effort that included high stakes accountable and the development of standards. Testing and tests have changed to meet the changing requirements. For example, performance-based testing which focused on application rather than on simple multiple choice answers was seen as an improvement which would allow for context. (Kruidenier, 2002). We appear to still be in this era. However what Linn described as applying to primary and secondary schools has also been extended to adult education. Over time, the federal mission has shifted to a demand for greater oversight and accountability in the disbursement of funds. In particular, organizations receiving funding need to prove that students are making adequate progress, usually defined in terms of improvement on test scores. In a process similar to that underway with the elementary and secondary system, adult education is becoming heavily reliant on these tests to prove that adequate progress is being made. Of course the federal government does not mandate any specific test, however states have been unable to develop their own tests and tend to rely on privately developed examinations. Additionally, in the 1990s, assessment was advanced as a way of proving the need for and the efficacy of adult education. (Askov, 1993). The adult education system is less controlled than the K-12 system. As adult educators remained unconvinced about the efficacy of high-stakes testing, it was not quickly adopted. For example, Kutner et al. (1996) found that in the early1990s, the adoption of standardized assessment was not a “priority for most programs” (p.2). Kutner et al. found that programs complained that the tests were not connected to what was taught and that instructors were unable to implement mandates because of lack of training. In addition, some critics have stated that testing is not appropriate for adult students because of the wide breadth of their experiences and contexts, making testing results difficult to assess. Kasworm and Marienau (1997) lay out five principles for assessment of adult learning: Recognition of multiple sources of “knowing”; “Recognizes and reinforces the cognitive, conative and affective domains of learning,”; recognizes the need for active involvement in learning; recognizes adult involvement in other life roles; “accommodates adult learners’ increasing differentiation from one another” (p. 8). The Adult Education and Family Literacy Act of 1998 codified assessment into the law by providing funding incentives to states based on performance on tests (Kruidenier, 2002). Kruidenier goes on to emphasize the low level of assessment in the states and the need for improvement in assessment approaches. In fact, this has been the thrust of reforms in the twenty-first century. Demetrion (2005) indicates that the emphasis on assessment coincided with 16

a renewed effort to focus on job skill development at the expense of general education. There are consistent worries about the use of standardized tests with an at-risk population. There have also been critiques that the adoption of these tests and of standards in general, has created a set of unrealistic expectations. There are increasing expectations for immediate measures of progress. Within adult education, the Comprehensive Adult Student Assessment System (CASAS) is the most widely used test. It was designed to measure the basic language skills needed to “function effectively at work and in life.” (CASAS, n.p.). It includes more than one hundred standardized assessments and a variety of instructional and supporting materials designed for adult basic education, workforce learning, special education, and adult ESOL. Some of the tests are specifically constructed for the adult English-language learner population. All of the items are tied to a list of competencies, and all of the tests are scaled to a single, uniform scale of proficiency. Ranges of the scale are associated with highly generalized statements of language proficiency of skill levels. However, many of the competencies are very narrowly defined. Finally, extensive training is required to use the CASAS materials (Van Duzer & Berdan, 1999). Critiques Certainly there have been many critiques of standardized testing. The Comprehensive Adult Student Assessment System, (CASAS), is the most widely used competency-based assessment examination system in the United States. It was designed to measure the basic language skills needed to function effectively at work and in life.” (CASAS). In general, the CASAS is used because states have not been able to develop their own standardized tests. Yet, there has been widespread unhappiness with all of the national tests that have been developed. There are consistent worries about the use of standardized tests with an at-risk population. There have also been critiques that the adoption of these tests and of standards in general, has created a set of unrealistic expectations. There are increasing expectations for immediate measures of progress. Even students enrolled in low level literacy programs and ABE/ESL are expected to show results within three months. Since many of these low-level students work with tutorvolunteers, it is especially difficult to fully understand their progress in terms of one simple measure. There is some indication that testing does not truly reflect abilities Dugan et al (1987) go on to note that subjecting English as a Second language students to standardized testing is ” unconscionable and objectionable” (pp. 110-111). There are other problems standardized testing for this population. Often they cannot interpret the language of the tests. (Celce-Murica and Olshtan, 2006). In addition, standardized tests do not provide information about affective and metacognitive factors in literacy acquisition. The impact of literacy on students’ family life, personal growth, effectiveness at work, or ability to make changes in their life is not reflected by test scores, although these effects are among the most important from students’ perspectives. The type of assessment used in all standardized tests focuses on product rather than process; therefore they have little explanatory power for the teacher or the student. There are also concerns about the use of CASAS, especially its use with at risk populations. There has been some discussion about the basic validity and reliability of the tests and of the 17

competencies used. There have also been critiques that the adoption of these tests and of standards in general, has created a set of unrealistic expectations. There are increasing expectations for immediate measures of progress. Even students enrolled in low level literacy programs and ABE/ESL are expected to show results within three months. Since many of these low-level students work with tutor-volunteers, it is especially difficult to fully understand their progress in terms of one simple measure. There is some indication that testing does not truly reflect abilities Dugan et al (1987) go on to note that subjecting English as a Second language students to standardized testing is ” unconscionable and objectionable” (pp. 110-111). Finally, one of the main critiques is that these examinations are culture specific, presupposing culturespecific knowledge and vocabulary (Auerbach, 1992). Others echo this critique, by pointing to problems with standardized testing for this population. For example, often learners cannot interpret the language of the tests (Celce-Murica and Olshtan, 2006). Finally, and perhaps most important, critics emphasize the point that the tests do not measure the competencies necessary for literacy or even progression to the next level. Futures areas of study So far, there has been little sustained study of the ways that high stakes testing affect or influence outcomes among adult English Language learners. Anecdotal evidence seems to indicate a profound cultural bias but the exact way that this plays out within a programmatic context is unclear. It is clear that the very structure of the experience of testing can be a problem. For example, in the CASAS has a Listening and a reading component. These two areas often include cultural subtexts that might hinder a student’s ability to answer questions correctly. Thus, they might answer incorrectly, not because they do not understand the question, but because of a limited exposure to culture of driving, eating in a fast food restaurants, or dealing with American measurement system. The test is sometimes more a test of cultural competence within the American context than a test of language skills. On the other hand, those who have been in the country for many years, sometimes score higher place into Level 3 and higher even though their speaking and writing skills may really be lower than needed. In short, the score does not show the real picture and does not inform the teacher about the students’ level or the areas which need improvement. Most importantly however, the constant testing appears to be a disincentive to attendance and completion. Additionally, there is some concern among educators about how the tests are being used. They are often used without any context, solely as a means of managing accountability reports rather than as a form of formative assessment. The test stands outside of the educational process and is contrary to the theory and practice of second language acquisition and adult education. Finally, the expansion of the testing during a period of with limited funding has forced programs to experiment as a form of cost cutting. Thus we see adult education programs run multilevel classes, raise the caps for the maximum number of students in class, and rely even more heavily on a combination of part-time teachers and tutor-volunteers. Additionally, the costs of testing materials and of administering the tests are straining already underfunded programs. Finally, on the surface it appears that this type of testing is unpopular among both students and teachers because it emphasizes deficiency rather than strength.

18

Conclusion While the teachers in K-12 are actively seeking new ways of validating the knowledge of their students beyond the Math tests and WAESOL by protesting openly and organizing strikes, their counterparts in AE seems numb or silenced when addressing the same problems. Educators in AE have the power to make a difference directly and indirectly to the public good. In diverse society like the USA, it is not enough to claim cultural competence, but act on it. The Biggest step for a man will be to come to recognition of human complexity and deficiency of standardized testing that rob the learners of meaningful education directly and indirectly, by taking time, energy, and resources from learners and educators. This paper questions the view that accountability is synonymous with continuous assessment. As many have indicated, such a view ignores the learning context and profoundly limits the teaching learning interaction. While outside standards may be of value the current focus on high stakes testing particularly impacts vulnerable populations. There is a need for the national dialogue about adult education and its increasingly critical role in today’s society, specifically the implementation of alternative ways of assessment, including interview, self-assessment, or portfolio-assessment. It can solve unemployment problems in this country by focusing on knowledge and skills necessary for the 21 century. It is important for all parties involved in decision-making to understand that assessment choices impact equity and ultimately the functioning of a democratic society. References Askov, E. N. (1993). Approaches to Assessment in Workplace Literacy Programs: Meeting the Needs of All the Clients. Journal of Reading, 36(7), 550-554. Auerbach, E. R. Making Meaning, Making Change: Participatory Curriculum Development for ESL Literacy. Washington, DC: National Clearinghouse on Literacy Education, Center for Applied Linguistics, 1992. (ED 356 688). CASAS (Comprehensive Adult Student Assessment System). (n.d.). http://www.casas.org Celce-Murica, M., & Olshtan, E. (2006). Discourse and context in Language Teaching. A guide for Language Teachers. Cambridge: Cambridge University Press. Demetrion, G. (2005). Conflicting paradigms in adult literacy education: In quest of a U. S. Democratic politics of literacy. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Heilig, J. V. (2011). Understanding the interaction between high-stakes graduation tests and English learners. Teachers College Record, 113(12), 2633-2669. Retrieved from http://search.proquest.com.proxy.lib.umich.edu/docview/1312425199?accountid=14667 Kasworm, C. & Marienau, C. (1997). Principles for assessment of adult learning. In A. D. Rose and M. A. Leahy (Eds). Assessing adult learning in diverse settings: Current issues and

19

approaches. New Directions for Adult and Continuing Education, 75 (pp. 5-16). San Francisco: Jossey-Bass. Kruidenier, J. (2002). Literacy assessment in adult basic education. In J. Comings, B. Garner, and C. Smith, (Eds). Annual Review of Adult Learning and Literacy, Vol. 3 (pp. 84-151). San Francisco, CA: Jossey-Bass. Kutner, M., Webb, L., & Mathesons, N. (1996). A review of statewide learner competency and assessment systems. Washington D. C.: Pelavin Research Institute. Linn, R. L. (2000). Assessment and accountability. Educational Researcher, 29 (2), 207-218. Office of Vocational and Adult Education. (n.d.) http://www2.ed.gov/about/offices/list/ovae/index.html Rose, A.D. (1991). Ends or Means: An Overview of the History of the Adult Education Act. Columbus OH: ERIC Clearinghouse on Adult, Career, and Vocational Education. Information Series No. 346. Van Duzer, C.H., & Berdan, R. (1999). Perspectives on Assessment in Adult ESOL Instruction. In J. Comings, B. Garner, & C. Smith (Eds.), Annual review of adult learning and literacy: Volume 1 (pp. 200-242). San Francisco: Jossey-Bass. Back to Table of Contents

20

Crossing the Chasm from Classroom Training to High-Performance Virtual Delivery Cynthia Clay, NetSpeed Learning Solutions Abstract: Many facilitators are reluctant to make the change from face-to-face classroom training due to lack of knowledge or even simply replicate their classroom presentations intact over the Internet, resulting in unengaged learners. Applying best practices to the virtual classroom and taking steps to mitigate the impact of technology snafus can result in good learning transfer and retention. Technology has allowed the business world to expand geographically, but economic challenges have caused our travel budgets to compact. This presents challenges for educators. They can’t always get participants in the same room, or even the same country. Webinars hold promise as a replacement for the physical classroom, but presenting an engaging, effective learning experience is not as easy as simply taking your face-to-face classroom material, eliminating the interaction, and presenting a lecture over the Internet. Without a solid understanding of how to effectively use web conferencing platforms, facilitators may avoid taking advantage of this new teaching opportunity or may present dull training that fails to engage learners. Unfortunately, there is not much information available about how to deliver effective virtual training. And so we find plenty of negative statistics about virtual training, especially self-paced virtual learning. General Physics Corporation found that only 20% of 114 respondents considered their companies to be very effective at using virtual instructor-led training (General Physics Corporation, 2010). Jennifer Hoffman found that “the completion rate for self-paced training is quite poor—most people are not self-directed enough to actually complete their training…” (Jennifer Hoffmann, 2005). According to Jack Phillips, Ph.D, “Our studies have shown that participants in e-learning programs are less likely to follow through than in an instructor-led program. In an instructor-led process, there is often a commitment made between the participant and the instructor that might increase the likelihood of participants applying what was learned” (Phillips & Burkett, 2007–2008). Reasons Many Facilitators Avoid Virtual Learning We routinely ask participants in our virtual training courses—typically highly experienced classroom facilitators—why they are taking the course and what has prevented them from making the jump from classroom to virtual training. Most participants respond that opportunities for face-to-face training are steadily shrinking in corporations and formal education organizations. Indeed, in the 2012 Training Magazine industry report, it was reported that 60% of training is delivered in the traditional instructor-led training format. This is down from over 90% of training delivered in the traditional instructor-led training format 20 years ago (Training Magazine, 2012). And we hear again and again concerns from trainers about the lack of feedback from participants, discomfort with using the web conferencing platform’s interactivity tools, a conviction that participants are multitasking rather than paying attention, and worries that technology problems will damage or ruin the session. It doesn’t have to be that way.

21

Common Virtual Delivery Errors Before considering how to create a great webinar, what mistakes are commonly made? We have gathered hundreds of responses from learners evaluating webinars, and the single most common complaint is a talking head or lecture-style presentation. Other commonly cited failures are not allowing participants to chat with one another and using dull PowerPoint slides that are hard to read and lack graphics. It is important to bear in mind that adult learners have different expectations of the experience than do non-adult learners (Knowles, 1973). Knowles proposes four principles of adult learning: 1. Adults need to be involved in the planning and evaluation of their learning 2. Experience (including mistakes) provides the basis for learning activities 3. Adults are most interested in subjects that have immediate relevance to their work or personal lives 4. Adult learning is problem-centered, not content-oriented. One can see how these principles relate to virtual delivery errors. Adults normally do not want to sit passively while a lecture is delivered. They want involvement—sharing their own experiences and knowledge and getting information that helps solve real problems they have. John Medina is even more explicit: His rule #4 of twelve rules or principles based on brain research studies that govern how we work and learn says succinctly: We don’t pay attention to boring things (John Medina, 2008). Five Techniques to Transform the Learning Experience To produce effective learning transfer and retention, we need to use the interactivity tools to keep participants engaged, take advantage of their experience and knowledge, present colorful and relevant slides, and avoid boring them. We recommend the following five techniques: 1. Allow no more than about three minutes between participant interaction opportunities. Using chat, polls, and status icons are quick and easy ways to keep their attention, and it allows the facilitator to monitor engagement. Although the facilitator can’t see the participants, it will be clear whether or not they are engaged in the session based on how they respond to interaction opportunities. 2. Use polling in diverse ways. Certainly you can use polls as quizzes, but there are other beneficial ways to use polls. You might introduce a discussion by having participants answer a poll, forcing them to think about what they already know or don’t know about that topic. You can use polls to find out your participants’ level of experience or opinions on a particular topic. Polls can be used as a self-assessment tool. Or seeing how other people answer a poll could be learning point that increases the likelihood they will change their own behavior as a result. 3. Build opportunities for participants to exchange ideas and opinions using a chat feature or telephone connection. You might ask participants to brainstorm solutions to a problem or

22

offer their opinions about the topic under discussion. This allows them to share and learn from one another, and helps meet the goal of keeping the learning problem-centered. 4. Create learning experiences with real issues and problems. We want to make sure the learning points are anchored to the participants’ real world concerns. Allison Rossett of San Diego State University says, “Transfer of learning is the application of skills and knowledge learned in one context to the context that matters” (Allison Rossett, 2006). In other words, the class content needs to relate to the participants’ real issues and problems. 5. Request learner feedback regularly. The facilitator should never have the feeling of not knowing what the participants are thinking. By regularly checking in with participants, it helps keep participants engaged and provides another way to fill in the missing visual and verbal feedback that would be expected in a face-to-face classroom. Handling Technology Failures If you deliver webinars or online training, then you know that no matter what web conference platform you use, things can go wrong. While snafus can’t be completely avoided, there are ways to plan ahead to mitigate the impact of technology problems. Three simple practices will head off many potential disasters. The first is for the facilitator to become comfortable with the features of the chosen web conferencing platform. There are differences—often dramatic differences—between the various commercial platforms. Understanding what a particular platform can do and what it cannot do will help the facilitator to avoid attempting something with a moderate or high likelihood of failure. For example, if the facilitator’s platform tends to freeze whenever more than three or four participants use annotation tools, then the facilitator will know not to ask ten participants to use the annotation tools simultaneously. If the platform has touchy breakout rooms, the facilitator should choose a different activity that doesn’t involve breakout rooms. The second practice is to have a host or producer assisting. This person can work in the background to answer participant questions or assist them with technical difficulties, open and close polls, move participants in and out of breakout rooms, and troubleshoot glitches. The host may even be able to take over for the facilitator in the event of certain types of technical failures, such advancing the slide deck should the facilitator lose Internet connection. The third practice is for the facilitator and host to conduct a dry run one or two days before the presentation. This will ensure they both understand what is to be done and who is doing it, and they will have the opportunity to discover certain kinds of problems in advance, such as that the wrong PowerPoint slide has been loaded or that the meeting room was not tidied up after the last webinar that was presented in that room. Once the facilitator is comfortable with the platform, the next step is to brainstorm all the things that could go wrong, assess the likelihood and repercussions of the risk, and develop a contingency plan. The facilitator might choose to focus especially on things that have a higher likelihood of happening, as well as things that are less likely but could be disastrous if they occurred.

23

For example, if the facilitator is delivering a webinar during a storm, he might anticipate a power outage that would cause a loss of Internet connection. The likelihood is fairly high and the repercussions could be severe if no plan is in place. Anticipating the problem, the host or producer might be in a different location and the facilitator could be armed with a printed copy of the slide deck. Should a power outage actually occur, the facilitator would work from the printed copy and the host would advance the slide deck. The participants might not even be aware that anything had happened. If the problem is something that is obvious to the audience, the facilitator or host should comment on the technical difficulty calmly and let participants know that the facilitator and host/producer are working to resolve it. The contingency plan should be activated, if one exists. The facilitator must recognize that participants don’t want to hear frantic, frustrated or angry comments from the presenter or host. Instead, the facilitator and host can model peaceful professionalism by responding to the existing circumstances without wasting effort wishing things were different. The facilitator should always keep the focus on the objective, which is creating an effective learning experience no matter what the technology is doing. A technical failure does not mean that the webinar must fail. An ability to stay cool and focused on the objectives, while activating a revised plan, will make the difference between a failed or successful learning experience. Conclusion Armed with an understanding of what adult learners are seeking, how the web conferencing platform works, and a contingency plan for technology problems, a facilitator can deliver interactive training that captivates, informs, energizes and leads to learning transfer. References General Physics Corporation. (2010). General Physics Corporation Virtual Instructor-Led Training. Retrieved from http://www.bloomberg.com/apps/news?pid=conewsstory&tkr=GPX:US&sid=ar_u5qrIA yJI Hoffmann, Jennifer. (2005). How to Design for the Live Online Classroom. Retrieved from: http://www.techrepublic.com/resource-library/webcasts/how-to-design-for-the-liveonline-classroom-creating-great-interactive-and-collaborative-training-using-web/ Knowles, Malcolm. (1973). The Adult Learner. Burlington, MA: Butterworth-Heinemann. Medina, John. (2008). Brain Rules. Seattle, WA: Pear Press. Phillips, J., & Burkett, H. (2007–2008). The Business Value of d-Learning. Retrieved from: http://media.roiinstitute.net/articles/pdf/2008/01/18/el1207_ROI.pdf

24

Rossett, Allison. (2006). Never Worry About Transfer Again: Five Strategies for Converging Learning and Work. Retrieved from http://edweb.sdsu.edu/Courses/ED795A/key1/never_worry_795a.ppt Training Magazine. (2012). 2012 Training Industry Report. Retrieved from http://www.trainingmag.com/content/2012-training-industry-report Back to Table of Contents

25

What Makes A Socially Responsible College Student Leader? A Research Perspective Joanne DeMark, Western Washington University Christina Van Wingerden, Western Washington University Abstract: The purpose of this research was to examine social responsibility, as measured by the social responsibility leadership scale (SRLS) through the Multi-Institutional Study of Leadership (MSL). Additionally, student and staff perceptions of what contributes to paraprofessional student leader and student leadership scholar social responsibility was explored. Introduction and Background What environmental, education and development experiences make for socially responsible college student leaders? Western Washington University, along with a number of other universities nationwide, participated in the Multi-Institutional Study of Leadership (MSL), a national survey to examine influences of higher education on college student leadership development. The study also directs significant attention to the examination of college experiences and their influences on leadership-related outcomes (e.g., complex cognitive skills, social perspective-taking, leadership efficacy). The study examines the influence of environmental factors—such as educational interventions, faculty and peer relationships, diverse peer interactions, and co-curricular involvement—on leadership development. Our participation was in hopes of participating and collaborating in a platform from which to engage in evidencebased practice while contributing to the knowledgebase on college student leadership development as well. Approximately 250 institutions in the U.S., Canada, Mexico, and Jamaica with over 300,000 student respondents have participated in the MSL. National and university researcher practitioners have presented and published numerous time, although this is the first time that Western Washington University staff has studied and is describing our students and student leaders. Additionally, WWU staff has conducted an additional qualitative study to understand the results on WWU paraprofessional student leaders. The key research questions are these: • How do WWU college students, and more particularly, WWU paraprofessional student leaders (a subset comparison sample) score on the eight leadership values associated with the Social Change Model? • How do scores compare across particular demographic factors, such as gender, race/ethnicity, and class-standing? • What environmental factors (e.g., cocurricular involvement, study abroad) contribute to higher scores on the leadership outcomes? 26

Background The education and development of students as leaders has long served as a central purpose for institutions of higher education as evidenced in mission statements and the increased presence of both curricular and co-curricular leadership development programs on college and university campuses (Astin & Astin, 2000; Zimmerman-Oster & Burkhardt, 1999). Astin and Astin go as far as to suggest that, “higher education plays a major part in shaping the quality of leadership in modern society” (p. 1) and a growing number of scholars and professional associations have identified socially responsible leadership as a core college outcome (Association of American Colleges & Universities, 2007; Astin & Astin, 2000; Hoy & Meisel, 2008; National Association of Student Personnel Administrators & American College Personnel Association, 2004). Yet, research on the topic continues to reflect an incomplete picture suffering from a lack of theoretical grounding consistent with contemporary conceptualizations (Dugan & Komives, 2007; Posner, 2004) as well as a lack of clarity regarding individual and institutional factors influencing leadership development (Kezar, Carducci, & Contreras-McGavin, 2006). If higher education institutions could begin to address these issues, the ability to enhance leadership development and the preparation of civically engaged citizens would increase dramatically. This study was designed to contribute to both theory and practice on college student leadership development at the national and single university levels. At WWU, the hope was that results would add further understanding of the leadership development needs of our students. This should advance the ability of student leadership program staff and leadership educators at our institution to deliver programs that directly build leadership capacities. A secondary benefit was that participants might reflect more on their leadership development and college involvement experiences after completing the study. A qualitative follow-up study was conducted to gain more knowledge about how students and staff perceived the results from the quantitative study, while at the same time providing them with an intentional time and space to reflect upon their campus experiences and development as leaders. Methodology Instrument The MSL instrument is derived from the Socially Responsible Leadership Scale-Revised 2 (SRLS) (Dugan, 2006; Tyree, 1998), a 68-item tool designed to measure leadership development across the eight critical values associated with the Social Change Model of Leadership. Each construct comprises 6- 11 items; participants respond using a 5-point Likert-scale response continuum. The Social Change Model scales used in the SRLS are these: Consciousness of self, congruence, commitment, collaboration, common purpose, controversy with civility, citizenship, change, and an OMNIBUS SRLS (total score 8Cs) score and a Leadership efficacy score. Leadership efficacy is how much one has confidence in one’s leadership. This instrument contains supplemental questions informed by the I-E-O model (Astin, 1991) that control for pre-college inputs, examine the college environment, and assess additional outcome variables—such as cognitive complexity, diversity appreciation, and leadership identity development stage. The MSL is administered online and the time to complete the instrument is 20 minutes for the average college student respondent.

27

Sample The research is designed to study undergraduate students attending Western Washington University. Number of subjects is 4, 000 randomly selected undergraduates, with a comparison sample of up to 1,000 identified undergraduate students who are in leadership roles on campus, such as Associated Students and Resident Advisors. A random sample of 4,000 WWU undergraduate students will be drawn from the general student population. WWU, like the other universities in the study, identified potential respondents through the registrar’s existing record-keeping system. A secondary sample of 855 participants was used for comparative purposes. Participants in this sample represent campus student leaders, and were identified by interested campus units, such as Residence Life, Associated Students. The potential college student respondents received e-mail invitations, with a link to the research instrument and a randomly assigned identification number. Students received up to three scheduled reminder e-mails, spaced according to the discretion of each school. They may opt out of future contact by replying to the e-mail and requesting to be removed from the study. Data Collection Methods All data was collected using empirically proven standards for web–based survey research. MSL’s online format allowed students to share their experiences at any time or place that was convenient to them. They could respond to the MSL when they felt comfortable, secure, and unhurried. Respondents further enjoyed the option of leaving the survey and resuming it from their last completed answer. Students were invited to participate via personalized e–mails. Each correspondence outlined the study, addressed issues of confidentiality and consent, and supplied a link to the survey instrument. Students received up to four total contacts that occurred in a three-week window. Results Response Rate WWU participated and was in the top ten of respondents for the MSL in the 2011 collection. Nearly one out of three WWU students in the random sample who was invited to participate did. For the comparison sample, paraprofessional student leaders (e.g. resident advisors and residence hall council representatives, Associated Students officers and employees, orientation student advisors, student outreach peer mentors, admissions representatives, recreation center leaders, health peer educators, career ambassadors, academic student advisors, student tutors, ethnic student center club leaders, outdoor center student leaders, challenge course facilitators, student union employees, student leadership scholars), one out of two WWU students who was invited to participate did. 28

Social Responsibility Generally WWU random sample results on the SRLS were on par with national results and with comparison benchmark groups comparable to WWU. The highlighted area indicates where WWU was statistically significantly lower than peer comparison groups. Leadership efficacy is defined as how much one has confidence in one’s leadership. Environmental experiences can influence these outcomes as well. In the following table indicates the most impactful environmental variables that influenced the SRLS outcome measures. For example, living on or off campus had a significant impact on consciousness of self. Highly impactful college experiences on the random sample of students from WWU include community service, involvement and leadership in campus organizations, engaging in socio-cultural discussions, and mentor relationships with faculty. These all had impact across all scales. The MSL was able to report on change over time for seniors. The following chart identifies the outcomes in which students report a significant difference between pre-college and senior year. Significant gains (identified by the X in the chart) in both leadership capacity and leadership efficacy were made from freshman to senior year with greater gains than national norms on Consciousness of Self, Common Purpose, and Citizenship. There were some differences by gender, race and class standing that will be discussed in the conference session.

29

The results of greatest significance are indicated in the data for the comparison sample, the paraprofessional student leaders. The comparison sample was higher on all accounts by large margins to the WWU random sample and the national random sample. According to the principal investigator at the national level, the results were generally statistically higher than most comparison cohorts at other schools. Here is what those numbers look like for the WWU comparison cohort (Enrollment and Student Services paraprofessional student leaders and those in the leader scholar program) compared to the WWU random sample and the MSL national sample: Qualitative Study A follow-up qualitative study was conducted to further explore what students and staff attributed to the significant findings. The students most often cited a strong, welcoming, close-knit sense of campus co-curricular community, a culture of social responsibility, strong mentor influences, encouragement to get actively involved in the surrounding community, and expectations and opportunities for self-reflection as their explanations for the study results on the paraprofessional comparison sample. The staff also cited strong sense of community, and the intentionality involved in creating community in the extra-curricular environment in which the paraprofessional students operate. The other factors that staff highlighted included the variety of leadership opportunities for students, and the expectation that the college students were adults, with the concomitant treatment and expectations that could be associated with adult-to-adult campus and workplace professional relationships. Implications and Discussion Interesting findings for further exploration include the following: What factors would contribute to WWU college students having comparable leadership for social change scores while having lower scores for self-efficacy that peer samples. What exactly contributes to self-efficacy for WWU paraprofessional student leaders, while not being gained in the campus student experience? Subgroups within the comparison sample had significant differences from each other. For example, Associated Students student government officers and employees had higher marks for citizenship; is that a natural outgrowth of student government leadership experiences at WWU. Data on the resident advisors and peer health educators indicated scores higher on consciousness of self; are the self-authorship and self-reflection activities built into these unit training/activities more likely to attain these results? MSL data on Collaboration in the student leader scholars and the outdoor center/challenge course facilitators was significant by comparison; has the relational 30

activity central in each of these programs connected to this result? Additional data, considerations and implications will be reviewed in the conference session. References Association of American Colleges and Universities (2007). College learning for the new global century. Washington, DC: Author. Astin, A.W. (1991). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. New York: American Council on Education/Macmillan. Astin, A. W., & Astin, H. S. (2000). Leadership reconsidered: Engaging higher education in social change. Battle Creek, MI: W. K. Kellogg Foundation. Dugan, J. (2006). SRLS-Rev 2: The second revision of SRLS. College Park, MD: National Clearinghouse for Leadership programs. Dugan, J. P., & Komives, S. R. (2007). Developing leadership capacity in college students: Findings from a national study. A Report from the Multi-Institutional Study of Leadership. College Park, MD: National Clearinghouse for Leadership Programs. Hoy, A, & Meisel, W. (2008). Civic engagement at the center: Building democracy through integrated co-curricular and curricular experiences. Washington, DC: Association of American Colleges and Universities. Kezar, A. J., Carducci, R., Contreras-McGavin, M. (2006). Rethinking the “L” word in higher education: The revolution in research on leadership. ASHE Higher Education Report 31(6). National Association of Student Personnel Administrators, & American College Personnel Association (2004). Learning reconsidered: A campus-wide focus on the student experience. Washington, DC: Author. Posner, B. Z. (2004). A leadership development instrument for students: Updated. Journal of College Student Development, 45, 443-456. Tyree, T. M. (1998). Designing an instrument to measure socially responsible leadership using the social change model of leadership development. Dissertation Abstracts International 59(6), 1945. Zimmerman-Oster, K., & Burkhardt, J. C. (1999). Leadership in the making: Impact and insights from leadership development programs in U.S. colleges and universities. Battle Creek, MI: W. K. Kellogg Foundation. Back to Table of Contents

31

Evaluation of a Technology-Enhanced Adult Literacy Program in Turkey: The Initial Findings Aydin Yücesan Durgunoğlu, University of Minnesota Duluth Hilal Gençay, Mother Child Education Foundation, Istanbul, Turkey Abstract: An important aspect of education is to enable individuals to access the knowledge resources in their communities. Given the digital literacy needs expressed by the learners, we developed a web-based version of our existing adult literacy program in Turkey. The program is available to learners in our face-to-face classes as well as any adult who wants to complete the online program on their own. In a pilot study, two groups of learners in face-to-face adult literacy classes were compared. The technology–enhanced group had a third of their regular classwork replaced with parallel web-based activities from this new program. The control group had practice as usual. Both groups received pre- and post-tests of math, word recognition, spelling and comprehension. The post-test scores of the two groups were similar, indicating that the technology-enhanced group developed digital literacy while improving their literacy and math proficiencies in a manner similar to the control group. Across the globe, higher educational levels are associated with multiple positive outcomes such as better health, better earning potential, higher achievement of children, stronger civic participation (Education for All, 2006; Kabeer, 1999, 2005; Kutner, Greenberg, Jin, Boyle, Hsu, & Dunleavy, 2007; National Research Council, 2011; Wagner, 1986;). Given these positive outcomes, it is especially important to reduce the gaps in educational access and achievement that are mostly due to cultural and socioeconomic factors (Kutner, et al 2007; Rogers, 2008; Sirin, 2005) Adult education has a special role in this endeavor because these programs directly provide opportunities for adolescents and adults who have been left out of mainstream education for various reasons (National Research Council, 2011). Since 1995, we have developed and implemented an adult literacy program in Turkey. The theoretical foundations of this program are summarized in several publications (Durgunoğlu, 2006; Durgunoğlu, Öney & Kuşcul, 2003). The program has now reached more than 120,000 participants, predominantly women, who had no or very limited schooling because of economic and sociocultural difficulties. Our program consists of two courses: Level 1 develops basic literacy proficiencies such as word recognition, spelling, listening comprehension, critical thinking and mathematics. Level 2 builds on this foundation and focuses on further developing literacy proficiencies, more specifically comprehension. The second course also has an empowerment component that covers topics such as legal rights, healthcare, financial skills, reproductive rights, among others. The courses have been systematically evaluated and the curriculum and textbooks have been revised multiple times. More recently, some new developments necessitated another important change. In the last decade, Turkey has undergone a significant electronic shift and has adopted technology in many facets of life. For example, Turkey ranks sixth in the world on Facebook usage, with 30 million users (http://www.socialbakers.com/facebook-statistics/). More recently the Turkish government has established a site named e-government. In 2008, this site was used for 22 bureaucratic functions (e.g. birth certificates, social security paperwork). This has 32

increased to 292 functions in 2011. Likewise the user numbers jumped from 10,000 in 2008 to approximately 13 million in 2011. As another example, in the centralized health care system, the hospital and doctor appointments can only be made using another online system. An important aspect of education is to enable individuals to access the resources and knowledge bases in their communities. In an information age, knowledge is truly power and to facilitate the well-being and prosperity of individuals, education needs to also include the awareness of and easy access to resources. In our adult literacy courses, participants started articulating this issue and expressed a desire to learn how to use computers and to access the internet in order to make medical appointments, download information, use Skype and Facebook, etc. To understand this situation better, we surveyed the learners in our classes about their technology use and needs. Of the 196 participants, 83% reported using cell phones, 58% had a computer at home, but very few (about 10.5%) knew how to use the computer or how to access the web resources. The most dramatic result was that an overwhelming majority (96%) expressed a desire to develop computer skills. Given this need expressed by the learners themselves, we decided to develop an online version of our curriculum and thus include digital literacy in our program. BITOP (Bilgisayar Tabanlı Okuryazarlık Programı, Turkish acronym for “Web-Based Literacy Program”) Using the platform developed by a Swiss company, Avallain (www.avallain.com), we developed an online curriculum consisting of two main components: (A) Basic Literacy and Math component consisting of four learning areas: • Foundations of literacy • Comprehension (oral, written, digital) • Expression (oral, written, digital) • Math (B) School Preparation component consisting of four learning areas: • Language Arts • Math • Social Studies • Science and Technology Within each learning area there are many modules. Working with a group of writers who were experienced adult literacy educators, we created approximately 5000 screens for these learning areas with many different types of activities. The activities have text-to-speech capabilities to make it easier to read instructions and any difficult sections. Learners can also post their writings to share with the whole community. Finally, each learner has an assigned online teacher who is in communication with them. BITOP targets several different learner groups. In hybrid courses, the learners in our regular face-to-face adult literacy courses use this program to practice their literacy and math skills while they also become familiar with technology. However, this online program is also open to any user across the country. Thus the target audience also includes individuals who may have 33

the basic literacy proficiencies but need more sophisticated comprehension, expression, math and technological skills in a society that is beginning to require higher order cognitive processes. In Turkey, until 1997, the required minimum schooling was 5 years. In 1997, it was changed to 8 years-- which was just revised as 4+4 this year. Hence there are quite a few people who may have completed the minimum required schooling, but need to practice or improve their skills for employment or to continue their schooling through national “Open Education” programs. These individuals are able to use this program on their own and work at their own pace. At the beginning, they take several online placement tests and then start at a level that is appropriate given their existing proficiencies. Pilot implementation and evaluation In the first implementation and evaluation of BITOP, we focused on the hybrid curriculum. Our normal literacy classes meet three times per week. In hybrid classes, one of these classes was replaced by work in a computer lab. The teachers were provided with a curriculum guide describing the online modules that supplemented the classroom content they were covering. All teachers followed this plan although the completion rates varied. In labs the learners worked on their own as the teacher walked around and helped as needed. The performance of this technology group was compared to a control group which had three face-to-face meetings and followed the usual curriculum. Table 1 shows the distribution of these classrooms and the number of learners in the study. As the table Illustrates, both Level 1 and Level 2 courses were included in the study. Evaluation materials All participants were given the following tasks at the beginning and at the end of the course. The tasks had two levels of difficulty and some participants who could not complete the first level tests usually did not do the second level tests. (The numbers by the task name indicate the level of difficulty of the task). For beginners who were struggling with letters and words, the comprehension test was not given. 1. Math-1. This test had 10 multiple choice questions on basic counting, addition, reading a clock and counting money, with a maximum score of 10. 2. Math-2: This test had 10 multiple choice word problems that required ordering numbers, addition, subtraction as well as interpreting a table, with a maximum score of 10. 3. Spelling-1. The participants were given 8 words that were 3-5 letters long and asked to write the words. In scoring, the actual letter as well as its location was considered. This test had a maximum score of 62. 4. Spelling-2. The participants were given 8 words ranging in length between 4-8 letters. In scoring, the actual letter as well as its location was considered, with a maximum score of 86. 5. Word recognition-1. In 3 sets of materials, participants were given 5 pictures and 5 single-syllable words to match to the pictures, with a maximum score of 15. 6. Word recognition-2. The participants were given 8 pictures and for each picture had to choose from three alternatives the correct word that represented the picture. The alternatives were real words that were spelled similarly to the target word. For example 34

for the target word tavuk (hen) the incorrect alternatives were tavus (peacock) and tavsan (rabbit). The maximum score was 8. 7. Picture naming-1. Participants were given pictures of four common objects (basket, tree, candle, stove) and asked to write down the names of those objects. In scoring, the actual letter as well as its location was considered, with a maximum score of 32. 8. Picture naming-2: Participants were given pictures of 10 common objects (rice, scissors, star, button, flag, leaf, apple, carrot, bottle, dog) and asked to write down the names of those objects. In scoring, the actual letter as well as its location was considered, with a maximum score of 104. 9. Reading comprehension: The participants read two short texts, one on diabetes management and the other on life of Ataturk and answered 15 textual and inference questions about the texts. The maximum score was 15. Results Given the realities of adult education classrooms, the timing of the pretest was not always consistent, some pretests were given at the very beginning and some were given several weeks after the courses started. Therefore in the analyses, instead of using pre-post test differences, the posttest scores of the two groups were analyzed after including the pretest scores as a covariate. This approach also enabled us to combine the two course levels, thus gaining more power since not every participant completed every task due to time limitations. All of the posttest scores were evaluated with ANOVAS that included the pretest as a covariate and the group (experimental or control) as the between-subjects variable. The results can be summarized very simply. As summarized in Table 2, in none of the 10 tasks, was there a group effect (all Fs