#EDU16 – Seeking Evidence of Impact: Methodologies in Teaching and Learning

img_1035Session Description: Benchmarking and collecting evidence of impact is important in any undertaking. In this respect, postsecondary teaching and learning presents a unique set of challenges and complexities, so the key is to identify those methods that will produce useful and actionable results. This session will consist of overviews of evaluation and research techniques and methods in three important domains in teaching and learning: online and blended learning, classrooms and learning spaces, and the LMS. We’ll conclude the session with a discussion with our domain experts about the relative strengths of the approaches they presented and considerations with respect to implementing them.
Outcomes: Learn about evaluation and research methods in key teaching and learning domains * Discover new methods that will help you conduct evaluations at your campus * Understand how these methods could be used at your institution

Speakers:

  • Tanya Joosten

    Director, Digital Learning R&D, DETA Center,University of Wisconsin-Milwaukee
  • Adam Finkelstein

    Academic Associate,McGill University
  • Malcolm Brown

    Director, EDUCAUSE Learning Initiative,EDUCAUSE
  • Terence O’Heron

    Director of Operations, Teaching & Learning w/Technology,The Pennsylvania State University

Notes:

  • EDUCAUSE Assessment Resources
    • Learning Space Rating System
    • Information Security
    • Information Security Program Assessment Tool
    • IT Risk Register
  • EDUCAUSE Benchmarking Resources
    • Technology Research in the Academic Community
    • Core Data Service
    • Benchmarking Service

The big question… “Are these (active learning) rooms worth it?”

  • A Framework for Evaluation
  • Level of Impact (Kirkpatrick)
    • Reaction, Learning, Behavior, Results
      img_1030
  • Potential vs Actual – There is always potential but it may not be realized.
    img_1032
  • LSRS Sections – Institutional Readiness, and Features of Physical Spaces
    img_1031
  • Measuring Actual Use of Learning Spaces
    • Surveys
      img_1033
    • Observation forms were used to see how the classrooms were actually being used.
    • Heat Maps to track movement of instructor and students in the classroom.
  • Think about…
    • Providing the right people , the right information, at the right time
    • Focus on high quality faculty development in high quality spaces
    • Instructors are allowed to seek the potential and lead toward the actual
  • Online and Blended Learning
    • DETA – Toolkit Download is Available
    • The toolkit includes: access, learning effectiveness, satisfaction, instructional effectiveness
    • Research model:
      img_1036
    • Research questions include:
      img_1037
    • Framework of Inquiry:
      img_1038
  • Penn State LMS Research
    • Criterion
    • Assessment Strategy – Direct observations, focus groups (faculty and student), surveys (faculty and student), vendor review (support, training, and ID staff)
    • Business Requirements – Functional, technical, support, training, governance, partnership, exit strategy, cost

Faculty development is key across all implementation and research…

#EDU16 – Only Blackboard: Innovation to Drive Student Success through the LMS and Beyond

 

Session Description: Your institution’s needs are unique, and as the industry’s most experienced education technology provider, Blackboard provides innovative solutions that extend beyond our foundation in the LMS. In this session, hear about innovations across our flagship Learn product line as well as new advancements in the use of predictive analytics and mobile technologies designed to help your institution promote student success.

Outcomes: Leverage data from your LMS to improve student outcomes * Empower students to improve pathway planning * Use technology to scale academic advisory services

Speakers:

Brook Bock, Vice President of Mobile,Blackboard Inc.

Mike Sharkey, Vice President, Analytics,Blackboard Inc.

Notes: 

  • Need to Innovate in Higher Education
    • Education is evolving and students are experiencing a different
    • Students are increasingly non-traditional
    • The students’ journey is different from what has beenimg_1007
    • Students have a lot of educational opportunities
    • Students have high consumer expectations
    • At Blackboard we are taking a user-centered approach in designing our products.
    • Students, faculty, advisors, administration, recruiters and there needs to be a unified experience.
    • Consistent and integrated experience – a new Ultra interface is beyond the look and feel but it’s about a connected experience
  • Student Planning
    • There are 12.5 million with some colelge credits and no degree. – Harvard Business Review
    • The most popular major among incoming freshman is undecided
    • Students take an average of 134 credits when 120 are sufficient for a bachelors degree
    • 30% of non-returning students left because they couldn’t find the classes they needed at a time that would fit into their schedule.
    • Introducing Blackboard Planner
      • This empowers students with the information they need to chart the course of the path to graduation
      • One app for all planning and learning activities – Bb Planner is integrated into the Bb Student mobile app.advise_02
      • Students navigate through the Planner app to enter information about their interest and then discover careers based on interests.
      • Bb has integrated with Burning Glass to power the apps decision tree.
      • Partnered with Road Trip Nation – to give students an idea of what it is like to work in a specific career.
      • Reaching out to a campus advisor is also easy, right from the device.
      • Planner also provides their status as to their progress toward a degree and then there is a “what if” feature that allows students to discover a new degree or pathway based on what classes they have already taken.
      • At the institutional level the system can help advisors identify at-risk students.
      • Advisors can also connect through Collaborate to have a real-time session through mobile.
  • Predictive Analytics
    • Bb Predict is based on a solution created by Blue Canary.
    • Date is collected from SIS and LMS – and asking questions like will the student pass the class, will the student show up in class, and then putting this information in front of advisors, faculty, and students.predicthero_616x418_tcm21-40961
    • The goal is to take the raw data and distill that into of something of value – with the end goal: Student Success.
    • PROBLEM: What is the question you are trying to answer? This is the place to start when looking at data/analytics.
    • INGEST: Aggregate the data from multiple sources.
    • MODELING: Take and turn the data into valuable info (answer the question)
    • DISTRIBUTE: Put the valuable info into the hands of those who can act on it
  • Data Science
    • How can we use data to make things better. Blackboard’s data science team is to further the state of the art in research, share that research, and help to improve products for that institutions can become more effective.
    • Student activity in courses was reviewed by Blackboard from 70,000 courses from 927 institutions, with 3,374,462 unique learners that resulted in the following categories of “course types”:course-archetype-717x359
    • Bb Collaborate data is also used to evaluate student activity in the creation of an engagement score.

See Blackboard Blog Post: Patterns in Course Design: How instructors ACTUALLY use the LMS