 |
 |
2002 ALA Midwinter Conference
Emerging Technologies in Instruction Committee: Meeting I
Saturday, 19 January 2002
Members Present: Peter Tagtmeyer (ch), Marianne Bracke, Erin Daix, John Hickok, Melissa Koenig, Carrie Kruse (intern, Robin Lockerby, Kathy Magarrell, Leslie Murtha, Francesca Lane Rasmus, Judy Smith, Sharon Verba, Janelle Wertzberger
Members Absent: Lori DuBois
Visitors: Jeffrey Barnett, Tina Benedek, Norman Buchwald, James Caufield, Coco Halverson, Gina Matesic, Ericka Raber, Jane Rowland, Sharon Saulmon, Lisa Spagnolo, Eileen Stec, Kathleen Turner
Agenda:
- Introductions
Peter gave an overview of the Internet Education Project for visitors. He then distributed the IEP Selection Criteria (to be reviewed during the meeting) for all to see, as well as the most recent review sessions scoring, for committee members only.
- Announcements
The Chair made the following announcements:
- ALA and ACRL will be hosting a WebCT server. Peter has requested that the Emerging Technologies Committee move to their server in the future. WebCT allows such things as bulletin board discussions, which E-Tech has been doing on Colgate's Blackboard courseware. This would give the committee continuity once Peter is no longer chair.
- IS committees need to provide Midwinter progress reports and planning reports, including the goals and objectives of the committee, report on progress toward those goals, and assessment of how well the committees are meeting their goals.
- IS Dinner plans for Annual Conference in Atlanta are underway.
- The Communications Committee is looking at issues relating to the Web presence of the Instruction Section. There is a push to move committee sites to the ALA server. Peter raised questions with them regarding access to maintenance and updates of the sites as well as style guidelines.
- Membership Committee will be changing the IS brochure and will be sending a welcome letter to new IS members.
- PreConference Committee reported that they have speakers lined up for the Atlanta IS PreConference. The theme this year is instruction for first-year students.
- Deb Gilchrist is soliciting ideas for an IS program at the ACRL conference in 2003.
Handouts announcing other IS Midwinter events of possible interest:
- The IS Discussion forum (Sunday, 1/20/02, 4:30-5:30) will be about Assessment and Diverse Populations.
- The Teaching Methods Committee will hold a brainstorming session (Sunday 1/20/02, 2-3:30) on Learning with Laughter.
- Internet Education Project - Review Process:
In the recent review session held: 37 sites were submitted, of which 34 qualified for review. The 34 sites were divided among teams of committee members for review: each team reviewed 8-9 sites. Of the 34 sites reviewed, 9 met the qualifying criteria.
The committee's decision at Annual was to have another "closing date" for a review session on Jan. 15th. No announcements of this review session have been made yet. 8 sites have been submitted already for the new review session.
The chair explained that this was the first time the IEP database was put into use. Although the review itself was the same as before, previously committee members would submit their scores via email and the chair would manually tally the results. Peter set up a database for viewing the IEP sites, accepting new submissions for review, and for the reviewers to submit their scores, automatically tallying the results.
Discussion: How did the review process go?
The following comments and suggestions were made:
Mechanics:
The mechanics seemed to go well.
Could there be some sort of reminder to print one's scores and comments after doing each review. If you don't remember to do this, you won't have a copy of your scores and comments. Although the breakdown of scores is sent out to everyone, the comments are not. We could either put a text reminder to print if you want this for your records on the final page that gets submitted, or find a way to automate this somehow; either have the scores and comments output to a page that's accessible to committee members, or have the submission of scores send an email to the committee member.
Perhaps review teams should be a mix of "old" committee members and "new" members.
It would be helpful to have the online (bulletin board) discussions synthesized so that committee members could have an easy record of what decisions were made through online discussion.
Submission requirements:
Could there be a better description of what is appropriate to submit to prevent getting things that aren't appropriate? Do we tighten up submission criteria? Make it clear that online instruction is a must at this point? Originally, emerging technologies included online syllabus or class handouts - simply having instruction materials on the Web isn't new anymore.
Current description:
The Emerging Technologies in Instruction Committee (ETech) of the ACRL Instruction Section maintains a database of exemplary user education and training materials that address the discovery, access and evaluation of information in digital network environments. The database is known as the ACRL Internet Education Project (IEP). The IEP is a means for librarians to share instructional materials judged to have merit. Committee members select materials for inclusion in the IEP using these selection criteria. The variety of materials may include Web pages, tutorials, texts, course materials, syllabi, workbooks, and self-instructional materials.
http://cooley.colgate.edu/dbs/iepsubmit/
There is a potential problem with doing too much definition or using specific examples of emerging technologies, because such technologies are, by their nature, constantly changing; we might prevent submission of something that's so new we haven't found a way to fit it in the definition. No matter how well things gets explained, there will always be a few things that don't fit what we mean that get submitted anyway.
Decision: Melissa will take charge of looking at the description on the submission page for potential revision.
Review Timeline:
There are concerns about the turn-around time for review. When all the sites are batch-processed in one big group, it's harder to get them done in a timely manner because there are so many to do.
Could there be some sort of rolling date for review? As submissions come in, assign them to some committee members to review at that time. Doing one or two at a time works well and isn't as overwhelming as doing eight or nine.
We still need publicity for submitting projects that generated interest. Also we need to be better established, with a strong reputation, in order for people to make submissions on their own (without reminders from us). Deadlines can help people be motivated to make the effort.
Scoring:
The scoring of how things meet the criteria was uncertain for many.
Suggestions:
- we create a scoring rubric to define the criteria
- we provide a better method for training new members on the review process
- when scores are given, especially lower scores, the comments reflect why the scores were what they were
- Internet Education Project - Selection Criteria:
Copies of a sample rubric from Melissa were distributed. Melissa explained that a rubric provides a checklist or a grading method that defines scores. The criteria are set and described.
Decision: we need to rewrite the criteria first, then work on the rubric.
Question: was there discussion as the criteria were established that all are weighed equally?
Weighting scores could get complicated. Suggestion: this can be dealt with by grouping together criteria that are "less" important as one criterion.
Rewriting of Criteria:
Original #1: All information included within the material is accurate and current.
Concerns:
We need to be realistic about submitters having ample chance to keep up with resources (e.g. a tutorial on a database and the screen shots need regular updates). Do we need to separate accuracy from currency? Is currency a part of accuracy?
Suggestion:
Remove "currency" from this criterion.
Decision:
New wording: All information included within the material is accurate.
Original #2: This material presents a unique or creative approach to content.
Concerns:
How is this different from criteria #7 and #8? #2 refers to content, #7 refers to design, #8 refers to technology
What does "approach" refer to? It's murky. What do we want to achieve by asking about a unique or creative approach to content? Is this about pedagogy? Effectiveness is more important than uniqueness when talking pedagogy.
#5 is also about meeting learning objectives. Can we add that in?
Suggestion:
Rewrite #2 to be about pedagogical approach: does it teach well? Add wording from #5 about meeting goals and objectives.
Decision:
New wording: The instructional design is pedagogically effective, i.e. it teaches well according to the scope and learning objectives stated by the submitter.
Original #3: The organization of the material is clear and easy to navigate.
Concerns:
Is this asking about aesthetics or usability?
Suggestion:
Focus on usability: easily navigated, organized well, user friendly, good help available.
Decision:
New wording: The organization of the material is clear and easy to use.
Original #4: The content and the language of the material is clear and effective.
Concerns:
This seems fine. We just need to add examples to the rubric: free of jargon, at an appropriate level, etc.
Decision:
Leave wording the same.
Original #5: This material matches the scope and learning objectives stated by the submitter.
See #2
Original #6: This material is relevant to those outside of the developer's institution either because of content or the presentation of a model for other developers.
Concerns:
This seems fine; it just needs a slight rewording to flow better
Decision:
New wording: This material is relevant to those outside of the developer's institution either because of content or presentation of a model for other developers.
Original #7: This material is designed in a unique or creative way.
Concerns:
Is this about aesthetics or instructional design? Aesthetics. (#2 is about instructional effectiveness)
Do we need a separate criterion about unique design? Can we drop this criterion?
If we drop it, there's no mention of "unique or creative" in any of the criteria; that wording was already dropped from #2.
Suggestion:
Add wording to #10 about "unique or creative"
Decision: Drop #7. See #10 for rewording that incorporates idea of "unique or creative"
Original #8: This material provides instruction using emerging technology, or existing technology in an innovative manner.
Concerns:
This one seems fine. We need examples for the rubric.
Decision:
Leave wording the same.
Original #9: The technology used to create the learning material enhances the learning experience, i.e. is appropriate and effective.
Concerns:
Is this the same as #2? No, this is about use of the technology.
Suggestion:
Remove the first instance of "learning"
Decision:
New wording: The technology used to create the material enhances the learning experience, i.e. is appropriate and effective.
Original #10: This material makes effective use of graphics, examples, interactive elements such as programmed feedback and flexible learning paths, and other supporting elements.
Concerns:
Seems fine, once we add in "unique or creative"
Decision:
New wording: This material demonstrates unique or creative use of graphics, examples, interactive elements such as programmed feedback and flexible learning paths, and other supporting elements.
Re-ordering of the criteria: General categories: Pedagogy, Technology, Content, Clarity, Creativity, Relevance to others.
New order:
- The instructional design is pedagogically effective, i.e. it teaches well according to the scope and learning objectives stated by the submitter.
- The technology used to create the material enhances the learning experience, i.e. is appropriate and effective.
- This material provides instruction using emerging technology, or existing technology in an innovative manner.
- The content and the language of the material is clear and effective.
- All information included within the material is accurate.
- The organization of the material is clear and easy to use.
- This material demonstrates unique or creative use of graphics, examples, interactive elements such as programmed feedback and flexible learning paths, and other supporting elements.
- This material is relevant to those outside of the developer's institution either because of content or presentation of a model for other developers.
Questions about the criteria in general:
Will the sites recently accepted be re-reviewed with the new criteria?
Decision: No. Only new submissions will be reviewed with the new criteria.
What about the submissions already received but not yet reviewed?
Decision: They will be contacted once the new criteria are in place and notified of the change.
Clarification: The rewordings agreed upon are not substantial changes; we see them as helping to clarify what was meant in the original criteria.
Should we add a criterion about accessibility? (ADA accessibility issues.)
Although accessibility is extremely important, this project is looking at emerging technologies, and the capabilities to make such technologies completely accessibly is often lagging behind.
Decision: We won't add a criterion about accessibility at this time.
Rubric: Will be worked on online.
Deferred to Sunday's meeting:
Decision about Review Schedule
Setting up new Review Teams
Technology Survey
Survey to IEP accepted sites
|
|
