Training at Artomatic 2012

Artomatic 2012 officially closed yesterday.  If you don’t know already, Artomatic is an occasional DC metro event wherein a massive amount of art is rapidly installed and deinstalled into an otherwise vacant building. I had the opportunity to work there as a site manager, which meant I was onsite nearly every day of the event. I worked alongside gallery managers every day and was helped by a fabulous leadership team behind the scenes. This year it was held in an 11- story building in Crystal City. The show was open for five weeks in May and June. Before opening, we built three bars, dozens of tables, five performance stages with light and sound, and facilitated 1000+ visual art installations. On the weekends, we had 5000 to 8000 people come through. This means several thousand people in the building at any given moment. Total attendance was close to 80,000 people in five weeks.

Another opportunity disguised as a disaster.

The 1200+ participating artists provided all the labor for the event, each artist generally performing three five-hour shifts. Training was an ongoing, everyday task. Some days I was the only person in the building familiar with any of our operating procedures. Every day from Wed to Sun, two or three shifts of about 20-45 artists arrived for their mandatory volunteer shifts. I was not delivering planned, structured training. I was verbally explaining tasks that the learner would have to immediately implement. I usually had less than a minute to debrief a volunteer on their duties before leaving them at their station. I was constantly floating to monitor task performance throughout a 380,000 square foot building. This was a very short, very real-time training cycle!

Defining the Essential: What was important versus not important?

Rapidly assessing my learners and knowing what to communicate to them was essential. Moreover, I was training adults, many of whom were older than I am. I worked in schools for four years. I’ve worked with other adults for 12 years. However, I have never been in the position of telling adults exactly what they needed to be doing at every moment for five hours.  I have trouble imagining a more intensive introduction to training adults. I tried to apply what I know about Adult Learning Theory to my task. I had to know if this volunteer needed to know how his task was important to the overall functioning of Artomatic that day, or this person could perform without that knowledge. I had to know what problems they might encounter (or create!) and adequately explain the solutions. When I failed to do any part of this, the “cogs” did not work well together. Volunteers became flustered, visitors perturbed. When I succeeded, volunteers were able to perform their tasks and visitors’ experience was pleasant.

Previously, I have done volunteer work such as disaster relief on the Gulf Coast and the Smithsonian Folklife Festival. But I have never participated on such a people-intensive event as Artomatic. Definitely a transformative experience in my life. I would do it again in a heartbeat.

Stereotype Threat

Reactivity is a phenomenon that occurs when a subject’s performance changes because he knows his performance is being evaluated. As an educator, I am interested in the effect of reactivity on standardized test performance. In particular, my interest centers on the effect this may have on the persisting difference in test scores between between white and black students in America. Evidence suggests that a particular type of reactivity called stereotype threat may be a contributing factor.

Professor Jeff Stone, a social psychologist at the  University of Arizona, studies stereotype threat. In one experiment, three groups made a series of identical golf puts. But each group was told a different characteristic was being measured: Group 1 was told “sports psycology”; group 2 was told the round would measure “natural athletic ability”; group 3 was told “sports intelligence”. The researchers were not measuring these characteristics. They were looking for correlations between race of the subject, common racial stereotypes, and effects on the subject’s performance. The researchers found the following: being told what was being measured had a statistically significant effect on performance.

  • Group 1: control group. The term “sports psychology” was chosen because it had no bearing on stereotypes. No measured effect on performance between white and blacks subjects.
  • Group 2: “natural athletic ability”. Black subjects’ performance was measurably than white subjects.
  • Group 3: “sports intelligence”. White subjects’ performance was measurably better than black subjects.

The data showed a correlation between a subject’s race, stereotype of that race, and performance when the subject believed stereotyped characteristic was being tested for. In other words, individual performance on this test regardless of actual ability, improved or declined based on racial stereotype, even if it was a negative result. Many other studies reach similar conclusions.

Despite the strong possibility that stereotype threat is affecting childrens’ test scores in every state, lawmakers and administrators use standardized tests to make decisions every day. Decisions that affect children’s beliefs about themselves, about school funding, and applying to college. White/black race relations are inextricably woven into the fabric of American culture and history. As Roger Wilkins said to me in 2001 after a talk he gave at NVCC, “The problem is 400 years old. Is 40 years [referring to civil rights movement] enough to fix it”? So while we may not have a short-term solution to the achievement gap, we shouldn’t exacerbate it by making decisions based on invalid test results.

Theory and Methods Resources for Training Professionals

I find that becoming immersed in particular projects sometimes requires a refresher of what I learned during teaching certification and the Masters program. Here are a few sites I use:

  • Don Clark maintains an excellent, comprehensive site about learning and performance improvement. I have been using it as a reference for years.

  • Another go-to of mine. It has become ad-heavy recently, but the content is still pretty good.  Entries are listed under one of five paradigms, i.e. the underlying epistemological theory about how people gain knowledge.

Student-centered Learning

Lecturing is easy. Successful teaching is hard. Teacher-centered classrooms support the teacher’s impression that “information is being distributed to students”. Student-centered classrooms support the ways student’s successfully receive the information, the way student’s feel about the information, and the way student’s go on to apply that information in meaningful ways. It’s definitely much more difficult.

Not only must the teacher master the content being delivered, but also understand and accept the many ways that students receive it. Despite the administrative efficiency of well-funded schools, learning itself is not an ordered process. Even with desks neatly lined in rows, learning is messy, full of mistakes, wrong concepts that must be corrected, errors of procedure, as well as frustration, anger, and hopelessness. Think back to your own education experiences, subjects that you struggled with. When were you afraid to raise your hand to ask a question? Or wonder why the rest of the class seemed to know what was happening when you did not? Were you ever angry at yourself for not getting it or jealous of others easy success? Alternatively, did you ever dismiss another student’s question as foolish? Wonder why they could not understand something so simple? Did drawing help you understand science better than taking notes? Which was easier: geometry or algebra? To be as successful as possible, the teacher at the front of the room is expected to deal with all of these setbacks. See my post on flipping the classroom to understand why training and supporting teachers ability to work one-on-one and in small groups is so important to ensuring success for all learners.

What Is Instructional Design?

There are long answers to this. I would like to give a simple answer by focusing on the word design. Design is the practice of creating products that are both easy to use and functional. Principles of design have long been applied to consumer products. With consumer products, a good design makes the product’s use intuitive and powerful. A bad design forces the user to waste time, effort, or is unpleasant to use.

Designing the Human/Machine Interface

A human/machine interface is where person and tool meet. The meeting point can be anything from the buttons on a microwave oven to the shape of a shovel’s handle. Consider the amount of effort Apple has put into designing good human/machine interfaces.  Users of Apple products experience the products as pleasant to use, easy to learn to use, and performing valuable functions. It’s no secret that good design has made Apple successful.

Human/Learning Environment Interface

Thinking of the learners as product users is valuable to the designer. Users are active. Users have rights. Users needs must be considered. Traditionally, learners have been thought of as either passive or needing to increase effort when experiencing failure. Increasingly, principles of design are being applied to learning environments: kids/adults in classrooms, an apprentice and a journeyman in a workshop, summer interns at a law firm, basic military training, or Microsoft Flight Simulator. However, considering the needs of the user does not mean compromising the objectives of the course. The needs of the user are considered so that the assessments, the measurements of learning, are valid.  For example, a welder needs to pass a certification test after successfully working as an apprentice. The test evaluates the ability of the welder to follow standard safety procedures. He is a proficient and safe welder, but functionally illiterate. Is a written test a valid way to assess his ability to weld safely? I would say it is not valid. An observational or verbal test would be a better choice. Certainly, gaining literacy would also be good, but the certification was intended to measure ability to weld safely, not literacy. Our welder’s characteristics must be considered, since he is a user of the certification, a type of assessment.

Verifying that assessments are valid is critical to having a pass/fail mark that meets the needs of the user. The users need to pass, but you do not want to pass users who have not learned whatever knowledge, skills, or attitudes (KSAs) the environment is meant to teach. Conversely, you do not want to fail users who have learned what was required. By knowing your users, you know the KSAs each user begins and ends with. Distilled to its essence, the process goes thusly:

  1. pre-assessment of specific KSAs
  2. intervention (i.e. the learning environment imposed on the user)
  3. post-assessment of specific KSAs
  4. repeat as necessary, with appropriate modifications
Any difference in results between step 1 and step 3 are, hopefully, the positive effects of the learning environment. These four steps are supported by many theories, methods, and tools of varying complexity. However, the underlying principle of instructional design, like consumer product design, is knowing your user’s needs.

SCORM and the Value of Metadata to eLearning

What is SCORM? Not the bad guy in a sci-fi film, but a standard for sharing digital content. Developed by Advanced Distributed Learning (ADL), the Sharable Content Object Reference Model (SCORM) was written to simplify inclusion of learning objects in a learning management system (LMS). SCORM is a set of technical specifications that allows SCORM-compliant products to be loaded quickly into an LMS for students to access. Although SCORM was created for the Department of Defense, it is used outside of DOD by many organizations because it is one of the only standards for sharing digital learning content.

SCORM does not currently include metadata tagging for categories like learning activity type, teaching method, differentiation category, or many other pedagogical considerations. So while SCORM is ‘smart’ about IT issues, it is “dumb” when it comes to issues teachers and instructors face when developming courses. For example, I have  often found myself searching the Internet blindly for a well-packaged teaching activity that I know is out there, is better than what I can create in one afternoon, but not sure what search terms to use to find it. Learning Object Metadata (LOM) is a standard that would solve this problem, but it is not used very widely. LOM tags would make it simple to locate an activity that is, for example,

  • in English with Spanish/Vietnamese supplements for English Language Learners
  • falls under relevant state or Common Core Standards
  • is constructionist in theory and student-centered in method
  • about soil erosion and watersheds
  • includes short video with pre and post-quizzes
  • list of materials for hands-on, small group, 20 minute classroom activity
  • relevant to a particular school district’s watershed (e.g. Chesapeake Bay, the Ogallala aquifer, or the Rio Grande)

The state of LOM and metadata is still in limbo. Researchers at MIT report that “The Metadata Unit has found that even with all the forward thinking and cutting edge tech…it is the traditional cataloger’s sensibilities regarding good descriptions and access…that is most valuable in discovering access to the library’s new class of electronic objects”. <sigh> The problem is the same: search terms on the Internet are analogous to the terms chosen by the archivist in any metadata system. Here in the Information Age, awash in information, we struggle to catalogue it all so that we can use actually use and share it efficiently.

Teaching With Stories

Storytelling is fundamental to human communication. Consider how much of your conversation with others consists of swapping stories about personal experience. People have been learning from each other’s experiences for as long as we have been people. Anthropological and linguistic  research suggests that storytelling has been the primary means of learning in hunter/gatherer societies for over 100,000 years at least. With this in mind, it seems important to include narrative when designing effective learning environments.  While facts might capture curiosity about the unknown, narrative is what hooks emotive interest.

In transitioning from hunter/gatherer society to more complex civilizations, when does formal education become important? History indicates that until only recently, few people underwent any type of formal education. These few included specialists like priests and scribes, living in societies where most people were field laborers or craftsmen. These laborers learned through apprenticeship, probably more similar to learning in a hunter/gatherer society than learning in a classroom. In short, education for the masses has only come about in the last couple hundred years. From my perspective, the idea that we are still working out the kinks in education is the understatement of the year; I would guess that the worst kink is that classrooms can be really boring. Do you remember how much better sitting through lecture was when the teacher or professor told great stories? Or how you remember a great novel or film because the story kept you rapt for the next chapter or scene?

I have come across the idea teaching through storytelling several times over the years. While working as a science writer, one of my editors stressed the importance of adding narrative to our dry, fact-saturated science content. As an anthropology major, one of my professors lectured on the power stories (read myth, cosmology, origin stories, etc) have over our understanding of ourselves and our place in the world. As much as I believe in the effectiveness of stories on learning, it is not always easy to do. Combining content with a good story is twice the work of just delivering content. In short, experts (the teacher) deliver what they learned (the content) without the interesting bits about what they went through to learn it. Efficient, in its own way, but not effective for holding the attention of most learners. I came across one interesting method of telling the story over the weekend while watching TED.

I am a big fan of TED, founded to be “a nonprofit devoted to Ideas Worth Spreading”. Every idea is presented on stage, by one person, sometimes with a multimedia slide show in the background. The presenter tells a story of how their idea came about and why  it is important for other people to know about this idea. Already, you see why I like TED. The talk by Nancy Duarte is doubly interesting because she is talking about effective delivery of ideas. She provides a template for storytelling to effectively convey the impact of a new idea. It’s worth watching, but the sum of her talk can be seen in this screen capture. Don’t look at the screen cap below if you want to learn through her story!


3/19 update: TedTalks available for streaming on Netflix. Still available free on YouTube.

Learning with Microworlds

Microworld is the academic term for a simple learning simulation. The simulation will focus on a dynamic but limited environment with only a handful of operating variables with at least one variables under the learner’s control. In my post on using simple games to teach linear algebra, each game I discuss is based around a microworld. Add gaming elements to a microworld, and you have a simple game.

One of my favorite microworlds allows the learner to play with momentum and gravity. Unlike the linear algebra games, this simulation of gravity in a solar system has no mathematical or gaming elements. The learner manipulates velocity and size of objects orbiting around a star to see the effects.

If students took measurements from a planetary gravity simulation such as this, they could derive principles of gravitation and orbital mechanics in a way similar to the way early astronomers did. Research shows that this type of learning, often called discovery learning or recapitulation, is profoundly more effective than instruction received via lecture or traditional textbook. The affordances of digital devices, broadband, and wi-fi make including elements like microworlds in classrooms easy.

Experiencing Failure Is Important to Learning


A teacher I know shared this article on the effectiveness of lecture in the classroom, commenting that this made him rethink his teaching technique. The articles states that lecture quality does little to aid learning. Students who master the objectives of a given class do so with or without the professor. The problem is that this number is minority of students. Test results suggest that peer learning in an experimental environment scaffold learning much better than lectures do. Why is this? It seems that the fundamental concepts are understood at a deeper level when students experience them together instead of than hearing them through the words of the lecturer.

A second article  explores another aspect of learning: making mistakes. According to this article, mastery of a subject requires learning how not to do things at least as learning how. In other words, experiencing both the how and the how not go hand in hand when it comes to building effective learning environments. Consider the possibility of failure while listening to a lecture. It doesn’t even make sense, unless you count falling asleep. From this perspective, the lecture model really fails students.

Lesson for learning: build options for both success and failure into interactive learning environments.