In the last installment of Kerby the Herding Dog, I wrote about bringing Kerby to a friend’s farm to see how he felt about sheep. Discovering that he is, in fact, very enthusiastic about sheep and farms in general, it was time to start training. But what to train and how to train it? I’ve already written quite a bit about my desire for a positive reinforcement-based approach, but positive reinforcement training plans can look a lot of different ways.
Trial-and-error learning
I suspect that most of us have had some experience with trial-and-error learning, a situation characterized by the opportunity to make either “correct” or “incorrect” choices in a learning context. The idea with this type of experience is that the learner makes the correct response because it has been followed by something “good”, a reinforcer; and does not make an incorrect response because doing so has resulted in no reinforcement (extinction) or something “bad” (punishment). In the case of herding, a trial-and-error approach might look like putting the dog in with sheep and, either through the sheep’s movement or the delivery of reinforcers from handler, the dog will contact reinforcement for “correct” herding behavior, and the handler will punish “incorrect” herding behavior.
When I reached out to sheepherding instructors in my area, they told me that this kind of approach was not only desirable but necessary. However, as you’ve heard us say plenty of times before, with procedures like extinction and punishment we know there are a slew of negative, damaging side effects we can expect (see references below).
What I wanted for myself and Kerby in our herding training was, first and foremost, joy! I wanted training to be fun. Errors that lead to extinction or punishment are not part of a joyful training experience. To that end, my goal in herding training, and in all training really, is to take an errorless approach.
Errorless teaching
Errorless teaching was first described in an article by Terrace (1963a). Terrace aimed to train pigeons to peck a red light (the correct response), and not to peck a green light (the incorrect response). Instead of a trial-and-error approach in which the researcher presents both lights at the same time and provides reinforcement for pecking the red light, and extinction or punishment for pecking the green light, Terrace first presented the red light only. When the pigeons were consistently pecking the red light, the green light was very gradually faded in. The green light didn’t start at its full intensity, but instead was presented briefly and dimly. Over time, the green light was presented for longer periods at brighter intensities until the pigeon was presented with both the red and green light, and the pigeon continued to peck the red light. In this case, the pigeon didn’t have to learn not to peck the green light by pecking it and contacting extinction or punishment. Instead, the pigeon learned the task (peck red light not green) through this gradual introduction of the green light. This important study demonstrated this method resulted in greater accuracy of responding compared to a trial-and-error method, and that because of a lack of errors, the incorrect stimulus (green light) did not acquire aversive properties.
That is to say, pigeons who learned the discrimination through this method got it “right” more often than ones who were exposed to trial-and-error learning, and they engaged in fewer behaviors suggesting stress and frustration compared to the trial-and-error birds.
**Note: For all of us used to red being the “stop” signal and green being the “go” signal, I kept getting tripped up reading this study where red was the correct choice and green the incorrect! Come on Terrace!
Researchers have gone on to explore the many applications of the errorless approach (Mueller, Palkovic, & Maynard, 2007). In the case of training my little herding dog, what I hoped to do for each skill was to arrange the teaching conditions such that Kerby would be very likely to make the “right” choice – the target behavior – and be very unlikely to make a different choice – an error.
But errors will happen, that’s just life isn’t it?
I should note here that errorless learning is something to aspire to. None of us are perfect. I make mistakes all the time, they don’t call me Stephanie Raining-Treats-From-The-Sky Keesey-Phelan for no reason (this came after a few too many times of bending over to feed a dog and spilling the contents of my treat pouch everywhere. I became very popular with the shelter dogs!). If I could just get a treat from my hand to my dog’s mouth in one fluid motion it would be a miracle, though I’d like to point out that I do not consider this a personal fault, just something I have to work really hard on! Anyway, I digress. Mistakes and errors will happen, but that doesn’t mean we should just throw in the towel and not strive to improve our training methods.
I keep two quotes close at hand for times when errors occur. The first is from Skinner, “The rat is always right.” In this case, we may translate it to: “The dog is always right.” This boils down to mean that any errors that occur are really mine as the trainer. I am the one orchestrating the training environment and the dog, Kerby, is always responding as a result of 1) his learning history, 2) the current context, and 3) his behavioral genetics as a mini Aussie. The second quote, though I’m not sure who to attribute it to, is that “errors are information.” So when errors pop up, my plan is to use that information to consider how I can make changes in the next repetition or session to ensure Kerby’s success and as a result (hopefully!) joy in training with me.
But you can’t control everything!
Not all training with positive reinforcement is designed to be errorless, but that is the goalpost I set for myself. One common misconception when it comes to both positive reinforcement and errorless learning is described by a sheepdog trainer I reached out to:
“Corrections, appropriate ones, are expected to teach the dog what is not acceptable vs letting them try a bunch of stuff until they get a reward marker.”
But if we think back to the pigeons and their red and green lights (Terrace, 1963a) you can see that errorless teaching is not about letting the dog try a bunch of stuff until they get a reward marker. The statement above reflects a misunderstanding about how positive reinforcement works. R+ training generally and errorless learning more specifically is about teaching that is carefully designed and crafted to ensure that the “correct” response is the most likely one.
Going “Errorless”: An Example
So if we’re going to go for errorless, what does that mean practically speaking? There are plenty of ways to go about setting up an errorless learning session but I thought it might be helpful to talk through an example that demonstrates what the process has looked like for Kerby and me. But before I do that, I want to take a moment to provide a few important clarifications:
-
Although I strive for Kerby to have an errorless experience as a learner, my design of the training session is far from “errorless.” As you’ll see below, I start with a behavioral definition and then try to figure out if there are any current situations in which the behavior is already happening that I could pull into my session. But as noted above, I’m just a human and I don’t always observe every relevant variable. Sometimes I miss something and get an “oops” moment because Kerby does something I didn’t expect or wasn’t planning for. In those cases, I typically shake my head at myself, laugh, and take a break to figure out what a better training set up might be. In that regard, though I’m aiming for Kerby to get it right with each repetition of a skill, my process of designing a training session remains very much trial-and-error.
-
The next thing is, I find it enormously valuable to tackle this sort of thing as a group project. In this case my colleague and friend Maia Huff-Owen, fellow behavior nerd and herding trainer/ enthusiast, graciously worked with me to develop these training plans. The benefits of another set of eyes are innumerable. Maia and I worked together via Zoom and she observed things I missed, asked insightful questions, and brainstormed with me until we found a procedure that resulted in an errorless approach (and a pretty darn good version of the behavior, if I do say so myself!). Locally, I was fortunate to work with another wonderful colleague and friend, Kim Palermo who is training her lovely collie, Jane, to herd. Kim and I set up training dates together with our dogs to practice herding skills at Kim’s training facility and at her home with her flock of ducks. This in-the-moment training, feedback, and brainstorming was invaluable. If you have the opportunity, I can’t recommend working with a trusted friend and trainer enough.
Okay enough with the caveats, let’s look at the creation of a real herding training plan!