Friday, November 23, 2012

Two Easy Peer Feedback Methods

Peer feedback can be extremely valuable in the BE classroom, especially when it comes to communication skills.  There are two reasons for this.

First, my learners are typically all in the same company and/or department and have a better grasp of the conventions within the discourse community.  Second, the learners have years of experience and training in various areas which they can draw upon to give feedback.  For example, several of my learners include project managers who have taken part in many training sessions on relationship building and giving feedback.  It is great to spread that knowledge.

Of course there are many other great reasons for constructive peer feedback, but there are also dangers, too.  Without direction and some limits, peer feedback can be overly positive or only highlight shortcomings.

For example, here is some peer feedback I received on a proposal I wrote for a university class.



·         Your introduction is very wordy.  I would consider consolidating some of the paragraphs and cut back on so many words.  Once you write the Letter of Transmittal you will realize most of what you wrote in your introduction will also be in your Letter of Transmittal.  Your introduction should be concise and to the point. You use too much detail for the reader in the first two paragraphs, which led to repeating most in the body of your proposal.  
·         Your title page doesn’t include who you ultimately want to read your proposal.  In our textbook on page 289 the title page lists who it is prepared by and who it is prepared for.  The prepared for individual will also be the name you address your Letter of Transmittal to. 
·         I would also consider spelling out what R&D because readers may not understand the acronym. 
·         You are also missing table of contents, which is a requirement for the report.
·         When using visuals you should name them in the proposal i.e. Figure 1, Figure 2, etc. Also if you pulled your illustrations from somewhere else you need to cite those as well. 

Hardly motivating... did I do anything well?  By the way... I did fine on the final assignment.

A Simple 3-2-1

I like to use a simple 3-2-1 feedback format for peer feedback.  I simply write on the board prior to a presentation, meeting, email, etc.

Write:
3 things they did well.
2 things they can improve.
1 thing you want to take and use in your presentations, emails, etc.

Then after the simulation/role play, I give them time to fully write out their feedback for the person.  I do not read them and let the learner look at them without pressure after the lesson.  Typically, the learner will come back the next week and thank their classmates for the excellent responses.  Then when we are giving class presentations, all participants are more likely to give complete, honest, and constructive feedback because they will receive the same in return.

Email Workshops

A second method for extensive peer feedback is email workshops.  I will set up pair groups and give each pair the task to write an email.  Each situation will be similar.  Note:  I will change the emails based on the target function.

For example: (the learners are told to fill in details to fit their job/situation)


Group A
  • To introduce yourself to a new business contact.  You will be working together in the future.
Group B
  • To follow up on a conference.  You met the person for the first time and talked shortly, exchanged cards and agreed to stay in touch.
Group C
  • To get in touch with a former friend / colleague.  You were close before but lost touch after several years.  But now you may need some help from him / her.
Then, the pairs compare and contrast their emails based on subject line, greeting, opening, structure, opening for discussion/response, closing, formality, and length.

Then I will have someone run to the copy machine and make copies for everyone.  Note: I give them an email template on A4 paper with all the top fields and a writing area.

Then the members of the three different groups will read the other emails.  While they are reading, I will mark the emails for accuracy and vocabulary.

To conclude, the members of the groups will meet together and discuss dos and don'ts in there situations, good structure, appropriate phrases, etc.  We will bring all the information together on a powerpoint slide and that will go out to the participants.

The participants absolutely love it.

So, two ideas on how I use peer feedback in my classrooms.

Monday, November 19, 2012

The Communicative Event... Session Recap

I heard my approach to needs analysis mentioned many times this past weekend at the BESIG conference and I am more than a little thrilled to have struck an issue which reached BE trainers.  As a relative new-comer to ELT I am always hesitant about saying how things should be done.  And at a place like BESIG, I am looking out at an audience which, in the case of the PCS, had well over a century of experience.

The purpose of this post is to recap my talk on Communicative Event Analysis, to add some background to the idea, and to reanswer some of the questions which were dealt with rather poorly in the session.

The Talk

During the talk I first outlined what I had learned from the needs analysis tools I see most often on the Internet, in trainer handbooks, or in course materials.  Unfortunately, due to a technical issue, the bullet points did not appear on-screen.  Please take a look at this version of the presentation for those items.




Context

It appears that the term 'communicative event' is gaining acceptance among some readers (and even non-readers) of my previous post "Why Needs Analysis Isn't Working".  Let me provide a little context about how I use this term.

For me a communicative event is anytime the learner is either the sender or receiver in the S/R communication model.  This is different than "an English situation".  In some cases, such as reading a document on the company intranet, the event "understanding and interpreting the information" is the entire situation.  However in many cases, such as a longer meeting or a company visit, there are many events in one situation.

Example 1

I have learners who make customer visits (the situation).  In this case there may be several events.
  • Reporting to reception
  • The initial greeting when being welcomed
  • Relationship building through small talk while walking to the conference room (or other area)
  • Meeting the others for the meeting
  • Talking about the agenda or plan for the visit
  • Starting the meeting
  • Presentation phase
  • Question phase
  • Discussion phase
  • and so on...
The idea with the form is to find the situation and start to define the communicative events.  But different cultures, discourse communities, job functions, and conventions will change the events conducted, the duration, and importance of those events.  My goal from the forms and the follow-up questions is to the find the order and importance of these events.  Artifacts such as old meeting agendas, mintues, slide decks, and emails make this much easier.  In general, however, I will try to focus on those events which are high value.  Reporting to reception is generally not high value.  For a secretary the small talk while going to the conference room will be much higher value than some others to create a welcoming first impression on the guest.

Example 2

Sometimes the communicative event is not directly linked to the situation.  I had a learner who worked in a call center.  At first thought, we needed to be working on telephoning, troubleshooting, politeness, etc.  However, her needs were actually quite different.

She was receiving the calls from German customers about problems with their telephone service.  But the company also had call centers in India.  Therefore, all the troublshooting guides on the Intranet were in English and all the incident tickets had to be written in English.  But she was speaking German on the phone.

So for the training, we needed to focus on quickly searching, reading, and interpreting the troubleshooting guides, and on writing short incident reports in English... all while speaking German.

Munby

Recently, I stumbled upon some past work on the communicative event from 1978 by John Munby.  It just goes to show that there is rarely a truly new idea.  Sadly, I have not even scratched the surface on his research, but it appears he advocated this approach to syllabus design long ago.  My initial impression is that he takes it a bit further (down to sentence level) with "micro-functions".

My apologies to Mr. Munby if I have inadvertently plagiarised his theories.  This was completely new to me until only recently.

Expertise, Assumptions and Materials

The main message I would like to give is three fold.  First, we are the language and communication experts in the room.  Asking the learners to map their own way to success is simply not effective.  Of course, we need to accomodate their goals and expectations.  But they do not know what a function is, they cannot name the words they don't know, and they cannot identify what makes their language different than their target.

Second, we cannot make assumptions that we know meetings, presentations, negotiations, etc.  Course books do an excellent job of providing functional phrases, but often assume that all meetings are the same and all presentations are alike.

Third, we cannot limit the learners' needs to what we know how to teach.  Many times what we know how to teach is either of marginal importance (e.g. making arrangements on the phone) or will not help alleviate their communication difficulties.  Tailor made training means just that... going out and developing original lessons and materials which will help our learners.  Of course, some recycling happens, but trotting out a different permutation of the same stuff is not ideal.

Question and Answer

I'd like to apologize to the audience (both around the world and in the room) for my poor responses to your excellent questions.  I was a bit overwhelmed by the simulcast, the technology challenges, being filmed, the time limit, and staying on script.  Thank you to the gentleman who politely helped me deal with one question.  So here is my second attempt.

Identifying needs for pre-experience learners or 'just-in-case' training

Very valid question... this method is generally limited to learners who currently use English in their workplace and have the goal of improving their immediate performance.  However, after having conducted this type of needs analysis for over a year, I do see patterns which can assist pre-experience learners.  Please check out my post "What I Don't Teach and Why".

One general thing I see in my market is that most communication is internal and virtual.  This means lots of information exchange on progress, processes, and rules but less persuasion and general conversation.  Communication is often reading and writing messages to conduct some kind of transactional information exchange.

For external contacts, I see more voice communication, but typically also little face-to-face contact.  Also, these external contacts are typically long-standing customers and preferred suppliers.  These conversations are more like colleagues than the more traditional customer-provider relationship.

I will also use my gained knowledge of communicative events from other learners to extrapolate what pre-experience learners are likely to be doing.  Sometimes these tasks are similar to what they are already doing in L1, and sometimes we have to make a few assumption based on job function (e.g. accounting, project management, etc.)  One note here is that most course books written for business management schools are heavy on strategy and business theory.  Most daily workplace communication is not about these topics.

Specfic lesson ideas from communicative events

The first thing is that it gives me a clear idea of which functions, grammar, and vocabulary to practice in certain contexts.  I know precisely what kind of authentic materials I should be looking and asking for.  Then, I have a pretty good idea of what to do with them when I have them.

During a role-play it gives a good idea of what roles to assign to whom, or how to change them to fit their real-world needs.  It also gives me an idea of how to design specific lessons.  For example, I have a group of secretaries and we recently rehearsed that long walk from reception to our conference room.  We made role cards for the 'visitors' like: 

You missed your connection in Amsterdam.  You are tired and hungry.  You want to be nice, but would rather not have small talk.  You are looking forward to the end of the meeting and you are dying for a cup of coffee and to know where the restroom is.

This is your first trip to Germany and you are really looking forward to it.  You want to do some sightseeing while you are here and pick up a few gifts.  You really hope the meetings can be short so you that you can look around on your own.

So, I hope that helps cover those two questions better than my spontaneous answers.

Sorry for the long post, below you can find the handout for the session and thanks to all who attended from around the world and in Stuttgart.

Handout - The Communicative Event

Sunday, November 18, 2012

Are We Fulfilling Our Promise?

I would first like to thank everyone who joined us for the PCS at the BESIG 2012 Annual Conference.  I know it was a financial and time committment on your part and I hope that the sessions were worthwhile.

For those who could not attend, I believe you missed a very valuble session and I hope you will be able to make the next one.  But I understand that distance, financial, and training constraints prevented you from joining the session.  So I will do my best here to recap my workshop on assessing and reporting training quality.

Here is the available video of the presentation.  Note, it starts when I am speaking about the benefits of a quality assessment with clients.

http://www.youtube.com/watch?v=ktZiYB9DIbY&feature=youtu.be

Let's start with the presentation and follow with some of the explanation.




Are In-Company Trainers Afraid of Assessment?

As expected at a BESIG conference many of the trainers came from the educational setting in which assessment is a part of life.  However, I see that in the in-company setting assessment is avoided.  As long as the learners leave with smiles and the manager seems satified then we carry on as though everything is hunky-dory.  But there are considerable benefits to a comprehensive assessment program.

Business terminology:
cost-plus pricing
value-based pricing

Kirkpatrick's Four Levels

This is nothing new.  Donald Kirkpatrick described these levels long ago, but they continue to be the gold standard in training assessment for corporate training.  I think we need to be able to accomodate these client expectations of results with quantitative and qualitative data.

Impressions from Workshop

First, I would like to commend Target Training (one of the key sponsors of the conference) for supporting their staff to achieve certification on the Kirkpatrick model.  During the workshop one mentioned that I was not presenting the most recent developments on this.  He is correct, for more info check some of the more recent references.  However, in the sense of ELT and assessing Business English training, I feel that the traditional framework is already a significant step in the right direction. 

To invert the model (as is currently being taught) or to add a fifth level of monetary ROI (as has been advocated) are simply not steps either our profession or our clients are ready to accept.  And unless we are going out and setting up massive training programs, maybe is it unnecessary.  Therefore, it is more practical to focus on the traditional four levels approach.  However, I find it outstanding that this company is not only taking this approach to corporate training, but also developing their people.  It is far too rare in our industry.

Horton's External Factors

The problem with adopting the four levels without consideration is that is can lead to distortions.  It tends to ignore external factors.  I believe the Holton's simple and effective organization resonates which the BE trainer because we can fully identify with these challenges.  Now, Holton actually does not think Kirkpatrick model is effective at all (and they have a personal dislike for each other).  But strangely, his own 'model' looks extremely similar.  So for the sake of simplicity I just super-imposed Holton ideas on the pyramid.

Surveys

A quick note about surveys because we talked a lot about this in the sessions.  These are not the end-all-be-all of assessment.  They are certainly valuable and quite easy to administer, but do not generally tell the whole story.  On one of the first slides, I showed the menu of assessment tools I see being used.  All have their place and all are valid, we simply need to understand which level they are assessing and how external factors can influence them.  I went to the talk by Judith Mader on performance-based testing which reveal some of the challenges with setting criteria.  This is what I use to judge learning, albeit on a smaller scale than her university.

But in response to questions about how to operationalize this I have uploaded an example survey that I use.  This is by no means perfect and I customize certain sections depending on who, what and when I am conducting the assessment.

English Training Feedback Form (Email)

Putting it Into Practice

It would be impossible for me to understand each training situation of the audience and we saw from the feedback that some have never thought about this, some have taken on part of this in their work, and some are already using these methods daily.  Additionally, some have no control over the assessment methods used in their organization.  However, it was very nice to hear some trainers talking about how they planned to change the way they speak with the learners to either get information on the transfer environment or gain insights on behavior/results.

Some other ideas were to review their feedback form, conduct some sort of before and after assessment, and to use a simple method like the workshop notes page in the handout.  I was really happy to hear that suggestion because, of course, this is the way the workshop was designed.

Reporting

This was not really discussed that much in the groups but I think it may be the most important step, especially for training companies running many classes with many trainers.  Because the information for the report will come from many sources it needs to be organized to help drive improvement.  I also think it is the best tool for initiating trainer cross-talk.

For example, Trainer A consistently gets great feedback on reaction.  The learners love her, she plays games and there are lots of laughs.  On the other side, Trainer B scores great on learning and preparing people for meetings.  Sit the two down together and Trainer A gives a few lesson ideas for more fun and relaxation in the classroom, and Trainer B shares how she builds simulations to help for meetings.

I know that reporting sounds like tons of work and a boring admin task.  It is if there is no point, it is actually very motivating if everyone knows that this report will generate suggestions and action points to improve.

So... thanks to all who came!

Handout - Are We Fulfilling Our Promise

Monday, November 5, 2012

Making an Impact by Understanding the Learner's Goal

I am about to share a lesson I taught this morning in a one-to-one lesson.  This lesson flies in the face of many of the 'tenants' of ESL and learning in general.  The intent is to show how teaching Business English in-company can be markedly different from other environments.

Learner profile

The one-to-one lesson takes place once a week for one hour.  It was not set up as an individual training, but the woman is level A1 and there were no other employees with such a low level.  She works in the accounting department for a multi-national and handles a range of international tax issues, mostly around withholding tax.  The learner is in her mid- to late- fifties and generally doubts her ability to learn English.

More importantly, however, is the fact that she doesn't really want to 'speak' English, she only wants to handle her international tasks until she retires.  I repeat this often, but it is definitely true in this case; she does not have a language problem, she has a communication problem.  We have done a communicative events analysis and found that emails to inform others about processes are the most routine situations.  She must also receive and understand emails asking questions about payment status and how to apply for withholding tax exemptions.

Lesson

I had planned to start looking at the passive in the lesson to help her explain a process.  We had just finished looking at adjectives as a method toward the past participle because it fits nicely into L1 German.  However, I started the lesson by asking, "Is there anything specific you want to talk about this week."

She told me that she had to write an email to a supplier about why an invoice has not yet been paid.  Okay... let's do it.  She explained the situation in German and I did a little graphic representation on the flip chart in English to confirm my understanding.  She agreed that I had.

Template

I wanted to give her a template for this type of email.  I boarded a template I use for writing emails of bad news.

1.  Unfortunately / I'm afraid / Sadly...
2.  Why
3.  Give options
4.  Offer help
5.  Apologize for the situation

I clarified that we did not want to take responsibility for the situation because it is not our fault.  She dutifully noted the template.

Then I filled it by writing her email (while eliciting things she should know from our training).  To check her understanding of the sentences, I asked her to translate them into German.

Dear _____,

Thank you for your email.  (Her sentence.  We have worked on emails before)

Unfortunately, we are still waiting for the withholding tax exemption from the central tax office.  We forwarded all your information to the government on/in _________.  But it takes some time for processing.  I cannot say when it will be confirmed.

We can either wait for the confirmation or we can pay the invoice now.  But I must withhold the 15.8%.

I can try to call the tax authority but sadly, I cannot speed up the process.

I apologize for the situation, and I will do everything on my side to complete the transaction.

If you have any questions, please contact me. (Again, her sentence.)

Best regards,

I wrote the email for her on the flipchart (filling the template) and she copied it.  I then asked her to read it aloud for me to check her dictation and to drill a few words.

We then discussed a few elements from the email.

1.  Performatives - We honed in on "I apologize" as this was a new word for her.  Then we looked at other performatives like suggest, propose, invite, request.

2.  either... or... - We will see this construction as we continue looking at processes.  She wrote 5 sentences using either... or....

3.  Forward - This was also a new word for her, although I'm sure we've covered it before.  We made a quick mind map of email words:  forward, reply, attach, confirm, sign.  Okay, they're not all email words, but they fit together for her.

Outcome

To close the lesson, we took a look at the email again and I elicited ways she could use the email as a model for other situations.  In the end, she decided that she would save it in Word and use it whenever she had to give bad news about the withholding tax process.  In fact, she writes this type of email about twice a month.

To conclude, I doubt she can speak anymore English that she could before she came in.  She is undoubtly confused about most of the grammar in her example.  But she does understand exactly what each sentence says.  It will be much easier for us to cover this later.  In fact, we can dissect our example even further in future lessons.

But most importantly, she has a communication tool.  Remember, she doesn't need the English... she only wants to communicate the idea.  I consider this a successful way to spend 1 hour.