Strategies Proposed for Listening Comprehension in PBT TOEFL: Tricks that Do Not Work

: The Longman Preparation Course for the TOEFL, written by Phillips in 2003, is the most widely used coursebook for TOEFL test preparation. This book provides 17 strategies for Listening Comprehension Part A. Therefore, the current study examines whether the strategies provided in the book are applicable in a real test. The data were obtained from Part A of the TOEFL listening section in nine tests designed by Educational Testing Service (ETS). Each item in the test was tested against the proposed strategies. The results reveal that many items did not successfully match the strategies. Only two proposed strategies had matches of higher than 80%, while others were less than 10%. There are four strategies that did not have a match at all. All in all, very few strategies can be applied to answer the questions correctly without adequate comprehension. With comprehension, test takers can answer the questions without needing to rely on any of the proposed strategies. Therefore, the study concludes that most of the strategies proposed by Phillips (2003) are not successful in real test.


INTRODUCTION
Listening received little attention in second and foreign language instruction before the 1950s. However, it is now considered a very significant part of language learning because experts have discovered that it facilitates language acquisition (Munoz, 2013, p. 3). Listening is perhaps the most important language skill to develop (Selamat & Sidhu, 2013, p. 422). Therefore, researchers have shifted their focus strategies that improve listening skills (Dong, 2016, p. 150). In order to measure learners' listening skills, the Educational Testing Service (ETS) provides a standardized language test known as the Test of English as a Foreign Language (TOEFL). The test is regarded highly by academicians since it is one of the admission requirements for most universities in the world (Golubovich, Tolentino & Papageorgiou, 2018, p. 8). Paper-based TOEFL (PBT TOEFL) tests listening, structure, reading, and writing. Among those language skills, listening is the most difficult skill to teach (Walker, 2014). This skill requires much practice and adequate exposure to the language (Megawati, Mustafa & Bahri Ys, 2016, pp. 352-353). For general listening, many suggest accessing videos such as movies to develop the skill (Muthalib, Bahri Ys, Mustafa, 2019;Safranj, 2015, p. 172). According to Bisson et al. (2012, p. 415), this process takes a long time to improve listening comprehension. To teach listening for TOEFL, the TOEFL preparation coursebook provides test-taking strategies. The most popular TOEFL preparation coursebook is Longman Preparation Course for the TOEFL Test: The Paper Test written by Deborah Phillips. It provides 17 strategies meant to be learned and practiced by TOEFL test candidates to help them answer the listening comprehension questions in the TOEFL test. However, to what extent these strategies are helpful is a question which needs to be answered through research. Therefore, the current study analyzed whether the proposed strategies are applicable in a real PBT TOEFL. The results of this research are significant for both TOEFL test candidates in planning their test preparation and instructors in designing syllabi for TOEFL training.

A. Test of English as a Foreign Language (TOEFL)
English language ability can be measured in many ways. One way is through the Test of English as a Foreign Language, which was published for the first time in 1960 (ETS, 2011, p. 2). The test itself is provided by a nongovernment organization named ETS (Educational Testing Service) (Elfiondri et al., 2020, pp. 54-55). The test is used all over the world. With more than 4,000 test centers in more than 165 countries, the TOEFL is used as the primary English standardized test (ETS, 2020, p. 4). There are four versions of the TOEFL: computer-based, internet-based, paper-based, and institutional.
The Computer-Based TOEFL (CBT TOEFL), introduced in 1998, consists of four sections, listening, structure, reading, and writing, with possible scores between 0 and 300 (ETS, 2020, p. 6). With the discontinuation of the CBT TOEFL in 2005, the Internet-based TOEFL was introduced (ETS, 2011, p. 3). The Paper-based TOEFL (PBT TOEFL), also a product of ETS, has only three sections with possible scores between 310 and 677.
The first section in the PBT TOEFL is the listening comprehension section, which consists of three parts -short conversations, long conversations, and short talks and lectures (Mustafa & Sofyan, 2018, p. 275). The second section deals with 40 grammatical questions divided into two partsincomplete sentences and error analysis (Mustafa, 2015, pp. 347-348). The third section is reading comprehension consisting of 5 passages, with about ten questions per passage (Elfiondri et al., 2020, p. 55). The PBT TOEFL, unless it is the Institutional PBT TOEFL, has a writing section called TWE -Test of Written English (Salma, 2018), which is given to the test-takers before the other sections.

B. Listening Comprehension Section in PBT TOEFL
Listening comprehension in the paper-based test consists of 50 multiplechoice questions that are divided into three sections (Kasim, Muslem & Mustafa, 2019, p. 482). Phillips (2003, p. 22) stated that the first section has 30 questions based on a short conversation between two people. The second section is based on two 60-90-second conversations between two speakers in which 4-5 questions come after each of the conversations. The last section is three 60-90-second talks with 3-4 follow-up questions after each talk. Nemati, Qareqani, & Fumani (2016) proposed specific factors which predict item difficulties in listening comprehension of the PBT TOEFL. They recognize two major factors including infrequent vocabulary and the role of speakers. For the first factor, when unfamiliar vocabulary was used, the number of participants who could correctly answer the question was low compared to when only familiar vocabulary was used. Finally, the role of the speaker determines the difficulty of a question in the TOEFL PBT. When the speaker's role is not familiar to the participants, the question is significantly more difficult.
However, considering the three parts of the listening comprehension test, the first part was predicted to be the least difficult part because one of the problems in listening proposed by Nushi & Orouji (2020, p. 10), i.e. the problem of difficulty in focusing, is very minimum in this part. It is because the conversation is short, and thus does not require intense concentration. In addition, when the conversations are longer, as in the second and third parts of the test, students feel discouraged and bored (Hamouda, 2013, p. 125).

C. Listening Strategies Proposed in Literature
There is some debate about whether listening strategies need to be taught (Chen, 2010;Fathi, Derakhshan & Torabi, 2020;López, 2017;Sharaf, Mobaraki & Nia, 2018). Nation and Newton (2009, p. 66) claimed that listening strategies are not applicable for a test taker. However, Fathi & Hamidizadeh (2019) were convinced that mastering strategies for listening can improve student's listening skills. Some language teachers also favor the idea that listening strategies are helpful for students (Golubovich, Tolentino & Papageorgiou, 2018, p. 5). See Table 1 for the listening strategies that was proposed by Nation and Newton (2009.) The listening strategies proposed in Table 1 mostly apply to non-test situations. Oxford (1990, p. 49) provided a listening strategy which is applicable in testing situations, i.e. guessing intelligently. Oxford (1990, p. 91) listed linguistic clues and other clues as methods of intelligent guessing. Linguistic clues include suffixes, prefixes, and word order. Other clues which can help learners in making an intelligent guess are provided in the following table. Previous research has studied the application of listening strategies in language tests. Among others, Dong (2016) found that listening strategies did not correlate with listening performance among highly motivated learners. However, Moradi (2013) found that undergraduate freshmen given listening strategy instruction outperformed those who were not given any instruction in a TOEFL listening test. Furthermore, a research study also discovered that, in terms of listening proficiency, there were no evident differences between those who were given strategy training and those who were not (Chen, 2010). Gholaminejad (2014) showed that strategy instruction benefits low-proficiency learners less than others.

D. Listening Strategies Proposed in Phillips (2003)
Preparation Course for the TOEFL Test written by Deborah Phillips provides strategies for each of the 3 parts of the listening comprehension section of the TOEFL test as follows: 1. Part A. Short Conversation Phillips (2003) proposed 17 strategies termed "Skill," that are categorized into six groups. Each strategy is provided with a definition and steps, as in the following example. b. Listen to the first line of the conversation. If you understand it, that's good. If you don't understand it, don't worry because it probably does not contain the answer.
c. Be ready to focus on the second line of the conversation because it probably contains the answer. Repeat the second line in your mind as you read through the answer in the text.
The complete list of strategies proposed by Phillips (2003) for Part A is presented in the following table.  Phillips (2003) are similar to those of Part C. The strategies are divided into two phases, i.e. before listening and while listening. Table 4 presents these strategies.

METHOD
To collect the data for this research, nine samples of the test published by the Educational Testing Service (ETS) were used. The tests were once used for official tests and, therefore, authentic and suitable for the purposes of the study. The tests come in three different publications, as presented in Table 5. This research comprised two stages of analysis. In stage one, each number in each test from all publications in Table 5 was matched to the strategies proposed in Preparation Course for the TOEFL Test written by Deborah Phillips. This book was popular because it is very easy to use, and the topics presented in the books are systematic. Both practice tests in each topic and the complete practice tests provided in the book are relevant to the topics discussed in the book. The earlier version of this book, published in 1989, provided practice tests which are very close to the real TOEFL designed by ETS from the early to mid-1990s, though only for the Structure and Reading Comprehension sections.
In the second stage, the items that have matches in the strategies were analyzed to find out whether the strategies can actually be applied to answer the questions. In the analysis, the writer answered the following questions: 1. Will the strategy narrow down the number of potentially-correct options? 2. How many distractors can the strategies eliminate?
The questions above were constructed with the consideration that if the strategies are effective, they need to give a high chance of obtaining good scores when the strategies are applied in taking the test. To answer the questions, the authors read the recording script, either the second speaker or both speakers as suggested by the strategies, and the options in the test material.
Using the strategy, the authors decided which distractors can be eliminated. The strategies were considered to work perfectly when all three distractors could be eliminated. The listening comprehension section in the PBT TOEFL comprises three parts, i.e. Part A, Part B, and Part C. Phillips (2003) did not propose specific strategies for Parts B and Part C. Therefore, the analysis in this study was restricted to the 17 strategies of Part A.
The results of both stages of analysis are presented via percentages in the following sections. The percentage for applicable strategies was obtained by dividing the number of questions for which the strategies were applicable by the total number of questions, i.e. 270 (30 questions in nine tests). The percentage for distractor elimination was calculated by dividing the number of strategies which can eliminate a certain number of distractors by the number of questions for which the strategies are applicable.

RESULTS
Based on the data collected from test materials published by ETS, not all strategies proposed by Phillips are potentially applicable in completing the real TOEFL. The results are presented in two tables. The first table reveals how many strategies can be potentially implemented during the test. The latter explains how many distractors that the strategies can eliminate.  Table 6 shows that the majority of questions can potentially be answered using the strategy of focusing on the second speaker (88%) and the strategy of avoiding similar sounds (87%). Two of the strategies can sometimes be applied in the real TOEFL, i.e. drawing conclusion (8%) and expression of suggestion and uncertainty (8%). However, the other strategies proposed by Phillips (2003) can rarely be applied (<3%). Regardless of their potential applicability, few strategies can actually be applied to correctly answer the questions in the TOEFL. These results are presented in Table 7.  Table 7 shows that when strategies are applicable, the strategies can eliminate all distractors except for negative expression and idioms, none of which can be eliminated. In Table 7, some strategies can only eliminate one distractor, which narrows down the chance of selecting the correct answer. Other strategies can eliminate two distractors, and the others can eliminate all distractors, leaving only the correct answer. However, regardless of their potential application, many strategies cannot be used to eliminate any distractors.

DISCUSSION
The objective of this research was to find out whether the strategies for answering listening comprehension questions in the PBT TOEFL as proposed by Phillips (2003) are applicable in taking the real TOEFL. The data were collected by finding matches between strategies proposed and nine historic listening tests designed by ETS. The results show that most strategies are not applicable. Some strategies are potentially applicable but they cannot eliminate distractors and thus cannot lead to correct answers.
The most potentially-applicable strategy was "focus on the second speaker." The study found that 88 percent of the questions can be answered only by listening to the second speaker in the conversation of Part A. However, this strategy can only be applied when a test taker can comprehend what the speaker says. In the case that the test taker does not completely understand what the second speaker says, the book suggests that he or she select an answer which has the synonym of keywords that they recognized from the conversation. This strategy can be applied to 28% of the questions and all distractors from 87% of those questions can be eliminated. When the test taker does not understand any of the conversations, the book recommends that the test taker avoid choosing the options which sound similar to what the test taker hears in the conversation. This strategy is potentially applicable to 87% of the questions, but all distractors can be eliminated in only 23% of them, two distractors in 48%, and one distractor in 29% (See Table 7).
The book provides a strategy to answer questions about who, what and where. The test taker is suggested to pay attention to keywords in the conversation in order to draw conclusions about who the speakers are, what they are speaking about, and where the conversation takes place. Based on the analysis, this strategy has matches in 8.15% of the questions in the real test. This strategy is very effective because it can eliminate all distractors in 87% of those questions. However, test-takers need to have a wide range of vocabulary to make such conclusions. In fact, Wang and Treffers-Daller (2017, p. 148) found that vocabulary is one of the strongest predictors of listening comprehension. For the strategy of listen for uncertainty and suggestion, the book only provides the way to identify the skill tested. For expressions of uncertainty, the speaker usually uses ... isn't it (tag), as far as I know, or as far as I can tell. In the real test, however, only two out of 270 questions can be identified by using this strategy. For expressions of suggestion, the book suggests to listen to the expressions Why not and Let's because those expressions signal suggestion. However, among 20 questions (7.4%), only 6 of them (30%) use Why not and Let's. In short, this strategy is not very useful for a test taker. In addition, the strategy does not state how identifying the expressions helps the test taker to determine the answer to the questions.
Idioms are found in the listening section of every TOEFL. There were seven questions found in the nine tests analyzed in this research. Phillips (2003) provided a list of 48 idioms. However, none listed in her book appears in the actual tests. According to Rafatbakhsh & Ahmadi (2020, p. 212), there are more than 5,000 idioms in English. Therefore, the chance that the idioms in Phillips' list appear in the TOEFL is less than 1%. However, the meaning of idioms can be determined if the test taker understands the context (Rafatbakhsh & Ahmadi, 2019, p. 17). Therefore, high-achieving listeners will most likely be able to answer the questions about idioms although they are not familiar with the tested idioms. Oakhill, Cain, & Nesi (2016, p. 133) also found that the understanding of idioms is not related to the understanding of the expression, suggesting that learning the meaning of idioms in isolation is not necessary. In short, the list provided by Phillips (2003) is not very useful. Other strategies proposed by Phillips (2003), as presented in Table 7, have no better applicability in answering the questions than the above strategies because they have limited matches in the real tests. In addition, most of the strategies that have matches cannot eliminate distractors. For a strategy to work in the test, it should be able to eliminate all the distractors in the tests, leaving only the correct answer.

CONCLUSION
The research results have found that only two strategies (i.e. focus on the second speaker and avoid similar sounds) have matches in more than 80% of the questions in the real TOEFL. Three strategies (i.e. choosing answers with synonyms, draw conclusion about who, what, and where, and listen for expression of uncertainty and suggestion) have matches between 8% and 28%. The other strategies rarely have matches (<3%). The most successful strategy, which had the most frequent matches in the real test, can only be applied if the test-takers understand the conversation. The second most successful strategy is only applicable if the test takers have sufficient vocabulary. The next three strategies, which have matches between 8% and 28%, can rarely eliminate all distractors. To conclude, strategies proposed by Phillips (2003) cannot successfully be applied to answer the questions in Part A of the Listening Comprehension section in the real TOEFL. However, the decision regarding whether the strategies matched the test items was based solely on the researchers' judgment. No real test takers were involved in confirming the findings. To overcome this limitation, further study should experiment by teaching students the strategies proposed by Phillips (2003) and compare their scores before and after they are taught the strategies.
As a pedagogical implication, the listening sections in standardized tests such as the TOEFL and IELTS are intended to measure the students' listening comprehension level. Therefore, in preparation courses, students should be taught to improve the level of their listening ability. It is suggested that teachers help improve their students' listening comprehension by using movies, involving pre-viewing and post-viewing (Amalia, Kusrini, Ramadhani, 2019). Movies are also recommended for independent learning (Safranj, 2015). Another useful technique of improving students' listening comprehension is through transcribing tasks (Shafiee & Rahimi, 2012). Students are provided with a podcast or other audio and are asked to transcribe it. According to Dinmore and Gao (2016, p. 198), not only is it useful in forming an awareness and for feedbacks, transcribing authentic texts also improves listening skills through word recognition enhancement.