Corps de l’article

1. Introduction

To what extent conference interpreters are a product of nature or nurture is still a moot point in the interpreting community. However, the need for candidates to harbour legitimate academic and professional aspirations and for the Interpreting Schools to optimise their human and financial resources have spurred many efforts to devise and implement selection practices (Gerver et al. 1984, Gran and Dodds 1989, Lambert 1992, Arjona-Tseng 1994, Moser-Mercer 1994).

Intuition and experience amongst professionals and/or trainers indicate that basic pre-requisites of a good professional interpreter are: profound knowledge of active and passive languages and cultures, ability to grasp the original meaning quickly and to convey the essential meaning of what is being said, ability to project information with confidence, coupled with good voice, wide general knowledge and interests and, finally, ability to work as a member of the team (Gerver et al. 1984, Lambert 1992). The purpose of our research is to investigate the second pre-requisite, i.e., the cognitive-linguistic component. In order to do so rather than concentrate on separate task components through separate tests as authoritatively done by previous researchers, especially by Gerver et al. (1984), we followed an ecological approach and chose a task as close as possible to actual simultaneous interpreting which involves connected discourse processing.

The testing methodology we are suggesting is based on an on-line oral paraphrase from Italian into Italian[1]. It was presented during the International Conference Interpreting in the 21st Century. Challenges and Opportunities as work in progress (Pippa and Russo, forthcoming). In that presentation, paraphrasing as a comprehension and production evaluation tool (Mortara Garavelli 1979, Marinetto 1998, Lumbelli and Mortara Garavelli 1999) and its similarities with the interpreting process (Moser 1983, Lambert 1989, Malakoff and Hakuta1991, Anderson 1994, Danks et al.1997) were also explained in detail. In the literature, the validity of paraphrase as a promising diagnostic tool was already suggested by Moser (1983) who observed that her students who performed poorly in paraphrasing obtained the lowest final score in the introductory course to simultaneous interpreting. Later on Moser-Mercer (1985) published data on paraphrasing as an item of a promising multi-task aptitude test. However, no specific indication of its load to the overall positive results was provided. Assuming the predictive validity of paraphrasing per se for its psycholinguistic implications, we focussed on it and set out to find objective categories to describe the manyfold linguistic reformulations with related semantic and pragmatic shifts and to assess the predictive value of the methodology suggested.

In the present paper we will briefly present the paraphrase evaluation model as it has been further developed since its first presentation and its empirical application to 46 subjects with the results obtained.

2. Development of the model: from a synthetic score to a three-tier analytical assessment

2.1 Early model: description, application and results

The proposed evaluation model is based on a three-level analysis: syntax and lexicon (text manipulation), semantics (meaning manipulation) and pragmatics (speaker’s communicative intention manipulation). It is the result of a long process entailing reflection and empirical work on the most suitable evaluation criteria as what follows shows.

Originally (Russo 1989), the variable ‘aptitude to interpreting’ had been operationalised into negative and positive operations both cognitive and linguistic, scored either positively or negatively (+1, +0,5 and –0,5, –1 respectively) in order to obtain a synthetic, i.e., a single numerical value (test score).

The categories envisaged were:

  1. Application of macrorules on the incoming pieces of information (+1):

    • for semantic abstraction via construction (SA): any single piece or sequence of pieces of information may be replaced or conceptualized by one denoting a global fact of which they are constituents (i.e.: the segment “Italians cast their vote to elect their representatives” could be substituted with “Italian elections”)

    • for semantic abstraction via generalisation (Ge): any single piece or sequence of pieces of information may be replaced or conceptualized by a general one denoting an immediate superset (i.e., in June, July and August → during Summer);

  2. Active storage (AS) of incoming linguistic occurrences in short-term memory so as to facilitate their processing, with subsequent permutation of the sentence segments during the message output (+1). If the original utterance starts as follows “The proposal we have endorsed and therefore will vote […]” there is evidence of AS if the subject waits before speaking and then produces an utterance like “We will certainly vote for the proposal we fully support […];

  3. Syntactic and/or semantic anticipation ability (+1);

  4. Paradigmatic[2] replacement of a phrase (PR) (“The response of the European Parliament” → The stance taken up by the European Parliament) (+1);

  5. Ability to detect and modify a paradigmatic structure from the syntactic point of view so as to be more concise (+0,5) (“If Europe wants to wage an effective war against terrorism, if Europe wants to isolate and deter its supporters she must follow another strategy […]” –> “For a successful war against terrorism and its supporters, Europe must follow […]);

  6. Deletion (De) of:

    • a main nucleus (noun phrase and modifiers) (-0,5)

    • an important secondary nucleus (specification, etc.) (-0,5)

    • advisable deletion (AD), i.e., deletion of an unimportant secondary nucleus;

  7. Form-linked errors (Fo) of various types, including stylistic imperfections, wrong collocations, violations of combinatory rules or grammar mistakes, provided comprehension of the output text is not hampered (-0,5);

  8. Loss of cohesion (LC): wrong reproduction of the grammatical links in the target text to the extent that its content becomes difficult to make out (i.e., incorrect agreement of subject and verb separated by an embedded sentence) (-1);

  9. Loss of coherence (CoL): the text produced by the subject lacks its internal logical links between the pieces of information, consequently, the text world reconstructed by the receiver is incomplete and lacks its consequential and informative aspects. This is the cause and at the same time the effect of the previous condition (-1);

  10. Semantic error (SE): the comparison between the source and target texts shows that original pieces of information have been replaced by incorrect ones (including misinterpretation) (-1).

The following features were also taken into consideration:

  1. Deletion of a whole sentence: for each deleted sentence (-1);

  2. Length of the performance (i.e., all of the 12 sentences of the original text were reproduced) (+1);

  3. Fulfilment of the task with breaks: each request to stop the tape-recorder and consequent break received a score of (-1);

  4. Fulfilment of the task without breaks (+1).

Some of the cognitive operations envisaged by this model were taken from Kintsch and Van Djik (1978) (1) and from Beaugrande and Dressler (1981) (6 and 9). They were adopted because they suitably highlight the interpretation process usually considered a “black box” (Mackintosh 1985). On the one hand, macro-rules are often interpreters’ strategies too. On the other, the text world with primary and secondary concepts is a logico-semantic network which helps detect interpreters’ comprehension and information selection processes, provided that the source text (ST) and the target text (TT) are analysed and matched accordingly.

In the academic year 1988-1989 the original test was administered to 20 second-year students of the Scuola Superiore di Lingue Moderne per Interpreti e Traduttori (SSLMIT) of the University of Trieste, whose gender composition and age appears to be fairly representative of the overall second year population (tab.1):

Tab.1

Student sample (Subjects) vs. total SSLMIT second-year student population (All)

Student sample (Subjects) vs. total SSLMIT second-year student population (All)

-> Voir la liste des tableaux

The same test was also administered to four interpreting students who had passed their final exams (advanced subjects) and who were asked to deliver both a paraphase and a simultaneous interpretation into English of the same text, so as to observe possible similarities in the processing skills of the two groups, and within the same group for both tasks.

The application of this model provided the following results (Russo 1989): the test score obtained by the twenty 2nd-year subjects ranged between +8 and -8. During the first observation based on a comparison between the performance of the 2nd-year subjects and that of the advanced subjects, the most salient results were: 1) in many cases, both groups (2nd-year vs. advanced subjects) performed the same operations on given target information nuclei (1989:62); 2) as expected, the performance of the advanced subjects was more faithful and complete in both tasks (they reproduced all main information nuclei), which did not always occur in the performance of the 2nd-year subjects. This seemed to indicate that selection and production skills were developed by training; 3) memory resources were better tapped by advanced subjects, as shown by their frequent recourse to active storage – this provided an indication of the effects of memory jogging; 4) on the whole, only a few 2nd-year subjects showed a remarkable aptitude to select the incoming message autonomously while preserving formal and semantic continuity throughout the whole text; 5) generalisation was the strategy most frequently implemented by both groups of subjects.

During the second observation in March 1991 (Russo 1991), only twelve 2nd-year subjects out of the original twenty were taken into consideration because they had sat at least one interpreting exam (5 had not passed the diploma exam required to start the interpreting course (3rd year), 2 had abandoned the interpreting course and 1 had not sat any exam yet). Expressed in percentage terms, as a sheer indication, of the group of 2nd-year subjects who had a test score ž0 (group A, 8 subjects), 4 had sat one or more exams successfully (50%), whereas none of the other group (group B, 4 subjects) had taken any exam (100%).

During the third observation in March 1992 (Russo 1993), it was noticed that out of the original sample of 20 2nd-year subjects only 19 had actually started the two-year interpreting course. However, during the first year of training there was a 11% drop-out rate (from 9 to 8) in group A and a 50% rate in group B (from 10 to 5). Therefore the sample went down to 13 subjects, and then to 12 when another student subsequently abandoned the course.

In order to measure the possible predictive value of the subjects’ performances as assessed by the test proposed originally, their test score was correlated with the following measures: a) number of 2nd-year subjects who had completed the course (or were about to do so) and number of students who had abandoned it; b) average mark obtained by each student in interpretation exams[3] throughout the two-year course[4]; c) success rate, expressed as the difference between the number of exams sat and the number of those actually passed; d) number of passed exams out of the total number envisaged by the curriculum; e) number of sessions actually needed to complete the course.

Only the correlation between the test score and the number of exams passed yielded statistically significant results r(12)=.56; p<.03.

This result seemed to indicate a meaningful correlation between the time required to complete the interpreter course as indicated by the number of exam sessions needed (performance) and the cognitive and linguistic skills displayed in performing a task similar to simultaneous interpreting (with the exception of the code-switching factor) such as paraphrasing (aptitude).

In other words, this observational study provided a measure of school efficiency for candidates wishing to start their training in Conference interpreting, given their natural linguistic and cognitive abilities as assessed by the test.

The small sample, however, did not warrant any reliable generalisation about the predictive value of the test. Therefore, in order to obtain a larger sample, between the academic years 1990/1 and 1993/4[5], the same test was administered to a larger number of 3rd-year students[6] who had not been exposed to interpreting techniques yet. The research project was longitudinal, as the test results were to be correlated with the time needed to complete the interpreting exams in the curriculum and with the corresponding average mark. Therefore the test had to be administered to the largest possible number of subjects, some of whom would presumably abandon the interpreting course (in which case their test score would no longer be taken into consideration, since it could not be correlated with the data of the two above-mentioned variables).

The sample was taken from the total 3rd year interpreting population according to the following procedure. To make sure that the sample would reflect all the subgroups present in the overall population, an “intentional” sample was selected, i.e., a deliberately chosen sample and not a casual one (Boncori 1993: 81). The random choice method would not have been feasible given the scant interpreting population. Attention was paid, however, to ensure that these subgroups were proportionally representative of the specific strata taken into consideration (female/male; B[7] language: English/French/German). Our sample was therefore intentional, stratified and proportional (1993:81). The following tables show data relative to the total student interpreter population (Italian mother-tongue students enrolled in the academic years 88-89; 90-94, tab. 2) and to the sample taken from it (tab.3)

Table 3 shows that only 91 students remained in the interpreting course. However, the number of students who had sat at least one exam by 1998, when the analysis was carried out, was only 84. The correlation between test scores and exams passed provided results which confirmed the trend in the pilot study: students with the highest test scores were quicker in completing the two-year curriculum (best performing students). In this case, the test scores were calculated by two separate raters. A very good inter-judge agreement was found because the two independent raters were in agreement in 88% of the cases. On this wider sample population, however, the correlation between test scores and exams passed was weaker than the results obtained in the initial study: r(84)=.25; p<.02 vs. r(12)=.56; p<.03. Interestingly, from this larger study it emerged that the quickest students also obtained the highest marks: r(83)=-.40; p<.001 (negative correlation between average mark and sessions) and r(83)=-.35; p<.001 (negative correlation between exams passed and number of sessions).

Tab.2

Total 3rd year interpreting population in the academic years 1988-89 and 1990-94

Total 3rd year interpreting population in the academic years 1988-89 and 1990-94

-> Voir la liste des tableaux

Tab.3

Sample population

Sample population

-> Voir la liste des tableaux

When the total sample population was broken down by students’ B language (English, French, German), the strongest correlation between test scores and exams passed was found for the French group (r(19)=.55; p<.02). This finding confirms the strong correlation found in the pilot study and suggests that perhaps this tool could be more predictive for students working between cognate languages where the morpho-syntactic operations are indeed more similar to an intra-linguistic translation in Italian). The English group only confirmed the trend of the best performing students already observed in the total sample population: the quickest r(55)=-.51; p<.001 (negative correlation between exams passed and sessions) were also more successful markwise r(55)=-.32; p<.02 (negative correlation between average mark and sessions). The German group (only 9 students) did not provide any statistically significant result.

Finally, to verify the speed of best performers over the total population, the subjects of the 3 different B languages who completed the course regularly (6 sessions) were correlated with the total number of sessions. The result confirmed the trend: r(73)=-.53; p<.001 vs. r(83)=-.35; p<.001.

However, the assessment paradigm with one single numerical score, albeit promising, did not clearly differentiate between the many relevant components involved in the task. For this reason it was felt that the paraphrase evaluation model had to be developed further.

2.2 The present three-tier model

The suggested three-tier model (syntax and lexicon, semantics and pragmatics) with its finite number of text-processing categories allows for consistent classification of a wide array of possible reformulations of the same linguistic chunk. It also makes it possible to keep the effects at the three levels separate and, consequently, to rate them differently.

For each level of analysis relevant categories have been developed and applied (see definitions in annex 1):

Seven general categories for syntax and lexicon: (1) reduction (further specified in type 1 and 2), (2) expansion (further specified in type 1 and 2), (3) syntactic transformation, (4) lexico-syntactic transformation, (5) lexical transformation, (6) permutation, (7) production disorder;

Four for semantics: (1) deletion (further specified as: deletion, deletion type 1, type 2 and Ø), (2) addition (further specified as addition, Ø, non-relevant addition), (3) substitution (further specified as non-relevant, synonymic, paradigmatic substitution), (4) loss of coherence;

Four for pragmatics: (1) pragmatic loss (further specified in loss of communicative efficiency, loss of communicative effectiveness because the original was more specific, loss of communicative effectiveness because the original was more emphatic), (2) pragmatic gain (further specified in gain of communicative efficiency, gain of communicative effectiveness because the TT is more specific, gain of communicative effectiveness because the TT is more emphatic), (3) theme, (4) focus.

The following table shows the model and the categories of analysis. The analysis is carried out at three levels simultaneously. Each paraphrasing attempt is assessed together with the associated effects. For instance, in the case of a syntactic reduction, at semantic level the concept may be totally or partially omitted (deletion) or preserved (Ø); at pragmatic level some type of loss or gain may result. The syntactic, semantic and pragmatic categories provided in the evaluation model are not listed in a ranking order. A classification of positive and negative categories, according to the degree of TT coherence and deviation from the ST, will be provided in the second part of our paper when dealing with the experimental application of the model.

Tab.4

The model

The model

-> Voir la liste des tableaux

With respect to the model in progress previously presented (Pippa and Russo, forthcoming), the general category “Interpretative paraphrase” has been eliminated because it lent itself to an excessive degree of subjectivity. So a more “objective” measure as substitution (non-relevant, synonymic and paradigmatic substitution) was adopted for the sake of replicability.

3. Methodology

3.1 Sample

To validate the present model 46 subjects (40 females, 6 males, mean age 21,42) were drawn from the intentional, stratified and proportional sample previously described. They were enrolled in the academic years 88-89, 90-91 and 91-92.

3.2 Data codification and processing

The 46 on-line oral paraphrases were evaluated according to the following procedure.

The source Italian speech (a five-minute political speech) was first analysed from the point of view of its logico-semantic structure (Leech 1974, Chierchia 1997) and segmented into syntactic units (Renzi 1988). Then the subjects’ versions were matched to the corresponding units and analysed from the threefold perspective envisaged by the general model: lexico-syntactic, semantic and pragmatic (an example of the analysis is provided in annex 2).

Theoretically, the cross combination of the 15 general categories of the model presented in Table 4 results into 113 triplets[8] of operations, but only 75 are realistically possible in our view. Therefore the performances were analysed applying these 75 operations (considered items and codified for analysis and statistical purposes starting with the semantic category[9], see annex 3).

Prior to data codification, two raters jointly analysed 3 paraphrased texts to test the model application and reach an agreement on those cases which were more difficult to categorise. Subsequently, the two raters separately analysed 5 performances. The results obtained were compared and an inter-judge agreement calculated using the “K-coefficient” (Areni et al. 1994:181-182). The resulting inter-judge agreement was K=.75, which for tests such as this – entailing categories which are complex or significantly dependent on the examiners’ assessment- is quite satisfactory. Upon verifying the inter-judge agreement, the two raters separately analysed the 46 on-line paraphrases and classified the subjects’ operations according to the general evaluation model suggested above, articulated into the 75 items (see annex 3). The individual subjects’ operations classified accordingly were subsequently processed, by feeding into a matrix the frequency of occurrence of the 75 items for all subjects.

The classification of all the subjects’ operations and the subsequent data codification have enabled us to obtain a test score for each one of the 75 items, corresponding to the total item frequency in the subjects’ paraphrases. In order to test the predictive power of the methodology suggested, the total codified data were processed statistically. Pearson correlation coefficients were calculated between the test score (corresponding to the item frequency detected in the on-line paraphrases) and the number of exam sessions[10] and between the test score and the average mark for each student. What we obtained is the measure of the predictive value of each item, as measured in the total sample of 46 subjects.

4. Results

In order to gain a comprehensive view of the statistically significant results obtained, the individual items were grouped into sets and subsets of operations. The sets of operations were either positive or negative:

Set of positive operations:

  1. Paradigmatic substitution with pragmatic loss

  2. Synonymic substitution with pragmatic loss

  3. Operation with Ø semantic value and pragmatic loss

  4. Operation with Ø semantic value and pragmatic gain

  5. Addition

  6. Synonymic substitution

  7. Deletion with gain in communicative efficiency

  8. Paradigmatic substitution

Set of negative operations

  1. Loss of coherence

  2. Non-relevant substitution

  3. Non-relevant addition

  4. Deletion

Some sets of operations were further subdivided into subsets of operations (es. Ø-SS-Ø-Expansions etc.) in order to highlight the underlying syntactic operations.

We shall first consider the results of the correlations with the total of positive operations (§ 4.1), then the single positive operations (§ 4.2) and finally the most significant positive items (§ 4.3). These will be followed by the results of the correlations with the negative operations in the same order of presentation (§§ 4.4, 4.5, 4.6).

The first interesting general result is the correlation between the number of sessions and the average mark (r(46)=.66; p<.001). The number of sessions envisaged by the interpreting curriculum is 6 (3rd year Summer, Autumn and February sessions and 4th year Summer, Autumn and February sessions). However, theoretically, it is possible to pass all exams by the 4th year Summer session, thus using only four sessions. At the time of the test administration, there were 12 interpreting proficiency exams: 3 consecutive interpretation exams and 3 simultaneous interpretation exams (all BA, AB and CA) for the 3rd year and the same number for the 4th year (today after the curriculum reform, only 6 exams over two years are envisaged).

The strong correlation observed between the time necessary to finish the exams and the average mark indicate that the quickest students (those who need fewer exam sessions) are also the best ones since they obtain better marks (best performers). Conversely, the slowest students also got lower marks. This result confirms the data obtained in the previous version of the test (see § 2.1). The detection of a group of best performers suggests that certain skills, needed to perform an interpreting task, are acquired in a shorter period of time in those same students who then use them in a more efficient way and obtain better evaluations.

After this general result, let’s now turn to the correlations with the 75 items. The 75 operations were divided into 52 positive and 23 negative (see the list in annex 3). As was already mentioned, the results of the correlations with the positive items will be examined first, followed by those with the negative ones.

4.1 Results of the correlations with the total of positive operations

We have first correlated the total positive operations with the two variables considered and obtained the following result:

Tab. 5

Correlations between the total of positive operations, sessions and average mark

Correlations between the total of positive operations, sessions and average mark
*

p<.05

-> Voir la liste des tableaux

As we can see, the total (sum of frequencies) of the positive operations (of each item per subject) which consists of 52 items has a correlation coefficient with the number of sessions equal to r(46)=.32; p<.05 and with the average mark obtained in interpreting exams equal to r(46)=.38; p<0.5 (see tab. 5). Both correlation coefficients are statistically significant. This result shows that the linguistic and cognitive operations considered as a whole as predictors of interpreting skills have a direct and positive incidence on subsequent academic results (school efficiency). The subjects who performed positive operations completed their training faster and with better results.

4.2 Results of the correlations with single positive operations

If we look into the correlations with each of the 8 positive operations the following significant results were obtained:

Tab. 6

Correlations between single positive operations, sessions and average mark

Correlations between single positive operations, sessions and average mark

Tab. 6 (suite)

Correlations between single positive operations, sessions and average mark

*p<.05 **p<.01 ***p<.001

-> Voir la liste des tableaux

The Pearson correlation coefficients between brackets are below r critical value =.296; p<.05 established for 46 subjects to find a significant correlation. These coefficients, however, have been reported here because they highlight a trend which gets stronger and worth stressing as it nears r critical value.

A.The Set Synonymic substitution with pragmatic loss includes the following 4 statistically significant items:

  • SS-LSTc-EFLE= Synonymic substitution obtained via syntactic and lexical transformation of a clause with loss of ST emphasis.

  • SS-LTc-EFLE= Synonymic substitution obtained via lexical transformation of a clause with loss of ST emphasis.

  • SS-LSTp-EFLE= Synonymic substitution obtained via syntactic and lexical transformation of a phrase with loss of ST emphasis.

  • SS-LTp-EFLE= Synonymic substitution obtained via lexical transformation of a phrase with loss of ST emphasis.

In this case the correlation coefficient is statistically significant for both the number of sessions (r=.30;p<05) and the average mark (r=.32; p<05). This shows that fluency and a large vocabulary, as evidenced by the skillful use of synonymic expressions, is a pre-requisite for an interpreter. It is worth noting that the production of synonyms, intended here as the production of a meaning similar to the original one (slips in register use have not been taken into account) implies a considerable ability to re-arrange the ST surface structure. The degree of “manipulation” complexity might be ranked in ascending order as follows:

  1. lexical transformations of phrases

  2. lexical transformations of clauses

  3. lexico-syntactic transformation of phrases

  4. lexico-syntactic transformation of clauses

It should, however, be noted that these synonymic substitutions caused a loss of the emphasis present in the ST. This result seems to indicate that an inadequate use of language register (pragmatic level) at this stage has no adverse effects on outcome of subsequent training.

B. The category Synonymic substitution without pragmatic loss includes the three following items:

  • SS-LTp-Ø= Synonimic substitution obtained by lexical transformation of a phrase.

  • SS-LSTp-Ø= Synonymic substitution obtained by lexico-syntactic transformation of a phrase.

  • SS-LSTc-Ø= Synonymic substitution obtained by lexico-syntactic transformation of a clause.

A strong correlation was found both with the number of sessions (r=-.31; p<05) and with the average mark (r=.34; p<05). This type of operation has already been demonstrated to be predictive of school efficiency. Yet, while in the previous case it was associated with pragmatic loss, due to inadequate synonym choice from the point of view of register, here the register is unaffected. Therefore the importance of the ability to produce synonyms as an indicator not only of lexical fluency but also of ST comprehension (two pre-requisites for “fast and top performers”) is confirmed.

C. The Set Operation with Ø semantic value with or without pragmatic gain includes 13 statistically significant items. They are grouped as follows:

a) Operation with Ø semantic value without pragmatic gain- Subset: Expansions (7 items):

  • Ø-EXP2-Ø= Operation without any semantic and pragmatic value (i.e., the meaning and the pragmatic levels are not affected) following expansion2.

  • Ø-EXP2-EYG= Operation without any semantic value following expansion2, with consequent gain in communicative efficiency.

  • Ø-EXP1p-Ø= Operation without any semantic and pragmatic value following expansion1 of a phrase.

  • Ø-EXP1p-EYG= Operation without any semantic value following expansion1 of a phrase, with consequent gain in communicative efficiency.

  • Ø-EXP1c-Ø= Operation without any semantic and pragmatic value following expansion1 of a clause.

  • Ø-EXP1p-EYG= Operation without any semantic value following expansion1 of a phrase, with consequent gain in communicative efficiency.

  • Ø-EXP1s-Ø= Operation without any semantic and pragmatic value following expansion1 of a sentence.

The correlation with the Set Expansion with Ø semantic value is statistically significant for the average mark (r=.45;p<.01) and highly indicative for the number of sessions (r=-.28; n.s.). This seems to indicate that expansion of the surface ST in the paraphrased text is an important cohesive device which improves fluency and oral delivery. It makes the text more efficient, and therefore easier to understand for the listener. This could explain the strong correlation between this operation and the average mark.

b) Operation with Ø semantic value with/without pragmatic gain- Subset: Permutations (3 statistically significant items):

  • Ø-P-T= Operation without any semantic value following permutation and thematic change.

  • Ø-P-F= Operation without any semantic value following permutation and focus changes.

  • Ø-P-EFGE= Operation without any semantic value following permutation with consequent gain in emphasis with respect to the original.

The correlation with permutation (with or without pragmatic gain) is highly significant for the average mark (r=.42; p<.01) and not significant for the number of sessions (r=-.18; n.s.). In this case the ability shown in controlling textual organization by manipulating the word order seems necessary both to improve delivery (for example by stressing specific elements via theme and focus selection mechanisms), and to enhance ST emphasis, thus providing more illocutory force to the utterance. This latter ability is not necessarily related to the time needed to complete the training, but it is undoubtedly essential to improve oral delivery.

c) Operation with Ø semantic value with pragmatic gain- Subset: Syntactic Transformations (3 significant items):

  • Ø-SyTs-EFGE= Operation without any semantic value following syntactic transformation of a sentence, with consequent gain in emphasis with respect to the original.

  • Ø-SyTc-EFGE= Operation without any semantic value following syntactic transformation of a clause, with consequent gain in emphasis with respect to the original.

  • Ø-SyTp-EFGE= Operation without any semantic value following syntactic transformation of a phrase with consequent gain in emphasis compared to the original.

In this case the correlation with “Syntactic Transformations and pragmatic gain” is highly significant for the average mark (r(46)=.49; p<.001) and only indicative for the number of sessions (r(46)=-.23; n.s.). The ability displayed in transforming syntactic structures while preserving the lexical items is a promising indicator of the student’s skill in rapid re-arranging of the original text with competence and linguistic flexibility. This seems to help the trainee obtain better marks in the exams and apparently slightly reduces the time needed to complete the training.

4.3 Correlations with single positive items

With regard to the correlations with the single items considered, the following significant results were obtained:

Tab.7

Correlations between single positive items, sessions and average mark

Correlations between single positive items, sessions and average mark

*p<.05 ***p<.001

-> Voir la liste des tableaux

Concerning A-Exp1c-EFGS (Addition obtained by expansion1 of a clause, with consequent gain in communicative effectiveness vs. the ST), a strong correlation between this item the number of sessions (r(46)=-.35; p<.05) is observed. What was previously mentioned about expansions is further borne out by this result. Whilst in the case previously observed, this operation did not imply any “addition” from the point of view of the content, here the supplementary clause to the ST makes the text more fluent and the content clearer and more explicit. Whilst in the previous case the ability to “expand” was deemed useful for the sake of the exam mark, in this case content addition shows a direct correlation only with the speedy completion of the training.

Concerning Ø-STp-EFGE (Operation with no semantic value following syntactic transformation of a phrase and associated gain in emphasis vs. the ST), a very strong correlation with the average mark (r(46)=.50; p<.001) and a not significant correlation with the number of sessions (r=-.23; n.s.) were observed. The impact of the ability to produce syntactic transformations on the average mark has already been observed. This is specifically confirmed by the syntactic transformation of phrases. A phrase is the smallest content unit in the spoken chain. It is smaller than the clause and the sentence which obviously tend to be more difficult to manipulate. Furthermore, one has to consider that when interpreting/paraphrasing simultaneously it is not always possible, albeit desirable, to mantain a décalage long enough to reformulate very long text segments.

Concerning SS-LTp-Ø (Synonymic substitution obtained by lexical transformation of a phrase, without pragmatic effect), a significant correlation was found with the average mark (r(46)=.30; p<05) and an indicative correlation with the number of sessions (r(46)=.25; n.s.).

Concerning SS-LSTc-Ø, (Synonymic substitution obtained by lexico-syntactic transformation of a clause, without pragmatic effect), the correlation is significant for both variables (r(46)=.30;p<.05 for the number of sessions and r(46)=.33; p<05 for the average mark). For these results the same remarks made for the category SS containing both items (A.) apply.

Similarly, for the item D2-R1p-EFLE+EYG (Deletion2 obtained by reduction1 of a phrase, with associated loss of the ST emphasis, compensated by a gain in communicative efficiency), a strong correlation is observed with the number of sessions (r(46)=-.33; p<.05) and the average mark (r(46)=.31; p<.05). The ability to establish a hierarchy among informative items and consequently skip secondary ones is a prerequisite for successful interpreter training. Syntactically prominent elements are not always equally prominent semantically. The ability to detect redundancies or, as in this case, the presence of secondary informative elements, and to “delete” them to streamline the TT are strategic for an interpreter. It does not seem to be a coincidence, therefore, that if these abilities are already present before training, a better school efficiency is to be expected both in terms of course length and marks.

4.4 Correlations with the total of negative operations

We have correlated the total negative operations and obtained the following result:

Tab.8

Correlations between the total of negative operations, sessions and average mark

Correlations between the total of negative operations, sessions and average mark

-> Voir la liste des tableaux

The data concerning the total of negative operations do not seem to have a significant impact on academic success/failure as measured against time (number of sessions needed to complete interpreting exams) or performance (examination marks). Yet, significant data were observed when the categories and the items were considered separately.

5.5 Correlations with single negative operations

The correlations with each of the 4 negative operations have produced the following significant results:

Tab. 9

Correlations between single negative operations, sessions and average mark

Correlations between single negative operations, sessions and average mark

*p<.05

-> Voir la liste des tableaux

The category Loss of coherence includes 3 statistically significant items:

  • LCis-LTp-EYL= Loss of coherence between sentences associated to lexical transformation of a phrase, with consequent loss of communicative efficiency.

  • LCis-PD-EYL= Loss of coherence between sentences associated to production disorder and consequent loss of communicative efficiency.

  • LCs-PD-EYL= Loss of coherence within a sentence associated to production disorder and consequent loss of communicative efficiency.

As can be observed, the correlations found for these items are statistically significant. The loss of text coherence, both within and between sentences, whether due to inappropriate lexical choice or to a “disturbed” linguistic production, is an unacceptable fault in an interpreter’s performance. These correlations show that detecting this type of fault before training is predictive of unsuccessful training outcome. “Unsuccessful” is to be intended here as the need for a longer period of training and as worse exam performance. This type of shortcoming highlights students’ difficulties in grasping the speaker’s intended meaning and in delivering a fluent and coherent speech (that is a connected spoken discourse). It seems to be a good indicator of processing difficulties which apparently are not completely overcome with the passing of time, or at least not with brilliant results, as evidenced by correlation coefficients. It is therefore sensible to hypothesize that candidates without this type of shortcoming from the start stand a better chance of succeeding in their interpreter training.

5.6 Correlations with single negative items

If we observe the correlations with the single items considered we obtain the following significant results:

Tab. 10

Correlations between single negative items, sessions and average mark

Correlations between single negative items, sessions and average mark

*p<.05

-> Voir la liste des tableaux

With regard to LCs-PD-EYL (Loss of coherence within a sentence associated to production disorder and consequent loss of communicative efficiency), very strong correlations were found. What was stated about the category “Loss of coherence” is further validated in this specific case, that is, when control over linguistic output is lost within a sentence, the receiver’s comprehension is jeopardised.

With reference to NS-LSTc-EFLS/EFLE (Non-relevant substitution associated to lexico-syntactic transformation of a clause, with consequent loss of ST communicative effectiveness and emphasis), the correlation is stronger with the average mark than with the number of sessions. This type of item highlights the negative outcome of an intrinsically positive move, namely the attempt to reformulate both syntax and lexicon in the TT. Unfortunately, the result is semantically inappropriate, since the meaning produced is not equivalent to the original one. To overcome this shortcoming a longer training period seems to be required (although the result r(46)=.27 does not reach statistical significance). However, this does not necessarily imply better performance results (r(46)=-.32; p<.05).

With regard to NS-LTp-EFLS/EFLE (Non-relevant substitution following lexical transformation of a phrase, with consequent loss of ST communicative effectiveness and emphasis), the time required to complete the training is considerably lengthened (r(46)=.35; p<.05), but no appreciable results are found in relation to the mark (r(46)=-.007; n.s.).

The last interesting correlation concerns D1-R2p-EFLS/EFLE (Deletion1 following reduction2 of a phrase, with loss of communicative effectiveness and emphasis). In this case we observe the opposite of what was seen with D2-R1p-EFLE/EYG (Deletion2 following reduction1 of a phrase with loss of emphasis, compensated by a gain in communicative efficiency). Here we observe the inability to detect primary semantic nuclei, irrespective of their syntactic prominence, and to establish an information hierarchy. This does not seem to affect so much the time needed to complete the interpreter training, but rather performance evaluation (average mark). A comparison between the correlations of the previous positive item and the present one indicates that the “ability” detected in the former item seems to have a greater effect than the “inability” detected in the latter in relation to the training time needed.

5. Discussion

To summarize, the following picture emerges from our research findings. What we described as positive operations are both generally and individually predictive of a good aptitude to complete the interpreter course rapidly and successfully. In particular, the results shed light on which operations significantly display a greater predicting value, as indicated by the relevant evaluation criteria. These concern lexical, semantic and reformulation abilities, deletion of secondary information, and additions. In other words they concern: the ability to grasp and convey the meaning properly, the ability to detect information hierarchy (and select items accordingly) and a communicative effort by providing supplementary information.

On the other hand, negative operations are generally less predictive, except for loss of coherence, non-relevant substitution and deletion of important information which displayed a statistically significant correlation with the number of sessions and the average mark. In other words these concern the inability to perceive the speech continuum as logico-semantic units and preserve their links, the inability to be semantically faithful and the inability to select basic informative constituents.

Against this backdrop, it is now interesting to draw a comparison with the results published so far on aptitude testing and see what contribution our research could provide.

The main results of our experiment correlating test and interpreting exam performances seem to confirm empirically what had been intuitively stated by the interpreting community: the ability to grasp the original meaning quickly and to convey the essential meaning of what is being said is undoubtedly a fundamental pre-requisite for interpreting. This has to be accompanied by fluency, namely full command of expressive means in one’s native language. Indeed, the strongest significant correlations with positive and negative items have been found at the level of meaning and text manipulation. Control over logico-semantic text coherence and semantic equivalence, as highlighted by Loss of coherence (LC) and Non-relevant (NS) or Synonymic substitution (SS) respectively and control of the mother tongue, as indicated by Lexico-Syntactic Transformation (LST) and Expansion (Exp) emerge as the most prominent predictive factors.

These are the skills which have been investigated more, as the literature shows. In particular, our results are in line with Gerver’s assumptions and findings concerning associational and expressional fluencies tested with specific tests (Synonym test and Rewriting test) which discriminated significantly between students who passed and those who failed interpreting examinations:

According to Carroll (1976), associational fluency entails a search of a major portion of long-term memory with special attention to its semantic and associational aspects. The test manual indicates that subjects score higher on this type of test if they have more associations tied to a word and more flexibility in interpreting similarity. The fluency entailed by this subskill would seem an important requirement for interpreting.[…] Like associational fluency, expressional fluency also includes a search of lexico-semantic memory but, as pointed out by Carroll (1976), with special attention to the grammatical features of lexical items and different syntactic patterns of phrases and sentences. Re-expression is basic to interpreter’s task.

Gerver 1984: 23

It appears therefore that synonym production as an indicator of both mental and expressive flexibility is a factor that emerges strongly as a significant predictor. In our research this assumption is confirmed, as already pointed out by the significant correlation between synonymic substitution (with pragmatic loss) and speed of acquisition (number of sessions r=-.30; p<.05) and evaluations (average mark r=.32; p<.05) on the one hand, and by the significant correlation between synonymic substitution (without pragmatic loss) and speed of acquisition and evaluations (r=-.31 and r=.34; p<.05 respectively) on the other. This confirms Gerver’s findings (1984: 27): “of the subskills and speed stress tests, only Synonyms significantly reflected differences between candidates who passed and those who failed the examinations.” Understanding and associating meaning quickly, however, does not seem to be only a strong aptitude predictor. Roy (2000: 125) stresses its pedagogical value too. Interpreting courses should emphasise how people create meaning within their social, cultural and individual styles and include exercises such as paraphrasing, substituting and clarifying meaning to become more aware of how various meanings can be interpreted. Recent studies (Mack and Cataruzza 1995) have empirically demonstrated how inadequate linguistic and encyclopaedic knowledge can hamper comprehension in a teaching environment.

More recently, two studies have further confirmed this hypothesis. The positive correlation between lexical fluency and interpreting aptitude has indeed been corroborated by the pilot study carried out by Anne Skaade (1999) on the impact of written lexical knowledge on aptitude testing.

Concerning the relevance of substitutions in assessing aptitude, Barbara Moser-Mercer (2000) found in her study testing both students and professional interpreters in a shadowing task (where the former proved more faithful) that

“Interpreters, however, made greater use of substitutions, producing nonetheless sentences that were grammatically correct, and did not alter the meaning of the sentences. These substitutions resemble what Lambert (1992) called ‘sophisticated corrections.’ One might wonder whether at some level they were treating the shadowing task as if it were simultaneous interpretation, paraphrasing the input in the same language”.

2000: 118

Given this further evidence, detecting this spontaneous ability in candidates in conditions similar to SI, in terms of time and dual-task constraints, might further corroborate the predicting validity of an adequately targeted test investigating this ability.

So far, our findings reflect similar results in the literature on interpreting research, albeit arrived at with different approaches and methodologies. Yet another significant result emerged from our research, namely the correlation between loss of coherence (especially at sentence level), associated to production disorder and efficiency loss, and number of sessions (r=.37; p<.05) and average mark (r=-.34; p<.05). This finding does not seem to be present in earlier empirical studies. In our view this feature ought to be investigated in depth because it plays an important role in detecting aptitude. Indeed, the ability to perceive a speech (while simultaneously paraphrasing it) as a single conceptual framework, either at sentence or inter-sentence level, helps preserve logico-semantic coherence, good linguistic output and communicative efficiency. It could be hypothesised that this ability might be related to the expert’s performance in the expert-novice paradigm applied by Moser-Mercer (2000), in that, like the expert, the more promising would-be trainees “use more global plans, where novices tend to favour low-level-microcontextual-plans” (2000:109). Preserving coherence between larger information chunks in a simultaneous task also has a strong bearing on memory capacity. Undoubtedly, both short- and long-term memory are efficiently activated in these subjects. It would seem, therefore, that slower and less proficient candidates mainly concentrate on and process information micro-states, probably as a result of poorer memory span and reformulating skills.

With regard to the other skills and inabilities which proved to be statistically significant predictors in our research, unfortunately no parallels can be drawn from the literature because these have not been investigated at all, or where they have, neither the methodology nor corresponding data are provided. Yet strategies connected to information analysis and synthesis and features connected to pragmatic competence to enhance communication are generally considered pre-requisites of an interpreter’s performance, as is the ability to be consistent and faithful. The need is felt for more explicit and objective methodologies to investigate these important sub-skills.

6. Conclusion

In the selection practices implemented worldwide, many aspects are tested to assess aptitude to interpreting. Various approaches are followed focussing on: knowledge (mother tongue/foreign languages and general education), skills (comprehension, speed of comprehension and memory capacity, simultaneity of listening and speaking, voice and diction) and personality traits (stress tolerance, resilience and learning curves) (Moser-Mercer 1994: 58-60). Moser-Mercer rightly stresses the importance of singling out sub-skills which should be given different weight when deciding admissions according to the course curriculum and length.

In our proposal we focussed on cognitive-linguistic skills because we intended to measure the effect of spontaneous oral text processing strategies on school efficiency (time needed to acquire specific skills and evaluations obtained), leaving the other equally important aptitude features to future test developments.

In our view the detection of a group of best performers warrants efforts to devise reliable candidate screening procedures, so as to optimise personal and institutional resources, both human and material.

In the light of the significant correlations observed between several of the 75 operations considered, the number of interpreting exam sessions and the average mark, we have reason to believe that the linguistic and cognitive abilities considered as predictors of interpreting skills have a direct and positive incidence on students’ subsequent academic results (school efficiency). Indeed, the subjects who performed positive operations completed their training faster and with better results.

Furthermore, a strong correlation between the number of sessions and the average mark in interpreting exams was observed. This led to infer that the best performers markwise are also the fastest ones and vice versa.

This suggests a preliminary important consideration. The very fact that there are students who complete the degree course quicker than their colleagues and with better interpreting exam results (simultaneous and consecutive) leads us to believe that the acquisition of these skills in some subjects occurs within the period of time envisaged by the training Institution and with a high degree of proficiency. For these subjects the course duration or a slightly longer time is generally enough to master the necessary ability to become a professional interpreter. After this “reasonably short” training period, these students are able to exploit what was learnt during their training to the full. The same teaching was provided to the other students as well who, albeit undergoing the same “treatment” during the same period of time, take longer to learn and never achieve the level of competence and performance of the former (on the basis of the examination marks). It would seem that the longer period they benefited from did not enable them to produce brilliant results. Against this backdrop, we wonder whether the training provided, the same for everybody, has a direct impact on students’ performance. This is not to question the quality or effectiveness of the training provided. We simply wish to stress the fact that there are preliminary conditions and starting points differing from one another which directly affect school efficiency. While acknowledging the existence of a wide variety of factors influencing school efficiency in general, and studying interpreting in particular (external and personal conditions at psychological, cognitive and linguistic level), the highly significant correlation emerged, however, strongly indicating that there are measurable pre-training conditions which affect either positively or negatively students’ future academic performance. This finding encouraged us to investigate through an on-line oral intra-linguistic task the possible linguistic and cognitive abilities or inabilities which reflected a starting condition likely to affect the future performance either positively or negative. We deem important to offer such a service to interpreting course candidates, so as to inform them in advance of their chance of success or failure since the psychological and financial costs implied are considerable.

Within the framework of the present longitudinal research, the test results have shown also that a series of items relative to linguistic operations during paraphrasing did not reach a statistically significant correlation with the variables ‘time necessary’ and ‘average mark.’ Therefore, focussing only on stronger predictors would save time for aptitude testing purposes. This is not tantamount to saying, however, that detecting these abilities or inabilities before training is pointless. Even if they do not seem to affect school efficiency, because they are likely to be strengthened and adequately made up for during training, these findings might prove their usefulness at other levels, for instance for teaching purposes. The trainer could tackle such weaknesses with more targeted remedial approaches.

Going back to the test and to the predictive abilities and inabilities observed, our results show that the positive abilities as a whole have a greater impact on future interpreting performance than the negative operations. This seems to suggest that interpreting aptitude manifests itself especially in the skills required to restructure speech and to perform analysis and synthesis. These skills are displayed through linguistic means and are certainly “pre-existing” at the beginning of training. Conversely, this seems to suggest that “pre-existing inabilities” are generally not an obstacle, since, once detected, they may be overcome through training.

Amongst positive operations, the most predictive have proven to be: (a) Synonymic substitution in general; (b) some operations which do not imply ST semantic modification and occur at language surface level, i.e., Expansions, Permutations, Syntactic Transformation.

As already stated, all these operations are positive abilities which, although bound to improve over time, are already detectable in a candidate for an interpreter course. In the case of operation (a), this implies the ability to understand the meaning of ST chunks and reformulate them into a TT via a synonymic expression replacing one lexeme with another or via a lexico-syntactic transformation. In other words, this ability is expressed through synonym generation, but at times also via syntactic re-arrangement of the ST. Therefore, Synonymic substitution operations confirm the ability to understand the ST and to rapidly tap the linguistic knowledge required to find the synonym, in order to properly “translate” the concept.

Operations (b) do not imply semantic alterations, but only surface structure manipulation. None of them is significantly correlated with the number of sessions utilized. Yet the fact that each of these operations shows a strong correlation with the average mark provides some valuable indications. Indeed, producing extra linguistic material (without adding any new meaning or distorting the ST sense), shifting the order of the sentence components in compliance with TT syntagmatic rules (while preserving the original meaning) and, finally, re-organizing ST syntax using the same lexical input are strong indicators of “linguistic flexibility” (verbal fluency, Moser-Mercer 2000) and text manipulation ability. During interpreting exams, these turn out to be excellent strategies for a more fluent TT delivery to the benefit of the receiver (the teacher in this case). Indeed, anaphoric links, typical of SI and Subject-Verb-Object structure reproduction, for instance, enhance oral discourse cohesion and, consequently, receiver’s comprehension. On the other hand, stressing specific sentence elements through theme and focus selection and attaching greater communicative emphasis to the original increase the pragmatic strength of the TT and the impact on the receiver. In any case, as experience shows, evaluators tend to appreciate students’ ability not only to “interpret” the ST meaning, but to convey it through her/his own words in order to make comprehension easier for the listener.

In particular, the research results show that more specific operations have a strong correlation with the time needed. First, the ability to add new contents by means of a clause contributes to supplement the ST meaning so as to make it clearer for the listener. In this case, the addition of linguistic material conveying new contents indicates a remarkable aptitude in the interpreting candidate. This is confirmed by the number of exam sessions sat. So the candidate proves her/his ability to understand the message and to decide when it needs to be made more explicit. This behaviour shows that the candidate is aware of the need to be faithful but also communicative. The potential receiver’s requirements for clarifications and effective communication are thus fully accounted for.

An equally important predictor of school efficiency is the pre-training ability “to cut” what is semantically marginal or even redundant, even if syntactically prominent. The operation of deletion highlights an ability symmetrical to the previous one (adding contents for elucidating purposes). Both abilities prove to be extremely important for school efficiency, especially with regard to the time needed to finish training.

Amongst negative operations, especially predictive are those implying a loss of coherence in the paraphrased text. As already seen, positive operations as a whole indicate the skill to grasp the ST meaning and organise it into the TT in a cohesive and coherent way. It has been observed how these operations correlate significantly with school efficiency. This is further borne out by the inability expressed by negative operations: a production disorder (i.e., a “false start,” an inadequate anaphorical link, etc.) or a non-relevant substitution may disrupt speech consistency. This may affect coherence at the level of one or more sentences, or even of the whole text. This highlights processing and reformulation difficulties which will not be overcome within the period of time foreseen by the training Institution. It is likely that the many students who abandoned the course may have had difficulties of this nature.

More specifically, inabilities resulting into loss of text coherence have been found associated to the inability to properly perform operations which are positive per se, such as to produce a synonymic expression by transforming a clause both syntactically and lexically. The intended attempt to transform ST syntax seems not to be supported by total message comprehension.

What was stated regarding the ability to wisely “cut” semantic redundancies conveyed by syntactically prominent units is further confirmed by the symmetrical inability which is negatively correlated with school efficiency: deletion of prominent semantic units conveyed by lesser syntactic elements. Also in this case, the candidate seems to find it difficult to follow the semantic development of the text and to identify the most relevant information. In all, all these operations which were considered negative confirm the predictive value of the previously considered positive ones.

To conclude, the research results indicate that the model and the criteria suggested to evaluate interpreting aptitude by means of paraphrasing can offer a reliable operational tool to assess interpreting candidates’ chances of success/failure from the linguistic and cognitive viewpoint. Indeed, the positive and negative parameters which correlated significantly with time needed and level of performance achieved are limited in number and therefore easy to apply. However, albeit highly promising, the present methodology would need the support of a greater sample to further confirm the predictive validity observed in our investigation.