BINATIONAL WORKING GROUPS OF THE OTO: A COLLABORATIVE FORM OF TECHNICAL ASSISTANCE. Harvard Institute for International Development Development Discussion Paper No. 247, October 1987

ABSTRACT: A form of technical cooperation devised for a training project in Indonesia has proven successful and may have usefulness elsewhere. The Binational Working Groups paired Indonesian and American consultants in the design and development of selection and pre-departure preparatory programs for a new overseas training office (OTO) in the Government of Indonesia. Some of the key features of the mechanism are the following: 1) the Indonesian member is generally the senior partner and chair of the joint effort; 2) the Indonesian member remained to implement the Group’s recommendations after the departure of the foreign member; 3) exceptionally well-qualified Indonesians were attracted from the University community because the Working Group gave them visibility and a rare opportunity to execute their ideas; and 4) the joint effort yielded opportunities and insights that neither member would have been able to achieve alone.

Courtney A. Nelson served as a Resident Advisor to the General Participant Training II Project in Indonesia.


Courtney A Nelson - Beirut 1973

Petra, Jordan 1973


          Indonesia at independence was not richly endowed with educated people. A few sparkling intellectuals trained abroad led the nationalist movement, but the masses of the people received little or no education, and no local institution was yet dignified by the title of university.

          Indonesia has scored many noteworthy achievements in the last forty years, but few accomplishments are more impressive than the steady progress made in education. Placing its priorities in accordance with accepted development theory, universal primary education came first, a considerable accomplishment for so geographically and culturally diverse a country.

          Higher education has also expanded at a rapid rate, although with varying standards of quality. Several institutions, most of them located on Java, are able to produce people with a high level of skills, but the nation must still rely to a considerable extent on training abroad for the most advanced levels.

          Overseas training has a long and proud history in Indonesia. The success of a generation of senior economists, trained abroad with Ford Foundation assistance (a group sometimes known affectionately as the “Berkeley Mafia”), in dealing with Indonesia’s complex development problems, has an honored place in international development literature. The Government’s ability to contain runaway inflation, decentralize economic activity, achieve self-sufficiency in rice production, and deal with fluctuating oil prices without becoming one of the world’s major debtors, provides some of the more convincing evidence the economics profession has to offer of the utility of their craft. Clearly, economists are not alone to be credited with these accomplishments, depending as they do on effective and wise administration, but a solid technical policy framework was an indispensable element of success.

          In addition to that well-known program, hundreds of trainees have gone abroad on routine short-term or degree courses and returned to contribute to their country’s development. In one USAID program alone, Generalized Participant Training (GPT-I), which ran from 1967 to 1981, 1429 participants went to the United States to study. It is a remarkable tribute to their abilities and to the devotion of Indonesians to their country that only two of them failed to complete their courses and return to Indonesia. Brain-drain was never a problem for this country.

          These training programs, and many others with similar purposes, were generally organized and managed by donor agencies. Donors knew best the offerings of their own institutions, and they had, through their offices in Indonesia, a sense of what additional skills were most needed for development. In addition, well-trained Indonesians were confronted by other tasks, which seemed more crucial to progress than the relatively mundane administration of overseas training programs. They were faced with the historical challenges of unifying the country, building solid institutions, and reaching consensus on overall national objectives.

          One problem with donor-managed overseas training programs, despite their evident successes, was that at the end of a particular project the executing agency would fold up shop and go home, leaving little behind in the way of accumulated experience. The next training project to come along had to begin anew, establishing its own requirements and procedures for selecting and preparing people to go abroad.


          At the time USAID and the Government of Indonesia (GOI) negotiated GPT-II, in 1983 and 1984, they decided the time had come to create an institutional capacity within the Government to administer programs of this type, an Overseas Training Office (OTO) that could administer other projects in addition to GPT-II and would carry on after GPT-II came to an end. The OTO would not only retain its accumulated experience in-country, it would be an efficient means of handling small projects which did not warrant the creation of special implementing mechanisms.

          In addition, the Government saw the OTO as a means of gaining more control over its training process. Control in this sense does not mean the simple exercise of power or authority over who goes where to study. Control means the ability to plan for the satisfaction of national requirements in a positive way rather than merely reacting to the proposals or offers of donor organizations. As Indonesia assumes a larger and larger share of its overseas training costs, it becomes increasingly important to ensure that the process is both efficient and effective.

          In a very real sense, overseas training has a higher place now on the scale of national priorities than ever in the past, and a higher place than it is destined to have in the future. The critical tasks of unifying the nation and building a national consensus remain of major importance, but accomplishments have rendered them less urgent. The nation can now afford to devote more of its attentions to the shaping of the competences on which the prosperity of its people will rest.

          At this stage, training abroad is a vital means of acquiring these skills. In the distant future training will become, as it is in the industrialized countries, primarily a maintenance function; a means of passing on accepted ways of doing things to the next generation. In present day Indonesia, however, overseas training has more the character of Research and Development in the industrially advanced states: it is a means for acquiring new and better systems and technologies. These technologies often require adaptation and modification to make them fit Indonesian needs, and that process, too, demands a high degree of skill. These factors account for the special importance attached to overseas training by the GOI.


          The GOI has demonstrated in many ways the priority it attaches to overseas training, including the unusual decision to contract for nearly $200 million in loans from the World Bank to finance graduate training in science and technology for the research institutions, and for strengthening university faculties. High priority is also evident in the organizational location accorded to OTO, and in the form of its governance.

          Organizationally, OTO is attached to the national planning agency, BAPPENAS. This location lent stature to the fledgling office, and probably helped to protect its budget in the austere period following the collapse of oil prices. OTO is not a part of BAPPENAS, and does not share its responsibility for controlling and monitoring the affairs of other ministries, but it is physically proximate to the planning agency and organically linked by virtue of the fact that the OTO Director, Drs. Saadillah Mursjid, is also Secretary General and Deputy Chairman of BAPPENAS.

          The OTO also benefited by being made responsible to a National Steering Committee for Overseas Training (NSCOT). Instead of appointing a project steering committee for GPT-II, as is common practice, the GOI named a National Steering Committee, which has a policy-making role extending beyond the project. This in a way disappointed AID, which normally had a seat on the project steering committee, but had no regular entree to NSCOT, but on balance the greater stature of a national rather than project approach served all concerned well.

          NSCOT currently has the following membership:


Prof. Bintoro Tjokroamindjojo, MA, Chairman of the National Institute of Administration


Mrs. Lamtiur Panggabean, SH, MPIA, Assistant I to the Minister of State for Administrative Reform


Dr. Burhanuddin Anwar Tajipnapis, MPH, Assistant II to the Minister of State for Administrative Reform

Drs. Saadillah Mursjid, MPA, Deputy Chairman of BAPPENAS and Director of the OTO

Mr. Moh. Widodo Condowardojo, SH, Head of the Bureau for Technical Assistance Cooperation, SERKAB

Dr. Dono Iskandar Djojosubroto, Special Assistant to the Minister and Head of the Center for State Budget Formulation and Analysis, Ministry of Finance

Dr. Buchari Zainun, Director, School for Senior Staff, National Institute of Administration (LAN)


          GPT-II was launched in late 1984 by the appointment of Drs. Mursjid as Director and the arrival of Sherwood 0. Berg of MUCIA and Richard Pagett of HIID, representing the prime and sub-contractors on the project, respectively. MUCIA and HIID shared responsibility for placing and monitoring participants in educational and training institutions. An anticipated 25 PhDs, 175 MAs and 225 short-term trainees would be financed by the program, a number which grew substantially through supplementation as AID and the GOI gained confidence in the program. HIID agreed to place candidates in management and economics, while MUCIA, through its support office at the University of Wisconsin, would handle other fields. Berg was Chief of Party, while Pagett’s task was primarily to facilitate the establishment of the OTO as an entity.

          The early months of GPT-II were largely occupied with establishing procedures for the orderly processing of candidates and with making the existence of the project known to those who might benefit from it. A number of participants already studying abroad, or in the process of preparing for departure, were transferred to GPT-II from other AID projects, thus sending GPT-II off to a running start.

          Family health problems caused Pagett to leave Jakarta early in 1985. Two months ensued before he was replaced by Courtney Nelson, also of HIID. By then the project was six months old and it was time to focus on the broader responsibilities of OTO.

          OTO was to have both administrative and policy responsibilities. It was designed to implement selected projects, such as GPT-II, and others that might from time to time be assigned to it by NSCOT. At an early date, NSCOT determined that no economies of scale would be achieved by bringing the large World Bank-supported graduate training projects under the OTO. Each of them was larger than GPT-II, and each was designed to serve the needs of a single ministry. OTO would be assigned to implement general training projects, those designed to serve a number of ministries, and smaller projects that could benefit from the provision of common services.

          The broader task of the OTO was to define national overseas training issues and offer staff support for NSCOT policy-making functions; and to provide services to other government departments sending trainees abroad. The most critical issues facing the OTO had to do with selection and pre­departure preparation, particularly language training and academic orientation. In addition, OTO needed to assist agencies to draw up long-range training plans.

          Within the GPT-II project, funds for 17 months of consultants’ services had been provided for work on issues such as these. At that point, however, the senior staff positions in the OTO had yet to be filled. To bring in expatriate consultants under these circumstances was to run the risk of producing a set of reports that would serve as shelf ornaments, because the officials who could act on the recommendations were not in place. Yet, delays in the deployment of consultants risked a slow start on the task of establishing the OTO, for which only a little over a year had been allotted.

          The solution was to recruit senior local consultants to work with the visitors and then be available to help implement the recommendations. This idea solved the immediate problem, and it turned out to have other advantages as well. When the binational team worked out well, as it did in most cases, a synergistic relationship resulted. The foreign expert could not have produced as useful a report alone as the Group produced together, and the local expert, even if able to produce the report, would have had difficulty gaining the audience necessary for it to be acted upon.

          Another by-product of the Working Group device was that it gave OTO access on a continuing basis to a high level of expertise in critical areas. None of the Working Group chairs was prepared to give up his or her university appointment, but all were quite ready to devote substantial portions of their time to the OTO, where their knowledge could be translated into action. The first local consultant identified was Dr. Amran Halim, Professor of Linguistics at Sriwijaya University. Dr. Halim is former head of the National Language Center, and he has since become rector of his university. In selection testing, Dr. Sumadi Suryabrata of Gadjah Mada University was a recognized leader in the field. Dr. Saparinah Sadli, Professor of Psychology at the University of Indonesia, a former dean of the faculty, took the lead in academic and cross-cultural orientation. And Dr. Suryono, who recently received his doctorate in public administration at the University of Southern California, was lent to the OTO by the National Institute of Administration (LAN) to work on training plans.

          A flurry of activity burst upon the OTO in August 1985, as Working Groups in English Language Teaching (ELT), Selection Testing (ST), and Training Plans (TP) swung into action. The academic and cross-cultural orientation group began the following month.


          Foreign language speaking ability is generally considered to be one of the most severe constraints on overseas training for Indonesians. In some parts of the world, the mastery of a foreign tongue is quite common, almost a regular part of growing up. In Indonesia, as in America, this is not the case; foreign language speaking ability is the exception rather than the rule. In 1985, it was the common perception among donor agencies that the pool of Indonesians linguistically qualified for graduate study abroad was diminishing. Even with six months’ intensive study at either of the excellent ELT centers in Jakarta, the British Council and the Australian Language Center, a distressingly high percentage of candidates were failing to achieve threshold levels for admission to universities in those countries or the United States.

          The British Council sponsored a study and issued a report in May 1985, entitled “Survey of Language Needs in Indonesian Public Administration and Public Sector Management,” more conveniently referred to as the Webb Report. The most frequently cited figure in the Report was the estimate that only 42% of the available opportunities for overseas study had been utilized in the past two years, largely because of the shortage of candidates with adequate English language competence. Some knowledgeable people suggested that 25% was nearer the mark, but that only emphasized the seriousness of a problem bound to get worse as the two major World Bank-supported training programs became active.

          The OTO called a meeting in mid-July of representatives of donor agencies interested in ELT. This was the first of a useful series of gatherings at which common problems could be discussed and donor groups informed of OTO activities. Not all of the Webb Report’s recommendations were endorsed at the meeting, but the gravity of the problem was agreed.

          The Report admonished that development projects should be scrutinized to ensure their language-teaching requirements were adequately financed. This point had particular meaning for GPT-II because no AID funds had been included for ELT, that responsibility falling to the GOI. It was already clear that qualified candidates for GPT-II were not abundant and that the GOI was not then organized to substantially increase ELT. Several years earlier, BAPPENAS had determined that the provision of ELT funds in regular budgets was often abused. Language preparation for specific overseas training opportunities was permitted, but this required a greater lead-time than was practical in most cases.

          Dr. Halim determined that two consultants could usefully be employed to work with him, in defining the dimensions of the ELT problem with regard to overseas training, and in devising policy recommendations for alleviating the problem. One consultant, with management skills, would quantify the demand for ELT in terms of opportunities available to the GOI, estimate the costs of meeting the demand, and suggesting how the costs should be shared between donor and GOI. The other, with ELT skills, would help to assess in-country language teaching capacity and estimate the time and resource requirements for achieving English proficiency from various levels of preparedness.

          Richard Pagett returned to Indonesia for the management task. MUCIA recruited Dr. Jeffrey Dreyfuss of the Wisconsin English Language Institute for the other position. The team, having only three weeks in which to prepare its report, relied heavily on other studies for basic data: the Webb Report, and documents produced for their own purposes on ELT needs and sources of supply by the Australian Language Center and the Canadian CIDA.

          The Group first gathered the most reliable information on the numbers of overseas training opportunities available to the GOI, the number of people who actually go abroad, and the kind of pre-departure training they receive, especially in the English language. They then identified which Government policies might unintentionally constrain language teaching and learning. Finally, they formulated a set of strategic recommendations through which the GOI and the international community could cooperate to improve the situation.

          The Group presented its quantitative data and strategic recommendations to a second donor meeting in late August, and then held meetings with Drs. Mursjid and Dr. William Fuller, the USAID Mission Director, to plan follow-up action. The key recommendations were that candidates for overseas training should be selected without regard to previous language training and then given intensive instruction to the level required for graduate school admission, around 550 on the TOEFL. The GOI should be responsible for bringing candidates to the 475 TOEFL level, but beyond that ELT should be considered an integral part of the overseas training process and financed by the donor accordingly. It was not proposed that the OTO organize in-house ELT classes, but that the Office could play a coordinating and brokering role, arranging intensive instruction for candidates sponsored by different training programs but at the same level of language-learning. The training model proposed involved three four-month ELT courses in-country for a candidate entering with a TOEFL score in the 300s, followed by a two-month course in the country of study. Those with better English could enter whichever course level was appropriate. The GOI would be responsible for the first two levels of training and the donor the last two.


          In the past, each project devised its own selection procedures for training at the graduate level. As could be expected, some were more effective than others. Typically, undergraduate grade-point averages, personal interviews and job performance were taken into account, although factors such as seniority and favoritism could not always be excluded. The Ministry of Education and Culture was confident that its candidates had the opportunity to demonstrate high potential before being selected for advanced training, but the rate of expansion projected for the university system in Repelita IV would require increasingly early selections to be made. Other ministries generally had less academic experience on which to judge candidates.

          Perhaps the main criterion for selection in reality often turned out to be foreign language familiarity. Most training opportunities offered by donor assistance agencies, if they provide language training at all, limit it to no more than six months. In some projects, 90% of the candidates proposed were rejected by the language centers as having inadequate backgrounds to be able to achieve graduate school entrance within that period.

          The need for an objective measurement device for graduate-level candidates was evident before the start of GPT-II. At least three training programs, one at the Ministry of Finance, one at BPPT and the third in computer sciences at UI, had improvised by translating items from published GRE or GMAT sample tests into the Indonesian language. A more systematic process of devising, standardizing and validating tests would surely meet an important need.

          The Ministry of Education and Culture suggested Dr. Sumadi Suryabrata, professor of psychology at Gadjah Mada University, to work with the MUCIA consultant, Dr. Daniel Mueller of the University of Indiana, in determining the merits of constructing an Indonesian language academic aptitude test and figuring out how to get it done. Dr. Sumadi has a PhD from the University of Iowa and has been involved in virtually every major initiative in the mental measurements field in Indonesia for the past 20 years.

          This Group quickly realized that the more difficult task was not the construction of the initial academic aptitude test, but the development of a process for renewing and revising the test, ensuring reliability, validity, and comparability of the different versions, and maintaining test security. They recommended a three-stage process culminating in the creation of a national testing service.

          Stage One, lasting only two months in the Fall of 1985, would be devoted to the modification of existing tests in Bahasa Indonesia, principally the ones used by the Ministry of Finance and BPPT, and the translation of additional items in order to produce an instrument for use on an experimental basis in November, 1985. Stage Two, lasting a year, would see the construction of a new test utilizing appropriate empirical test construction procedures, and Stage Three would comprise a two-year period in which test revisions and validations would take place. The Group recommended the formation of a national testing service at the end of Stage Three, in mid-1988, quite possibly encompassing the national university selection process as well as graduate-level selection.

          The Group discussed its findings with selected members of the mental measurement community in a seminar at BAPPENAS in late August. They also presented their recommendations to senior officials of the Ministry of Education and Culture, the Ministry of Finance, BPPT, NSCOT, and USAID.


          Ministries and agencies of the Government of Indonesia have formally been required to construct training plans since 1974, but in practice few do so. NSCOT made the existence of a plan a requirement for any agency to participate in overseas training, but in August 1985, nine months after the announcement of this policy, only ten agencies had complied and their efforts were extremely uneven.

          The OTO identified C. David Esch of the Institute of Public Administration (IPA) in Washington, D.C., as consultant. Esch had recently assisted LAN in organizing a course for senior civil servants and he understood the administrative system. Dr. Soerjono, a LAN employee recently returned with a PhD in public administration from USC, was assigned to work with Esch for the two weeks he was in Jakarta.

          The Working Group studied the training plans submitted, talked with administrative authorities in LAN and elsewhere, and held two seminars with over 30 departmental officials at each to discuss the planning process. Their report stressed process rather than product. The departments must do the initial training plans because only they know the content of their organizational tasks. OTO can and should provide technical assistance to the gradual upgrading of these plans, but NSCOT was advised not to be overly critical of first-round efforts. Each year, the planning process could be improved until the point when it would become a valued element in the organizational management process.

          The Report recommended that OTO consultants, seconded from LAN, work with not more than three or four ministries in 1985 to improve their training plans and that a major attempt to upgrade the process be deferred until 1986.


          By the end of August, when the first three Working Groups had completed their initial reports, the OTO had moved to a higher plane of action and significance. Common to each report was the theme that achieving greater quality of performance in planning, selection and the preparation of candidates for overseas training was within the range of possibility for the OTO, but a year of solid effort would be required in each field to develop adequate support systems.

          It might be possible, and less costly in the short run, to continue selecting candidates on the basis of foreign language ability and occupational seniority -- schools could as always be found to accept them and the failure rate might be low, in the sense that they would eventually receive their intended degrees.

          NSCOT and USAID became convinced, however, on the basis of the Working Group reports, that the opportunity costs of this approach would be substantial in that the people with the most innate ability might not be selected, and those selected would often be placed in inferior schools relative to their abilities because they had not been adequately prepared.

          The Working Groups were therefore granted the funds and authority to pursue their recommendations. The binational character of the Groups diminished with the departure of the foreign consultants, but they or others reappeared from time to time, and in a way their main tasks had been accomplished: they helped to empower their counterparts to perform the essential tasks for which they had been trained, by clarifying the need for these activities and bringing the case to the attention of national policy-makers. This role is most difficult to play from within a system.

          Before examining the actions followed by these three initial Groups, we look at the fourth, which began its activities two months later than the others.


          One of the early accomplishments of this Group was the change of its name. Recognizing that the full title was cumbersome and the acronym no better, they devised the term Cross-cultural Orientation for Participant Effectiveness (COPE). This term actually refers to an intensive one-week program the Group devised for participants immediately before their departure for study abroad, but the acronym is so punchy that it has come to apply to the Group itself.

          Dr. R. Michael Paige, Associate Director of the Office of International Education at the University of Minnesota, was unavailable in August but was able to join Professor Sadli for two months beginning in late September. She was acquainted with the efforts of the other Groups, but found her Group dealing with a more amorphous task.

          In the process of preparing Indonesian students for graduate study in the United States, the language training requirements are measurable and quite specific. Beyond language training, however, are a set of equally important readiness variables that are somewhat less quantifiable and less well understood. These variables range from the quality of academic preparation up to and including undergraduate work, the differences in educational methods and practices between Indonesia and the US, and cultural factors that may affect performance as evaluated by US educators.

          The COPE Group set itself to assessing the quality of orientation offered in existing programs; interviewing returned students and searching the literature for evidence of the nature of orientation problems encountered by Indonesian participants; and defining experimental techniques for dealing with them. Their task was complicated by the general perception that Indonesian students do well abroad academically and seldom have to return prematurely due to maladaptation to their circumstances. This view was countered by equal recognition that Indonesian students tend to cluster to themselves socially, frequently refrain from the give-and-take of classroom exchanges which typify US graduate education, and experience difficulty with writing assignments, particularly those of an analytical character.

          The Group contacted leading local language centers and found them very interested in academic skills upgrading and cultural orientation. The sponsors of students preparing for study in the US were often, however, mainly interested in improved TOEFL scores. That test does not measure writing skills, and the amount of time that can be devoted to non-testable skills is limited. The students themselves often question whether a given activity will help them pass the TOEFL.

          The COPE Group and the language centers worked out a pattern of cooperation that continues. In addition, they determined that a separate, intense, 60-hour program should be offered, in which participants would be prepared for cross-cultural experience just before they embarked. In developing the content for the week-long COPE, the Group decided to call on talents from two groups in Indonesia: the Faculty of Psychology at the University of Indonesia, and Bina Antar Budaya. The latter is a yayasan, or private foundation, which grew out of an international student exchange program called American Field Service (now known simply as AFS).

          Bina Antar Budaya members have all traveled abroad, many spending one of their high school years living with a family in another country. Over the years, they have gained a great deal of experience in preparing young people for cross-cultural experiences. AFS groups in many countries share their knowledge and training techniques, making the organization the repository of vast practical information about intercultural exchanges.

          The people from the Faculty of Psychology brought a theoretical underpinning to the effort. They were often less experienced in actually participating in and preparing people for cross-cultural living and studying, but they were better grounded in the literature, such as it is, than those from the yayasan. In reality, the literature pertaining to methods of preparing graduate students for work in another culture is not abundant. The COPE Working Group is almost from the beginning breaking new ground. The fact that they are Indonesians preparing their students for experience abroad is particularly unusual, although it seems curious that this should be so. There are many foreign student advisors employed by US universities working on the problem from the other end, helping foreign students adapt to conditions once they are already in the university.

          In January, Professor Joseph Mestenhauser, Director of the Office of International Education at the University of Minnesota, came out to join Dr. Sadli in carrying forward the work of the Group started with the assistance of his colleague, Paige. Mestenhauser is 1987-88 President of the National Association of Foreign Student Advisors, (NAFSA), and has a strong grounding in cross-cultural psychology. During his visit, the COPE Group, including prospective trainers from UI and Bina Antar Budaya, held a weekend retreat at Ancol to discuss the literature in the field and the psychological theory that might be useful in preparing materials for the COPE.

          Then Dr. Sadli and the trainers set to work developing training materials in the Indonesian language. In May 1986 they held a practice COPE of two and a half days, attended by participants en route to the US for September enrolment. The participants were divided into two groups, one led by Faculty people and the other by Bina Antar Budaya trainers. That turned out to be unwise; a synthesis of theory and experience was needed, not a program featuring one approach or the other.

          In the summer of 1986, Dr. Sadli and five of the trainers attended a two-week seninar on cross-cultural research at the University of Hawaii. From there, most of them went on to the mainland, where they visited many Indonesian students at various universities and got first-hand knowledge of the kinds of problems that most concerned them. This was an excellent learning experience for the COPE group.

          Back in Jakarta, at a meeting with staff from the language training centers, the problem of poor study habits arose. The ELT staff found the single-minded pursuit of TOEFL scores by the students to be a constraint on learning many of the other aspects of the use of English that are important to success in graduate studies. The language teachers thought the COPE Working Group could help bring the participants to understand the need for a balanced approach to language, not just a TOEFL preparatory course.

          The COPE Group presented a 20-hour weekend workshop on study skills to advanced language students in late October. Paige, who is fluent in Bahasa Indonesia, returned to observe, but not participate, in the session, and to prepare an evaluation process to measure the results. The balance of the COPE, an additional 40 hours, is to be given to the participants near their departure dates.

          The COPE Group has recently been called upon for another interesting undertaking. The issue arose of how much intensive language instruction is required by candidates for undergraduate training in the Netherlands. Sadli and Nani Nurrahman, one of the COPE trainers from UI, were asked to do a brief study of the issue and report to the negotiators of a bilateral study program. Working Groups such as COPE can readily handle this kind of applied research on immediate policy issues, because of the expertise they command and their freedom from routine managerial responsibilities.

          COPE continues to be an experimental activity. There is the practical problem of scheduling the full COPE workshop: advanced language training ends in January and the departure of participants can be anywhere from May to August. January is thought to be too far in advance of departure for the training to be most effective, but it is logistically and financially difficult to re-assemble participants just before departure. In addition, it is difficult to measure the results of the COPE. The language centers were very pleased with the results of the study skills workshop, but solid evidence of achievement is hard to come by because of the nature of the task.

          The path-breaking nature of this work is gaining recognition internationally, as evidenced by the invitation by NAFSA for COPE to send a representative to its annual meeting. Nani Nurrahman made a presentation on COPE in San Diego in May 1987.


          Professor Sumadi’s test was ready for administration in November 1985, as planned. He was able to use the preliminary efforts of the Ministry of Finance and BPPT as a starting point, and to supplement these materials with translations of items from old GREs and GMATs as well as by devising new items. He called the result Tes Potensi Akademik (TPA).

          On November 30, 1985, the TPA was administered to 941 candidates at 18 locations around the country. The Institute for International Education (IIE) operates a network of testing centers in Indonesia for the periodic administration of TOEFL, GRE and other Educational Testing Service (ETS) instruments, and the local IIE office contracted to administer the TPA in the morning and the pre-TOEFL in the afternoon at the designated sites. To provide broader geographic coverage, OTO staff members set up four sites in addition to the IIE network, those in Bali and Lombok receiving particularly generous attention.

          The test is composed of three parts, Verbal, Quantitative, and Analytical, as is the GRE. The results of the November tests revealed a normal distribution of quantitative scores, a mean of 45.29 out of 90 and a standard deviation of 15.8. Lower than normal scores were recorded on the verbal and analytical sections. The mean verbal score was 36.38 out of 90 and the mean analytical score was 24.57 out of 70 or 35%. The highest scores achieved were 85 on the quantitative section, 64 on the verbal and 51 on the analytical.

          These results are similar to the experience of most Asian students taking the GRE in the United States. Lack of familiarity with English, a factor in the GRE, would not pertain in the TPA since it was offered in Bahasa Indonesia. The verbal section of the TPA did, however, contain a number of recently coined words dealing with scientific phenomena that may not have found general acceptance or awareness as yet.

          Low analytical scores present an intriguing problem. Quite possibly, the verbal analogies employed in Western culture are different in kind from those employed in the Indonesian culture. Also, Indonesian students have not usually encountered tests of this sort before. More practice may improve their performance. OTO flagged this issue as one deserving more investigation in the future.

          The value of the TPA was demonstrated and enhanced by the administration of the Pre-TOEFL at the same time. Some candidates scored high on both tests, and they would presumably have been selected for overseas training no matter what selection process was used. Others did well on the Pre-TOEFL but poorly on the TPA. They may still be selected by various programs because of the ease with which they can pass language thresholds, and their academic records will be worth tracing to see how well TPA serves as a predictor. The most rewarding result of the test, however, was the identification of people with high academic aptitude and low pre-TOEFL scores. These were the people unlikely to have been selected using other methods, despite their intellectual promise. Five candidates out of the top 20, excluding university people, had pre-TOEFL scores of under 400. They would need from four to eight months of intensive instruction before they could be expected to be admitted to most preparatory language courses.

          The principal purpose of offering the TPA at that time was to test the test, to provide data for checking item reliability. The ministries responded so well to the invitation to participate that the results were useful for the initial screening of candidates as well.

          USAID agreed to support experimental language courses at the basic, intermediate and advanced levels in the first six months of 1986. This allowed time for the GOI fiscal year to end and another to begin, at which time the GOI would assume the costs of basic and intermediate level training and USAID would continue to support advanced level training in Indonesia and the US. On the basis of TPA and pre-TOEFL scores, OTO selected 48 candidates to begin basic ELT, 54 for intermediate, and 48 for advanced training.

          It was up to the ELT Working Group to arrange courses for these 150 people. The Australian Language Center and the British Council, the premier ELT facilities in Indonesia, offered full cooperation. The BC provided OTO with a staff member on loan to help manage the program, and OTO hired another ELT specialist from ALC.

          Advanced courses, in which native-speakers are generally the most effective teachers, were allotted to ALC and BC. The BC offered one intermediate course, and ALC had one intermediate and one at the basic level. This allowed the centers the opportunity to develop curricula at more elementary levels than they normally offered courses, and put them in a position to provide technical assistance to regional universities mounting courses at these levels. After the first round, it was intended that all basic and elementary courses should be offered at universities rather than the more costly language centers.

          The Bahasa Lembaga at Sriwijaya University in Palembang, Sumatra, admitted 36 candidates at the basic level in January, and Hasanuddin University in Ujung Pandang, Sulawesi, began courses at both basic and intermediate levels in March. Sriwijaya had an on-going program that was expanded to admit the CPT-II candidates, but the Hasanuddin courses had to be organized from scratch. Six teachers from Hasanuddin spent a week working with staff at BC and ALC in preparation for these courses, and staff exchanges and curriculum development meetings have been held frequently among all four institutions since then.

          Most of the classes began in late January or early February in order to be able to complete one full four-month cycle before the start of Ramadan, the holy month. Delays in obtaining pre-TOEFL test results led to the notification of ministries of the eligibility of their candidates for language training in some cases only a week before the course began. Of the 36 identified for admission to the Sriwijaya course, 34 showed up in Sumatra one week later to begin studies.

          The extraordinary alacrity with which these ELT opportunities were taken up indicates the pent-up demand for such training on the part of the candidates and their employers. It testifies to the remarkable ability of the machinery of Government to move quickly when it really wants to, and it is a strong demonstration of the discipline of the Indonesian civil servants.

          Not all were happy to be in Palembang for four months, away from family and friends in a provincial city. Nevertheless, they, like the people at the other three sites, persevered with their studies until relieved by the fasting month of Ramadan. Since that cycle of courses, a BELT course has been inaugurated at LAN, and other sites in Jakarta and elsewhere in Java are currently being explored.

          After the first ELT cycle was over, the Working Group assessed results, particularly the fit between where one course left off and the next was supposed to begin. In this they had the full cooperation of professionals from ALC, BC, Sriwijaya and Hasanuddin, plus the assistance of Dr. Richard Tucker, President of the Center for Applied Linguistics in Washington, who came out as a consultant. Tucker was much impressed with the accomplishments of the program and the utility of the TPA in selecting candidates who warrant the investment of intensive ELT. He and the others found, however, that the initial estimates of the time it takes to achieve substantial gains in TOEFL scores, particularly at the higher levels, had been optimistic. As a result, the course levels were re-calibrated and optional loops built in to permit slower students to repeat certain segments.

          The annual cycle of TPA administration and ELT courses was also worked out so that candidates will no longer have only a week to pack their bags. The TPA is now offered regularly twice each year, the number of takers increasing each time until now, when about 2000 show up. IIE still administers the test using an expanded TOEFL network.

          Fewer candidates are selected for training now, because GPT-II funds are largely committed. Even a supplement of $10,000,000 made by USAID in 1986 is already obligated for candidates. Other programs are using the TPA for selection, however, including some USAID projects and some other bilateral and international projects. In addition, one program has used it for selection to an Indonesian graduate school, and BAPPENAS has used it in evaluating candidates for professional employment.

          The TPA clearly does not suffer from neglect. If anything, the tendency is for it to be relied upon too heavily. No selection test should be the sole or primary criterion for choosing people for training or employment. It can become, and the TPA is rapidly becoming, a necessary element in the choice, but other factors such as work experience, grade-point average and interviews must play a part in the final choice.

          The Working Group on Training Plans has had less evident success. This may in part be due to the fact that the Indonesian member was not in practice the head of the team, as was the case in the other Groups, and was not able to continue on a part-time basis with OTO after the departure of the foreign consultant. More importantly, perhaps, training plans only become genuinely meaningful when they are substantially funded. Projects drawn up for strengthening the capacity of an agency or institute will often include funds for overseas training, the uses of which will be carefully defined. It is less likely that the same care will be devoted to constructing a training plan in the hope that one or more employees will qualify for a government-wide selection process. Moreover, one could make the case that it is more important to select individuals on the basis of their academic potential, and assume that they will find a way to use the training they receive, than to invest in training the cadre of a department and assume that they will remain in the positions for which they were trained.

          Other consultative activities could be called Working Groups but lack the essential dynamics of the ELT, TPA and COPE Groups. For example, Herbert Roberts, a retired AID training officer, assisted the OTO on two occasions to perfect a D Base III system for handling participant data. He works closely with Bambang Widlanto, an able BAPPENAS electronic engineer, and together they have produced a highly useful system that may be adopted by other projects administered in Indonesia. This is a discrete technical activity, however, rather than a continuing process of development under Indonesian leadership, as in the other cases.


          The TPA, ELT and COPE Groups all have substantial agendas facing them. Until now, Professor Sumadi, with the part-time assistance of Dr. Soetarlinah Soekadji, has revised the TPA and created new items each time the test was to be administered. The TPA needs to be institutionalized, preferably by being broadened into a national testing center that could handle university admission tests as well as the TPA. Additional staff need to be hired, and more sent off for advanced training. Scoring machines and computers for use in item banking are needed. Security needs to be improved. The course ahead is clear, if not easy.

          ELT has a similarly long list of tasks before it. Additional regional universities need help in developing basic and intermediary ELT programs. Sriwijaya and UI may need the capacity to offer advanced courses.

          Curriculum development must continue at all levels, and some thought should be given to a program for training ELT trainers abroad.

          COPE remains in an experimental mode. In addition to testing and refining the COPE itself, the group will continue to be called upon for advice or research on particular intercultural problems. The COPE Group will remain on the frontier of the intercultural field in the foreseeable future.

          Beyond the boundaries of the current array of Working Groups, the OTO faces additional challenges for which this collaborative approach may be suitable. A Working Group on Academic Upgrading has already been formed under the chairmanship of the Rector of Sacha Wachana University, Dr. Willi Toisuta. Two senior American consultants helped to define the problem: Dr. Michael Moravcik, a physicist from the University of Oregon, and Dr. David Cole, an economist from HIID. Special offerings in mathematics, statistics and analytical writing are to be added to the advanced ELT course at some future date.

          The materials designed for this purpose could in time have an important impact on university education in Indonesia generally. They are characterized by a questioning, problem-solving approach that is often thought to be lacking in Indonesian higher education.

          Other subjects for future attention include overseas placement, support and monitoring of participants, and the re-entry process. Until now, OTO has relied upon support services provided by MUCIA at the University of Wisconsin and by HIID for the placement, support and monitoring of students. Both offices have long experience assisting Indonesian students; they are easy to deal with and familiar with the educational backgrounds of the candidates. In time the OTO will of course wish to review its options for handling students abroad, including exploring the possibility of opening its own offices. Drs. Mursjid will make a preliminary review of service options late in 1987.

          The re-entry issue is as multidimensional and experimental as COPE, and a Working Group is needed on the subject. The complexity arises not just from the individual adjustment that the return to Indonesia entails. Re-entry into the workplace, where the benefits of the overseas experience may not be recognized, could be facilitated. The adaptation of newly learned skills and techniques to Indonesian realities is a vast, unexamined problem.

          A beginning of the process of understanding re-entry problems should result from an assessment of the impact of overseas training being undertaken by OTO with AID support. This study is intended to deal analytically with the benefits of overseas graduate training to the individuals and to the organizations employing them. Interviews with several hundred returnees will be the basis for the study.


          The Working Groups are a story in progress. Fortunately, no end to their usefulness to the OTO is in sight. Whether the binational approach used by OTO would be useful to other Indonesian institutions, or institutions in other countries, is an open question, but our assumption is that if care is taken in the selection of both local and foreign participants, the device has some valuable features that could be replicable.

          To some extent the Working Groups take advantage of apparent flaws in the Indonesian system. The necessity of most Indonesian professionals, especially in the university system, to supplement their incomes through outside employment is widely and justly deplored. Yet this feature served OTO well by allowing leading individuals from the university world to work several days each week for the OTO without damage to their academic standing.

          In general, the pairing of local and foreign experts on a single task is likely to be very productive. The particular knowledge of the one complements the more general knowledge of the other. Access to policy-makers and other key figures is likely to be greater together than either would have separately. In some cases the status of a foreign visitor is useful, and at other times intimate local knowledge of whom it is essential to see about what is of great importance.

          The Working Groups continue to be binational in character, even though the foreign visitors come only once or twice each year. The consultants who proved most valuable tend to be invited to return periodically, a considerable saving in the orientation period necessary for a visitor to become useful. The OTO, essentially an interface between Indonesia and countries in which Indonesians study, should, I think, cherish its multinational character and not aspire to be an organization wholly staffed by Indonesians.

          And so the Working Group story is in part the story of Mueller and Mestenhauser, Pagett and Paige; but it is even more the story of Sumadi, Sadli, Amran Halim and Willi Toisuta. It is a story of successful technical assistance; that is, the creation of an organization where the local people do the real work and their society reaps the benefits.

© 2000 Development Strategies Inc. | Contact | Disclaimer | Privacy Policy