On 19th November 2005, ACALAN, Linguasphere Observatory and Language Observatory had a meeting at Hotel Acropole, nearby the WSIS Conference Complex, to discuss about a joint action plan.

On 17th November 2005, Language Observatory signed Memorandom of Understanding (MOU) with the Linguasphere Observatory at the UNESCO booth in the Exibition Hall. The Linguasphere Observatory (the Observatoire Linguistique in French and the Wylfa Ieithoedd in Welsh) is a transnational research network devoted to the worldwide study and promotion of multilingualism. It is an independant non-profit organization, created in France in 1983 based currently in Wales, UK. The signing ceremony was witnessed by the British Minister of State for Industry, Rt. Hon. Alun Michael and Ms.Elizabeth Longworth, the representative from UNESCO.






Mikami, Ms.Debbie Garside and Ms.Carla Salem

15th November 2005, Dr. Zavarsky, Ms. Carla nd myself attended the Round Table meeting organized by African Academy of Languages (ACALAN). The meeting was held as one of parallel events registered at World Summit on Information Society (WSIS), Tunis phase (see official record of the events here). As shown in photo, the event was also held as a part of "THE AFRICAN ICT WEEK".



The meeting was attended by various groups from all around the globe. In the front line from right (in front of screen), Daniel Pimienta from FUNREDES (Dominican Republic), Daniel Prado from UNION LATIN (France), David Pearson from SIL International, myself, Antoni Mir i Fullana from Casa de les Llengues (Barcelona), Viola Krebs from ICvolunteers (Geneve). The main table on the back was seated by Adama Samassekou (President of ACALAN) at the center, and Claudio Menezes from UNESCO on the left, etc.

Our research partner in Thailand, Thai Computational Linguistics Laboratory (TCL) has started a new collaborative experiment a week before. TCL is a partnership-laboratory of Computational Linguistics Group of Keihanna Human Info-Communication Research Center (KICR) / National Institute of Information and Communications Technology (NICT), a national research institute under the Japanese Ministry of Interior and Communications (MIC).

The experiment is a part of TCL's research project called "Web Language Engineering - Open Collaborative Archiving (WLE-OCA)". The idea behind this experiment is to create a distributed crawler network, where each crawler is assigned a queue list depending upon time proximity criteria, instead of predetermined algorithm like URL-hashing, domain-hashing, etc. At this moment, three crawlers located at Bangkok(TCL), Keihanna(KICR) and Nagaoka(NUT) are running. You can monitor their data collection process through here.

Advantage of this architecture is:

1) Scalability: Crawler network is fully scalable, depending upon the availability of computing resources. Taking into account the rapidly increasing web resources (see my previous blog "Growth of indexed pages by Google"), it seems almost impossible to crawl entire web-space by a single cluster of servers. This situation is necessitating scalable architecture of crawling.

2) Optimum allocation of tasks among the network: When servers are distributed widely through the network, it is needed to realize an optimum allocation of tasks among servers, where assignment criteria should be given by their access time to target servers. It will contribute the improvement of data collection speed as a whole cluster.

3) Fault-tolerance: This architecture enables high fault-tolerance of crawling mechanism.

Dr. Virach and his research team at Thai Computational Linguistics Laboratory (TCL) visited us last week to have a series of technical discussions on various issues of our common interests and to develop our joint research program in future.

His team has developed a framework of Web Language Engineering for Open Collaborative Archiving (WLE-OCA). The framework contains three components:

1) collaborative cralwer
2) language identifier
3) multi-lingual search engine.

Collaborative crawler architecture has a unique feature. It allocates crawling tasks of a newly found URL based on proximity to the target URL. Distributed and collaborative clusters of crawlers are expected to work most efficient manner through this mechanism. NUT allocate a part of gii servers to this joint experiment.

Also we will challenge several other interesting joint programs. These will be reported later. Our collaboration was reported by "The Nation" on 10th Ocotober 2005.