Yet another uninformative blog title

2011-11-02T02:23:00.000Z

I tried out DBpedia Spotlight with part of the introduction to "With Fire and Sword":

Now let us see what this western history was. In the middle of the ninth century Slav tribes of various denominations occupied the entire Baltic coast west of the Vistula; a line drawn from Lubeck to the Elbe, ascending the river to Magdeburg, thence to the western ridge of the Bohemian mountains, and passing on in a somewhat irregular course, leaving Carinthia and Styria on the east, gives the boundary between the Germans and the Slavs at that period. Very nearly in the centre of the territory north of Bohemia and the Carpathians lived one of a number of Slav tribes, the Polyane (or men of the plain), who occupied the region afterwards called Great Poland by the Poles, and now called South Prussia by the Germans. In this Great Poland political life among the Northwestern Slavs began in the second half of the ninth century. About the middle of the tenth, Mechislav (Mieczislaw), the ruler, received Christianity, and the modest title of Count of the German Empire. Boleslav the Brave, his son and successor, extended his territory to the upper Elbe, from which region its boundary line passed through or near Berlin, whence it followed the Oder to the sea. Before his death, in 1025, Boleslav wished to be anointed king by the Pope. The ceremony was denied him, therefore he had it performed by bishops at home. About a century later the western boundary was pushed forward by Boleslav Wry-mouth (1132-1139) to a point on the Baltic about half-way between Stettin and Lubeck. This was the greatest extension of Poland to the west. Between this line and the Elbe were Slav tribes; but the region had already become marken (marches) where the intrusive Germans were struggling for the lands and persons of the Slavs.

Corrected:

Now let us see what this western history was. In the middle of the ninth century Slav tribes of various denominations occupied the entire Baltic coast west of the Vistula; a line drawn from Lubeck to the Elbe, ascending the river to Magdeburg, thence to the western ridge of the Bohemian mountains, and passing on in a somewhat irregular course, leaving Carinthia and Styria on the east, gives the boundary between the Germans and the Slavs at that period. Very nearly in the centre of the territory north of Bohemia and the Carpathians lived one of a number of Slav tribes, the Polyane (or men of the plain), who occupied the region afterwards called Great Poland by the Poles, and now called South Prussia by the Germans. In this Great Poland political life among the Northwestern Slavs began in the second half of the ninth century. About the middle of the tenth, Mechislav (Mieczislaw), the ruler, received Christianity, and the modest title of Count of the German Empire. Boleslav the Brave, his son and successor, extended his territory to the upper Elbe, from which region its boundary line passed through or near Berlin, whence it followed the Oder to the sea. Before his death, in 1025, Boleslav wished to be anointed king by the Pope. The ceremony was denied him, therefore he had it performed by bishops at home. About a century later the western boundary was pushed forward by Boleslav Wry-mouth (1132-1139) to a point on the Baltic about half-way between Stettin and Lubeck. This was the greatest extension of Poland to the west. Between this line and the Elbe were Slav tribes; but the region had already become marken (marches) where the intrusive Germans were struggling for the lands and persons of the Slavs.

stupid-unknown-extractor

2011-08-26T00:12:00.008+01:00

This is a small utility for extracting unknowns from a pair of tagged streams. It's based on that idea that, if each sentence has a single unknown word, then those words are likely to be translations of each other. It's stupid because it does nothing if there is more than one unknown.

Usage:

$ cat en.txt | apertium -d [path-to]/apertium-en-es/ en-es-tagger > en.tagged

$ cat es.txt | apertium -d [path-to]/apertium-en-es/ es-en-tagger > es.tagged

$ stupid-unknown-extractor en.tagged es.tagged

Sample output:

buckler	adarga	old<adj><sint> *buckler ,<cm> :: *adarga antiguo<adj><f><sg>

homespun	velludo	good<adj><sint><sup> *homespun .<sent> :: de<pr> *velludo para<pr>

gaunt	recia	*gaunt -<guio> :: complexión<n><f><sg> *recia ,<cm>

Compile:

g++ stupid_unknown_extractor.cc -o stupid-unknown-extractor

Code:

#include <iostream>

#include <string>

#include <cstdio>

#include <list>

#include <vector>



using namespace std;



//Set to true to also split at <cm>

bool split_cm = true;



inline bool

is_sent(wstring &in)

{

return ((in.size() > 6) && (in.compare(in.size()-6, 6, L"<sent>") == 0));

}



inline bool

is_cm(wstring &in)

{

return ((in.size() > 4) && (in.compare(in.size()-4, 4, L"<cm>") == 0));

}



inline bool

is_split (wstring &in)

{

if (split_cm)

return (is_sent(in) || is_cm(in));

else

return (is_sent(in));

}



wstring

read_word(FILE *input)

{

wstring out = L"";

wchar_t c;

bool inword = false;



while(!feof(input))

{

c = static_cast<wchar_t>(fgetwc(input));

if (!inword)

{

if (c == L'^')

{

inword = true;

}

if (c == L'\\')

{

c = static_cast<wchar_t>(fgetwc(input));

}

}

else

{

if (c == L'$')

{

return out;

}

if(c == L'\\')

{

out += L'\\';

c = static_cast<wchar_t>(fgetwc(input));

out += c;

}

else

{

out += c;

}

}

}



return L"";

}



void usage()

{

wcout << L"usage: stupid-unknown-extractor file1 file2 [output]" << endl;

}



bool

read_sentence (FILE* file, vector<wstring> &tokens)

{

wstring word;

while (!feof(file))

{

word = read_word(file);

tokens.push_back(word);

if (is_split(word))

{

return true;

}

}

return false;

}



vector<int>

unknown_indices(vector<wstring> sentence)

{

vector<int> index;

vector<wstring>::iterator it;

int count = 0;



for (it=sentence.begin(); it < sentence.end(); it++)

{

if ((*it)[0] == L'*')

{

index.push_back(count);

}

count++;

}

return index;

}



void

print_context(FILE* out, vector<wstring> &sent, int index)

{

wstring tmp;

if (index >= 1)

{

fputws(sent[index - 1].c_str(), out);

fputwc(L' ', out);

}

fputws(sent[index].c_str(), out);

fputwc(L' ', out);

fputws(sent[index + 1].c_str(), out);

}



void

print_unk(FILE* out, vector<wstring> &sent, int index)

{

if (sent.at(index)[0] != L'*')

{

wcerr << L"Error with unknown: " << sent.at(index) << endl;

}

fputws(sent.at(index).substr(1, sent.at(index).length() - 1).c_str(), out);

}



void

try_output(FILE* out, vector<wstring> &left, vector<wstring> &right)

{

vector<int> lvec;

vector<int> rvec;

lvec = unknown_indices(left);

rvec = unknown_indices(right);



if ((lvec.size() == 1) && (rvec.size() == 1))

{

print_unk(out, left, lvec.at(0));

fputwc(L'\t', out);

print_unk(out, right, rvec.at(0));

fputwc(L'\t', out);



// context

print_context(out, left, lvec.at(0));

fputws(L" :: ", out);

print_context(out, right, rvec.at(0));

fputwc(L'\n', out);

}

}



int main (int argc, char** argv)

{

FILE* left;

FILE* right;

FILE* out;



if (argc < 3 || argc > 4)

{

usage();

exit(1);

}

if (argc == 3)

{

out = stdout;

}

else

{

out = fopen(argv[3], "wb");

}



left = fopen(argv[1], "rb");

right = fopen(argv[2], "rb");



vector<wstring> sentl;

vector<wstring> sentr;



while (read_sentence(left, sentl) && read_sentence(right, sentr))

{

try_output(out, sentl, sentr);

sentl.clear();

sentr.clear();

}



fclose(left);

fclose(right);

fclose(out);

exit(0);

}

Tenancy agreement, GFDL

2011-06-28T23:39:00.001+01:00

Bitext in my pocket

2011-05-07T23:06:00.001+01:00

Smoking is a bad habit blah blah blah but being involved in MT gives me a convenient excuse - parallel text

The missing slide/The first Apertium nursery rhyme

2011-05-05T10:11:00.004+01:00

In my first talk on day one of the workshop, I forgot the slide that explained the most important part of the dictionaries. I promised to include it the next day (but day one was so disastrous, and I presented everything so badly, that instead I just worked through the notes again, explaining things slowly and adding the new material at the end).

So, we had a meeting to discuss how things went (so badly wrong), when I decided not to use a presentation at all, and towards the end, someone said "it's getting late, we should go to sleep". I said I was too wired to sleep, and Juan Antonio offered to read me a nursery rhyme to help me sleep.

A few minutes later, I went outside to smoke, and thought... hey, I can explain this as a nursery rhyme... so I thought of something quickly, then looked for a cutesy template to use for the slide, and put it on my phone to show the others. They liked the rhyme, but not the picture -- thought it would be insultingly childish. They were right, so I skipped it.

After the workshop, we had the social evening (Chinese buffet), and everyone was talking, drinking, laughing, so I thought I'd show it to a few of the participants. They all loved it. Most of the women had congregated at one table, so I showed them all at the same time. Gema's assessment later was something like "with that slide, Jim chatted up every woman at the table. Including me!".

Day 3 went pretty smoothly (we'd made enough changes for day 2 to recover from day 1), but Tomas decided to rant about XML again (he had launched a ~30 minute discussion about it on day 2. He clearly dislikes XML.) and Felipe asked me if I had the text, I told him I'd put it on the wiki, and he asked me to put it on screen. Some of the women who had been at the restaurant spotted it, and laughed, but most eyes were on Tomas, so I took the first chance to interrupt him, and point at the screen as an example that "we don't think it's too hard to remember".

Then Felipe called on me to read it. Bastard. So I did, and, thankfully, everyone laughed. And applauded, which was odd.

So, that's the story of the first (and hopefully last) Apertium nursery rhyme.

How Google Translate turns millions into a billion

2010-08-18T17:45:00.003+01:00

I was using some stories from the Spanish Wikinews to see how Apertium's Spanish to English translation compares against other MT systems, and noticed something a little entertaining in this article with Google Translate:

El complejo industrial es uno de los cuatro más grandes de China, y se estima que tiene reservas de 19 millones de barriles.

became:

The industrial complex is one of the four largest in China, and is estimated to have reserves of 19 billion barrels.

So, although I work with rule-based MT, I also use SMT tools, so let's see if I can explain this.

The really high-level overview of statistical MT is that it uses existing translations to provide new ones: by looking at thousands of sentences that are (assumed to be - important point!) translations of each other, software (usually giza++) slowly determines which words are probable translations of each other.

One common technique to make the bilingual text more general is to replace all numbers with a pseudo-word, like __NUMBER__, in the training text, and to pass them through unchanged at run time; that is, to assume that numbers don't need to be translated.

However, that's not always true, because of the long and short scales. In English-speaking countries, we generally write '2 billion', whereas in Spanish-speaking countries, they write '2.000 millones'... well, I think it should now be clear what happened.

This is why I like RBMT. We can't approach SMT in terms of (so-called) fluency, but at least we can avoid this sort of stupid inaccuracy.

Note to spectie/explaining an obscure commit message

2009-01-19T22:30:00.003Z

I've had too much 19th century Polish poetry here... now for something completely different:

<spectie> There is no indefinite article in Manx. The definite article precedes the noun and is differentiated by number but not gender. For singular nouns, it is yn (‘n if following a vowel, y if between two consonants), and for plural nouns, it is ny.

interesting
only celtic language which has a number differentiated article
that i know of
<jimregan> what???
<spectie> does irish ?
<jimregan> both ga and gd have that
<spectie> i thought it was 'an' for everything
<jimregan> an fear - the man; na fir - the men
* spectie rings the bell of ignorance
<spectie> aha
<jimregan> heh
the next time this happens, I'm going to make you keep a chart of linguistic discoveries, so I can definitively say 'we talked about this before, more than once' :)

(spectie, consider this that reminder).

Then there's this obscure commit message:
apertium: ftyers * r8244 /trunk/incubator/apertium-ga-gv.gv.dix: and yea, one more entry be thrust upward

Easily explained:

<spectie> this grammar is too old :(
<jimregan> what gave that away?
'if the Noun be of the feminine gender'
<spectie> :D
yep basically
<jimregan> heh
English's appendix
'quick! amputate the subjunctive!'

Back to the nonsense...

To B-- (attempt 2)

2009-01-17T04:22:00.001Z

To surround me in dance, the world now conspires
On butterfly wings, it would make me an angel
Place blooms in my hands, in my eyes kindles fires
But no means can it find, my faith to unravel

I have walked on this road of joy and suffering
By which I came to the soul's own rich bounties
The home of the heart, whose ideal is changing
Because the heart knows it - the beauty of beauties

As I loved once, still to love I continue
The same form, that once I so madly pursued
The same soul for whom my affection was real

Both too in spirit, white and holy as one
In loving so I have eternal become
Through my faith in this, my eternal ideal

Do Beatr...

2009-01-16T19:28:00.003Z

Do Beatr...
Zygmunt Krasiński

Darmo świat ziemski pląsem mnie otoczy,
Motylim skrzydłem chce udać anioła,
Pcha kwiaty w ręce, ciska iskry w oczy -
On wiary mojej przetworzyć nie zdoła.

Przebyłem drogę szczęścia i cierpienia
I nią doszedłem do tych ducha włości,
Gdzie się ideał serca już nie zmienia,
Bo zna go serce - pięknością piękności.

Do dziś dnia kocham, jak dawniej kochałem,
Tę samą postać, którąm ścigał szałem,
Tę samą duszę, którąm wielbił szczerze;

Obie i widmem - jednem - świętem - białem -
Tylko w kochaniu wieczniejszym się stałem,
Bo w wieczność mego ideału wierzę!

To Beat...

Freely the earthly world will surround me with dance,
on butterfly wings it wants to imitate an angel,
pushing flowers into my hands, pressing sparks into my eyes -
it won't manage to transform my faith.

I experienced the road of happiness and suffering
and by it I reached these estates of the spirit,
where the ideal of the heart is already changing,
because the heart knows it - by the beauty of beauty.

To this day I love, as I loved before,
that same form, that I chased in a frenzy,
that same soul, that I adored sincerely;

Both also in spirit - one - holy - white -
only in loving I became more eternal,
because I believe in the eternity of my ideal!

A warning...

2008-08-23T14:52:00.005+01:00

I was warned by nicknamed:

“tęsknię po tobie” would sound pretty awkward to a girl… but I presume you know that? it’s “tęsknię za tobą”

Yeah... I already knew that - practical knowledge, so to speak.

Here's a list of some other things that occur in this edition:

'w ów czas' is now one word: 'wówczas'
'wtém' is 'w tym'
the letter 'é' no longer exists in Polish: 'daléj' is 'dalej', etc.
'iak' and 'iuż' each appear ('jak' and 'już' in modern Polish). These were written this way in very old Polish, but here, they are normally written as usual, so these are probably just printer's errors.
'nie' is written separately from most verb forms: 'niewiedział' should be 'nie wiedział'

Pan Tadeusz, Book 1 #1

2008-08-23T11:46:00.003+01:00

LITHUANIA, my country, thou art like health; how much
thou shouldst be prized only he can learn who has
lost thee. To-day thy beauty in all its splendour I see
and describe, for I yearn for thee.

Litwo! Ojczyzno moja! ty jesteś jak zdrowie;
Ile cię trzeba cenić, ten tylko się dowie
Kto cię stracił. Dziś piękność twą w całéj ozdobie
Widzę i opisuję, bo tęsknię po tobie.

Holy Virgin, who protectest bright Czenstochowa
and shinest above the Ostra Gate in Wilno! Thou
who dost shelter the castle of Nowogrodek with its
faithful folk! As by miracle thou didst restore me to
health in my childhood--when, offered by my weeping
mother to thy protection, I raised my dead eyelids, and
could straightway walk to the threshold of thy shrine
to thank God for the life returned me--so by miracle
thou wilt return us to the bosom of our country. Meanwhile
bear my grief-stricken soul to those wooded hills,
to those green meadows stretched far and wide along
the blue Niemen; to those fields painted with various
grain, gilded with wheat, silvered with rye; where
grows the amber mustard, the buckwheat white as
snow, where the clover glows with a maiden's blush,
where all is girdled as with a ribbon by a strip of green
turf on which here and there rest quiet pear-trees.

Panno Święta, co jasnej bronisz Częstochowy
I w Ostréj świecisz Bramie! Ty, co gród zamkowy
Nowogródzki ochraniasz z jego wiernym ludem!
Jak mnie dziecko do zdrowia powróciłaś cudem,
(Gdy od płaczącéj matki pod Twoją opiekę
Ofiarowany, martwą podniosłem powiekę;
I zaraz mogłem pieszo do, Twych świątyń progu
Iść za wrócone życie podziękować Bogu;)
Tak nas powrócisz cudem na Ojczyzny łono.
Tymczasem przenoś moję duszę utęsknioną
Do tych pagórków leśnych, do tych łąk zielonych,
Szeroko nad błękitnym Niemnem rozciągnionych;
Do tych pól malowanych zbożem rozmaitém,
Wyzłacanych pszenicą, posrebrzanych żytem;
Gdzie bursztynowy świerzop, gryka jak śnieg biała,
Gdzie panieńskim rumieńcem dzięcielina pała,
A wszystko przepasane, jakby wstęgą, miedzą
Zieloną, na niéj z rzadka ciche grusze siedzą.

Amid such fields years ago, by the border of a brook,
on a low hill, in a grove of birches, stood a gentleman's
mansion, of wood, but with a stone foundation; the
white walls shone from afar, the whiter since they were
relieved against the dark green of the poplars that
sheltered it against the winds of autumn. The dwelling-house
was not large, but it was spotlessly neat, and it
had a mighty barn, and near it were three stacks of hay
that could not be contained beneath the roof; one
could see that the neighbourhood was rich and fertile.
And one could see from the number of sheaves that up
and down the meadows shone thick as stars--one could
see from the number of ploughs turning up early
the immense tracts of black fallow land that evidently
belonged to the mansion, and were tilled well
like garden beds, that in that house dwelt plenty and
order. The gate wide-open proclaimed to passers-by
that it was hospitable, and invited all to enter as guests.

Śród takich pól przed laty, nad brzegiem ruczaju,
Na pagórku niewielkim, we brzozowym gaju,
Stał dwór szlachecki, z drzewa, lecz podmurowany;
Świeciły się zdaleka pobielane ściany,
Tym bielsze że odbite od ciemnéj zieleni
Topoli, co go bronią od wiatrów jesieni.
Dóm mieszkalny niewielki, lecz zewsząd chędogi,
I stodołę miał wielką i przy niéj trzy stogi
Użątku, co pod strzechą zmieścić się niemoże;
Widać że okolica obfita we zboże,
I widać z liczby kopic, co wzdłuż i wszerz smugów
Świecą gęsto jak gwiazdy; widać z liczby pługów
Orzących wcześnie łany ogromne ugoru
Czarnoziemne, zapewne należne do dworu,
Uprawne dobrze nakształt ogrodowych grządek:
Że w tym domu dostatek mieszka i porządek.
Brama na wciąż otwarta przechodniom ogłasza,
Że gościnna i wszystkich w gościnę zaprasza.

A young gentleman had just entered in a two-horse
carriage, and, after making a turn about the yard, he
stopped before the porch and descended; his horses,
left to themselves, slowly moved towards the gate,
nibbling the grass. The mansion was deserted, for the
porch doors were barred and the bar fastened with a pin.
The traveller did not run to make inquiries at the farmhouse
but opened the door and ran into the mansion,
for he was eager to greet it. It was long since he had
seen the house, for he had been studying in a distant
city and had at last finished his course. He ran in and
gazed with eager emotion upon the ancient walls, his
old friends. He sees the same furniture, the same
hangings with which he had loved to amuse himself
from babyhood, but they seemed less beautiful and not
so large as of old. And the same portraits hung upon
the walls. Here Kosciuszko, in his Cracow coat, with
his eyes raised to heaven, held his two-handed sword;
such was he when on the steps of the altar he swore
that with this sword he would drive the three powers
from Poland or himself would fall upon it. Farther on
sat Rejtan, in Polish costume, mourning the loss of
liberty; in his hands he held a knife with the point
turned against his breast, and before him lay Phaedo
and The Life of Cato. Still farther on Jasinski, a fair
and melancholy youth, and his faithful comrade Korsak
stand side by side on the entrenchments of Praga, on
heaps of Muscovites, hewing down the enemies of
their country--but around them Praga is already
burning.

Właśnie dwókonną bryką wjechał młody panek
I obiegłszy dziedziniec zawrócił przed ganek,
Wysiadł s powozu; konie porzucone same,
Szczypiąc trawę ciągnęły powoli pod bramę.
We dworze pusto: bo drzwi od ganku zamknięto
Zaszczepkami i kołkiem zaszczepki przetknięto.
Podróżny do folwarku nie biegł sług zapytać,
Odemknął, wbiegł do domu, pragnął go powitać,
Dawno domu niewidział; bo w dalekiém mieście
Kończył nauki, końca doczekał nareszcie.
Wbiega i okiem chciwie ściany starodawne
Ogląda czule, jako swe znajome dawne.
Też same widzi sprzęty, też same obicia,
S któremi się zabawiać lubił od powicia;
Lecz mniéj wielkie, mniéj piękne, niż się dawniéj zdały.
I też same portrety na ścianach wisiały.
Tu Kościuszko w czamarce krakowskiéj, z oczyma
Podniesionymi w niebo, miecz oburącz trzyma;
Takim był, gdy przysięgał na stopniach ołtarzów,
Że tym mieczem wypędzi s Polski trzech mocarzów,
Albo sam na nim padnie. Daléj w polskiéj szacie
Siedzi Rejtan żałośny po wolności stracie,
W ręku trzyma nóż, ostrzem zwrócony do łona,
A przed nim leży Fedon i żywot Katona.
Daléj Jasiński, młodzian piękny i posępny;
Obok Korsak towarzysz jego nieodstępny
Stoją na szańcach Pragi, na stosach moskali
Siekąc wrogów a Praga już się w koło pali.

Pan Tadeusz

2008-08-22T19:32:00.003+01:00

Aside from manuals and grammars, I haven't really read anything so far this year. So, it's about time I changed that. My friends keep asking me what I'm doing to improve my Polish, so I've decided to tackle the big one: to read Pan Tadeusz in Polish.

My Polish isn't really up to that, so essentially, I have to read it first in English, then in Polish, and see what sticks.

Despite the fact that Pan Tadeusz is a poem, the English translation is quite readable, and relatively true to the original:

Przypadkiem oczy podniósł, i tuż na parkanie
Stała młoda dziewczyna--białe jéj ubranie
Wysmukłą postać tylko aż do piersi kryje,
Odsłaniając ramiona i łabędzią szyję.

By chance he raised his eyes, and there on the wall
stood a young girl--her white garment
hid her slender form only to the breast,
leaving bare her shoulders and her swan's neck.

The Polish text is not really representative of modern Polish: jéj in the text above is now written jej, the preposition 'z', although pronounced as 's' where appropriate is never in modern Polish written 's' - but these are small differences, and the English translation I'm working with contains similar archaisms, such as 'to-day' instead of 'today'.

Anyway; in linguistic processing, the moans of the lack of parallel text are frequently heard: I think it's more productive to try to do something about it, so I'll be posting book 1 of Pan Tadeusz in both English and Polish. It'll probably be more suitable for language learners than for NLP, but it's worth a try.

Until I'm finished, here are a few selections:

Rumienił się, serce mu biło nadzwyczajnie;
Więc rozwiązane widział swych domysłów tajnie!
Więc było przeznaczono, by przy jego boku
Usiadła owa piękność widziana w pomroku.

He blushed, and his heart beat faster than its wont.
So he now beheld the solution of the mystery upon which he had pondered.
So it had been ordained that by his side
should sit that beauty whom he had seen in the twilight

Lecz młodzież o piękności metrykę nie pyta,
Bo młodzieńcowi młodą jest każda kobiéta,

But youth never asks beauty for its baptismal certificate;
to a young man every woman is young

I w twarz spójrzała, z której wytryskał rumieniec,
Ilekroć z jej oczyma spotkał się młodzieniec:

and she looked into his face, on which a blush rose
as often as the young man met her eyes.

Podróżny zląkł się, spójrzał, lecz już jéj niebyło,
Wyszedł zmieszany i czuł że serce mu biło
Głośno, i sam niewiedział czy go miało śmieszyć
To dziwaczne spotkanie, czy wstydzić, czy cieszyć.

the traveller looked up in alarm, but she was there no longer;
he departed in confusion and felt the loud beating of his
heart; he knew not whether this strange meeting should
cause him amusement or shame or joy.

Apertium in 'The Guardian'

2008-08-15T18:41:00.003+01:00

There was an article about our Welsh translator in yesterday's Guardian: http://www.guardian.co.uk/technology/2008/aug/14/freeourdata.opensource

"Flummoxed by a document in Welsh? Now you can get a free translation at cymraeg.org.uk. The Apertium-cy software, described as the first free automatic translator from Welsh to English, is the fruit of a multilingual effort involving developers in Spain, Wales and Ireland pushing forward the possibilities of open-source software and, they hope, free public-sector data."

The focus of the article is on how we weren't able to use public data compiled by the Welsh Language Board:

'When we contacted the Welsh Language Board, however, it said the Apertium team couldn't be more wrong. "We welcome re-use," it said. Although the small print forbids unauthorised reproduction, the board says it would be delighted to consider requests. Where feasible, it will make products available under what it says would be "a suitable free non-commercial agreement".'

Well, if they had ever returned any of my phone calls, maybe we could have used their data. Maybe they'll give me an answer now :)

apertium-cy-en in the wild

2008-08-15T16:43:00.002+01:00

We released the first version of our Welsh to English translator recently (announcement).

First thing... I get a little thrill every time I see something I've written translated into another language, so I just want to point out the part of the announcement I wrote:

This package constitutes a number of 'firsts' for the Apertium platform: it is the first package targetting a Celtic language (moreover, this is the first release of linguistic data for Apertium that does not include a Romance language); it is also the first 'community' developed package: i.e., developed in a 'bazaar'-like fashion using volunteer contributions - hopefully, the
first of many!

And the Welsh translation:

Hwn yw'r pecyn cyntaf sy'n targedu iaith Celtaidd (hefyd hwn yw'r rhyddhad cyntaf o ddata ieithyddol sydd ddim yn cynnwys iaith rhamant); mae e hefyd y pecyn cyntaf wedi ei ddatblygu gan 'cymuned': h.y. wedi ei ddatblygu mewn ffordd 'basar'-aidd gan ddefnyddio cyfraniadau gan gwirfoddolwyr - gan obeithio mai hwn yw'r cyntaf o nifer!

Kevin, who contributed almost all of the data in the bilingual dictionary and the Welsh morphological analyser, has a dedicated Welsh to English site set up: www.kymraeg.org.uk

Francis, who did most of the implementation work on the rules, pointed out this article - specifically, this section:

"There aren't any mutations in Catalan or Spanish, so there needed to add software that dealt with that. A man from Ireland had already developed simmilar software for Irish and Scottish Gaelic

I think the 'man from Ireland' is me, but that's not quite true: I did have another idea, but the idea that we ended up using was Francis's, and that's what we're now using in the ga-gd translator.

While I'm quoting things, Kevin's appeal for assistance is worth repeating:

You can also help by asking your Assembly Member to help ensure that resources that receive public funds are made available under a free licence. The sad thing is that the development of these translators could proceed much more quickly if we didn't have to create freely-distributable word lists for them from scratch. Public money has gone into compiling Welsh dictionaries and lists of terms, and yet apertium-cy and similar projects cannot use these because they are not available under terms which allow them to be freely redistributed. Every minute we spend adding words is a minute we can't spend writing software. It would be a tremendous help to our work if the Welsh Language Board could look again at this issue - after all, if developers in Spain and the USA are prepared to spend time on Welsh, we should surely give them all the help we can!

It is a great pity that more language boards aren't interested in even providing assistance to open source projects like ours, but unfortunately it's the rule, rather than the exception.

(Oh, and let me wave a flag and say 'developers in Spain, the USA and Ireland' :)

Mówimy po irlandzku!

2008-07-28T01:54:00.002+01:00

Wiem, że Polacy dziwią się, że my Irlandczycy nie mówimy po naszemu... tylko w szkole, kiedy uczymy się naszej ojczyzny jako język obcy. A wiele Polaków w Irlandii chcą znać trochę irlandzki. Mimo że piszę brzydko po polsku, spróbuję czegoś nauczyć po irlandzku; a teraz jestem za zmęczony, i tylko daję parę słów:

fear (far): mężczyzna
bean (ban): kobieta
leabhar (liałer): książka
bóthar (boher): ulica
úll (ul): jabłko
ainm (anm): imię
cailín (kalin): dziewczyna
buachail (bułakl): chłopiec
ceol (kiol): muzyka
focal (fakl): słowo

Currently...

2008-07-28T01:30:00.000+01:00

Currently, I'm ripping apart the adjective paradigms in Apertium's Irish analyser. We got a good head start on this, thanks to Kevin Scannell's donation of material from his ga2gd project, but we've got a basic problem with the paradigms. The basic thing is, we need to over generate where Kevin doesn't, and vice-versa.

It's a long and not-particularly fascinating story about what we need to know at the time the words come to the reordering rules, but it's a hell of a lot better than starting from scratch.