When I originally set up the database back last September, I used the default latin collation for SQL Server, Latin1_General_CI_AI. I knew at the time I would need to consider a different collation for the dhivehi content but then forgot about this issue until the other day when I tried a search against fields holding dhivehi as unicode. My first search returned all records in the table, irrespective of the filter string. For instance searching the dhivehi firstname field for the dhivehi Bart, ބާރޓް
in a table containing records for {Bart, Homer, Lisa, Maggie, Marge} as { ބާރޓް, ހޯމަރ , މެގީ , މާރޖް , ލިސާ } using
SELECT * FROM tblPeople WHERE D_FirstName LIKE N'%ބާރޓް%'
records were returned for all the Simpson family. I was sort of expecting something like this might happen, though if I'd had to put money on it, I would have guessed I'd get no matching records.
By changing the collation on this field to one of the dhivehi set supported by SQL Server 2008, only the record for bart was returned.
The choice of collation took some consideration. The set of divehi collations in SQL Server 2008 numbers over 30 variations:
We can reduce this list by half by using the 100s in preference to the 90s. According to sql 2008 books online:
SQL Server 2008 has introduced new collations that are in full alignment with collations that Windows Server 2008 provides. These 80 new collations are denoted by *_100 version references. They provide users with the most up-to-date and linguistically accurate cultural sorting conventions.
We don't want to use the BIN collations.
Dhivehi doesn't have cases. A blog at http://www.jawish.org/blog/plugin/tag/how+to, referring to MySQL suggests a collation of utf8_unicode_ci, which isn't divehi specific but does indicate that case insensitive is good enough. Also, a blog at http://blogs.msdn.com/michkap/archive/2005/05/11/416293.aspx suggests that case sensitive can lead to problems. So case insensitive seems a good option.
The filis are stored as separate characters rather than accented versions of consonants. The choice between accents sensitive and insensitive probably makes no difference but accent insensitive seems a safe bet.
Kanatype indicates whether the data in the dataset is kanatype sensitive, which distinguishes between the two types of Japanese kana characters: Hiragana and Katakana. This is of no relevance to Dhivehi, so insensitive sounds a good option.
Width isn't an issue but insensitive seems the safest option.
This brings the choice down to
Divehi_100_CI_AI Divehi-100, case-insensitive, accent-insensitive, kanatype-insensitive, width-insensitive
To test this I created the table
CREATE TABLE [dbo].[tblPeople](
[PersonId] [int] IDENTITY(1,1) NOT NULL,
[E_Firstname] [nvarchar](40) NULL,
[E_Lastname] [nvarchar](40) NULL,
[D_FirstName] [nvarchar](40) COLLATE Divehi_100_CI_AINULL ,
[D_LastName] [nvarchar](40) COLLATE Divehi_100_CI_AI NULL)
GO
populated d_firestname with the simpsons o=in dhivehi and repeated the earlier query
SELECT * FROM tblPeople WHERE D_FirstName LIKE N'%ބާރޓް%'
which now just returns Bart's record.
Note, it is important to prefix the search string with N to indicate a unicode string.
Wednesday, 24 March 2010
A ship the size of an island
The P &O liner Aurora arrived off Male' last night. At almost 900' long, it's bigger than the next door island of Funadhoo.
The P & O Aurora moored opposite the President's Jetty on Bodu Thakurufaanu Magu, Male'
It's not a pidgin
There's a road close to the hotel called Filigas Magu (Filigas Street). At the end of it is the only petrol (gas) station on Male'. Initially, I wondered if it was pidgin but it turns out to be just an amusing coincidence. Filigas is a kind of tree. Fili is also the name given to the 'squiggles' above and below the main characters in dhivehi, which represent the vowel sounds.
Not pidgin
Pigeon
Wednesday, 2 December 2009
Meanwhile back on the database...
Latest news from the project is that I added another 800 islands to the database yesterday and lost 200 health centres in the process.
The project plan for phase 2 has been redesigned and considerably slimmed down with only two more trips planned and an official launch on May 10'th which is Maldives Childrens' Day.
More meetings with social workers and Dept of National Planning to discuss reporting requirements.
When not in meetings I've spent the last two days finishing the end of project report for phase 1, rather than finishing off the project. I do believe I may have fallen into the unicef way of life.
Coast Guard boat sailing between Male' and Dhoonidhoo
For no other reason than it's one of four streets on Male' I know the name of
The project plan for phase 2 has been redesigned and considerably slimmed down with only two more trips planned and an official launch on May 10'th which is Maldives Childrens' Day.
More meetings with social workers and Dept of National Planning to discuss reporting requirements.
When not in meetings I've spent the last two days finishing the end of project report for phase 1, rather than finishing off the project. I do believe I may have fallen into the unicef way of life.
Coast Guard boat sailing between Male' and Dhoonidhoo
On another note, the restuarant staff at the hotel have exhibited another bout of haplessness at lunchtime when the beef fried noodles I ordered arrived as beef fried rice, apparantly destined for another diner. I wonder if he got my chilli sauce as well since what I got was plain old tomato ketchup!
For no other reason than it's one of four streets on Male' I know the name of
Circular reasoning
Currently, mosques are the only places of worship in Maldives. I read in a newspaper here from last week that there was a proposal to permit places of worship for other faiths to be built in the country.
However one muslim leader noted that as Islam was the only religion in Maldives it wasn't necessary to allow this, which rather neatly brings closure to the subject.
A mosque on Male'
However one muslim leader noted that as Islam was the only religion in Maldives it wasn't necessary to allow this, which rather neatly brings closure to the subject.
A mosque on Male'
Abuse of a different kind
This time, it's abuse of the English language.
I was sitting in the unicef office yesterday and Latheef says to me, "Tony, can you explain what this means"? From the look on his face I could see he felt some disquiet in what he was reading. It was World Aids Day yesterday and he had found the following on an HIV/AIDS website. As soon as I read the second line I could see the cause of his concern.
While male to male sex is often discouraged from a moral or religious point of view, from the public health perspective it is desirable to engage men having sex with men in an attempt to help them reduce HIV transmission; not only between men...
It was the use of the term engage which has been adopted in recent years by many people as a replacement for a much simpler word, talk. The closest definitions of the word engaged for the meaning intended in the about sentance are
To attract and hold the attention of; engross: a hobby that engaged her for hours at a time.
To draw into; involve: engage a shy person in conversation.
But how much clearer it would have been to just write 'talk to'. Somehow by engaging with people it conveys more importance in the engager and what they are saying. I think it is probably meant to suggest that a two way discussion will take place but there is no guarantee that the engagee will actually be engaged.
What of course it does not mean in the context of the above is encourage men to become engaged to other men!
Unicef engaging the wider community
I was sitting in the unicef office yesterday and Latheef says to me, "Tony, can you explain what this means"? From the look on his face I could see he felt some disquiet in what he was reading. It was World Aids Day yesterday and he had found the following on an HIV/AIDS website. As soon as I read the second line I could see the cause of his concern.
While male to male sex is often discouraged from a moral or religious point of view, from the public health perspective it is desirable to engage men having sex with men in an attempt to help them reduce HIV transmission; not only between men...
It was the use of the term engage which has been adopted in recent years by many people as a replacement for a much simpler word, talk. The closest definitions of the word engaged for the meaning intended in the about sentance are
To attract and hold the attention of; engross: a hobby that engaged her for hours at a time.
To draw into; involve: engage a shy person in conversation.
But how much clearer it would have been to just write 'talk to'. Somehow by engaging with people it conveys more importance in the engager and what they are saying. I think it is probably meant to suggest that a two way discussion will take place but there is no guarantee that the engagee will actually be engaged.
What of course it does not mean in the context of the above is encourage men to become engaged to other men!
Unicef engaging the wider community
Child Sexual Abuse Act
There was a new child sexual abuse act passed recently in Maldives.
The act contained many good things to help protect vulnerable children, in particular young girls.
However one article was included which stated that if a girl is married under sharia law all the other articles in the act no longer stood because she is no longer considered a child but an adult.
I think under Sharia law in Maldives the age of consent is12. It might have been increased more recently by legislation in the parliament, but the general feeling here is that the new act may do more harm than good.
The act contained many good things to help protect vulnerable children, in particular young girls.
However one article was included which stated that if a girl is married under sharia law all the other articles in the act no longer stood because she is no longer considered a child but an adult.
I think under Sharia law in Maldives the age of consent is12. It might have been increased more recently by legislation in the parliament, but the general feeling here is that the new act may do more harm than good.
So this is one of the reasons we are here and this place is where we doing it from.
Whilst checking on the age for marriage on the web, I stumbled upon this item in another blog
Subscribe to:
Posts (Atom)
