Upon upon a time when some changes were made to the server hosting this blog, I lost the ability to display certain characters, including Chinese characters.
Hundreds of entries had words that were turned into gobbledygook.
I can’t tell you how irritating that was.
Do any of you who possess a Chinese language know how to change the code left behind back into Chinese characters?
Help!
Coat of Arms by D Burkart
on ROME 24/4– Day 36 (-5): Sacristy secrets, music from nuns, and Fr. Z’s Kitchen (real “all’amatriciana”): “Dinner looks exceptional. Newfoundland has great history, you should go. Will be listening to the Sisters.”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “I’ve heard it said there are as many galaxies as grains of sand. Here is interesting facts about our Milky…”
on ROME 24/4– Day 36 (-5): Sacristy secrets, music from nuns, and Fr. Z’s Kitchen (real “all’amatriciana”): “Thanks for the all’amatriciana”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “Thanks Father Z, cosmology lesson as only you can impart.”
on ROME 24/4– Day 35 (-6): adventures in symmetry: “Hmmm. Those kind of tassels have always reminded me of fancy curtain tiebacks so I have avoided making them but…”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “Fr.Z sez: …The Voyagers are destined—perhaps eternally—to wander the Milky Way and then become characters in Star Trek…. V’ger!! https://youtu.be/gxAaVqdz_Vk?t=96”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “Being from Florida, I remember when both Voyagers launched – you could see them from our house. At that time,…”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “Too bad the Curia has drifted too far from its power source…”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “Voyager 2 launched on 20 August 1977, from Cape Canaveral. Voyager 1 launched from Cape Canaveral on 5 September 1977.…” on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “I guess Voyager I gives new meaning to the poem High Flight. Oh! I have slipped the surly bonds of…”
on So very cool that it needs its own post! VOYAGER 1 is TALKING AGAIN!: “Now this is very interesting. Not just looking up at some small dots of light overhead!”
on ROME 24/4– Day 34 (-7): The burial place of… PLATO?!?: “The final resting place of Plato puts me in mind of a memorable excerpt I once read about Aristotle in…”
on ROME 24/4– Day 34 (-7): The burial place of… PLATO?!?: “Someone pointed out the St. George’s Day Google Doodle with its background account to me – with one of the…”
on Three items of interest: a conversion, a connection, and a con job: ““George Farmer, aka Candace Owen’s husband. He is a revert” If I’m reading the interview correctly, first a convert, then…”
on Three items of interest: a conversion, a connection, and a con job: “Not to detract from anything Imrahill said, but well, the papally-attested infallible ordinary magisterium about a catholic truth, not the…”
Loading ...
A Daily Prayer for Priests
Nota bene: I do not answer these numbers or this Skype address. You won't get me "live". I check for messages regularly.
WDTPRS
020 8133 4535
651-447-6265
PLEASE subscribe via PayPal if it is useful. Zelle and Wise are better, but PayPal is convenient.
A monthly subscription donation means I have steady income I can plan on. I put you my list of benefactors for whom I pray and for whom I often say Holy Mass.
In view of the rapidly changing challenges I now face, I would like to add more $10/month subscribers. Will you please help?
For a one time donation...
Loading ...If you travel internationally, this is a super useful gizmo for your mobile internet data. I use one. If you get one through my link, I get data rewards.
Loading ...
They find you a pro-life agent in your area who commits to giving a portion of the fee to a pro-life group!
Help Monks in Wyoming (coffee – UPDATED LINK) and Norcia (beer) and the wonderful "Soap Sisters" of Summit, NJ!
Please follow me on Twitter!
Use my code for a discount
Visits tracked by Statcounter
since Sat., 25 Nov. 2006:
Dui bu qi. No I don’t. :-(
I imagine it’s an encoding issue. The encoding that pages on the blog are being served with is currently UTF-8; I don’t know what it might have been before.
In firefox, at least, you should be able to go to one of the damaged pages, go to View > Character Encoding from the menu, and try different encodings until you find the right one (though you will need to know what one or two of the characters are supposed to be to make sure you’ve got the right one).
At that point you should be able to copy the text out and paste it into an editing form in a separate tab/window that is still set to UTF-8 in order to convert the text.
Hm, unfortunately it looks like the text of entries may actually be corrupted in the database and not simply transmitted with the wrong encoding, though it wouldn’t hurt to get a second opinion.
If the pages in question still do not display correctly with either DTF-8 or GB3212 then the flat files themselves have been corrupted and you are lost.
Let me try to type some Chinese here….
Father Z
Z ??
O! It didn’t work! I typed “Father” in Chinese but “??” appeared instead.
Fr. Ho: Yes… I know. I will see what can be done at the level of the server.
Here’s the problem: the original UTF-8 Chinese text was misinterpreted as though it were in Windows encoding (CP1256) and then “converted” into UTF-8. This happened *twice*. To recover the Chinese text, apply the reverse transformation. I only know how to do this under Unix using uconv (part of the ICU package from IBM):
cat garbled-file | uconv -f utf8 -t cp1252 | uconv -f utf8 -t cp1252
David C: I am not sure I entirely understand that, but I can pass that along.
Father, it’s as if someone found a scrap of paper that said “sex dies”, mistakenly thought it was in English, and translated it into Latin “sexus moritur”.
This must by why your Korean text has also been mangled for some time.
Hooray for Windows!
Apologies, I wrote “CP1256” above when I should have written “CP1252.” Father, if the recipe above doesn’t work for your admin, I’d be happy to ungarble the files if they can be sent to me conveniently.
I can confirm that David’s recipe works (with the substitution of cp1252), even for sequences of Chinese text copy-and-pasted directly from the blog. However, it doesn’t always appear to work when there are runs of English text interspersed — it looks like the runs of Chinese text may need to be converted individually, and a handful of them appear to have been irreversably mangled.
Here is the recovered text (using David’s method) from Father’s post of a couple years ago:
(We’ll see if this actually shows up correctly in the comments section — if it does, Father Ho’s problem may be that his browser is using an encoding other than UTF-8 for posting.)
Nope. Evidently some things are still(?) misconfigured on the server side with respect to character encodings. Oh well.
Just out of curiosity: how often exactly do you post in Chinese, Father?
Catholic terms in Chinese: Catholic – ???; Father – ??; Nun – ??; Sacrament of Confession – ????.
Chinese characters do not work. Now I will try Korean. Catholic words in Korean: Catholic – ???; Father – ??; Sister – ??; Sacrament of Confession – ????
Hm, let’s try accented european characters: áéíóúýñ àèìòù?ñ
It looks like, for posted comments, any character not covered in cp1252 is getting smashed into a ?. If “smart quotes” work, that pretty much confirms it. The really maddening thing is that all these characters show up fine in previews.
Yep. New comments are being shoehorned into cp1252 encoding, losing most non-European characters (and some European ones).
MenTaLguY: but the recovered characters displayed correctly before going into the comments system, yes?
MenTaLguY: sorry, I missed one of your comments which answers my question. Can you point me to some posts that don’t convert correctly? Thanks.
Dear Father,
I am Chinese and I hope I can help you.
But where are the passages? Perhaps you can send me them by email. And I will try to get them back into Chinese Characters, if they are not corrupted, they can be converted again.
If you can tell me the English meaning of the passages, I can translate them back into Chinese and type the text and send you by email.
And I will be happy to be able to help you in these kind of issues later too.
Just write me an email.
I am sending you an email to tell you my email address.
Greetings.
Just as a test:
???? ???, ??? ??? ?? ???????. (Russian)
????? ????, ? ?? ???? ????????. (Greek)
O?e naš, koji si na nebesima. (Serbian, Latin characters)
Ojcze nasz, który? jest w niebe. (Polish)
Pater noster, qui es in caelis. (Latin)
All of them appear correctly in the preview. Now for the post …
David: yes, the recovered comments show up correctly via preview. The post I linked to from which I recovered text actually doesn’t convert completely (try the title), at least partly due to the presence of some 8-bit characters (i.e. smart quotes). Also try some of the comments on that same post. In some cases it looks like the transformation is not reversible.
(Because it appears that the transformation is lossy, I’d definitely recommend going over any recovered text with a chinese speaker to make sure it hasn’t been mangled to say something weird…)
Roland: to save yourself some trouble, any it appears that any character not on this chart won’t work.
In any case (this is a _guess_; I’m not privy to the technical details of the site), it looks like one of the things which will need to be done is to change the database encoding from Microsoft cp1252, which can’t represent most international characters, to UTF-8. This will involve converting all of the data there from cp1252 to UTF-8. At that point it’s just a matter of making sure that WordPress is storing things with the correct encoding for the database (which it should do automatically, I think, but it wouldn’t hurt to check). That should take care of addressing problems for new posts and comments.
As far as the older posts, it would probably be a good idea to save the unconverted database, as converting the database to support international characters might mangle the existing mangled data further.
I suspect the reason that things used to work is that the software was simply passing things through without paying attention to character encodings, so things went in and out unconverted and usually happened to work. Then at some point updates to the infrastructure meant that parts of the system started paying proper attention to character encodings, which unfortunately meant that (since different parts of the system disagreed about character encodings, and data had already been stored in mismatched encodings) things started getting mangled.
This is useful for me, folks. Thanks.
Father, I’ve sent you an email with a hopefully robust solution, a script that attempts to circumvent the problems Mentalguy has pointed out. I absolutely agree that you should save a backup copy!