from strings.po to strings.xml?

[byron]

So as of the end of eden (I think) xbmc mainline started migrating to strings.po files instead of strings.xml...that's neither here nor there for us until you start trying to fork newer addons/skins that have their languages set up into strings.po format. I'm trying to figure out the best way to go about this without having to edit these files one single line at a time.

Here's mainline's main strings.po

Here is our strings.xml

I know Dan Dar3 had some suggestions for going about this on a different thread, if he or anyone else would care to assist me in this we need a couple of things for a script or something along those lines (to do batch editing ideally).

We need to change the header (not sure if all of their po files are identical here) from:

Code: Select all

# XBMC Media Center language file
msgid ""
msgstr ""
"Project-Id-Version: XBMC Main\n"
"Report-Msgid-Bugs-To: http://trac.xbmc.org/\n"
"POT-Creation-Date: YEAR-MO-DA HO:MI+ZONE\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: XBMC Translation Team\n"
"Language-Team: English (http://www.transifex.com/projects/p/xbmc-main/language/en/)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: en\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"

to:

Code: Select all

<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>
<!--$Revision$-->
<strings>

and then each item (respectively) from:

Code: Select all

msgctxt "#0"
msgid "Programs"
msgstr ""

to:

Code: Select all

<string id="0">Programs</string>

I'm not sure how to go about doing this quickly (thought we might put our heads together here), if needs be I'll correct each line for each language (while cursing) for what I'm working on, but if the thought of more current addons and skins is alluring (which I think it is for most), then this is definitely something that needs to be addressed in some manner at least...whether it be one line at a time, a batch editor/format, or possibly even a .module like xbmcaddon.py to use trickery on xbmc4xbox

[Dan Dar3]

Hey Byron,

Here's a Python script you can use with the batch on the other thread to convert strings.po to strings.xml - let me know how it works for you.

Code: Select all

'''
Filename : strings_po2xml.py 
Purpose  : Convert strings.po to strings.xml (XBMC4Xbox)
Author   : Dan Dar3 <dan.dar33@gmail.com>
Date     : 20 Jan 2014
'''

# Imports
import sys, os, re
from xml.sax.saxutils import escape
import codecs

#
# Convert...
#
def convert(path):
    # Check input file exists...
    if os.path.isfile(path):
        strings_po = path
        path       = os.path.dirname(path)
    else :
        strings_po = os.path.join(path, 'strings.po')
    if not os.path.exists(strings_po):
        print 'File not found "%s"' % strings_po
        return False
    
    # Read input (strings.po)...
    f = codecs.open(strings_po, mode='r', encoding='UTF-8')
    contents = f.read()
    f.close()
    
    # Write output (strings.xml)
    strings_xml = os.path.join(path, 'strings.xml') 
    f = codecs.open(strings_xml, mode='w+', encoding='UTF-8')
    
    # Message...
    print 'Converting "%s" to "%s"...' % (strings_po, strings_xml)  
    
    # Header...
    f.write('<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n')
    f.write('<!--$Revision$-->\n')
    f.write('<strings>\n')
    
    # Entries...
    pattern = re.compile('msgctxt "#(\d+)"\s+msgid "(.*)?"\s+msgstr "(.*)?"')
    for match in pattern.finditer(contents) :
        msgctxt =        match.group(1)
        msgid   = escape(match.group(2))
        msgstr  = escape(match.group(3))
        
        if (msgstr) :
            f.write('  <string id="%s">%s</string>\n' % ( msgctxt, msgstr ))
        else :
            f.write('  <string id="%s">%s</string>\n' % ( msgctxt, msgid ))
    
    # Footer...
    f.write('</strings>')
    f.close()
    
    # Return...
    return True

#
# Main
#
if len(sys.argv) == 1 :
    success = convert(os.getcwd())
    if not success :
        print ''
        print 'Convert strings.po to strings.xml (XBMC4Xbox)\n'
        print 'Syntax:'
        print '   python %s [path_to_strings.po]' % ( os.path.basename(sys.argv[0]) )
else:
    for argv in sys.argv[1:] :
        convert(argv)

[byron]

Excellent, thank you sir! So just to be clear, is this a file that I save in .py (foo.py?) or .bat? And then from the other thread....

Ok, very good, thanks for that. Now, up at the top, import urllib (either on it's own line or just add in the comma list), and then change line 91 to use urllib.quote() instead.

To change strings.po into strings.xml on Windows recursively, paste this code into a file next to default.py, I don't know call it fix or change.bat (PS: if you intend to run the command from a cmd.exe yourself, you need to change %% into a single %). If you're wondering where all that comes from, you can read all about it by running "for /?" in a command line yourself, but basically it recurses through and renames the files; double quotes are used cause some directories have spaces in their names, e.g. Chinese (Simple).

Code: Select all

fix.bat
~~~~
for /r %%i in (strings.po) do rename "%%i" "%%~ni.xml"

...are these things that can be placed in a skin/plugin folder and work in real time as a new language is selected, or is it work that I do on my pc to change all files to xml and then upload? I can't wait to see if works, but not sure exactly what to do...thanks for taking some time to help. As of right now I've just taken about 10 languages or so for transparency! and translated the english strings via m$ word...it sucks a lot doing that because most characters lose their upper-case which turns into a nightmare very quickly.

[Dan Dar3]

I should have clarified - yes you could run this on Xbox being Python and all, but the way I wrote it it can only do a file at the time so it's probably best to do it on your PC and let the OS do the looping and calling for each file.

Yes, that's the thread I was referring to. So to integrate, lets save the Python script as say "strings_po2xml.py" next to the "fix.bat" and change it to do:

Code: Select all

fix.bat
~~~~
for /r %%i in (strings.po) do python strings_po2xml.py "%%i"

You could remove these two when you're finished so you don't zip them up or upload them to your Xbox.

[byron]

Alright, so I can't figure out how you're suggesting I go about doing this. I've created both file (.py and .bat) and placed both into the main addon folder. Sorry for the crude diagram:

addon
------------->

• - (foo.files)
  - default.py
  - fix.bat
  - strings_po2xml.py
  
  -[resources]

• • |
    |
  • -[languages]
  • • |
      |
    • (language folders)

Are you proposing to run the fix.bat this way? I thought this was what you were suggesting, so I've tried running the batch by double clicking it...also tried dropping the p2x.py into the bat (tried placing both of them in the language folder, one in language folder and one in addon folder, etc)...the command shell just hangs on the directory that I've placed it in. Hate to be such a noob, but I really don't know where to go with this...

[Dan Dar3]

Ok, that' seems about what I was thinking...

I would try opening up a command line (cmd.exe), navigate to the directory (cd /d path_to_plugin) and then call the plugin directly and see if it can conver one file correctly, e.g.

Code: Select all

python strings_po2xml.py resources\languages\English\strings.po

If still doesn't work, please send me an email (dan.dar33 at gmail.com) and lets arrange for a remote session to have a look at it.

[byron]

Okay good news, that got it working...but now another problem, the script only translates the msgid (Dutch example):

Code: Select all

msgctxt "#32118"
msgid "Manual Search..."
msgstr "Manuel zoeken..."

into:

Code: Select all

<string id="32118">Manual Search...</string>

but should be:

Code: Select all

<string id="32118">Manuel zoeken...</string>

So is there anyway I could sweet talk you into adapting this a bit further to [exclude the msgid string and include the msgstr] for non english/english (us) languages? Thanks again btw, it took about one second for that script to do what it would have taken me 30 minutes

[Dan Dar3]

Hey Byron, just updated the code in the first post, hope it helps

[BuZz]

Just a heads up, as of r32687, strings.po files are supported (tested for core files, but should be ok for plugins too - I need to do more testing though).

My hope now is to get the core translations hooked up to transifex, so they can be more easily managed, and allow people to contribute etc.

this is all based on work by xbmc that I merged/backported etc.

[Dan Dar3]

Very nice, good work Buzz!