XBMC4Xbox

Posted: **Mon Dec 16, 2013 1:50 pm**

After a lot of time I tried again to update (successfully run from early days, at least from around 2009 to 31869). XBMC SVN Installer (now XBMC4Xbox Installer) fail with no apparently reason

Debugging a lot, using different source and destination version, still stop during phase 2: "(2 of 6) Copying user data..."
Manually updated from 31869 to 32590 copying profiles.xml, FileZilla Server.xml, UserData and skipping skin, scripts and plugins, cleaned plugin_data as seen on another thread, reinstalled only XBMC4Xbox Installer using provided Addons4Xbox Installer. Retried upgrade (from 32590 to 32598), same error:

Code: Select all

File "Q:\plugins\programs\XBMC4Xbox Installer\default.py", line 59, in <module>
  plugin.Main()
File "Q:\plugins\programs\XBMC4Xbox Installer\resources\lib\xbmcsvn_build_install.py", line 73, in __init__
  self.install_build( self.zip, self.install_path, build_dir )
File "Q:\plugins\programs\XBMC4Xbox Installer\resources\lib\xbmcsvn_build_install.py", line 314, in install_build
  shutil.copytree( old_userdata_path, new_userdata_path )
File "Q:\system\python\python27.zlib\shutil.py", line 208, in copytree
  raise Error, errors

Thinking about filesystem problems, doing a whole xbox F:\Apps ftp backup without any issue.
Thinking about disk full (more than 8GB free, removed old installations, now more than 11GB free)
Thinking about too many files in UserData (more than 6000), removing ALL video thumbnails [0-9a-f] and recreating empty dirs (down from 6552 files and 175 dirs to 468 files and 175 dirs)

Only one error on log, a very long line reformatted here for your convenience, all about "Permission denied", but ftp or internal file manager works fine:

Code: Select all

# grep ERROR xbmc.old.log | head -1 | tr '\(\)' '\n' | sort | sed -e 's/, /#   /g' | tr '#' '\n'
'Q:\\UserData\\'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData'"
'Q:\\UserData\\Database'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Database'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Database'"
'Q:\\UserData\\Database\\CDDB'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Database\\CDDB'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Database\\\\CDDB'"
[...]
'Q:\\UserData\\Thumbnails\\Music'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Thumbnails\\Music'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Thumbnails\\\\Music'"
'Q:\\UserData\\Thumbnails\\Music\\0'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Thumbnails\\Music\\0'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Thumbnails\\\\Music\\\\0'"
'Q:\\UserData\\Thumbnails\\Music\\1'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Thumbnails\\Music\\1'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Thumbnails\\\\Music\\\\1'"
[...]
'Q:\\UserData\\Thumbnails\\Video'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Thumbnails\\Video'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Thumbnails\\\\Video'"
'Q:\\UserData\\Thumbnails\\Video\\0'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Thumbnails\\Video\\0'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Thumbnails\\\\Video\\\\0'"
'Q:\\UserData\\Thumbnails\\Video\\1'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\Thumbnails\\Video\\1'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\Thumbnails\\\\Video\\\\1'"
[...]
'Q:\\UserData\\script_data'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\script_data'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\script_data'"
'Q:\\UserData\\visualisations'
        'F:\\Apps\\XBMC4XBOX-32598\\UserData\\visualisations'
        "[Errno 13] Permission denied: 'F:\\\\Apps\\\\XBMC4XBOX-32598\\\\UserData\\\\visualisations'"

Maybe a problem with empty dirs (or dirs containing only dirs)? How can I debug any further?
Using a loop for subdirs (like on skin) for UserData can be useful?
Thanks in advance.

PS: to speed up debugging would be nice a third option other than download/install: transfer user config, to skip time consuming zip expansion

--
Riccardo.

Posted: **Mon Dec 16, 2013 3:16 pm**

That would be mine.

python27.zlib\shutil.py doesn't seem to have changed in quite a while, so it's not the actual Python code behind it.
http://redmine.exotica.org.uk/projects/ ... tem/python

Also XBMC4Xbox Installer (or XBMC SVN Installer, as was used to be known) hasn't changed in a while, I just renamed it so the behavior should be the same.

So the remaining variables are the new XBMC4Xbox you're running it from (changes in internal behaviour?) or the actual storage (F:)...

What was the last revision from which you were able to upgrade correctly to the which one? Because if it's the first case (changes in XBMC4Xbox), to find out where it broke you could follow it by using the half-splitting method - what I mean is if upgrading from revision 0 to revision 10, and failing from 10 to 11, you might want to try to upgrade 0 to 5 and then 5 to 11 - then repeat the process and split the remaining interval until you find in which revision it brakes (hope it makes sense).

If the second scenario (I find odd the excessive escaping of \ characters in the error message, but that may be just a formatting issue), I would try to configure the plugin (go in it's settings) and change to another partition, lets say E:\Apps - and tried to upgrade again. If it works this time, then you have a problem writing to F:.

Let me know how it goes - btw, please upload a full debug log if you can (pastebin.com or xbmclogs.com).

Posted: **Tue Dec 17, 2013 1:03 am**

Dan Dar3 wrote:What was the last revision from which you were able to upgrade correctly to the which one?

Good question

I don't remember, was a lot of months ago, same problems from "something" to 31869 maybe. Also some problem during internal database schema upgrade from 36 to 37 (about one year and a half ago, july 2012 between 31117 and 31199) solved here: http://www.xbmc4xbox.org.uk/forum/viewt ... =114#p1278
I can try a fresh install of a 315xx-31600 (I saved all version I installed, SVN20121117_31600.zip is the last SVNxxx.zip version that I have, then they changed name to XBMC4XBOX-317xx.zip) and some upgrade using my local zip collection tomorrow in the afternoon. Stay tuned...
All "\" showed in my previous message are present into the log, see here for the complete one: http://pastebin.com/pDmv81nV (step was: reboot, install without download, upgrade=yes, waiting for fail, reboot, save log using ftp).

I also removed about 10 zip and 10 old versions, access to F:\ seem fine (no error during zip extract, neither using internal file explorer nor with the ftp client, only during copy with plugin).
Do you think that hard disk is about to leave for the Valhalla of HDD?

Can I do some HDD test using some program for XBOX without removing the disk?

EDIT: Update: I changed installation path from F:\Apps to E:\Apps and retried from 32590 to 32598 but update fails the same (also installed Notepad for a quick look to log file). Noticed all disk doubled when setting installation path ( C: C: DVD DVD E: E: F: F: G: G: Q: Q: ) is this normal? Anyway I tried both lines with F: and with E: (just in case).

--
Riccardo.

Posted: **Tue Dec 17, 2013 12:53 pm**

Thanks Ricardo, the log is perfect - I got the from revision, the to revision and the bunch of errors, so I can try the same thing myself and confirm. Will get back to you.

Well, the plugin can't do anything about upgrading the database structures internally, that's the job of XBMC4Xbox to do it on the fly if it can, otherwise it's recommended to do a clean install or an upgrade, but removing the database files after and re-scanning your content (having generated .nfo files with your content should speed it up if that's a concern).

The double entries in the file dialog, yes I notice them too - let me see if I can dig something in the code and we'll report it then.
Edit: Created issue 271 (http://redmine.exotica.org.uk/issues/271)

PS: Your suggestion re just an upgrade function, didn't overlook that - I think it would be nice but limited to very few people. It will probably be very confusing for most, with questions like "do I need to be in the previous version or in the new version to run that?". Hopefully we'll find a fix for this problem and you won't feel like you need it

Posted: **Tue Dec 17, 2013 2:25 pm**

Your log was showing an upgrade from r32590 to r32598 - I was able to replicate the issue doing the same.
So it's probably to do with the code at r32590, will come back with more later.

Posted: **Tue Dec 17, 2013 2:59 pm**

Dan Dar3 wrote:Well, the plugin can't do anything about upgrading the database structures internally, that's the job of XBMC4Xbox to do it on the fly if it can, otherwise it's recommended to do a clean install or an upgrade, but removing the database files after and re-scanning your content (having generated .nfo files with your content should speed it up if that's a concern).

I regularly export database to separated .nfo file (as a backup and because some my script rely on that for check for missing episodes).
Database upgrade was solved with a manual intervention on july 2012 because field order changed (thanks to Buzz for pointing me to the right source file).

Dan Dar3 wrote:The double entries in the file dialog, yes I notice them too - let me see if I can dig something in the code and we'll report it then.
Edit: Created issue 271 (http://redmine.exotica.org.uk/issues/271)

Perfect. About localized... I have all XBMC menu in English, only movie and tv serier scraping are in Italian.

Dan Dar3 wrote:PS: Your suggestion re just an upgrade function, didn't overlook that - I think it would be nice but limited to very few people. It will probably be very confusing for most, with questions like "do I need to be in the previous version or in the new version to run that?". Hopefully we'll find a fix for this problem and you won't feel like you need it

Maybe some hidden code enabled with a semaphore (an empty file for example) that skip remove/extract and only retry upgrade, but probably I can do it myself.

Posted: **Tue Dec 17, 2013 3:03 pm**

Dan Dar3 wrote:Your log was showing an upgrade from r32590 to r32598 - I was able to replicate the issue doing the same.
So it's probably to do with the code at r32590, will come back with more later.

Please note that r32590 was "manually upgraded" from r31869 because of the same problem, so the issue is between r31??? and r31869 I think...

Posted: **Tue Dec 17, 2013 3:19 pm**

No yeah, you're right, r32590 (trunk) has the problem, but it's not necessarily where it happened. I've also tried from r32598 to r32642 and it's still there.
I'm chasing it down the nightly builds as well, will keep you posted.

Posted: **Tue Dec 17, 2013 3:27 pm**

Actually, now looking again at Python shutil.copytree() it does say that destination must not exist and the plugin seems to remove the new UserData first using shutil.rmtree(), so copytree should not fail. I think there might be something wrong with the Python 2.7 implementation - I wrote a test script below (create a file say Q:\scripts\Python shutil test.py) and run from Programs > Scripts.

Code: Select all

import os
import shutil

SRC_PATH = "Q:\UserData"
DST_PATH = "F:\Temp\UserData"

if os.path.exists( DST_PATH ):
	shutil.rmtree   ( DST_PATH )
shutil.copytree ( SRC_PATH, DST_PATH )

Running from latest trunk builds (nightlies) test fails (although it appears to copy the stuff though ?!), but it works ok from 3.3.3.

So we might be onto something here, now we got a repeatable test we can log a ticket.

Posted: **Sat Dec 21, 2013 5:28 pm**

Ticket #273 created.

Posted: **Wed Jan 01, 2014 11:18 pm**

@vic3mila
Buzz just fixed the issue a few days ago and that should be included in 3.5 beta 1 (or any other build after r32644).
http://www.xbmc4xbox.org.uk/2013/12/xbm ... available/

XBMC4Xbox

XBMC4Xbox Installer - Error updating to any newer version

XBMC4Xbox Installer - Error updating to any newer version

Re: Unable to update to any newer version

Re: Unable to update to any newer version

Re: Unable to update to any newer version

Re: XBMC4Xbox Installer - Error updating to any newer versio

Re: Unable to update to any newer version

Re: XBMC4Xbox Installer - Error updating to any newer versio

Re: XBMC4Xbox Installer - Error updating to any newer versio

Re: XBMC4Xbox Installer - Error updating to any newer versio

Re: XBMC4Xbox Installer - Error updating to any newer versio

Re: XBMC4Xbox Installer - Error updating to any newer versio