Major lag when switching between views

[LSDArcade]

The problem is when switching between different views in any section, there is a 1 second lag or longer before the switch when normally it will switch almost instantaneously. Also going into a subfolders in the programs menu can cause the waiting dialog to pop up when normally it would be instantaneous as well. One other effect is a 1 or 2 second wait of clicking on a xbe before it launches. I have been trying to troubleshoot this for a week. I've tried everything I can think of to fix this problem and every time I think I know the cause, I find I am wrong shortly after. I have tried deleting files in E/Cache. That worked once or twice then it came back. I've tried deleting additional profiles, this works sometimes but definitely not every time. One time cleaning the music library fixed it. I even deleted some mp3's that I thought may have had problematic tags. This also fixed it but then the problem comes back again. I have tried changing the location of XBMC to E:\ XBMC, F:\XBMC, F:\Apps\XBMC I have the same problems on version XBMC4XBOX 3.2Stable as well as 3.3.3.

Here is a debug log where I am cycling through the views of Programs/Apps and the lag is terrible.
http://www.xbmclogs.com/show.php?id=93764

This is obviously not a huge problem but it's just completely baffling me as to what is causing it.

[Dan Dar3]

@LSDArcade
A few things that maybe you can try with an without, see if it makes any difference:

• custom advancedsettings.xml
• custom Brad PM3 Mod skin
• custom RSS feeds
• XBMC4Xbox 3.3.3 (yours is built on 25 July 2012 (last year).

You say you tried 3.3.3, but your log is from that, and it seems to be peppered with these warnings.

Code: Select all

12:06:33 M: 35909632 WARNING: old getenv from python.lib called, library check needed
12:06:33 M: 35909632 WARNING: JpegIO: Error 55: Not a JPEG file: starts with 0x%02x 0x%02x

Give us a log with 3.3.3 and I might be able to more accurately find those in the current code. If any fixes required, they will need to go on top of 3.3.3, testing with a 2012 version won't do much good.

If I were you, I would install the latest (3.3.3) version separately in a new folder, no need to set it as dashboard, run it from the File Manager from the old one. See if empty gives you any problems, then gradually add stuff (media sources, addons, advancedsettings etc) one by one with restart and see which one causes the issue.

[xman]

You may also like to try a new copy of the build and see if it has the same issues. I've had similar problems and although the original appeared to be correct, something just wasn't quite right resulting in every action to be bogged down. Leave your original copy as it is and just put another copy somewhere else on the HDD and manually boot it and try the same tests that you know are causing issues with your build you using now. You also get this same problems with sluggish performance as a result of a lot of uploading and deleting content in which case the only real answer is to format and start the HDD over again but that is the last option when all else fails. My son's machine is up to this now after 4 years of service. This machine has never faulted but is now up to no matter what XBMC build is used, it is just slow. I'll format his HDD and rebuild the content and all will be sweet guaranteed. Remember, there is no "DeFrag" program for the Xbox and the only way to achieve defragging an Xbox is by completely reloading it.

[byron]

I know I've stated this ad nauseum, but have you tried using ToDDS? I've found that it helps in every single instance where many thumbs are loaded (big or small), and in view changes this is most certainly the case.

http://www.xbmc4xbox.org.uk/forum/viewt ... f=14&t=993

++ edit ++

okay the more I think about it, the view switches first and then all of the images load so I'm not entirely sure compressing them would help the view change time (but it would help the page load time after switching for sure). Also, I can sort of guess when you're trying to change views but it's interesting because there isn't any thing in the log like I'm used to seeing from what I've been working on (http://www.xbmclogs.com/show.php?id=94086). I know every skin is different, but I would at least think that there should be some sort of information in the log about view changes (unless I missed it?). It seems that you're getting many jpeg i/o errors and I'm not certain, but even if they're just warnings I suspect that this could be a part of the problem as well.

Also, if you're actively editing files and then transferring them back to xbmc via ftp while xbmc is running you're asking for issues there too. It won't break anything afaik, but in my experience it eventually ends up where you are right now...but not just view changes, everything will slow down (windows closing/opening especially). Nothing that what xman and dandar3's recommendations won't remedy though, usually a new build adds the snappiness right back which is why I use two builds, one for developing files on and one for enjoying the work of art that is xbmc4xbox

[LSDArcade]

After dozens and dozens of experiments, I think I may know the cause. There were several times when I copied over the XBMC4Xbox directory complete with UserData and thousands of thumbnails from one partition to another using UnleashX because it copies faster than XBMC. The view changing after this would sometimes be faster but not always. So it hit me to try the file manager in XBMC to copy. Well surprisingly this worked well. The view change was as fast as I have ever seen. Cool, success. Well then I start copying some other random stuff on the same partition. Now I check the view changing speed and it is slower than I have ever seen, almost stops responding. Very bizarre behavior, no? Another thing I noticed, not totally sure on this but pretty sure, Is that the XBMC directory ate up a bit more disk free space when copying with the XBMC file manager over the UnleashX file manager. Something tells me that this is related to the 64k clusters size of these partitons and all these thousands of thumbnails. Maybe someone with more knowledge on this knows the cause, but that's my guess. So my final solution was to copy XBMC4Xbox directory on my G partition (with the XBMC file manager) that only has Xbox games (450). Not on my F partition that I occasionally swap out movies and music. It works beautifully now. View changes are fast, switching between profiles is fast, everything is very responsive.

I hope this helps someone else. I spent way to much time on this one.

[xman]

If it is your booting app you are talking of when you say XBMC4Xbox directory, I only ever use E drive because it is a fast to come on line drive as is C drive where as G drive is only marginally quicker than F drive and E drive is too small for anything else really. I used to use F drive till I started having massive issues with XBMC booting when using HDDs above about 400GB. C is good but can get maxed out easierly so E it is and I've never had issues since. I use E drive for nothing else actually, only apps. By the way, my 2TB machines have movies, music and pics with no games and are near full but I never have loading issues with thumbs other than the pics for the 1st time after loading as it scans them. Have you used Dan's "thumb cleaning app"?. I have used it in the past just to get rid of rubbish old , out of date thumbs and had good results. You can also go to your Q drive and manually delete all your thumbs and let the machine get them again. This way it is clean.

[LSDArcade]

Yes, I use XBMC as my main dashboard. To be a bit more clear, this is a softmodded Xbox v1.4 using SID 5.12, dual boot with a recovery dashboard, no shadow C, using a Seagate Barracuda 2TB HDD. Also has a HD cable and digital 5.1 speakers. I never touch the C partition. I have 450 Xbox games (895GB) on the G partition and 200 Xbox games (384GB) on the F partition. Also on the F partition are Apps (5GB), CoinOPs 5 (67GB), Emulators & Media (emuxtras, resurrection xtras) (174GB), Comic Books (35GB), 2,300 MP3 albums (198GB), Movies (24GB) and pictures (1GB).

I added 500 wma tracks to the built in Xbox library on the E partition for use with custom soundtracks in games, along with Backup, cache, game trainers, saves & screenshots it leaves about 1.5GB of free space on E. Now for my XMBC folder, I can really fill it up with thumbnails. There are 12,000 (400MB) of thumbnails. I set thumbsize to 320 in advancesettings.xml because it looks awesome. I also like using other skins but I don't like how each skin can't remember its own unique view settings. Well I figured a way around this by creating a separate profile for each skin I like to use plus another profile for my wife's music. that's 6 profiles x 400MB worth of thumbnails. That takes up 2.68GB when I transfer the XBMC directory to my computer. Since my G partition is static with Xbox games that wont move or change, it makes the most sense to put the XBMC directory there where things wont become fragmented. Like I said, XBMC on F became very slow after transferring more mp3s to the partition.

I hex edited evoxdash.xbe on the E partition to point to G:\XBMC\default.xbe

I also have 7 other Xboxes (no v1.6 crap) all with 2TB HDDs and all setup very similar. These are very clean installs where I have the whole layout setup on on my computer's HDD and simply ftp it all in one shot. It takes almost 3 days.

[LSDArcade]

After some more testing I have come to the conclusion that this is solely related to HDD fragmentation and possibly the64k clusters size of 2TB drives. My other theory of using a different file manager is bunk. I believe that if the thumbnails hit any kind of fragmentation it greatly reduces XBMC's responsiveness. Further, if I have successfully tranferred XBMC to a nice un- fragmented spot of HDD it behaves perfect... until you add anything else to the partition that will apparently sit next to it. The responsiveness drops dramatically. I don't know why this is but it definitely is. Here is my solution and it's worked on three different Xboxes... use a dummy file. I copied a 700mb movie to the partition after it was working great. This of course caused XBMC to act sluggish again. I then copied whatever else I wanted to put on the partition. Afterward, I delete the 700mb movie. Upon deletion, XBMC is again perfectly responsive. I'm sure I sound like a lunatic by now but there is definitely a huge difference in responsiveness.

[Dan Dar3]

Look, what you're proving here is hardly news, FAT file system was designed for small disks and simple folder structures. A large cluster size is what it allows you to have a large partitions and in fact it should help with fragmentation (imagine sifting through a large box in one case full with a million grains of sand and the other with a thousand smaller rocks). Your problem is large amounts of directories containing large amounts of files on a partition, making for a large and slow file allocation table. To give you an example, you say you have 12.000 thumbnails, UserData is split into 16 sub-folders (from 0 to F) to speed up seeking when looking for a thumb, meaning on average 750 entries per directory. Just for one thumb, it would need to seek through these 750 files to find the entry is looking for. Not to mention the similar work it had to do for each directory component in the path and find them in their respective directories in the file allocation table! And this is just to find out the cluster where the file's data starts on disk, before even reading the contents of the file! http://digital.ni.com/public.nsf/allkb/ ... 8000681F58

Your best bet is to place XBMC4Xbox on the smallest partition where you can fit it, allowing for future expansion (thumbnails, library dbs etc) and / or with the simplest folder / file structure. Or for a future test with one of the boxes, try on an empty partition uploading a full XBMC4Xbox copy with all thumbnails and everything in it first, then copy your other content files and directories after.

Your last test results don't make sense to me really, no offence - there is no "nice, un-fragmented spot of HDD". Copying a 700 MB movie makes it slow then copying "whatever else" (how many files / directories, how large / small?) on top of it then finally removing the 700 MB to make it fast again? it all sounds a bit odd, especially if noticing a "huge" difference. When comparing results, you should wait for FTP server to flush contents of disk to quiet down before starting to navigate around, in order to get consistent timings. I would also check the System > Hardisk settings, like Acoustic management level = Fast and Power management level = High power, as those can affect HDD performance overall.

[LSDArcade]

Well without formatting the near 1TB partition that that was all but 7GB full, I filled up the remaining blank space with dummy files (since this blank space may be fragmented) deleted 6GB of large movies that were put on the hdd when I 1st filled it up. I'm pretty sure that new free space would be un-fragmented. I then copied over XBMC, and deleted the dummy files. After this, XBMC runs with zero lag, even with all the thumbnails I have. From this point though, anything I add to the partition will induce the serious lag again. So I copy over one movie (dummy file) and add the movies I deleted for the unfragmented disk space. Delete the dummy file and all runs smooth. Yes of course it would make a lot more sense to start with a freshly formatted partition but I don't understand why adding anything after XBMC on to the partition so grossly effects the performance. I also didn't want to format and reload the partition for two days to find out it still does the same thing. I don't know why but having the blank space on the disk after xbmc keeps it from lagging. I know it sounds convoluted, I'm not a good communicator. It worked though.

[xman]

Thanks for educating me on how that works Dan, I never knew. I have zero issues ever but I also use a lot of sub-folders so it isn't just one drive drive but is split into about 10 sub-folders for each drive. Seems to work. I also clear out unused thumbnails and do this because one look through the userdata thumbs shows many of the same thumbs repeated but the machine only using one, probably from doing auto searches of movie data, that's what I put it down to anyway.