XBMC4Xbox

Posted: **Tue Jan 24, 2017 3:41 am**

This is a continuation of the thread here:
http://www.xbmc4xbox.org.uk/forum/viewt ... f=6&t=3468

It was left for dead over a year ago with no real follow up from OP, mods, or devs as far as I can see so I would like to revisit this issue.

The problem is long delays with the "Scanning Games" dialog on every entry into the Games menu.
This problem gets worse the more games you have on your system.

For those of us with large HDDs (1-2TB) the delay can be substantial. With ~500 games I am seeing about 25 seconds of delay *every* time I access the Games list. This is not the same as the initial scanning delay when new items are added. That process takes maybe 3-5 minutes to scan the entire library. But, even when that completes fully there is still this additional 20-25 second delay on every access.

This even happens *twice* when configuring a trainer. So, when you enter the games folder it takes 20-25 seconds to enumerate the games. Then, after selecting a game to enable a trainer it enumerates them all again. Thus it takes almost a full minute to be able to even begin to load the game!

I have played around with the number of games and it seems like somewhere between 25-70 games is the limit to not bring up this scanning dialog. Although I suspect the scanning *is* happening every time and is just complete before the scanning dialog has a chance to appear.

I have tried everything I can think of (including fresh load) and the problem persists. The only workaround I have found is to organize all my games into smaller folders (e.g. /A-B, /C-D, /E-F,...) and access them one at a time. This way only < 25-50 games need to be enumerated at once.

As per the final mod guidance on the other thread I have created all the logs requested.

Here is a log with only 25 games in my sources:
http://pastebin.com/uHrZrzwv

Here is 150 games:
http://pastebin.com/iwHvA0Sa

Here is 508 games:
http://pastebin.com/525FhJjF

In each of these logs I have xbmc configured to open directly to Programs/Games, so basically I simply boot up and allow enumeration and take the log.
I performed an un-logged boot between these to allow a full enumeration, so the logs are showing the enumeration *after* the check for new games would have taken place.

There is one additional log file here:
http://pastebin.com/CFALm74V

In this file it does the 508 games on boot-up, but then I exit the games menu and go back in immediately so that the games enumerate a second time. Finally, this log ends with the httpapi commands requested here:
http://www.xbmc4xbox.org.uk/forum/viewt ... 468#p27205

The output of that second command is here:
https://pste.eu/p/mymo.html

As you can see no -1 in the columns.

There is one bit of weirdness with that httpapi output in that it seems to be listing more than my 508 games as it seems to list duplicates from sources that no longer exist. Regardless, I did this test again (and all tests) after a clean install and the output was proper (508 games) but all the problems persist.

Thoughts?

Posted: **Tue Jan 24, 2017 12:21 pm**

The workaround currently is to use favourites instead of programs/games

Not sure if trainers are loaded ( after configuration in programs/games ) when launched from favourites but it don't see why they wouldn't.

I made a script to generate the favourites.xml using your programs db or your games directories.
( note the games directories are E,F,G:\Games\ so your current layout you need to use the programs6.db option )

https://github.com/Rocky5/XBMC4Kids/tre ... s%20Extras

You will need to download the hole source to get the script. Just the way GitHub works.

Posted: **Tue Jan 24, 2017 2:55 pm**

Rocky5 wrote:The workaround currently is to use favourites instead of programs/games

Not sure if trainers are loaded ( after configuration in programs/games ) when launched from favourites but it don't see why they wouldn't.

I made a script to generate the favourites.xml using your programs db or your games directories.
( note the games directories are E,F,G:\Games\ so your current layout you need to use the programs6.db option )

https://github.com/Rocky5/XBMC4Kids/tre ... s%20Extras

You will need to download the hole source to get the script. Just the way GitHub works.

Thanks for this solution Rocky. I'm not really a fan of having to use the favorites, but it is better than nothing.

Honestly I think I prefer my other solution of dividing the games up into subfolders based on alphabet (or possible genre, etc) so that there are only ~50 games per folder.

A couple reasons why I think this second method is superior:
- Favorite method still requires you go into Programs to configure trainers. So you will still have an issue to enumerate the games (twice) when you go to enable/disable trainer.
- Favorite method requires being rerun on any changes (though I assume you can automate this at startup?).
- Favorite method looses all other context menus (save game, edit, etc).

The favorite method does have the advantage of listing all the games in one screen, whereas my method requires going to each folder alphabetically (etc.).

Is there a way to automatically launch favorites at start?
Assign it to a hotkey or a menu item in a more prominent place?
What about have it take up a larger amount of the screen? One of the reasons I don't like it is it opens off to the side and the overlay almost makes it look like something is wrong. It is meant to be a "temporary" dialog, so it just seems out of place from the main menu system.

Is there another thread that is discussing this issue? I haven't really seen much talk of it.
It would be nice to know that the devs are aware of it, that they hopefully know what the issue is, and that we might expect a fix on next release.

Thanks for the info.

Posted: **Tue Jan 24, 2017 10:27 pm**

I'm sure BuZz is aware of this behavior, but even prior to BuZz saving the project, previous devs were not focused on the games side but rather the media. This is still true to this day, and I don't foresee this changing.

You can always submit a patch to improve functionality, but I don't guarantee the patch will be applied immediately or at all.

As for the scanning issue, in short; the programs section relies on directories and they are scanned dynamically and hits the file system more often.

With regards to you seeing more games than currently available; this is because, whilst xbmc scans dynamically and adds to the database, it does not remove items that are no longer found in the source directory, and there is no "clean database" function. At least not that I'm aware of.

As an alternative option, you can build a custom window and manually add the games to the container.

Pros:
- faster loading
- as much details as you wish; e.g. genre, overview, devs, etc
- custom artwork

Cons:
- initial setup may be time consuming if no skinning skills
- no trainer support
- manual entry/deletion

Sent from my iPhone using Tapatalk

Posted: **Tue Jan 24, 2017 11:30 pm**

Dom DXecutioner wrote:I'm sure BuZz is aware of this behavior, but even prior to BuZz saving the project, previous devs were not focused on the games side but rather the media. This is still true to this day, and I don't foresee this changing.

You can always submit a patch to improve functionality, but I don't guarantee the patch will be applied immediately or at all.
....
Cons:
- initial setup may be time consuming if no skinning skills
- no trainer support
- manual entry/deletion

That's sad to hear - I was just having a discussion on another forum where I was positing the superiority of xbmc for a dash above the competition - even if only used for games.
It is hard to back that claim up now, especially with confirmation here that games are just an afterthought.

While I understand that development time is limited to the whims of those working on the project and that as a user we have limited right to demand anything is fixed - I think it is a bit strange to believe that any random user has the ability to submit a patch to fix this. That being said, I would think that an option to manually scan this folder along with an option to turn off automatic scanning would be almost trivial for anyone familiar with the code to implement. Is there a better place for me to request this type of feature?

As for trainer support - this is the same as the "favorites" fix. Do you mean to say that you won't be able to configure trainers from here (seems obvious), or will trainers already configured from the programs menu also fail to run?

Posted: **Wed Jan 25, 2017 12:39 am**

bengalih wrote: While I understand that development time is limited to the whims of those working on the project...

I think is a bit strange to believe that by using this type of reference you're going to get someone to do it... Often is not a matter of a "whim," as you so eloquently put it. People have jobs, school, families, higher priority projects, and many real world challenges that prevent them from being more active on any given project; not to mention that the user base/interest for xbmc is at an all time low

bengalih wrote:I think it is a bit strange to believe that any random user has the ability to submit a patch to fix this.

Perhaps... True story: some devs, including BuZz - and I belive Rocky5 as well, knew nothing about how the project worked, but they found that they needed/wanted a particular feature/fix, so they researched, learned, fixed, and submitted such patches. Just food for thought!

bengalih wrote:...I would think that an option to manually scan this folder along with an option to turn off automatic scanning would be almost trivial for anyone familiar with the code to implement.

Agreed... When that person comes along, we'll definitely welcome him/her. Keep in mind that just because someone can make changes, to the project, does not necessarily mean that this person knows every aspect of this huge project. Just because you know how to drive a car, does not mean you know how to build the engine.

bengalih wrote:Is there a better place for me to request this type of feature?

Not to my knowledge; in all fairness, my knowledge is limited

bengalih wrote:As for trainer support - this is the same as the "favorites" fix. Do you mean to say that you won't be able to configure trainers from here (seems obvious), or will trainers already configured from the programs menu also fail to run?

I don't know, with specificity, how trainers work. What I can say, however, is that when you create a custom window in xbmc, you only have but the basics of the basics. No context menu, default onclick behaviour, etc. That said, this limitations are, somewhat, trivial when using python scripts.

My 2 cents...

Posted: **Wed Jan 25, 2017 1:49 am**

Dom DXecutioner wrote:
bengalih wrote: While I understand that development time is limited to the whims of those working on the project...
I think is a bit strange to believe that by using this type of reference you're going to get someone to do it... Often is not a matter of a "whim," as you so eloquently put it. People have jobs, school, families, higher priority projects, and many real world challenges that prevent them from being more active on any given project; not to mention that the user base/interest for xbmc is at an all time low

Exactly my point. I realize that on projects of this type it is often an unpaid (and often thankless) job and they have all the responsibilities you've mentioned above. That is why working on this is a whim/hobby of theirs. It wasn't meant to be derogatory.

Posted: **Wed Jan 25, 2017 10:00 am**

OK so i took a look in the source, the file you want to edit is "XBMC4Xbox SVN\xbmc\programs\GUIWindowPrograms.cpp" line 629.

It seems when you enter the games or any programs window for that matter it looks for xbe files and then flattens the folder structure. This is why it scans when entering the menu.

If you disable the flattening code you get directories and fast entry into the menu, but this isn't what you want.

Edit:
Though I find it strange that the xbe path is in the programs6.db but it seems to not be used

why flatten the directories if you there already scanned in.
( dont think I will be able to modify the code to make it use the db paths instead, above my head it is )

Ok if you remove the check for the default.xbe exists it no longer shows the dialog and I have 200 games and loads almost instantly.
Now this isn't ideal as if you have a folder in your games directory without and xbe XBMC will no longer be able to tell, as it just forces false so it thinks its a game

.

Code: Select all

if (item->m_bIsFolder && !item->IsParentFolder() && !item->IsPlugin())
    { // folder item - let's check for a default.xbe file, and flatten if we have one
      CStdString defaultXBE;
      URIUtils::AddFileToFolder(item->GetPath(), "default.xbe", defaultXBE);
      if (CFile::Exists(defaultXBE))
      { // yes, format the item up
        item->SetPath(defaultXBE);
        item->m_bIsFolder = false;
      }
    }

to

Code: Select all

if (item->m_bIsFolder && !item->IsParentFolder() && !item->IsPlugin())
    { // folder item - let's check for a default.xbe file, and flatten if we have one
      CStdString defaultXBE;
      URIUtils::AddFileToFolder(item->GetPath(), "default.xbe", defaultXBE);
      item->SetPath(defaultXBE);
      item->m_bIsFolder = false;
    }

Posted: **Thu Jan 26, 2017 8:18 pm**

bengalih you manage to test the xbe I sent you?

Posted: **Fri Jan 27, 2017 3:34 am**

Rocky5 wrote:bengalih you manage to test the xbe I sent you?

Hey sorry... It didn't show I had any PMs so I hadn't bothered to look until this post.
So..yes...it definitely speeds things up. There is still about a 4-5 second delay or so every time going into the games folder.
About 1-2 seconds of what appears to be nothing (visible) happening and then another 2-3 seconds of a scanning "working" toast pop-up at the bottom right. But comparatively it is world's better.

What is strange regarding the default.xbe information you provided was this:
1) The game still appears as a normal game, but when launched it just launches normally and then cycles back into XBMC (effectively a reboot).
2) I deletes the programs directory and did a full rescan of the games folder with one of the games tested as renamed to default.xbe.old (same game I tested for #1 above). Strangely enough it (or maybe not?) it still listed the game there with the appropriate icon.

Would you expect all of the above behavior? I guess I'm a little surprised that it found the game and the icon without a default.xbe. I thought all that data was contained within the default.xbe.

Posted: **Fri Jan 27, 2017 4:24 am**

It will treat folders as xbe files, ( as the check to see if it's a folder is removed ) so your games will need to be structured like so;

F:\Games\halo
F:\Games\splinter cell
G:\Games\voodoo vince
eg.....

The game information is in the xbe, you would need to remove myprograms6.db to clear all scanned info.
( Q:\USERDATA\database\ )

Posted: **Fri Jan 27, 2017 6:42 am**

Rocky5 wrote:It will treat folders as xbe files, ( as the check to see if it's a folder is removed ) so your games will need to be structured like so;

F:\Games\halo
F:\Games\splinter cell
G:\Games\voodoo vince
eg.....

The game information is in the xbe, you would need to remove myprograms6.db to clear all scanned info.
( Q:\USERDATA\database\ )

Yes my folders are structured like you mention.
After applying your .xbe I did a test and went into one of the games folders and renamed the default.xbe to default.xbe.old.
As you indicated, the game still showed up in the list but when I launched it xbmc just rebooted into itself (or rebooted the xbox in general).

I was curious to see if your patch would still scan the folders properly when new games were added, so I booted into another dash and deleted the MyPrograms database. I then went back into XBMC and it started to rescan all of the games.

However, after scanning it still shows the game ("folder"?) of the one I renamed to .old. I just thought this was strange since it shouldn't be able to find the xbe.

Perhaps what is happening is that it is just finding the folder still and the icon is somehow cached from something else?

Posted: **Fri Jan 27, 2017 9:24 am**

bengalih wrote:
Rocky5 wrote:It will treat folders as xbe files, ( as the check to see if it's a folder is removed ) so your games will need to be structured like so;

F:\Games\halo
F:\Games\splinter cell
G:\Games\voodoo vince
eg.....

The game information is in the xbe, you would need to remove myprograms6.db to clear all scanned info.
( Q:\USERDATA\database\ )
Yes my folders are structured like you mention.
After applying your .xbe I did a test and went into one of the games folders and renamed the default.xbe to default.xbe.old.
As you indicated, the game still showed up in the list but when I launched it xbmc just rebooted into itself (or rebooted the xbox in general).

I was curious to see if your patch would still scan the folders properly when new games were added, so I booted into another dash and deleted the MyPrograms database. I then went back into XBMC and it started to rescan all of the games.

However, after scanning it still shows the game ("folder"?) of the one I renamed to .old. I just thought this was strange since it shouldn't be able to find the xbe.

Perhaps what is happening is that it is just finding the folder still and the icon is somehow cached from something else?

Like I said the xbe exist check is disabled when entering the menu, so that means XBMC thinks all folders are xbe files regardless if one exists or not.
( it will be the folder name you see as the name and also it's the cached icon you see )

You will get strange things like that as it was just a quick hack to get what you wanted.
( just don't have empty folders in your games folder and you don't need to worry )

Also remove the thumbnail cache.
( it's in the USERDATA folder )

Posted: **Fri Jan 27, 2017 10:35 am**

Good too see you digging around the code Rocky5; one day you might just be the number 2 guy, after BuZz, of the project

Sent from my iPhone using Tapatalk

XBMC4Xbox

scanning games delay on every run

scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

scanning games delay on every run

Re: scanning games delay on every run

scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

Re: scanning games delay on every run

scanning games delay on every run