TMDB and TVDB Scrapers Not Working

[shady7]

Hi there any chance of TVDB and TMDB scrapers updated no longer working Cheers

[RextheC]

Yes.. I also noticed that TVDB webside has changed many things
Than i have tried to get some infos with the TVDB xbox scraper but that didnt work
I hope that someone can solve that problem once more

TMDB scraper only finds some of my movies - but not everything..
Perhaps IMDB works better?! I still have to try that.

Greetz

[RextheC]

Hydihou
IMDB scraper (same as TVDB and TMDB) only finds some movies too..
Where are the chiefs who can do something that would be great
Greetings

[grupy]

This would explain why re-scraping seasons for shows hasnt worked for me :\

Hope this can be resolved soon as I still love using my old Xbox

[Dan Dar3]

Grupy just PMed me, let's try and have a look, fingers crossed

Although for larger libraries and accidental wiping it's probably to use a library manager to generate the metadata files (.nfo, poster etc) next to the media files so they can be imported back very quickly.
* https://kodi.wiki/view/Library_managers

Or it can be exported and imported back, even if slightly outdated would save some time.
https://www.xbmc4xbox.org.uk/wiki/Impor ... rt_Library

[grupy]

I appreciate the reply back and again apologies again for the PM.

I have everything on my XBOX HDD already so in order to use the EMM option, I would need to transfer everything from my XBOX to another HDD on my PC and then run the scan as I dont think EMM can connect to my XBOX and scrape everything on there.

I hope there is a way to fix the scrapers so I can simply scan my movies and TV shows straight from my XBOX but failing to do so this, I suppose this another solution (moving forward).

[grupy]

Grupy just PMed me, let's try and have a look, fingers crossed

Although for larger libraries and accidental wiping it's probably to use a library manager to generate the metadata files (.nfo, poster etc) next to the media files so they can be imported back very quickly.
* https://kodi.wiki/view/Library_managers

Or it can be exported and imported back, even if slightly outdated would save some time.
https://www.xbmc4xbox.org.uk/wiki/Impor ... rt_Library

I used EMM and was able to scrape all movies and TV shows in my Library locally before using FXP to send the files back. I noticed posters and thumbs arent showing up automatically when I scanned my library again on my XBOX. Is there a quick manual way to fix this or should I just wait for the scapers (TMDB and TVDB) to be fixed so I can scrape directly off my XBOX again?

[Dan Dar3]

Oh, that is good for now (Y)

I'm not sure whether that is something that might have change in Kodi since or in Ember Media Manager, maybe give us a list of files that have been generated for one video in particular so we can look at.

Maybe still set the Content Type and TMDB as a scraper and try scanning one and see if the thumb gets updated from the newly generated files by EMM. https://www.xbmc4xbox.org.uk/wiki/Set_C ... o_Library)

Alternatively, you can use the Addons > Programs > Addons4Xbox Installer to install this Thumbnails plugin we wrote that has a Clean function that might help it recreate the thumbs on next scan. See more on this forum page: viewtopic.php?f=7&t=3036&p=23752#p23752

[grupy]

I used TinyMM this time and set the scaper for older version of Kodi and XBMC.

When I scraped Movies and TV shows, all posters showed up properly for TV shows but some fanart and posters are missing for Movies. But this time it was much better.

[Dan Dar3]

Sorry for the delay, my Xbox started acting funny, powering for 2 seconds and then powering off. Took me two hours to take it apart down to the casing, the motherboard looked ok, and now it boots up fine ocasionally, and other times goes into green-red flash, probably haven't position correctly the Xecuter2 or the adapter just moved when turning it on sides to pul screews back and *&%^, will have to take it apart again some other time.

Anyway, tried to reprocude your error, haven't asked for a debug log, but initially mine was on an older version of the TBDM scraper, specifially the r33032 where Buzz has create a new key for XBMC4Xbox and not having to use Kody's one.

Anyway, with the old key I tried the URL myself and it wasn't working:

Code: Select all

16:40:39 M: 36282368   DEBUG: scraper: Searching for '500 days of summer' using themoviedb.org scraper (file: 'tmdb.xml', content: 'movies', language: 'en', date: '2012-01-16', framework: '1.1')
16:40:39 M: 36282368   DEBUG: CurlFile::Open(D00E52A8) http://api.tmdb.org/3/search/movie?api_key=ecbc86c92da237cb9faff6d3ddc4be6d&query=500%20days%20of%20summer&year=2009&language=en
16:40:39 M: 36208640    INFO: XCURL::DllLibCurlGlobal::easy_aquire - Created session to http://api.tmdb.org
16:40:39 M: 36110336   ERROR: CCurlFile::FillBuffer - Failed: HTTP response code said error(22)

With the new code (simply download the files manually and place them in the exact same location on Q: drive), it works but there is an issue with the regular expressions, I'll have another look at that and try to find a fix and propose it to Buzz in a ticket. BRB soon.

Code: Select all

16:53:19 M: 34250752   DEBUG: scraper: Searching for '500 days of summer' using themoviedb.org scraper (file: 'tmdb.xml', content: 'movies', language: 'en', date: '2012-01-16', framework: '1.1')
16:53:19 M: 34250752   DEBUG: CurlFile::Open(D00E52A8) http://api.tmdb.org/3/search/movie?api_key=112978b163bca3ca7982bbdb9732827b&query=500%20days%20of%20summer&year=2009&language=en
16:53:19 M: 34242560   DEBUG: scraper: GetSearchResults returned <results></results>

[Dan Dar3]

Created issue #363 for TBDB with a proposed fix, you can try to download the archive and navingate into it (system/scrapers), find the tmdb.xml and replace into the similar location (Q:\system\scrapers), restart XBMC4Xbox and try again refreshing the contents with ignoring the NFO information (might ask you to use that instead of the server).
https://redmine.exotica.org.uk/issues/363

PS: You might not even need to restart XBMC4Xbox, try without restarting first.

PPS: If you don't mind please try the second fix as well (see the 2nd comment on the ticket above) where I've added HTTP GZIP compression, that might improve the responsiveness especially when scraping in bulk.

[Dan Dar3]

On TVDB might be a bigger issue. The information is provided by TBDB as a ZIP archive and its format may have changed and it doesn't seem to be suported by XBMC4Xbox.
Plus we used an HTTP URL that redirects to HTTPs on their website, and if XBMC4Xbox doesn't handle HTTPS it might just return the incorrect content that might be causing the above, we don't know yet.

I've created an issue #364 with my findings so far.
https://redmine.exotica.org.uk/issues/364

This is how my log file looks like, can you provide similar from yours.

Code: Select all

10:41:56 M: 32382976   DEBUG: VideoInfoScanner: Found episode match f:\tv shows\the mentalist\the.mentalist.s02e01.640x360.avi (s2e1) [[Ss]([0-9]+)[][ ._-]*[Ee]([0-9]+)([^\\/]*)$]
10:41:56 M: 32387072   DEBUG: CurlFile::Open(D00E52AC) http://www.thetvdb.com/api/1D62F2F90030C444/series/82459/all/en.zip
...
10:41:56 M: 31993856   DEBUG: advapi32.dll fake function dllCryptAcquireContext() called
10:41:56 M: 31993856   DEBUG: advapi32.dll fake function dllCryptReleaseContext() called
10:41:56 M: 31993856   DEBUG: advapi32.dll fake function dllCryptAcquireContext() called
...
10:43:21 M: 33951744   ERROR: FileZip: extended local header, not supported!
10:43:21 M: 33951744   DEBUG: scraper: GetEpisodeList returned <episodeguide></episodeguide>

[grupy]

Created issue #363 for TBDB with a proposed fix, you can try to download the archive and navingate into it (system/scrapers), find the tmdb.xml and replace into the similar location (Q:\system\scrapers), restart XBMC4Xbox and try again refreshing the contents with ignoring the NFO information (might ask you to use that instead of the server).
https://redmine.exotica.org.uk/issues/363

PS: You might not even need to restart XBMC4Xbox, try without restarting first.

PPS: If you don't mind please try the second fix as well (see the 2nd comment on the ticket above) where I've added HTTP GZIP compression, that might improve the responsiveness especially when scraping in bulk.

I added a couple of movies and with the GZIP compression file I was able to scan the two movies without any issues. I proceeded to delete all of the files downloaded by TinyMM and my library to so I can scan everything again with this file and it worked great! I was able to scan my Movie Library without any issues. I didnt test the file for TV Shows.

[Dan Dar3]

Thank you grupy for the feedback on TMDB, I will add it to the ticket.
TVDB unfortunatelly will still be an issue, waiting for Buzz's feedback as well.

[grupy]

Thank you grupy for the feedback on TMDB, I will add it to the ticket.
TVDB unfortunatelly will still be an issue, waiting for Buzz's feedback as well.

Thanks for all of your help.

Was there any updates for TVDB scraper?

[Dan Dar3]

No update on TVDB since, sorry.
https://redmine.exotica.org.uk/issues/364

[MasterChief2]

Parsers are fixed. I have opened issue and you can find it here: https://redmine.exotica.org.uk/issues/372 For TV Show parser to work properly there are some changes that needs to be done to source code of IMDB.cpp and IMDB.h. I have provided patch file in hope buzz will merged it. You can fine video here:

[GoTeamScotch]

That looks great! Good job.

I hope this becomes a new official version. We haven't had one in 7 years.

You should reach out to Rocky5 and see if you two can collaborate. A modern build of XBMC that includes your work + Rocky's mods (XBMC4Gamers) would be a dream come true.

[MasterChief2]

Thank you!
I doubt there will be new official release. I'm working on this from February ant there is a lot of changes. And BuZz is give up on this project, ha have moved focus on some other things. So I will probably release some sort of fork.

BTW, we had a really outdated guilib and many other things were dating from 2009. XBMC4Xbox code was some mix between 2009-2012 with couple patches from newer Kodi releases. Biggest changes on which Estuary is depending are from 2011-2013. I'm currently porting some InfoBools which we were missing (like String.IsEqual, Integer.IsGreater, etc.) and after that I have to optimize and port some features regarding Dynamic Contents and after that I hope I can finally start working on Estuary port.

I have some big plans and merging rocky5's scripts and artworks is one of them. I want to refsctor programs database and make it similar to videos/music database. For example we niw have movies sorted by sets, title, year, country, actors so I want to create something similar to programs. That also includes creating programs scraper which will pull artwork and metadata from some API. But before anything is done regarding all this I will need to try with backporting some PR which deals with sorting and database refactor. And that PR is huge just like the PR which deals with Dirty Regions