TMDB and TVDB Scrapers Not Working

If you need any help with XBMC4XBOX, such as installing or using the software, please post it in here.
Post Reply
shady7
Posts: 85
Joined: Fri Aug 03, 2012 6:45 am
Has thanked: 1 time
Been thanked: 4 times

TMDB and TVDB Scrapers Not Working

Post by shady7 » Sun Feb 02, 2020 9:39 pm

Hi there any chance of TVDB and TMDB scrapers updated no longer working Cheers

User avatar
RextheC
Posts: 40
Joined: Sun Feb 22, 2015 5:15 pm
Has thanked: 33 times
Been thanked: 8 times

Re: TMDB and TVDB Scrapers Not Working

Post by RextheC » Fri Feb 07, 2020 11:04 pm

Yes.. I also noticed that TVDB webside has changed many things :twisted:
Than i have tried to get some infos with the TVDB xbox scraper but that didnt work :(
I hope that someone can solve that problem once more :) :)

TMDB scraper only finds some of my movies - but not everything..
Perhaps IMDB works better?! I still have to try that.

Greetz
...THE TRUTH IS OUT THERE...

User avatar
RextheC
Posts: 40
Joined: Sun Feb 22, 2015 5:15 pm
Has thanked: 33 times
Been thanked: 8 times

Re: TMDB and TVDB Scrapers Not Working

Post by RextheC » Thu Mar 12, 2020 9:03 pm

Hydihou :P
IMDB scraper (same as TVDB and TMDB) only finds some movies too.. :cry:
Where are the chiefs who can do something :?: that would be great :)
Greetings
:mrgreen:
...THE TRUTH IS OUT THERE...

grupy
Posts: 30
Joined: Thu Aug 09, 2012 6:26 am
Has thanked: 5 times
Been thanked: 2 times

Re: TMDB and TVDB Scrapers Not Working

Post by grupy » Fri Mar 27, 2020 4:14 am

This would explain why re-scraping seasons for shows hasnt worked for me :\

Hope this can be resolved soon as I still love using my old Xbox :)

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Tue Mar 31, 2020 8:11 am

Grupy just PMed me, let's try and have a look, fingers crossed :-)

Although for larger libraries and accidental wiping it's probably to use a library manager to generate the metadata files (.nfo, poster etc) next to the media files so they can be imported back very quickly.
* https://kodi.wiki/view/Library_managers

Or it can be exported and imported back, even if slightly outdated would save some time.
https://www.xbmc4xbox.org.uk/wiki/Impor ... rt_Library

grupy
Posts: 30
Joined: Thu Aug 09, 2012 6:26 am
Has thanked: 5 times
Been thanked: 2 times

Re: TMDB and TVDB Scrapers Not Working

Post by grupy » Tue Mar 31, 2020 5:08 pm

I appreciate the reply back and again apologies again for the PM.

I have everything on my XBOX HDD already so in order to use the EMM option, I would need to transfer everything from my XBOX to another HDD on my PC and then run the scan as I dont think EMM can connect to my XBOX and scrape everything on there.

I hope there is a way to fix the scrapers so I can simply scan my movies and TV shows straight from my XBOX but failing to do so this, I suppose this another solution (moving forward).

grupy
Posts: 30
Joined: Thu Aug 09, 2012 6:26 am
Has thanked: 5 times
Been thanked: 2 times

Re: TMDB and TVDB Scrapers Not Working

Post by grupy » Fri Apr 03, 2020 7:29 pm

Dan Dar3 wrote:
Tue Mar 31, 2020 8:11 am
Grupy just PMed me, let's try and have a look, fingers crossed :-)

Although for larger libraries and accidental wiping it's probably to use a library manager to generate the metadata files (.nfo, poster etc) next to the media files so they can be imported back very quickly.
* https://kodi.wiki/view/Library_managers

Or it can be exported and imported back, even if slightly outdated would save some time.
https://www.xbmc4xbox.org.uk/wiki/Impor ... rt_Library
I used EMM and was able to scrape all movies and TV shows in my Library locally before using FXP to send the files back. I noticed posters and thumbs arent showing up automatically when I scanned my library again on my XBOX. Is there a quick manual way to fix this or should I just wait for the scapers (TMDB and TVDB) to be fixed so I can scrape directly off my XBOX again?

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Sun Apr 05, 2020 2:45 pm

Oh, that is good for now (Y)

I'm not sure whether that is something that might have change in Kodi since or in Ember Media Manager, maybe give us a list of files that have been generated for one video in particular so we can look at.

Maybe still set the Content Type and TMDB as a scraper and try scanning one and see if the thumb gets updated from the newly generated files by EMM. https://www.xbmc4xbox.org.uk/wiki/Set_C ... o_Library)

Alternatively, you can use the Addons > Programs > Addons4Xbox Installer to install this Thumbnails plugin we wrote that has a Clean function that might help it recreate the thumbs on next scan. See more on this forum page: viewtopic.php?f=7&t=3036&p=23752#p23752

grupy
Posts: 30
Joined: Thu Aug 09, 2012 6:26 am
Has thanked: 5 times
Been thanked: 2 times

Re: TMDB and TVDB Scrapers Not Working

Post by grupy » Mon Apr 06, 2020 8:56 pm

I used TinyMM this time and set the scaper for older version of Kodi and XBMC.

When I scraped Movies and TV shows, all posters showed up properly for TV shows but some fanart and posters are missing for Movies. But this time it was much better.

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Tue Apr 07, 2020 5:01 pm

Sorry for the delay, my Xbox started acting funny, powering for 2 seconds and then powering off. Took me two hours to take it apart down to the casing, the motherboard looked ok, and now it boots up fine ocasionally, and other times goes into green-red flash, probably haven't position correctly the Xecuter2 or the adapter just moved when turning it on sides to pul screews back and *&%^, will have to take it apart again some other time.

Anyway, tried to reprocude your error, haven't asked for a debug log, but initially mine was on an older version of the TBDM scraper, specifially the r33032 where Buzz has create a new key for XBMC4Xbox and not having to use Kody's one.

Anyway, with the old key I tried the URL myself and it wasn't working:

Code: Select all

16:40:39 M: 36282368   DEBUG: scraper: Searching for '500 days of summer' using themoviedb.org scraper (file: 'tmdb.xml', content: 'movies', language: 'en', date: '2012-01-16', framework: '1.1')
16:40:39 M: 36282368   DEBUG: CurlFile::Open(D00E52A8) http://api.tmdb.org/3/search/movie?api_key=ecbc86c92da237cb9faff6d3ddc4be6d&query=500%20days%20of%20summer&year=2009&language=en
16:40:39 M: 36208640    INFO: XCURL::DllLibCurlGlobal::easy_aquire - Created session to http://api.tmdb.org
16:40:39 M: 36110336   ERROR: CCurlFile::FillBuffer - Failed: HTTP response code said error(22)
With the new code (simply download the files manually and place them in the exact same location on Q: drive), it works but there is an issue with the regular expressions, I'll have another look at that and try to find a fix and propose it to Buzz in a ticket. BRB soon.

Code: Select all

16:53:19 M: 34250752   DEBUG: scraper: Searching for '500 days of summer' using themoviedb.org scraper (file: 'tmdb.xml', content: 'movies', language: 'en', date: '2012-01-16', framework: '1.1')
16:53:19 M: 34250752   DEBUG: CurlFile::Open(D00E52A8) http://api.tmdb.org/3/search/movie?api_key=112978b163bca3ca7982bbdb9732827b&query=500%20days%20of%20summer&year=2009&language=en
16:53:19 M: 34242560   DEBUG: scraper: GetSearchResults returned <results></results>

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Wed Apr 08, 2020 10:17 am

Created issue #363 for TBDB with a proposed fix, you can try to download the archive and navingate into it (system/scrapers), find the tmdb.xml and replace into the similar location (Q:\system\scrapers), restart XBMC4Xbox and try again refreshing the contents with ignoring the NFO information (might ask you to use that instead of the server).
https://redmine.exotica.org.uk/issues/363

PS: You might not even need to restart XBMC4Xbox, try without restarting first.

PPS: If you don't mind please try the second fix as well (see the 2nd comment on the ticket above) where I've added HTTP GZIP compression, that might improve the responsiveness especially when scraping in bulk.

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Wed Apr 08, 2020 11:09 am

On TVDB might be a bigger issue. The information is provided by TBDB as a ZIP archive and its format may have changed and it doesn't seem to be suported by XBMC4Xbox.
Plus we used an HTTP URL that redirects to HTTPs on their website, and if XBMC4Xbox doesn't handle HTTPS it might just return the incorrect content that might be causing the above, we don't know yet.

I've created an issue #364 with my findings so far.
https://redmine.exotica.org.uk/issues/364

This is how my log file looks like, can you provide similar from yours.

Code: Select all

10:41:56 M: 32382976   DEBUG: VideoInfoScanner: Found episode match f:\tv shows\the mentalist\the.mentalist.s02e01.640x360.avi (s2e1) [[Ss]([0-9]+)[][ ._-]*[Ee]([0-9]+)([^\\/]*)$]
10:41:56 M: 32387072   DEBUG: CurlFile::Open(D00E52AC) http://www.thetvdb.com/api/1D62F2F90030C444/series/82459/all/en.zip
...
10:41:56 M: 31993856   DEBUG: advapi32.dll fake function dllCryptAcquireContext() called
10:41:56 M: 31993856   DEBUG: advapi32.dll fake function dllCryptReleaseContext() called
10:41:56 M: 31993856   DEBUG: advapi32.dll fake function dllCryptAcquireContext() called
...
10:43:21 M: 33951744   ERROR: FileZip: extended local header, not supported!
10:43:21 M: 33951744   DEBUG: scraper: GetEpisodeList returned <episodeguide></episodeguide>

grupy
Posts: 30
Joined: Thu Aug 09, 2012 6:26 am
Has thanked: 5 times
Been thanked: 2 times

Re: TMDB and TVDB Scrapers Not Working

Post by grupy » Sat Apr 11, 2020 10:38 pm

Dan Dar3 wrote:
Wed Apr 08, 2020 10:17 am
Created issue #363 for TBDB with a proposed fix, you can try to download the archive and navingate into it (system/scrapers), find the tmdb.xml and replace into the similar location (Q:\system\scrapers), restart XBMC4Xbox and try again refreshing the contents with ignoring the NFO information (might ask you to use that instead of the server).
https://redmine.exotica.org.uk/issues/363

PS: You might not even need to restart XBMC4Xbox, try without restarting first.

PPS: If you don't mind please try the second fix as well (see the 2nd comment on the ticket above) where I've added HTTP GZIP compression, that might improve the responsiveness especially when scraping in bulk.
I added a couple of movies and with the GZIP compression file I was able to scan the two movies without any issues. I proceeded to delete all of the files downloaded by TinyMM and my library to so I can scan everything again with this file and it worked great! I was able to scan my Movie Library without any issues. I didnt test the file for TV Shows.

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Wed Apr 15, 2020 11:28 am

Thank you grupy for the feedback on TMDB, I will add it to the ticket.
TVDB unfortunatelly will still be an issue, waiting for Buzz's feedback as well.

grupy
Posts: 30
Joined: Thu Aug 09, 2012 6:26 am
Has thanked: 5 times
Been thanked: 2 times

Re: TMDB and TVDB Scrapers Not Working

Post by grupy » Wed May 27, 2020 12:04 am

Dan Dar3 wrote:
Wed Apr 15, 2020 11:28 am
Thank you grupy for the feedback on TMDB, I will add it to the ticket.
TVDB unfortunatelly will still be an issue, waiting for Buzz's feedback as well.
Thanks for all of your help.

Was there any updates for TVDB scraper?

User avatar
Dan Dar3
Posts: 1166
Joined: Sun Jul 08, 2012 4:09 pm
Has thanked: 255 times
Been thanked: 251 times
Contact:

Re: TMDB and TVDB Scrapers Not Working

Post by Dan Dar3 » Wed May 27, 2020 12:55 pm

No update on TVDB since, sorry.
https://redmine.exotica.org.uk/issues/364

Post Reply