Hi,
I’ve a folder with a couple of hundred movies which – until today – have been quite happily retrieving data using the TMDB scraper. However, today I tried and failed to add a couple of new ones which I could clearly see were on TMDB. It further turns out that if I remove then re-add (and rescan) the source, many of the movies that used to work are no longer identified. Some 90 or so are now failing.
An extract from the kodi.log for one that works and one that doesn’t shows:
Code:
18:04:55.801 T:12260 DEBUG: VideoInfoScanner: Scanning dir 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/' as not in the database
18:04:55.815 T:12260 DEBUG: VideoInfoScanner: No (new) information was found in dir nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/
18:04:55.925 T:12260 DEBUG: VideoInfoScanner: Scanning dir 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/' as not in the database
18:04:55.937 T:12260 DEBUG: CVideoDatabase::GetMovieId (nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/Pirates of the Caribbean At Worlds End (2007).mp4), query = select idMovie from movie where idFile=4391
18:04:55.944 T:12260 DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/Pirates of the Caribbean At Worlds End (2007).mp4'
18:04:55.944 T:12260 DEBUG: ADDON::CScraper::FindMovie: Searching for 'At World's End' using The Movie Database scraper (path: 'C:\Users\chris\AppData\Roaming\Kodi\addons\metadata.themoviedb.org', content: 'movies', version: '3.9.3')
18:04:55.959 T:12260 DEBUG: scraper: CreateSearchUrl returned <url>http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=At%20World%27s%20End&year=2007&language=en</url>
18:04:55.959 T:12260 DEBUG: CurlFile::Open(14AF8558) http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=At%20World%27s%20End&year=2007&language=en
18:04:55.998 T:12260 DEBUG: CScraperUrl::Get: Using "UTF-8" charset for "http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=At%20World%27s%20End&year=2007&language=en"
18:04:55.998 T:12260 DEBUG: scraper: GetSearchResults returned <results></results>
18:04:55.998 T:12260 DEBUG: ADDON::CScraper::FindMovie: Searching for 'At World's End' using The Movie Database scraper (path: 'C:\Users\chris\AppData\Roaming\Kodi\addons\metadata.themoviedb.org', content: 'movies', version: '3.9.3')
18:04:55.999 T:12260 DEBUG: scraper: CreateSearchUrl returned <url>http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=At%20World%27s%20End&year=&language=en</url>
18:04:55.999 T:12260 DEBUG: CurlFile::Open(14AF8558) http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=At%20World%27s%20End&year=&language=en
18:04:56.040 T:12260 DEBUG: CScraperUrl::Get: Using "UTF-8" charset for "http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=At%20World%27s%20End&year=&language=en"
18:04:56.041 T:12260 DEBUG: scraper: GetSearchResults returned <results></results>
18:04:56.041 T:12260 WARNING: No information found for item 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/Pirates of the Caribbean At Worlds End (2007).mp4', it won't be added to the library.
18:04:56.041 T:12260 DEBUG: VideoInfoScanner: No (new) information was found in dir nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/
18:04:56.080 T:12260 DEBUG: VideoInfoScanner: Scanning dir 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/extrafanart/' as not in the database
18:04:56.080 T:12260 DEBUG: VideoInfoScanner: No (new) information was found in dir nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/extrafanart/
18:04:56.109 T:12260 DEBUG: VideoInfoScanner: Scanning dir 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/extrathumbs/' as not in the database
18:04:56.109 T:12260 DEBUG: VideoInfoScanner: No (new) information was found in dir nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/At World's End (2007)/extrathumbs/
18:04:56.202 T:12260 DEBUG: VideoInfoScanner: Scanning dir 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/Dead Man's Chest (2006)/' as not in the database
18:04:56.208 T:12260 DEBUG: CVideoDatabase::GetMovieId (nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/Dead Man's Chest (2006)/Pirates of the Caribbean Dead Mans Chest (2006).mp4), query = select idMovie from movie where idFile=4392
18:04:56.214 T:12260 DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'nfs://192.168.0.2/mnt/HD/HD_a2/Public/Shared Videos/Movies/Pirates Of The Caribbean/Dead Man's Chest (2006)/Pirates of the Caribbean Dead Mans Chest (2006).mp4'
18:04:56.214 T:12260 DEBUG: ADDON::CScraper::FindMovie: Searching for 'Dead Man's Chest' using The Movie Database scraper (path: 'C:\Users\chris\AppData\Roaming\Kodi\addons\metadata.themoviedb.org', content: 'movies', version: '3.9.3')
18:04:56.223 T:12260 DEBUG: scraper: CreateSearchUrl returned <url>http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=Dead%20Man%27s%20Chest&year=2006&language=en</url>
18:04:56.223 T:12260 DEBUG: CurlFile::Open(14AFAC20) http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=Dead%20Man%27s%20Chest&year=2006&language=en
18:04:56.267 T:12260 DEBUG: CScraperUrl::Get: Using "UTF-8" charset for "http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=Dead%20Man%27s%20Chest&year=2006&language=en"
18:04:56.267 T:12260 DEBUG: scraper: GetSearchResults returned <results><entity><title>Pirates of the Caribbean: Dead Man's Chest</title><id>58</id><year>2006</year><url cache="tmdb-en-58.json">http://api.tmdb.org/3/movie/58?api_key=f7f51775877e0bb6703520952b3c7840&language=en</url></entity><entity><title>Pirates of the Caribbean: Dead Man's Chest</title><id>58</id><year>2006</year><url cache="tmdb-en-58.json">http://api.tmdb.org/3/movie/58?api_key=f7f51775877e0bb6703520952b3c7840&language=en</url></entity></results>In both cases, if I call the API from a browser I get a valid response back from TMDB. Admittedly in this example one does return multiple results and the other does not, but this factor isn’t consistent across all failures. However, what does seem to be consistent is the order in which the JSON data is return, and particularly whether the “total_results” field appears at the beginning or end.
E.g. for a failed result, the JSON is:
Code:
{"page":1,"total_results":4,"total_pages":1,"results":[{"vote_count":3612,"id":285,"video":false,"vote_average":6.9,"title":"Pirates of the Caribbean: At World's End","popularity":27.43969,"poster_path":"\/bXb00CkHqx7TPchTGG131sWV59y.jpg","original_language":"en","original_title":"Pirates of the Caribbean: At World's End","genre_ids":[12,14,28],"backdrop_path":"\/8ZgpAftUiYTU76IhUADITa3Ur9n.jpg","adult":false,"overview":"Captain Barbossa, long believed to be dead, has come back to life and is headed to the edge of the Earth with Will Turner and Elizabeth Swann. But nothing is quite as it seems.","release_date":"2007-05-19"},{"vote_count":0,"id":247646,"video":false,"vote_average":0,"title":"The Lion at World's End","popularity":1.065,"poster_path":"\/g03DkgBVjCw6V25m3elrw2NZkdI.jpg","original_language":"en","original_title":"The Lion at World's End","genre_ids":[99,12],"backdrop_path":"\/qe2532dMhyt1oJOOIxYtH7iS8tg.jpg","adult":false,"overview":"The astonishing true story of a zoo-born lion who found himself in the pet department of Harrods, then the cellar of an antique shop in London and, ultimately the wilds of Africa.","release_date":"1971-02-28"},{"vote_count":8,"id":19306,"video":false,"vote_average":5.4,"title":"At World's End","popularity":1.209159,"poster_path":"\/kyx08K6qBzXbepULUOu7EfXEOgO.jpg","original_language":"da","original_title":"Ved verdens ende","genre_ids":[12,28,35,53],"backdrop_path":"\/c1D9mKNYukz0q2ToifkaSor8KLJ.jpg","adult":false,"overview":"An english tv-team discovers a special white flower in the indonesian rainforest, but they never get to investegate it further, before the danish recluse Severin has shot them down. A danish special enforcement with a psychiatrist in front is sent off, and they find out, that Severin claims, that he is 129 years old, and it's the flower, which keeps him young. Soon all hunts the white flower, which apparently gives eternal life.","release_date":"2009-10-09"},{"vote_count":28,"id":203351,"video":false,"vote_average":2.1,"title":"Atlantic Rim","popularity":1.020435,"poster_path":"\/9kgK6GpzqreO8DUbLY1L4QD1nrC.jpg","original_language":"en","original_title":"Atlantic Rim","genre_ids":[28,878],"backdrop_path":"\/kX83lWzst7U7z4zRbykv1Zw3VVY.jpg","adult":false,"overview":"When monsters suddenly appear from the bottom of the Atlantic Ocean, a special team pilots giant robots to combat the new threat.","release_date":"2013-07-09"}]}Compared to a successful result where the total_results appears at the end of the JSON:
Code:
{"page":1,"results":[{"poster_path":"\/waFr5RVKaQ9dzOt3nQuIVB1FiPu.jpg","adult":false,"overview":"Captain Jack Sparrow works his way out of a blood debt with the ghostly Davey Jones, he also attempts to avoid eternal damnation.","release_date":"2006-06-20","genre_ids":[12,14,28],"id":58,"original_title":"Pirates of the Caribbean: Dead Man's Chest","original_language":"en","title":"Pirates of the Caribbean: Dead Man's Chest","backdrop_path":"\/hdHgIcljPHli4xaJGt0INz8Gn3J.jpg","popularity":21.639369,"vote_count":4154,"video":false,"vote_average":6.9}],"total_results":1,"total_pages":1}This behaviour seems consistent in all the results I’ve looked at – if the total_results comes at the end then Kodi can parse the response. If it comes at the start, Kodi cannot parse the response. The number of results returned seems to have no bearing on success or failure (or where the total_results field is placed).
Has anyone else seen this behaviour?
Chris