The return of youtube

Discussion of XBMC4XBOX development.
Post Reply
lolcool
Posts: 19
Joined: Fri Oct 31, 2014 3:49 pm
Been thanked: 4 times

The return of youtube

Post by lolcool »

I started making a iphone youtube remote control for the xbox last year.

I had it basically done when the XBMC YouTube plugin broke.

Well... I rewrote that part of the code so my iphone youtube remote works without the youtube plugin and uses my own plugin instead.

I plan to release very soon.

How it works:
- iphone uses web ui to connect to xbox
- google api is used to search for youtube videos
- when youtube video is selected, video id is sent to xbox, and the xbox saves youtube html page to xbox /web/ folder
- saved page is then analyzed and parsed on iphone (or any browser) with javascript to find the video .mp4 url
- video .mp4 url is sent to xbox to play

it is 100% functional at the moment - just need to create an interface for it :mrgreen:
lolcool
Posts: 19
Joined: Fri Oct 31, 2014 3:49 pm
Been thanked: 4 times

Re: The return of youtube

Post by lolcool »

i will release a working prototype sunday!!!!!!!!!!!!!
Geeba
Posts: 539
Joined: Wed Jul 04, 2012 11:32 am
Location: UK
Has thanked: 63 times
Been thanked: 31 times

Re: The return of youtube

Post by Geeba »

Nice! :D
whufclee
Posts: 922
Joined: Tue Jul 17, 2012 5:42 pm
Location: Essex, UK
Has thanked: 18 times
Been thanked: 109 times
Contact:

Re: The return of youtube

Post by whufclee »

Also if anyone wants to take a look at the yt.py module that spoyser created a few years ago that's always worked even when the api changed. Trouble is it doesn't seem to pick up anything lower than 720 and freezes on Xbox. I had a little look and couldn't understand how the links were getting pulled and the streams generated but if someone can find a way to get the 360/480 streams only we should have a working yt function again. You can find the module in the community portal add-on for main branch, it's in some of spoysers addons too but can't remember which ones off hand.
lolcool
Posts: 19
Joined: Fri Oct 31, 2014 3:49 pm
Been thanked: 4 times

Re: The return of youtube

Post by lolcool »

I'll check it
peanutismint
Posts: 3
Joined: Sun Jan 10, 2016 3:59 pm

Re: The return of youtube

Post by peanutismint »

This sounds awesome. Just wanted to register interest to let you know that there are still people out there who are wanting this! :-)

Any more progress since November?
User avatar
cashonly
Posts: 181
Joined: Thu Jul 19, 2012 5:12 pm
Has thanked: 20 times
Been thanked: 32 times

Re: The return of youtube

Post by cashonly »

same here, would like to se this!
These pretzels are making me thirsty!
whufclee
Posts: 922
Joined: Tue Jul 17, 2012 5:42 pm
Location: Essex, UK
Has thanked: 18 times
Been thanked: 109 times
Contact:

Re: The return of youtube

Post by whufclee »

I'll be back from holiday in 2 days time so will try and upload the latest version of Community Portal 4Xbox. Using spoysers module I had to edit it slightly but got it working well on Xbox. Should be simple enough for someone to edit the YouTube addon and use this to get links to play I think.
fxmech
Posts: 673
Joined: Wed Aug 01, 2012 9:15 am
Has thanked: 37 times
Been thanked: 46 times

Re: The return of youtube

Post by fxmech »

I have a 128mb so maybe the 720 will play for me :) I'll try and find this plugin
lolcool
Posts: 19
Joined: Fri Oct 31, 2014 3:49 pm
Been thanked: 4 times

Re: The return of youtube

Post by lolcool »

sorry, got a lil distracted. I can post an updated version this week!
peanutismint
Posts: 3
Joined: Sun Jan 10, 2016 3:59 pm

Re: The return of youtube

Post by peanutismint »

Awesome! Thanks, looking forward :-)
fxmech
Posts: 673
Joined: Wed Aug 01, 2012 9:15 am
Has thanked: 37 times
Been thanked: 46 times

Re: The return of youtube

Post by fxmech »

Thanks byron
whufclee
Posts: 922
Joined: Tue Jul 17, 2012 5:42 pm
Location: Essex, UK
Has thanked: 18 times
Been thanked: 109 times
Contact:

Re: The return of youtube

Post by whufclee »

I think the module has been updated slightly since that version. The one I'm using is called yt.py and can be found in some of his addons. I'm back from holiday now so just catching up on some work but will upload the version I've edited tomorrow for you to tinker with.
whufclee
Posts: 922
Joined: Tue Jul 17, 2012 5:42 pm
Location: Essex, UK
Has thanked: 18 times
Been thanked: 109 times
Contact:

Re: The return of youtube

Post by whufclee »

Ok so for anyone wanting to tinker about with the YouTube module here it is, unfortunately the best I've managed to get working so far is 360p. You can set the ones you want to try out in line 158 where I've already added a list of potential Xbox compatible ones but so far the only results I've had back are ID numbers 18, 43 and there's a 720p one that often comes back as a result (think it's ID 22). The 720 one just won't play on 64MB Xbox running stock Confluence skin unfortunately and just freezes. There was another one that I managed to get audio playing but no video, sorry can't remember which one that was - have a feeling it may have been 480 resolution.

To use this module just use yt.PlayVideo(url) whenever you want to play an item, where url is the YT ID.

Code: Select all

#
#  Copyright (C) 2013 Sean Poyser
#
#
#  This code is a derivative of the YouTube plugin for XBMC
#  released under the terms of the GNU General Public License as published by
#  the Free Software Foundation; version 3
#  Copyright (C) 2010-2012 Tobias Ussing And Henrik Mosgaard Jensen
#
#  This Program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 3, or (at your option)
#  any later version.
#
#  This Program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with XBMC; see the file COPYING.  If not, write to
#  the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
#  http://www.gnu.org/copyleft/gpl.html
#


#        5: "240p h263 flv container",
#        18: "360p h264 mp4 container | 270 for rtmpe?",
#        22: "720p h264 mp4 container",
#        26: "???",
#        33: "???",
#        34: "360p h264 flv container",
#        35: "480p h264 flv container",
#        37: "1080p h264 mp4 container",
#        38: "720p vp8 webm container",
#        43: "360p h264 flv container",
#        44: "480p vp8 webm container",
#        45: "720p vp8 webm container",
#        46: "520p vp8 webm stereo",
#        59: "480 for rtmpe",
#        78: "seems to be around 400 for rtmpe",
#        82: "360p h264 stereo",
#        83: "240p h264 stereo",
#        84: "720p h264 stereo",
#        85: "520p h264 stereo",
#        100: "360p vp8 webm stereo",
#        101: "480p vp8 webm stereo",
#        102: "720p vp8 webm stereo",
#        120: "hd720",
#        121: "hd1080"


import re
import urllib2
import urllib
import cgi
import HTMLParser
import xbmcgui

try: import simplejson as json

except ImportError: import json

dp            =  xbmcgui.DialogProgress()
MAX_REC_DEPTH = 5

def Clean(text):
    text = text.replace('–', '-')
    text = text.replace('’', '\'')
    text = text.replace('“', '"')
    text = text.replace('”', '"')
    text = text.replace(''',   '\'')
    text = text.replace('<b>',     '')
    text = text.replace('</b>',    '')
    text = text.replace('&',   '&')
    text = text.replace('\ufeff', '')
    return text

def PlayVideo(id, forcePlayer=False):
    import sys
    dp.create("Loading video",'','Please Wait','')

    video, links = GetVideoInformation(id)

    if 'best' not in video:
        return False

    url   = video['best']          
    title = video['title']
    image = video['thumbnail']

    liz = xbmcgui.ListItem(title, iconImage=image, thumbnailImage=image)

    liz.setInfo( type="Video", infoLabels={ "Title": title} )

    if forcePlayer or len(sys.argv) < 2 or int(sys.argv[1]) == -1:
        import xbmc
        pl = xbmc.PlayList(xbmc.PLAYLIST_VIDEO)
        pl.clear()
        pl.add(url, liz)
        dp.close()
        xbmc.Player().play(pl)
    
    else:
        import xbmcplugin
        liz.setPath(url)
        xbmcplugin.setResolvedUrl(int(sys.argv[1]), True, liz)

    return True


def GetVideoInformation(id):
    #id = 'H7iQ4sAf0OE' #test for HLSVP
    #id = 'ofHlUJuw8Ak' #test for stereo
    #id = 'ifZkeuSrNRc' #account closed
    #id = 'M7FIvfx5J10'
    #id = 'n-D1EB74Ckg' #vevo
    #id = 'lVMWEheQ2hU' #vevo

    video  = {}
    links  = []

    try:     video, links = GetVideoInfo(id)
    except : pass
    
    return video, links


def GetVideoInfo(id):
    url  = 'http://www.youtube.com/watch?v=%s&safeSearch=none' % id
    html = FetchPage(url)

    video, links = Scrape(html)

    video['videoid']   = id
    video['thumbnail'] = "http://i.ytimg.com/vi/%s/0.jpg" % video['videoid']
    video['title']     = GetVideoTitle(html)

    if len(links) == 0:
        if 'hlsvp' in video:
            video['best'] = video['hlsvp']
    
    else:
        video['best'] = links[0][1]

    return video, links


def GetVideoTitle(html):
    try:    return Clean(re.compile('<meta name="title" content="(.+?)">').search(html).groups(1)[0])
    except: pass

    return 'YouTube Video'

    
def Scrape(html):
    sd            = [5, 18, 26, 33, 34, 35, 44, 46, 59, 78, 82, 83, 85, 100, 101]
    # 43 is an option but it's ogg and that doesn't seem to be great, xbox best appears to be 18

    video  = {}
    links  = []

    flashvars = ExtractFlashVars(html)

    if not flashvars.has_key(u"url_encoded_fmt_stream_map"):
        return video, links

    if flashvars.has_key(u"ttsurl"):
        video[u"ttsurl"] = flashvars[u"ttsurl"]

    if flashvars.has_key(u"hlsvp"):                               
        video[u"hlsvp"] = flashvars[u"hlsvp"]    

    for url_desc in flashvars[u"url_encoded_fmt_stream_map"].split(u","):
        url_desc_map = cgi.parse_qs(url_desc)
        
        if not (url_desc_map.has_key(u"url") or url_desc_map.has_key(u"stream")):
            continue

        key = int(url_desc_map[u"itag"][0])
        url = u""
        
        if url_desc_map.has_key(u"url"):
            url = urllib.unquote(url_desc_map[u"url"][0])
        
        elif url_desc_map.has_key(u"conn") and url_desc_map.has_key(u"stream"):
            url = urllib.unquote(url_desc_map[u"conn"][0])
            
            if url.rfind("/") < len(url) -1:
                url = url + "/"
            
            url = url + urllib.unquote(url_desc_map[u"stream"][0])
        
        elif url_desc_map.has_key(u"stream") and not url_desc_map.has_key(u"conn"):
            url = urllib.unquote(url_desc_map[u"stream"][0])

        if url_desc_map.has_key(u"sig"):
            url = url + u"&signature=" + url_desc_map[u"sig"][0]
        
        elif url_desc_map.has_key(u"s"):
            sig = url_desc_map[u"s"][0]
            #url = url + u"&signature=" + DecryptSignature(sig)
           
            flashvars = ExtractFlashVars(html, assets=True)
            js        = flashvars[u"js"]    
            url      += u"&signature=" + DecryptSignatureNew(sig, js)          

        if key in sd:
            links.append([key, url])
    #    print"### LINKS: "+str(links)
    #links.sort(reverse=True)
    return video, links


def DecryptSignature(s):
    ''' use decryption solution by Youtube-DL project '''
    if len(s) == 88:
        return s[48] + s[81:67:-1] + s[82] + s[66:62:-1] + s[85] + s[61:48:-1] + s[67] + s[47:12:-1] + s[3] + s[11:3:-1] + s[2] + s[12]
    
    elif len(s) == 87:
        return s[62] + s[82:62:-1] + s[83] + s[61:52:-1] + s[0] + s[51:2:-1]
    
    elif len(s) == 86:
        return s[2:63] + s[82] + s[64:82] + s[63]
    
    elif len(s) == 85:
        return s[76] + s[82:76:-1] + s[83] + s[75:60:-1] + s[0] + s[59:50:-1] + s[1] + s[49:2:-1]
    
    elif len(s) == 84:
        return s[83:36:-1] + s[2] + s[35:26:-1] + s[3] + s[25:3:-1] + s[26]
    
    elif len(s) == 83:
        return s[6] + s[3:6] + s[33] + s[7:24] + s[0] + s[25:33] + s[53] + s[34:53] + s[24] + s[54:]
    
    elif len(s) == 82:
        return s[36] + s[79:67:-1] + s[81] + s[66:40:-1] + s[33] + s[39:36:-1] + s[40] + s[35] + s[0] + s[67] + s[32:0:-1] + s[34]
    
    elif len(s) == 81:
        return s[6] + s[3:6] + s[33] + s[7:24] + s[0] + s[25:33] + s[2] + s[34:53] + s[24] + s[54:81]
    
    elif len(s) == 92:
        return s[25] + s[3:25] + s[0] + s[26:42] + s[79] + s[43:79] + s[91] + s[80:83];
    #else:
    #    print ('Unable to decrypt signature, key length %d not supported; retrying might work' % (len(s)))


def ExtractFlashVars(data, assets=False):
    flashvars = {}
    found = False

    for line in data.split("\n"):
        
        if line.strip().find(";ytplayer.config = ") > 0:
            found = True
            p1 = line.find(";ytplayer.config = ") + len(";ytplayer.config = ") - 1
            p2 = line.rfind(";")
            
            if p1 <= 0 or p2 <= 0:
                continue
            data = line[p1 + 1:p2]
            break
    data = RemoveAdditionalEndingDelimiter(data)

    if found:
        data = json.loads(data)
        
        if assets:
            flashvars = data['assets']
        
        else:
            flashvars = data['args']

    return flashvars


def FetchPage(url):
    req = urllib2.Request(url)
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
    req.add_header('Referer',    'http://www.youtube.com/')

    return urllib2.urlopen(req).read().decode("utf-8")


def replaceHTMLCodes(txt):
    # Fix missing ; in &#<number>;
    txt = re.sub("(&#[0-9]+)([^;^0-9]+)", "\\1;\\2", txt)

    txt = HTMLParser.HTMLParser().unescape(txt)
    txt = txt.replace("&", "&")
    return txt


def RemoveAdditionalEndingDelimiter(data):
    pos = data.find("};")
    
    if pos != -1:
        data = data[:pos + 1]
    
    return data
    
####################################################

global playerData
global allLocalFunNamesTab
global allLocalVarNamesTab

def _extractVarLocalFuns(match):
    varName, objBody = match.groups()
    output = ''
    
    for func in objBody.split( '},' ):
        output += re.sub(
            r'^([^:]+):function\(([^)]*)\)',
            r'function %s__\1(\2,*args)' % varName,
            func
        ) + '\n'
    
    return output

def _jsToPy(jsFunBody):
    pythonFunBody = re.sub(r'var ([^=]+)={(.*?)}};', _extractVarLocalFuns, jsFunBody)
    pythonFunBody = re.sub(r'function (\w*)\$(\w*)', r'function \1_S_\2', pythonFunBody)
    pythonFunBody = pythonFunBody.replace('function', 'def').replace('{', ':\n\t').replace('}', '').replace(';', '\n\t').replace('var ', '')
    pythonFunBody = pythonFunBody.replace('.reverse()', '[::-1]')

    lines = pythonFunBody.split('\n')
    
    for i in range(len(lines)):
        # a.split("") -> list(a)
        match = re.search('(\w+?)\.split\(""\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), 'list(' + match.group(1)  + ')')
        # a.length -> len(a)
        
        match = re.search('(\w+?)\.length', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), 'len(' + match.group(1)  + ')')
        # a.slice(3) -> a[3:]
        
        match = re.search('(\w+?)\.slice\((\w+?)\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), match.group(1) + ('[%s:]' % match.group(2)) )
        # a.join("") -> "".join(a)
        
        match = re.search('(\w+?)\.join\(("[^"]*?")\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), match.group(2) + '.join(' + match.group(1) + ')' )
        # a.splice(b,c) -> del a[b:c]
        
        match = re.search('(\w+?)\.splice\(([^,]+),([^)]+)\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), 'del ' + match.group(1) + '[' + match.group(2) + ':' + match.group(3) + ']' )

    pythonFunBody = "\n".join(lines)
    pythonFunBody = re.sub(r'(\w+)\.(\w+)\(', r'\1__\2(', pythonFunBody)
    pythonFunBody = re.sub(r'([^=])(\w+)\[::-1\]', r'\1\2.reverse()', pythonFunBody)
    return pythonFunBody

def _jsToPy1(jsFunBody):
    pythonFunBody = jsFunBody.replace('function', 'def').replace('{', ':\n\t').replace('}', '').replace(';', '\n\t').replace('var ', '')
    pythonFunBody = pythonFunBody.replace('.reverse()', '[::-1]')

    lines = pythonFunBody.split('\n')
    for i in range(len(lines)):
        # a.split("") -> list(a)
        match = re.search('(\w+?)\.split\(""\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), 'list(' + match.group(1)  + ')')
        # a.length -> len(a)
        
        match = re.search('(\w+?)\.length', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), 'len(' + match.group(1)  + ')')
        # a.slice(3) -> a[3:]
        
        match = re.search('(\w+?)\.slice\(([0-9]+?)\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), match.group(1) + ('[%s:]' % match.group(2)) )
        # a.join("") -> "".join(a)
        
        match = re.search('(\w+?)\.join\(("[^"]*?")\)', lines[i])
        
        if match:
            lines[i] = lines[i].replace( match.group(0), match.group(2) + '.join(' + match.group(1) + ')' )
    
    return "\n".join(lines)

def _getLocalFunBody(funName):
    # get function body 
    funName = funName.replace('$', '\\$')
    match = re.search('(function %s\([^)]+?\){[^}]+?})' % funName, playerData)
    
    if match:
        return match.group(1)
    
    return ''

def _getAllLocalSubFunNames(mainFunBody):
    match = re.compile('[ =(,](\w+?)\([^)]*?\)').findall( mainFunBody )
    
    if len(match):
        # first item is name of main function, so omit it
        funNameTab = set( match[1:] )
        return funNameTab
    
    return set()
    
def _extractLocalVarNames(mainFunBody):
    valid_funcs = ( 'reverse', 'split', 'splice', 'slice', 'join' )
    match       = re.compile( r'[; =(,](\w+)\.(\w+)\(' ).findall( mainFunBody )
    local_vars  = []
    
    for name in match:
        
        if name[1] not in valid_funcs:
            local_vars.append( name[0] )
    
    return set(local_vars)

def _getLocalVarObjBody(varName):
    match = re.search( r'var %s={.*?}};' % varName, playerData )
    
    if match:
        return match.group(0)
    
    return ''

def DecryptSignatureNew(s, playerUrl):
    if not playerUrl.startswith('http:'):
        playerUrl = 'http:' + playerUrl
        
    #print "Decrypt_signature sign_len[%d] playerUrl[%s]" % (len(s), playerUrl)

    global allLocalFunNamesTab
    global allLocalVarNamesTab
    global playerData
                
    allLocalFunNamesTab = []
    allLocalVarNamesTab = []
    playerData          = ''    

    request = urllib2.Request(playerUrl)
    #res        = core._fetchPage({u"link": playerUrl})
    #playerData = res["content"]
            
    try:
        playerData = urllib2.urlopen(request).read()
        playerData = playerData.decode('utf-8', 'ignore')
    
    except Exception, e:
        #print str(e)
        print 'Failed to decode playerData'
        return ''
        
    # get main function name 
    match = re.search("signature=([$a-zA-Z]+)\([^)]\)", playerData)
    
    if match:
        mainFunName = match.group(1)
    
    else: 
        print('Failed to get main signature function name')
        return ''
        
    _mainFunName = mainFunName.replace('$','_S_')   
    fullAlgoCode = _getfullAlgoCode(mainFunName)    

    # wrap all local algo function into one function extractedSignatureAlgo()
    algoLines = fullAlgoCode.split('\n')
    
    for i in range(len(algoLines)):
        algoLines[i] = '\t' + algoLines[i]
    
    fullAlgoCode  = 'def extractedSignatureAlgo(param):'
    fullAlgoCode += '\n'.join(algoLines)
    fullAlgoCode += '\n\treturn %s(param)' % _mainFunName
    fullAlgoCode += '\noutSignature = extractedSignatureAlgo( inSignature )\n'

    # after this function we should have all needed code in fullAlgoCode

    #print '---------------------------------------'
    #print '|    ALGO FOR SIGNATURE DECRYPTION    |'
    #print '---------------------------------------'
    #print fullAlgoCode
    #print '---------------------------------------'

    try:
        algoCodeObj = compile(fullAlgoCode, '', 'exec')
    
    except:
        print 'Failed to obtain decryptSignature code'
        return ''

    # for security allow only flew python global function in algo code
    vGlobals = {"__builtins__": None, 'len': len, 'list': list}

    # local variable to pass encrypted sign and get decrypted sign
    vLocals = { 'inSignature': s, 'outSignature': '' }

    # execute prepared code
    try:
        exec(algoCodeObj, vGlobals, vLocals)
    
    except:
        print 'decryptSignature code failed to exceute correctly'
        return ''

    #print 'Decrypted signature = [%s]' % vLocals['outSignature']

    return vLocals['outSignature']

# Note, this method is using a recursion
def _getfullAlgoCode(mainFunName, recDepth=0):
    global playerData
    global allLocalFunNamesTab
    global allLocalVarNamesTab
    
    if MAX_REC_DEPTH <= recDepth:
        print '_getfullAlgoCode: Maximum recursion depth exceeded'
        return 

    funBody = _getLocalFunBody(mainFunName)
    if funBody != '':
        funNames = _getAllLocalSubFunNames(funBody)
        
        if len(funNames):
            
            for funName in funNames:
                funName_ = funName.replace('$','_S_')
                
                if funName not in allLocalFunNamesTab:
                    funBody=funBody.replace(funName,funName_)
                    allLocalFunNamesTab.append(funName)
                    #print 'Add local function %s to known functions' % mainFunName
                    funbody = _getfullAlgoCode(funName, recDepth+1) + "\n" + funBody
                    
        varNames = _extractLocalVarNames(funBody)
        
        if len(varNames):
            
            for varName in varNames:
                
                if varName not in allLocalVarNamesTab:
                    allLocalVarNamesTab.append(varName)
                    funBody = _getLocalVarObjBody(varName) + "\n" + funBody

        # convert code from javascript to python 
        funBody = _jsToPy(funBody)
        return '\n' + funBody + '\n'
    
    return funBody
peanutismint
Posts: 3
Joined: Sun Jan 10, 2016 3:59 pm

Re: The return of youtube

Post by peanutismint »

Sorry - for a complete novice user - how do I download/install/use this addon? Is it installable from the XBMC4XBOX GUI via an addon installer or adding a specific repo? Or will I need to download something on a computer and then copy the addon to the Xbox via FTP etc??
whufclee
Posts: 922
Joined: Tue Jul 17, 2012 5:42 pm
Location: Essex, UK
Has thanked: 18 times
Been thanked: 109 times
Contact:

Re: The return of youtube

Post by whufclee »

The code I posted will require someone to insert it into the YouTube add-on or create a new basic YouTube addon which may be easier as it could be made to be as basic as possible freeing up valuable resources.
Post Reply