Home > Uncategorized > SCPD stream “re-encoding” mencoder to save space

SCPD stream “re-encoding” mencoder to save space

October 15th, 2011 Leave a comment Go to comments

The SCPD downloading script posted several years back seems to use some interesting mencoder encoder and so there always appears to be extra information packaged in the video file. Unfortunately, I needed to reduce the size of the streams and it appears that by using ffmpeg and extract the audio and video, and then muxing it back together seems to reduce the overhead file size by about 100 Mb.

I’ve included a simple bash script which you can modify below which takes a list of past streams to re-process (for windows you will need do figure out the dos equivalent). There doesn’t seem to be any loss in video quality and sound quality seems fine, so it seems like a free win.

Note: I have tried using x264 with high10 encoding, but it seems I cannot get the x264 10-bit compilation to produce proper ts video. The windows x264 10-bit from x264.nl seems to do a perfectly fine job, and ffmpeg in linux can then mux it with the audio, but using the linux compiled 10-bit, there seems to be problems with muxing it in ffmpeg, although the actual video only encode works as it should. If both can be used in conjuction, perhaps one could even attain a smaller file size by using 10-bit compression and the reduction in overhead explained here.

while [ -n "$1" ]
do
    echo "$1"
    ffmpeg -threads 0 -i $1 -vn -acodec copy $1.wma
    ffmpeg -threads 0 -i $1 -an -vcodec copy $1.wmv
    ffmpeg -threads 0 -async 1 -i $1.wma -i $1.wmv -vcodec copy $1.mkv
    rm -rf $1.wma $1.wmv
    shift
done

For those of you that interested in 10-bit encoding with the profile high10 in x264 here are some snippets:

# x264 --threads 0 --profile high10 -o $1.264 $1                                
# ffmpeg -threads 0 -i $1 -vn -acodec copy $1.wma                               
# ffmpeg -async 1 -threads 0 -i $1.264 -i $1.wma  -vcodec copy $1.final.mp4
Categories: Uncategorized Tags: , , ,
  • bigdoce

    Hi Jon,

    I have extracted the video URLs using your greasemonkey script, but when I try to use the python scrip to download the videos, I get the following error:

    cd ‘/Fall2011/’ && ‘/usr/bin/pythonw’ ‘/Fall2011/getSCPD.py’ && echo Exit status: $? && exit 1
    dn0a206b05:~ BigMizAC$ cd ‘/Fall2011/’ && ‘/usr/bin/pythonw’ ‘/Fall2011/getSCPD.py’ && echo Exit status: $? && exit 1
    mms://cobb.stanford.edu/courses/msande293/111130/111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wmv
    mencoder mms://cobb.stanford.edu/courses/msande293/111130/111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wmv -ovc copy -oac copy -o 111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wm.avi
    Traceback (most recent call last):
    File “/Fall2011/getSCPD.py”, line 21, in
    downloadClass(‘Fall2011/mse293′)
    File “/Fall2011/getSCPD.py”, line 18, in downloadClass
    strip(url)
    File “/Fall2011/getSCPD.py”, line 11, in strip
    p1 = subprocess.call(["mencoder", newurl, "-ovc","copy","-oac","copy","-o", ("%s.avi" % (vidname,))])
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 444, in call
    return Popen(*popenargs, **kwargs).wait()
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 595, in __init__
    errread, errwrite)
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 1106, in _execute_child
    raise child_exception
    OSError: [Errno 2] No such file or directory

    Your help will be greatly appreciated.

    p.s.
    I am using a Mac (snow leopard) and the python script I am running is:

    #!/usr/bin/python

    import subprocess,sys,string

    # Grab the videos using mencoder. Do it one at a time using ‘call’ instead of ‘popen’
    def strip(url):
    newurl = “mms”+url[url.find(":"):]
    print newurl
    vidname = url[url.rfind("/")+1:url.rfind("?")]
    print “mencoder %s -ovc copy -oac copy -o %s.avi” % (newurl,vidname)
    p1 = subprocess.call(["mencoder", newurl, "-ovc","copy","-oac","copy","-o", ("%s.avi" % (vidname,))])

    # Download links that are stored in a file, one link per line
    def downloadClass(filename):
    f = open(‘/Fall2011/mse293′, ‘r’)
    for url in f:
    url = string.rstrip(url, ‘\n’)
    strip(url)

    downloadClass(‘Fall2011/mse293′)

    • http://jonblog.wong.lan Jon

      Looks like mencoder is not in your path. You can replace mencoder in the script with the path to the mencoder file. The script doesn’t really do anything except generate the command-line to run mencoder and strip part of the url.

  • bigdoce

    Hi Jon,
    Thank you very much for your reply. I added the path to the mencoder file, and I received a new error message indicating permission is denied for File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py. I added my useraccount and gave it a read/write permission for the file, and then I gave myself a read/write permission for the python2.6 folder, but to no avail. I still receive the same error. Any help will be greatly appreciated. Cheers.

    cd ‘/Fall2011/’ && ‘/usr/bin/pythonw’ ‘/Fall2011/getSCPD.py’ && echo Exit status: $? && exit 1
    rescomp-11-204838:~ BigMizAC$ cd ‘/Fall2011/’ && ‘/usr/bin/pythonw’ ‘/Fall2011/getSCPD.py’ && echo Exit status: $? && exit 1
    mms://cobb.stanford.edu/courses/msande293/111130/111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wmv
    mencoder mms://cobb.stanford.edu/courses/msande293/111130/111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wmv -ovc copy -oac copy -o 111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wm.avi
    Traceback (most recent call last):
    File “/Fall2011/getSCPD.py”, line 20, in
    downloadClass(‘Fall2011/mse293′)
    File “/Fall2011/getSCPD.py”, line 18, in downloadClass
    strip(url)
    File “/Fall2011/getSCPD.py”, line 11, in strip
    p1 = subprocess.call(["/Users/BigMAC/Desktop/Mencoder OS X.app", newurl, "-ovc","copy","-oac","copy","-o", ("%s.avi" % (vidname,))])
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 444, in call
    return Popen(*popenargs, **kwargs).wait()
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 595, in __init__
    errread, errwrite)
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 1106, in _execute_child
    raise child_exception
    OSError: [Errno 13] Permission denied

    • http://jonblog.wong.lan Jon

      I think if you paste in your app file into terminal and try to run it, I don’t think it will work. Generally there is a directory inside of *.app/bin that holds the actual executable. Try running what you think is the executable from terminal and if that works, then use that as the substitute string.

  • bigdoce

    Thank you for the prompt reply. I cannot open mencoder from the command line. However, I can open MPlayerOSX2 from terminal, but when I use its path in the python script, I get the same permission denied error. The script is below:

    #!/usr/bin/python

    import subprocess,sys,string

    # Grab the videos using mencoder. Do it one at a time using ‘call’ instead of ‘popen’
    def strip(url):
    newurl = “mms”+url[url.find(":"):]
    print newurl
    vidname = url[url.rfind("/")+1:url.rfind("?")]
    print “mencoder %s -ovc copy -oac copy -o %s.avi” % (newurl,vidname)
    p1 = subprocess.call(["/Users/BigMAC/Desktop/MPlayerOSX2.app", newurl, "-ovc","copy","-oac","copy","-o", ("%s.avi" % (vidname,))])

    # Download links that are stored in a file, one link per line
    def downloadClass(filename):
    f = open(‘/Fall2011/mse293′, ‘r’)
    for url in f:
    url = string.rstrip(url, ‘\n’)
    strip(url)

    downloadClass(‘Fall2011/mse293′)

  • Bigdoce

    Hey Jon,

    I can open up MPlayer from terminal–using the /Users/BigMAC/Desktop/MPlayerOSX2.app path in the script below–without an issue. However, I still encounter the permission denied error message. I’ll appreciate any insight you might have.
    Cheers.

    cd ‘/Fall2011/’ && ‘/usr/bin/pythonw’ ‘/Fall2011/getSCPD.py’ && echo Exit status: $? && exit 1
    dn0a208a77:~ BigMAC$ cd ‘/Fall2011/’ && ‘/usr/bin/pythonw’ ‘/Fall2011/getSCPD.py’ && echo Exit status: $? && exit 1
    mms://cobb.stanford.edu/courses/msande293/111130/111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wmv
    mencoder mms://cobb.stanford.edu/courses/msande293/111130/111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wmv -ovc copy -oac copy -o 111130-msande293-znPJvX1nNNUYpTeQAp1a-500.wm.avi
    Traceback (most recent call last):
    File “/Fall2011/getSCPD.py”, line 21, in
    downloadClass(‘Fall2011/mse293′)
    File “/Fall2011/getSCPD.py”, line 19, in downloadClass
    strip(url)
    File “/Fall2011/getSCPD.py”, line 11, in strip
    p1 = subprocess.call(["/Users/BigMAC/Desktop/MPlayerOSX2.app", newurl, "-ovc","copy","-oac","copy", "-o", ("%s.avi" % (vidname,))])
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 444, in call
    return Popen(*popenargs, **kwargs).wait()
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 595, in __init__
    errread, errwrite)
    File “/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py”, line 1106, in _execute_child
    raise child_exception
    OSError: [Errno 13] Permission denied

    • http://jonblog.wong.lan Jon

      I’d first not worry about the script. Just concentrate on the commandline. mencoder mms://link -ovc copy -oac copy -o video. If it doesn’t work in the terminal, it won’t work in the script. More broadly speaking, Permission denied means that you have an issue with file/folder permissions. More than likely, the program can’t write into a particular folder. Anyways generally “.app” files are folders. For example for AquaEmacs: /Applications/Aquamacs.app/Contents/MacOS/Aquamacs starts Acquamacs.