Jon’s Project Blog

The SCPD downloading script posted several years back seems to use some interesting mencoder encoder and so there always appears to be extra information packaged in the video file. Unfortunately, I needed to reduce the size of the streams and it appears that by using ffmpeg and extract the audio and video, and then muxing it back together seems to reduce the overhead file size by about 100 Mb.

I’ve included a simple bash script which you can modify below which takes a list of past streams to re-process (for windows you will need do figure out the dos equivalent). There doesn’t seem to be any loss in video quality and sound quality seems fine, so it seems like a free win.

Note: I have tried using x264 with high10 encoding, but it seems I cannot get the x264 10-bit compilation to produce proper ts video. The windows x264 10-bit from x264.nl seems to do a perfectly fine job, and ffmpeg in linux can then mux it with the audio, but using the linux compiled 10-bit, there seems to be problems with muxing it in ffmpeg, although the actual video only encode works as it should. If both can be used in conjuction, perhaps one could even attain a smaller file size by using 10-bit compression and the reduction in overhead explained here.

while [ -n "$1" ]
do
    echo "$1"
    ffmpeg -threads 0 -i $1 -vn -acodec copy $1.wma
    ffmpeg -threads 0 -i $1 -an -vcodec copy $1.wmv
    ffmpeg -threads 0 -async 1 -i $1.wma -i $1.wmv -vcodec copy $1.mkv
    rm -rf $1.wma $1.wmv
    shift
done

For those of you that interested in 10-bit encoding with the profile high10 in x264 here are some snippets:

# x264 --threads 0 --profile high10 -o $1.264 $1                                
# ffmpeg -threads 0 -i $1 -vn -acodec copy $1.wma                               
# ffmpeg -async 1 -threads 0 -i $1.264 -i $1.wma  -vcodec copy $1.final.mp4

Updated 1/13/11: Turns out, I had corrected the script sometime earlier in 2010, but had thought I had already uploaded it, so my script looked pretty similar to Joey’s. I just looked here, and it turns out the script is not the same as the old one. I apologize for the confusion.

At the end of every quarter I use the following greasemonkey script to extract the links of each of the classes.

Usage: Go to the current quarter SCPD page and click on each of the links. Extracting the links takes awhile, since it is based on the previous SCPD link extractor. The GM script simply goes to each lecture page and extracts the link. After extraction is finish a new tab will appear and will display all the links for you to copy and paste. (This uses the same idea as the Xanga2RSS extractor.)

It seems that the SCPD program keeps the lectures archived at these links for a pretty long time. At least for a quarter or two. (However I once googled for SCPD links and found an EE one from 07 and it still worked, so perhaps SCPD never really erases them.)

Here it is below:

// ==UserScript==
// @name           SCPD links
// @namespace      hawflakes
// @description    Strips video links off of SCPD
// @include        https://myvideosu.stanford.edu/*
// ==/UserScript==
 
var vidurls="";
var links;
 
// Main link page for a course
if (window.location.toString().indexOf("GradCourseInfo.aspx?")>=0)
{
// Get links to video page
 
links = findXPathNodes("/html/body//table/tbody/tr/td/a[text()='WMP']");
i=0;
 
var href=links.snapshotItem(i).getAttribute("href");
var url = href.substring(href.indexOf("'")+1);
url = url.substring(0,url.length-4);
get(url,"",getlink,i);
 
}
 
// helper functions
function findXPathNode(xpath, start,doc)
{
	var result = (doc == null ? document : doc).evaluate(xpath,(start == null ? document : start), null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE ,null);
	return (result.snapshotLength > 0 ? result.snapshotItem(0) : null);
}
 
function findXPathNodes(xpath, start,doc)
{
	return (doc == null ? document : doc).evaluate(xpath,(start == null ? document : start), null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE ,null);	
}
 
function elem(tagname,content)
{
	var ret = document.createElement(tagname);
	ret.innerHTML = content;
	return ret;
}
 
function get(url, data, cb,info) {
	var client = new XMLHttpRequest();
	client.open("GET",url,true);
 
	client.onreadystatechange = function () {
		if(client.readyState==4) {
 
			cb(client,info);
 
		}
	}
	client.send(null);  
}
 
function getlink(client,info)
{
 
 
	var tempurl=client.responseText.substring(client.responseText.indexOf("data=\"")+"data=\"".length,client.responseText.indexOf(".wmv\"")+4)+"\n";
	alert(tempurl);
 
	vidurls=vidurls+tempurl;
	GM_log(tempurl);
if (info<links .snapshotLength-1)
{
	var href=links.snapshotItem(info+1).getAttribute("href");
	var url = href.substring(href.indexOf("'")+1);
	url = url.substring(0,url.length-4);
	get(url,"",getlink,info+1);
}
else
{
 //Generate links page
GM_openInTab("data:text;charset=UTF-8," + encodeURI(vidurls));
}
 
}

Since I am currently attending Stanford, and I once searched the intraweb for a solution for downloading wmv streams, but couldn’t find anything specific, I’ll post my solution here.

Normally, SCPD requires you to login and then you watch the videos streamed using some wmv player. However, should you want to rewind or seek through a stream, this will prove to be impossible. Many commercial wmv recorders are not free, and simply play the video to download it. My solution is similar to this, but it is FREE (as in beer).

What you need:

1. Mencoder (An encoder that comes along with Mplayer) If you use windows, you should get the command-line non-gui download from the website since it will come with mencoder. Otherwise it will not.
2. Optional: Firefox + Greasemonkey for extracting the video URL if you’re lazy like me.
3. Optional: Python for executing mencoder and for converting http:// links to mms:// links.

To download from SCPD (or other unprotected wmv streams), simply execute the following at a command prompt:

mencoder mms://stream.wmv -ovc copy -oac copy -o output.avi

1. Replace mms://stream.wmv is simply the url with “http:” replace with “mms:”
2. Replace output.avi with the output file name. Extensions should govern the format that it spits out.

It will throw a bunch of text to the screen, and you simply leave the window open (or minimized) and wait for it to finish. However, since it’s free, I commonly download several streams at the same time, so when they are done I will have several streams downloaded instead of just one.

Download: Greasemonkey script for SCPD (gets links automatically when you view an SCPD video) (Updated 9/29/09)

If you are lazy like me, and don’t want to change the http://url to mms://url I’ve provided an mencoder script that, given an http url, will fill out the commandline and execute it for you. (Note: Many mac and linux users will have python already installed on their machines.)

Usage:

./stripSCPD http://url

Download: Python SCPD script

#!/usr/bin/python
 
import subprocess,sys
 
def strip(url):
    newurl = "mms"+url[url.find(":"):]
    print newurl
    vidname = url[url.rfind("/")+1:url.rfind("?")]
    print "mencoder %s -ovc copy -oac copy -o %s.avi" % (newurl,vidname)
    subprocess.Popen(["mencoder", newurl, "-ovc","copy","-oac","copy","-o", ("%s.avi" % (vidname,))])
 
if __name__=="__main__":
    strip(sys.argv[1])

Update (11/09/08):

I’ve just tried everything again in windows, and it seems to work. Note, for some reason mplayer doesn’t have the necessary codecs by default in the downloadable zip file. However, VLC, media player classic and Windows media Player seem to be able to seek. However only media player classic can increase the audio speed (in case you want to watch a stream 1.5 times fast and be able to comprehend what someone is saying even if they seem to be on helium!).