Home > Uncategorized > SCPD Class Link Extractor

SCPD Class Link Extractor

Updated 1/13/11: Turns out, I had corrected the script sometime earlier in 2010, but had thought I had already uploaded it, so my script looked pretty similar to Joey’s. I just looked here, and it turns out the script is not the same as the old one. I apologize for the confusion.

At the end of every quarter I use the following greasemonkey script to extract the links of each of the classes.

Usage: Go to the current quarter SCPD page and click on each of the links. Extracting the links takes awhile, since it is based on the previous SCPD link extractor. The GM script simply goes to each lecture page and extracts the link. After extraction is finish a new tab will appear and will display all the links for you to copy and paste. (This uses the same idea as the Xanga2RSS extractor.)

It seems that the SCPD program keeps the lectures archived at these links for a pretty long time. At least for a quarter or two. (However I once googled for SCPD links and found an EE one from 07 and it still worked, so perhaps SCPD never really erases them.)

Here it is below:

// ==UserScript==
// @name           SCPD links
// @namespace      hawflakes
// @description    Strips video links off of SCPD
// @include        https://myvideosu.stanford.edu/*
// ==/UserScript==
 
var vidurls="";
var links;
 
// Main link page for a course
if (window.location.toString().indexOf("GradCourseInfo.aspx?")>=0)
{
// Get links to video page
 
links = findXPathNodes("/html/body//table/tbody/tr/td/a[text()='WMP']");
i=0;
 
var href=links.snapshotItem(i).getAttribute("href");
var url = href.substring(href.indexOf("'")+1);
url = url.substring(0,url.length-4);
get(url,"",getlink,i);
 
}
 
// helper functions
function findXPathNode(xpath, start,doc)
{
	var result = (doc == null ? document : doc).evaluate(xpath,(start == null ? document : start), null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE ,null);
	return (result.snapshotLength > 0 ? result.snapshotItem(0) : null);
}
 
function findXPathNodes(xpath, start,doc)
{
	return (doc == null ? document : doc).evaluate(xpath,(start == null ? document : start), null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE ,null);	
}
 
function elem(tagname,content)
{
	var ret = document.createElement(tagname);
	ret.innerHTML = content;
	return ret;
}
 
function get(url, data, cb,info) {
	var client = new XMLHttpRequest();
	client.open("GET",url,true);
 
	client.onreadystatechange = function () {
		if(client.readyState==4) {
 
			cb(client,info);
 
		}
	}
	client.send(null);  
}
 
function getlink(client,info)
{
 
 
	var tempurl=client.responseText.substring(client.responseText.indexOf("data=\"")+"data=\"".length,client.responseText.indexOf(".wmv\"")+4)+"\n";
	alert(tempurl);
 
	vidurls=vidurls+tempurl;
	GM_log(tempurl);
if (info<links .snapshotLength-1)
{
	var href=links.snapshotItem(info+1).getAttribute("href");
	var url = href.substring(href.indexOf("'")+1);
	url = url.substring(0,url.length-4);
	get(url,"",getlink,info+1);
}
else
{
 //Generate links page
GM_openInTab("data:text;charset=UTF-8," + encodeURI(vidurls));
}
 
}
Categories: Uncategorized Tags: , ,
  • anonymous

    Hi, I have installed greasemonkey and also installed the user script..
    However, when i click on the links on the current quarter page, nothing happens.
    Plz help

  • anonymous

    This is wat i did:
    first installed greasemonkey addon
    -installed the script u have provided
    – went to https://myvideosu.stanford.edu
    – wanted to get the links of some electrical engineering courses
    so clicked on one of them.. but nothing seems to be happening
    plz help
    and i watch videos in SL if that is an issue.
    And can u tell how u got old SCPD links via google

  • anonymous

    Sorry for spammin the comments box
    The script is working fine now..
    However would still like to know how to find links for old courses
    Want links for previous quarter’s courses

    • http://jonblog.wong.lan Jon

      They don’t exist anymore. You may want to archive this quarters classes before SCPD gets rid of them.

  • anonymous

    Hi,
    I want SCPD video lectures. I have no SCPD account :(

    • http://jonblog.wong.lan Jon

      Sorry, I can’t help you.

  • http://n/a daemeon271

    I need some help with the GreaseMonkey script. What exactly do you mean when you say, “Go to the current quarter SCPD page and click on each of the links” does this mean click each course page, or does it mean click each link for each video? How long does it take to generate the page full or urls, seconds, minutes?

    • http://jonblog.wong.lan Jon

      usually several seconds to a few minutes

  • Mark

    I am SCPD student.

    Is there any way to access archived lecture videos?

    • http://jonblog.wong.lan Jon

      If SCPD gets rid of the videos so that the url’s don’t work there is no way to grab them.

  • Anonymous

    Hi Jon,
    Thanks so much for your wonderful idea. I am also a Stanford student. And recently when I wanted to watch the summer 2009 classes, they have taken the classes offline already. Could you please give me the links of those classes? Thank you so much!

    • http://jonblog.wong.lan Jon

      Unfortunately I cannot, since it’s against school policy.

  • Brandon

    I am not smart on greasemonkey and spent alittle bit of time trying to debug, but
    I get the following error with the script:

    Error: links.snapshotItem(i) is null
    Source File: file:///C:/Users/Brandon/AppData/Roaming/Mozilla/Firefox/Profiles/naq3ts1d.default/gm_scripts/scpd/scpd.user.js
    Line: 18

    Not sure if it is a issue with the script or something with my configuration…

    • http://jonblog.wong.lan Jon

      Probably it’s the scripts fault. It has to do with the xpath URL that points to the links. UCLA probably changed the website.

  • http://www.guyslikedolls.com Joey

    Hi –
    The greasemonkey script didn’t work out of the box because Stanford has modified the page format. But I was able to update two lines in the script and it worked beautifully, so wanted to share the changes with other script users. Here are the updated lines:

    links = findXPathNodes(“/html/body/div/div[2]/div/div/div[2]/div/div/table/tbody/tr/td/a[text()='WMP']“);

    var tempurl=client.responseText.substring( client.responseText.indexOf(“data=\”h”)+6, client.responseText.indexOf(“.wmv\””) +4) +”\n”;

    Thanks for the great script!

    – joey

    • http://jonblog.wong.lan Jon

      Thanks. I will take a look at that. I haven’t used it since the end of the quarter.

  • Jen

    @Joey
    Hi Joey, can you upload the full script? I edited the script, but it still doesn’t work for me. Thanks!

    • http://jonblog.wong.lan Jon

      I’ll take a look at it in the next few hours and update the script and the page.

  • Jen

    @Jon
    Thanks!

  • Allan

    Jon, I can confirm that your trick does NOT work anymore. It is now 24 March 2012, 1am PDT and THEY’VE ALREADY TAKEN DOWN THE VIDEOS for Winter Quarter 2012 (which officially ended 1 HOUR AGO)!! I had a window open from before midnight, clicked on a link, and got this error message: “The resource you have requested is not available. This class has not begun or it is no longer available after the end of the current Quarter.” F#$%*!! I really needed something from the lecture videos, and there was no way I could have downloaded everything in the few hours I had after my finals.

    • http://jonblog.wong.lan Jon

      I guess you need to download the videos while they are still up. Obviously if you cannot watch/access the videos, it will not work. Also it is not a “trick”. The script mainly does what your computer does. To be fair, it should not be the scripts fault that you were not able to download the videos. You technically have had the whole quarter to do so. I understand you are frustrated, but you should understand it’s technically up to you to save your videos.

  • Allan

    By “trick” I was actually referring to getting the direct links to the videos, not downloading entire copies of them onto my hard drives.

    You said “It seems that the SCPD program keeps the lectures archived at these links for a pretty long time. At least for a quarter or two. (However I once googled for SCPD links and found an EE one from 07 and it still worked, so perhaps SCPD never really erases them.)”

    This is not the case anymore. The links stop functioning after the quarter has ended.

    • http://jonblog.wong.lan Jon

      Yes. That’s happened for several years. It’s in the comments somewhere in one of the SCPD posts. I don’t actually take SCPD classes anymore, so I’m just maintaining the working scripts. Not all of the videos disappear, last time I checked last year. It depends on the server, of which there are several. The newer ones seem to all be from one of them so after the quarter you are out of luck. Everything seems to work fine before the end of the quarter (although I do not check every quarter). You should grab the videos beforehand, anyways. I believe there is some script somewhere on the blog that will do a list of links for you. You could easily just leave it running during finals week and grab a whole class.

  • Allan

    I’ve confirmed with the SCPD office that as of Winter 2012, videos are DELETED at the end of each quarter to make room for new ones. The SCPD office does not store any backups of the old videos.

    I’ve learned my lesson. I’ll begin downloading videos on a regular basis this quarter. You should revise your post and note that copying the links no longer works.

    • http://jonblog.wong.lan Jon

      Thanks for your “information”, but at this point, I won’t be replying to your comments anymore, unless you actually find out something new. I have mentioned that your “supposedly” new finding, has been known for awhile. The purpose of copying all the links, is and was, back then to download all of the links such that you could download all of the videos before they (the links AND videos) were taken away. While initially, I mentioned that the videos stay around, it has been known through the other comments that the videos are no longer available after the quarter officially ends. Given this information, the logical conclusion is NOT that the extractor no longer works as you have claimed. If the script still extracts links, it has done it’s job. It does not work for you, because you didn’t download the videos using the links in the time frame necessary, which is PURELY your issue. I also do not plan on making any revisions unless the scripts don’t work.