Home > Education, Releases > Download Stanford SCPD streams (wmv streams)

Download Stanford SCPD streams (wmv streams)

November 7th, 2008 Leave a comment Go to comments

Since I am currently attending Stanford, and I once searched the intraweb for a solution for downloading wmv streams, but couldn’t find anything specific, I’ll post my solution here.

Normally, SCPD requires you to login and then you watch the videos streamed using some wmv player. However, should you want to rewind or seek through a stream, this will prove to be impossible. Many commercial wmv recorders are not free, and simply play the video to download it. My solution is similar to this, but it is FREE (as in beer).

What you need:

  1. Mencoder (An encoder that comes along with Mplayer) If you use windows, you should get the command-line non-gui download from the website since it will come with mencoder. Otherwise it will not.
  2. Optional: Firefox + Greasemonkey for extracting the video URL if you’re lazy like me.
  3. Optional: Python for executing mencoder and for converting http:// links to mms:// links.

To download from SCPD (or other unprotected wmv streams), simply execute the following at a command prompt:

mencoder mms://stream.wmv -ovc copy -oac copy -o output.avi
  1. Replace mms://stream.wmv is simply the url with “http:” replace with “mms:”
  2. Replace output.avi with the output file name. Extensions should govern the format that it spits out.

It will throw a bunch of text to the screen, and you simply leave the window open (or minimized) and wait for it to finish. However, since it’s free, I commonly download several streams at the same time, so when they are done I will have several streams downloaded instead of just one.

Download: Greasemonkey script for SCPD (gets links automatically when you view an SCPD video) (Updated 9/29/09)

If you are lazy like me, and don’t want to change the http://url to mms://url I’ve provided an mencoder script that, given an http url, will fill out the commandline and execute it for you. (Note: Many mac and linux users will have python already installed on their machines.)

Usage:

./stripSCPD http://url

Download: Python SCPD script

#!/usr/bin/python
 
import subprocess,sys
 
def strip(url):
    newurl = "mms"+url[url.find(":"):]
    print newurl
    vidname = url[url.rfind("/")+1:url.rfind("?")]
    print "mencoder %s -ovc copy -oac copy -o %s.avi" % (newurl,vidname)
    subprocess.Popen(["mencoder", newurl, "-ovc","copy","-oac","copy","-o", ("%s.avi" % (vidname,))])
 
if __name__=="__main__":
    strip(sys.argv[1])

Update (11/09/08):

I’ve just tried everything again in windows, and it seems to work. Note, for some reason mplayer doesn’t have the necessary codecs by default in the downloadable zip file. However, VLC, media player classic and Windows media Player seem to be able to seek. However only media player classic can increase the audio speed (in case you want to watch a stream 1.5 times fast and be able to comprehend what someone is saying even if they seem to be on helium!).

  • Another SCPD

    Hi Jon,
    Congratulations! You are very talented. I tried my best to download SCPD videos, but couldn’t manage that.
    Still, I didn’t understand how to download videos. I tried on a Windows and on a Mac, but couldn’t get anything.
    First, which file do I have to download for Windows? Can you give exact link? Also, a link for Mac would be great.
    Second, when I write “mencoder …..” into the “run” window at Windows, it says that “Windows cannot find encoder”.
    When I click on the encoder.exe, nothing happens.
    Would you please help?
    Thanks a lot.

  • http://jonblog.wong.lan Jon

    I don’t own a mac, so I honestly couldn’t tell you.

    For windows, the non-gui version of mplayer contains mencoder. http://www.mplayerhq.hu/MPlayer/releases/win32/MPlayer-mingw32-1.0rc2.zip

    You have to have mencoder in order to download the videos with the scripts.

  • http://blog.fogel.ca Mike

    Thanks so much! I was not looking forward to getting this figured out myself.

    The only thing I’d add is if you’re running debian, you can use the debian-multimedia repository to get mencoder and if you need it the wmv codec.

    http://debian-multimedia.org/dists/testing/main/binary-i386/package/mencoder.php
    http://debian-multimedia.org/dists/testing/main/binary-i386/package/w32codecs.php

  • Another SCPD

    Jon,

    Thanks for the blog.
    I’ve been successful in streaming lectures in WMP format.
    Do you know of a solution to pull down lectures in SL format?

    It’d be nice to be able to stream the free seminars which appear to get invoked in SL format only. Any ideas?

    • http://jonblog.wong.lan Jon

      Seems that the SilverLight player on the SCPD previews page points to a wmv stream. So I’d just download the wmv stream, since that is what it seems to be streaming anyways.

  • Ups

    Jon, have you tried downloading any lectures recently? I’ve been trying to download some recently, but have been having a lot of trouble. Neither MEncoder nor the other recorders out there seem to work anymore. It looks like SCPD has tightened things up…

    • http://jonblog.wong.lan Jon

      I just tried downloading a video and it seems to work perfectly fine. Typically I just use the python script I’ve posted above. I also generally download using linux, but I’ve tested it on windows periodically.

      Check to make sure your URL’s are correct. I haven’t tried silverlight videos, but the wmv’s should work fine. When I get some time later today, I may post a greasy script that grabs URL’s in case Stanford students feel like archiving links to download later.

  • Ups

    hmmm…I don’t know what to say…

    I did the exact same thing this quarter that I had been doing for the past year and all of a sudden everything stopped working. So strange!

  • Thai

    Hi Jon,

    Your script is very helpful. Instead of mencoder, you can use mimms btw.

    PS: It seems that there’s no authentication in SCPD. Anybody knowing the links can download the videos. Has anybody tried to figure out the format of the links?

  • http://jonblog.wong.lan Jon

    @Thai
    I have tried using mimms, perhaps I passed it off prematurely (Only spent a few minutes trying to get it to work in the beginning). I use mencoder because almost everyone has it, since it comes with mencoder. It was just more convenient for me to use on Windows, Linux, Mac.

    I’m aware of the “authentication” issue. This is probably because it is troublesome to implement authentication with wmv streams. They can only use authentication to obscure the links. There is no format for the SCPD videos since the beginning of Fall 08, because they started adding nasty hashes to all of their videos. I suggest you try using the greasemonkey script I posted recently to rip links.

  • Ash

    Hi Jon,

    I have downloaded MPlayer using the link you posted (http://www.mplayerhq.hu/MPlayer/releases/win32/MPlayer-mingw32-1.0rc2.zip), and also added Grease Monkey to the firefox addons list. Could you please advise a detailed step-by-step approach on downloading scpd videos for computer novices :P. I clicked on mplayer.exe and mencoder.exe, but the command prompt opens and then closes instantly. How do we proceed after getting the MPlayer zip file and Grease Monkey? I am using Win-XP home and do not have Python.

    Thanks a lot!

  • unsown

    Hi Jon,

    Thanks for your sharing this method. As you have pointed out, the SCPD video has now been protected by certain hash method. I wondered if the links are still static. If it’s so, is it possible that you could phase the links for some courses for me? I really want to watch some of these video, and I’m not intending to get the credits. Thank you very much!

    Jon :
    I just tried downloading a video and it seems to work perfectly fine. Typically I just use the python script I’ve posted above. I also generally download using linux, but I’ve tested it on windows periodically.
    Check to make sure your URL’s are correct. I haven’t tried silverlight videos, but the wmv’s should work fine. When I get some time later today, I may post a greasy script that grabs URL’s in case Stanford students feel like archiving links to download later.

    • http://jonblog.wong.lan Jon

      Nope, I can’t repost links, since it’s against SCPD policy. For the record, the links to my knowledge are not static. Last I checked, they no longer work at the end of the quarter.

  • zman

    Thanks for posting the greasemonkey script and the instructions! What is the purpose of the ffmpeg step I see in the comments? (ffmpeg -i output.avi -ar 44100 -vcodec copy -acodec mp2 output2.avi) Are you trying to shrink the file size? If so, what are the tradeoffs as far as quality? Sorry if this is a dumb question but I’m not very familiar with audio/video formats and codecs.

    • http://jonblog.wong.lan Jon

      If it’s in the comments in the greasemonkey script, it must’ve been something I was simply trying out. I think I was probably trying to convert the mencoder video to a better conditioned avi file. I don’t think it makes a difference, which is why I didn’t bother including it.

  • Garibaldi

    Thanks man, this helped tremendously!

  • Austin Hill

    This seems to work really well– the one difficulty I’m having is outputing these to a format I can put on my iPhone through iTunes. Any suggestions there?

    • http://jonblog.wong.lan Jon

      Yes, convert to mp4. I don’t know if an iPhone can play it, but I know that quicktime supports it. Even my lesser phones seem to be able to handle mp4 so I’d assume an Iphone can handle it too. You can use ffmpeg to do it, or configure mencoder.

  • madler

    Great work! I would never have figured this out without your help.

    However, this is not completely working for me. I executed the Python script which does indeed fill out the command line and execute Mencoder, but Mencoder does not download the stream.

    This is the error that I get:

    Resolving myvideosu.stanford.edu for AF_INET…
    Connecting to server myvideosu.stanford.edu[171.67.216.22]: 1755…
    connection timeout
    Resolving myvideosu.stanford.edu for AF_INET…
    Connecting to server myvideosu.stanford.edu[171.67.216.22]: 80…
    Server returned 302: Found
    Failed to parse header.
    Failed, exiting.

    It then cycles back and forth indefinitely between the following two output statements:

    Resolving myvideosu.stanford.edu for AF_INET…
    Connecting to server myvideosu.stanford.edu[171.67.216.22]: 80…

    where the last address segment .22 may change to .21 or some other near value. I let it go for hours, to no avail.

    I’m not sure what I am doing wrong. I’ve tried sending other urls to the python script (such as youtube videos or other online video sites) and I get the same essential error.

    I should backup and state that I executed the python script by executing the command:

    strip(‘url’) where I manually pasted the url in between single quotes to force it into a string. If I try to simply Run the Module I get the error:

    Traceback (most recent call last)
    File “C:\*\stripSCPD.py”, line 14
    in
    strip(sys.argv[1])
    IndexError: list index out of range.

    So to my untrained non-expert eyes, it looks like it’s seeking an input argument (the url) but not getting one, which is why I tried to just call the function definition ‘strip’ as strip(‘url’).

    When I do so, it looks like the Python script works as advertised, since I get the following output which calls Mencoder:

    mencoder mms://myvideosu.stanford.edu/player/slplayer.aspx?coll=fd425772-6b73-40a0-a692-d4c08344c052&course=MSandE207&co=9f71d59f-02ae-48ce-b094-cc1f158bdf4a&lecture=100920%20&authtype=WA&lectureType=ps&wmp=true -ovc copy -oac copy -o slplayer.aspx.avi

    As an afterthought, it doesn’t look like the code is grabbing the correct video name (naming it slplayer.aspx.avi), but I think that’s moot for now.

    This is my first experience with Mencoder and Python, so sorry for being a bonehead. Do you see why I got no dice on my download?

    • http://jonblog.wong.lan Jon

      Which stream are you using? SL or WMV? The script was designed for WMV and not for SilverLight.

  • M

    Hi Jon,
    I am still a n00b and couldn’t really figure the whole script thing out, but I found my own way to download the lectures. My problem is that since 12:01am today the scpd links no longer point to the videos because the quarter ended. All the links I had still work, so the videos are there. Is there any way to figure out the urls of the rest of the videos? I can download them myself after that. I really need CS110 since I have to take it next quarter, but I can’t attend in person :(.
    Please help meeee.

    • http://jonblog.wong.lan Jon

      Nope. Unfortunately I can’t help you with this. I didn’t bother to save all the links for the classes either. Unless you know of a good way to backout hashes, it’s probably not possible to guess.

  • J

    @M
    I’ll also be taking cs110 next quarter and just realized the same thing that you did. Is there anyway you could send me the videos and links you already have. It would be greatly appreciated. My email is afroboy0731@gmail.com

    Thanks!

    • http://jonblog.wong.lan Jon

      I don’t have CS110 links since I didn’t take it this quarter. It’s best to just get it next quarter. In fact, someone could perhaps setup a link saving site that is Stanford Kereberos protected.

  • E

    what do you mean by output file name when you say “Replace output.avi with the output file name. Extensions should govern the format that it spits out.” I’ve been able to figure out most of your instructions, but I keep getting this msg from the cmd prompt: ” ‘mms:’ is not recognized as an internal or external command, operable program, or batch file.” I also have the same problems as Ash (above). Also, I’m using windows, so I don’t have Python, and have had difficulty figuring out what to do with it…

    • http://jonblog.wong.lan Jon

      1. Extensions do not govern the format that it spits out. This is merely a formality.
      2. Without python, execute the mencoder line. MMS:// is a URL using the MMS protocol, windows media player should be able to handle it. You can watch it in windows media player, you just can’t download it. Mencoder allows you to do that.
      3. I’ve run this script in windows using python. So I know it works. In fact, I’ve done it for all three operating systems.
      4. Do yourself a favor, just install python, it exists for Windows. Just save yourself the effort of having to type in the same thing over and over again. That’s the purpose of using the scripts. I plan to update it soon to handle batch scripting.

  • Student

    Hi,

    Has anyone tried this recently as of 3/17/2011 in Windows?

    Because I executed mencoder mencoder -ovc copy -oac copy -o output.avi
    and get the following:
    Resolving myvideosu.stanford.edu for AF_INET…
    Connecting to server myvideosu.stanford.edu[171.67.216.21]: 80…
    .
    .
    .
    Resolving myvideosu.stanford.edu for AF_INET…
    Connecting to server myvideosu.stanford.edu[171.67.216.21]: 80…
    connection timeout
    No stream found to handle url mms://
    Cannot open file/device.

    Exiting…
    ‘course’ is not recognized as an internal or external command,
    operable program or batch file.
    co: RCS/=433ec0f2-c886-4018-a359-e946c820cc2c,v: No such file or directory
    ‘lecture’ is not recognized as an internal or external command,
    operable program or batch file.
    ‘authtype’ is not recognized as an internal or external command,
    operable program or batch file.
    ‘wmp’ is not recognized as an internal or external command,
    operable program or batch file.

  • Student

    Hi Jon, thanks for providing these instructions. One issue that I keep running into with the SCPD downloads is that mencoder seems to only be able to grab/download the first 50 minutes or so of a lecture. Would you happen to know why that is and how to fix it?

  • E

    Jon, I seem to be having trouble with mencoder only downloading the first 50 minutes or so of lectures (when they’re 1.5 hrs long). any thoughts on why this is/how to fix this?

    • http://jonblog.wong.lan Jon

      Sorry, I haven’t run across this problem before. Can you watch the whole stream using mplayer (the wmv stream)? I’ve recorded 2 hour long lectures and haven’t had problems before. However, I haven’t been taking classes lately, so I haven’t actually tried downloading stuff myself.

  • ichad

    Hi Jon,

    I am facing below issue .Can you pls help ?

    mencoder “mms://…;course=EE271&lecture=111005&wmp=true” -ovc copy -oac copy -o output.
    vi
    MEncoder 1.0rc2-4.2.1 (C) 2000-2007 MPlayer Team
    CPU: Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz (Family: 6, Model: 23, Ste
    ping: 10)
    CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
    Compiled with runtime CPU detection.
    STREAM_ASF, URL: mms://…;authtype=OC
    &course=EE271&lecture=111005&wmp=true
    Resolving myvideos.stanford.edu for AF_INET…
    Connecting to server myvideos.stanford.edu[171.64.115.212]: 1755…
    connection timeout
    Resolving myvideos.stanford.edu for AF_INET…
    Connecting to server myvideos.stanford.edu[171.64.115.212]: 80…
    Server returned 302:Found
    Failed to parse header.
    Failed, exiting.
    Resolving myvideos.stanford.edu for AF_INET…
    Connecting to server myvideos.stanford.edu[171.64.115.212]: 80…
    Resolving myvideos.stanford.edu for AF_INET…
    Connecting to server myvideos.stanford.edu[171.64.115.212]: 80…
    Cache size set to 320 KBytes
    success: format: 0 data: 0x0 – 0x0

    ============ Sorry, this file format is not recognized/supported =============
    === If this file is an AVI, ASF or MPEG stream, please contact the author! ===
    Cannot open demuxer.

    Exiting…

    • http://jonblog.wong.lan Jon

      You cannot use the slplayer or silverlight link. You need to use the asf or the wmv links.

  • ichad

    Thanks Jon, Sometime download randomly fails on some header parsing issue . Is there a way to start downloading from in between

    • http://jonblog.wong.lan Jon

      I don’t think you can skip around quickly when using the wmv streaming. Sometimes it takes awhile for your media player to skip to a certain position, this is because it’s essentially loading the stream and waiting till it hits that point. If windows media player or vlc can skip to the location and play the stream, then you should likewise also be able to start downloading from that point. All mencoder is doing is saving the stream for you. It can only save the video as fast as the stream is willing to send the data over. There might be some ways of optimizing it or making it only start recording whence it gets to the correct location, but I haven’t tried experimenting with it. Just look at the mencoder command-line options and see if there is anything that suits your needs.

  • KP

    Hi all,

    I found that you can record the stream directly in VLC. Open the SL player and view the source code. Find the mms link and copy it. Open the File->Streaming/Exporting Wizard and choose transcode/save to file. Paste the mms link into the ‘select a stream’ field. For the codec I chose WMV 1 and for audio MPEG Audio. For encapsulation format I chose asf. Fifty minutes later you have a recorded video that you can play in VLC.

    • http://jonblog.wong.lan Jon

      While it is true that it is possible to use VLC instead of mencoder, the SL player stream and the WMP stream are actually the same. You can rewrite the greasemonkey script to automatically extract the mms url for you on the SL page too. If you look at the url in the WMP and SL page they actually have the same mms url you get from both the SL and WMP link. So you are downloading exactly same stream, merely using a different recording program, both of which are open source.

      Yes, with the updated VLC you may now be able to record wmv streams, but mencoder is more flexible in terms of recording, should you meet strange video streaming formats in hi-10 (10-bit encoding) or realplayer streams. Also the reason why mencoder was chosen initially was mainly because it was easier to script, and if you watch movies with mplayer, the fast forwarding features of mplayer are far more flexible and useful (i.e. can go past 1.77x) than VLC.

      Still thanks for investigating and verifying that it does work. VLC also works well for saving arbitrary RAW streams from websites. This was done for Starcraft 2 tournaments before they actively blocked the VLC player. Again it’s somewhere documented on this blog about how you do that.

  • MJ

    Hey Jon,
    I am a level zero noob when it comes to all these things. I have a macbook pro and desperately want to download videos off SCPD before they take then down. I tried following the instructions above but to no avail. Can you please help me by sending me ‘instructions for dummies’ to download these videos? Thanks a bunch!

  • MJ

    When I type in “mencoder mms://URL.wmv -ovc copy -oac copy -o Output.avi” in the terminal window in the macbookpro, it displays the message “mencoder: command not found”. Am I doing something wrong? Should I be changing some directory thing somewhere? @MJ

    • http://jonblog.wong.lan Jon

      It just means that mencoder is not in your “unix” PATH. You either need to find out where mencoder is located, and then put that directory in the UNIX path, or copy the location of mencoder and paste it into the commandline before mencoder.

      > /mencoder mms:…. -ovc copy … blah blah blah

      Once that works, modify the python script if you are using in a similar way.

  • wvxvw

    Hi Jon. I was trying to do something similar, another university in another country, but they too use proprietary and patented codecs… well, anyways, so what I was trying to do is to reduce the size of the video while it’s been downloaded. I tried to do something like this:

    mencoder “mms://video.wmv” … -fv scale=320:240

    but nothing happens. mencoder accepts this option, and, in fact, the manual says to do it this way, but the video is still 512×384

    Would you happen to have any idea if that is actually possible to do it in one go? (rather then download and resize later).

    Thanks!

    • http://jonblog.wong.lan Jon

      Some of the output container formats are restricted to certain file sizes. You could try using vlc instead of mencoder and use the rawfile option.

  • wvxvw

    Thanks for advise Jon. I searched for rawfile as you said, but didn’t find such option, however I found something that actually did what I wanted! So, not all is lost :)

    cvlc ‘mms://video.wmv’ –sout ‘#transcode{vcodec=mp2v,audio=mp2a,width=320,height=240}:standard{access=file,dst=movie.mpg}’

    so I could get it working, however, I’ve one nuisance now. The codec I’m trying to use is very unhappy with the wmv input it receives. It prints a lot of angry messages about it. Would you happen to know what other codecs may possibly do it better?

    Thanks!

    • http://jonblog.wong.lan Jon

      I don’t really have any input regarding this. Since you are just saving a stream, I’d worry about losing frames rather than the particular encoding. You can always convert it to something else later. Personally I would just keep it as wmv or just swap the container to avi.

  • Paul

    Hi Jon, I am also a zero-level noob. Been trying to download mencoder for my macbook pro, but couldn’t install it properly (with the ‘make’ command, it fails).

    It appears to me that MPlayer is designed for windows user and not mac.

    Do you have any recommendations as to which mencoder I should use? Thanks.

    • http://jonblog.wong.lan Jon

      You should be able to use VLC in pretty much the same way. There should be some comments on this post that talk about using vlc.

  • hao

    Hi, Jon.
    I use windows and don’t use the greasemonkey and python.
    after I enter the command, it shows:
    Connecting to server myvideosu.stanford.edu[171.67.216.23]: 80…
    Resolving myvideosu.stanford.edu for AF_INET… (these two lines repeatedly)
    Connecting to server myvideosu.stanford.edu[171.67.216.23]: 80…
    connection timeout
    No stream found to handle url mms://myvideosu.stanford.edu/player/slplayer.aspx?…
    Cannot open file/device.

    Exiting…

    and I use the wmp of the vedio, not the sl.
    could you please tell me what the problem is?
    Thank you very much!
    (The videos would be closed in two days but I really need them. Thank you)

  • http://jonblog.wong.lan Jon

    The problem is your link is incorrect. That’s why you need to use greasemonkey. If you really want the videos and you don’t use the scripts you will have to get the links manually. Open up the video page and search for “wmv” and copy the link. Change http to mms: and you are done.

  • hao

    @Jon
    Thank you very much!