Author Archive

Group Think

Now that the warm-up is finally over, it’s time to roll up my sleeve and start tearing Web-CAT apart down to its database core. Next on the agenda is to integrate partner and group associations into assignment submissions. The plan is to add the ability for users to setup groups and use it to collectively organize individual student submissions.

Currently, partner associations in Web-CAT is – excuse my language – practically unusable. For one thing, partners are done on a per-submission basis and only has a one-way relationship, which means that student A’s submission might be partnered up with student B, but student B’s submissions aren’t. This can both be tedious and can create duplicates, where it’s possible to have two groups with the same members. Also, the fact that the grader is responsible for specifying partners manually (without any automation tools) would certainly make a grown man (or woman) cry if done on a course with 300 students on a span of typically four assignments.

Overhauling the entire workflow on group submissions is a daunting task, which is what I’ve been doing for the past week. Since Karen is away, I’ve been working closely with Jennifer Campbell and Paul Gries to solicit very useful feedback on what they think would be useful for courses that involve group assignments. With the design we have so far, students take charge of creating their groups instead of the instructors, based on an invite model similar to Facebook. However, groups would be decentralized, meaning that an actual Group entity doesn’t exist, and is only formed according to students requesting and accepting invites. Implementing it this way allows us to impose restrictions such as no student can create and be in more than one group.

One of the major difficulties right now is handling group merging and taking into account all the different cases of how invites and joins should be handled. Putting control of group assignments in student’s hands involves some serious restrictions on what they’re allowed and not allowed to do. Privacy and sharing issues also presents problems that we’re looking into as well. The good news is that we should be able to reuse some of the underlying database for submissions to merge user submissions into group submissions. However this would still take a lot of non-trivial work to fully implement, while considering how our changes would affect existing Web-CAT installations. There’s a lot more to figure out along the way but with what we have so far, I think Web-CAT has a promising and bright future to look forward to.


July 10, 2008 at 6:51 pm 1 comment

Complete and Release

After a very eventful meeting with Gene Amdur last week, we’ve finally figured out what to do for the rest of the summer.  Karen decided to switch Qi and I, where Qi will now be working on the Eclipse plug-in while I get the reins of Web-CAT, and everything else that comes with it.

We’ve decided to tread slowly, fixing minor bugs to get us familiarized with the innards of Web-CAT.  This week, I’ve fixed the bugs regarding releasing of assignment results and viewing permissions of students.  Apparently, students can see comments and marks added by the TA that is still being marked and has not been released yet (a big no-no).  Interestingly enough, even if marks added by the TA are not explicitly shown if assignments are unreleased, it is still included in the calculation of final mark shown when students view their feedback, which is just a matter of simple arithmetic to find out the “hidden” TA mark.

Next week, I would then start doing more major surgery on Web-CAT, now adding a complete and release mechanism to marking assignments, typical to the way instructors here are used to OLM.  This would provide capabilities for markers to change the status of the student as complete, indicating that it is ready to be released by the instructor for student viewing.  We would then transfer the release capability to the instructor, rather than the TA, of making the completed submission result available to students.  This would involve more changes internally as it would mean adding attributes to the tables that is accessible only through WebObject’s EOF – which is like SQLAlchemy in python, except that it’s meaner and eats children.

Another problem that I have is providing granular restrictions on components in a WebObject page.  As far as we know, Web-CAT only provides access level restrictions on a per-page level and not on individual components.  What this probably means is that we can’t have conditions on components like “display this button only to instructors”, but rather create a whole new page with just that one change and put restrictions accordingly on the page.  This still requires investigation that I’m looking forward to next week.

June 21, 2008 at 4:23 am 1 comment

SWT Error in Eclipse on Firefox 3 RC1 update

Apparently, Eclipse has this weird dependency on Firefox through the SWT library.  After upgrading to Firefox 3 RC1 on Ubuntu 8.04 due to Firebug not working, Eclipse started throwing an SWT error with the following log in <workspace>/.metadata/.log file:

Failed to execute runnable (org.eclipse.swt.SWTError:
No more handles (java.lang.UnsatisfiedLinkError:
no swt-mozilla-gtk-3349 or swt-mozilla-gtk in 
swt.library.path, java.library.path or the jar file))

Luckily, I’m not alone and suggests solving the problem by typing:

sudo apt-get install xulrunner


sudo apt-get install xulrunner-gnome-support

on console, restart Eclipse, and problem solved.

June 18, 2008 at 5:34 pm Leave a comment

Bitten by the Version Bug

For the past week, I’ve been struggling with getting OLM installed in Ubuntu, while Qi struggles with getting Web-CAT built. Every time we figure out what’s wrong, it seems that the culprit is always the same: versions.

OLM documentation isn’t that bad, except that they forgot to include some of the dependencies involved. And it turned out that this is very important to make it run. The first mistake that I made was blithefully ignoring the current version of python Ubuntu was hooked on (which was 2.5) and proceeded to install it on the older version 2.4. Once I got to the point of installing modules, I then realize that using ‘apt-get install’ (which was another mistake on its own) installs the modules on the latest version. Turns out that you can specify which version of python you want to use by editing debian-defaults file and specify the version to use. This is when I’ve also discovered easy_install which makes life a bit rosy when installing modules, especially on a specific version. Just execute to install it on a specific version of python and voila! you got python modules at your fingertips.

Once I got out of the apt-get hell, I then ran into the problem of compiling one of the modules that I need: PyXML. Turns out that it needs another python module (python-dev) that I couldn’t figure out how to install on 2.4 Edit: turns out you can do this with python<version>-dev. After giving up, I finally decided to manage my own python installation (building from scratch) and installed everything in it. I then override my $PATH with my custom python installation. At the end of the day, I finally got OLM up and running (even took me a while to figure out the default authentication to log in), with one exception: nothing was running. This was frustrating after a few days of wrestling through it.

After some laborious debugging, I finally tracked down the error: the version of TurboJson I was using (1.1.1) was in conflict with OLM. Even if the OLM doc specified to use TurboGears, installing TurboGears have other module dependencies and that it automatically downloads all the latest versions. The worse part is that OLM hides the error with its AJAX front-end so there was no way to see what’s going on without debugging (firebug not working well with FF3 is another story). Turns out latest version of TurboJson is throwing some AmbiguousMethod that can only be resolved by either changing the OLM source or downgrading TurboJson. So I chose the latter, and everything seems to be working like it should be.

With all this, I think it’s safe to say that we programmers still have a long way to go in resolving dependencies that do not break on every upgrade. The terms design-by-contract and program-to-interface always comes to mind on these occasions. For now the best remedy is to document everything, until someone comes up with a solution to make dependencies resolve itself on its own and make them, as Greg always says, either famous or filthy rich.

May 29, 2008 at 11:07 pm 1 comment

So…what is Web-CAT?

In the spirit of preparation for our Web-CAT demonstration this Friday, I figure I’d try to sum up what we’re trying to do this summer to our non-existent readers out there. If you want the more official answer, then you should go here but succinctly put, Web-CAT (Web-based Center for Automated Testing) is a way for instructors to setup an automated marking portal for students to submit their assignment to.

What’s nice about it is that students get an almost instantaneous feedback as to whether their code is up to par with the expected test results.  We have only tested it for 5 stub users so far, so we have no idea how scalable it is, but apparently it is being used in several universities/colleges already.  Also, plug-ins are also available to submit assignments within popular IDEs such as Eclipse and BlueJ, which means students have less excuse to procrastinate while switching windows to submit through the browser (no more “Firefox made me do it” excuses).

So, why did we pick Web-CAT as a project to work on this summer? Well, a few years back, one of the course projects here at U of T was an Online Marking Tool (OLM), which is a way for students to get a more sophisticated feedback from instructors.  More importantly, it also allows easy marking for TAs by providing tools such as easy-to-click rubric, inline code comments, etc which helps them avoid throwing student code print-outs in the air and catching them to see who gets the A+ in the class.

As far as I know, the plan this summer is to pick out the features that are “best-of-both-worlds” and integrate them into this one supercool application that hopefully will take over the world someday.  Hopefully, we can start off this iteration with a bang and to see what we can come up with over the next several months.  And while we’re at it, maybe we’ll try to create an app that does the studen’ts assignments for them as well.

May 15, 2008 at 5:42 pm 2 comments

Time Machine

August 2019
« Aug    

RSS Qi’s Utterances

  • An error has occurred; the feed is probably down. Try again later.