ThingsLeft

list of things that we need to do between now and the end of the evaluation

Put "DONE" in front of things once they're done (and/or move them to the DONE section). Let's try to keep the list comprehensive and up to date!

before the test data deadline

• DONE

before the paper deadline (or later)

• consider moving to public github because they're talking about cutting access to iu github.
• try training L2 classifiers with the estimated labels instead of the real ones
• write the paper (due April 9)
• do the Joshua Baseline. What if that beats everything else?
• OPTIONAL to be extremely rude, we could also do a Google Translate baseline.
• OPTIONAL consider SVMs or other classifiers?

---

done things, could be revisited for later.

• DONE pickle all the classifiers we need
• DONE (or at least done enough) how many iterations to use for the MRF? Does it matter?
• SKIPPED (done enough?) what value should we use for unseen sense pairs in the MRF inference?
• DONE EVALUATE ON THE NEW TEST DATA, WHICH WE HAVE. DUE FRIDAY MARCH 8, 3PM EST.
• DONE don't forget to actually upload the results of the eval. on the ftp server, y'know.
• DONE decide on the feature set that we're going to use (use the bigram features)
• be able to evaluate for Two Level classifiers
  • same for One Level: need features and pickled one-level classifiers
  • also need to have extracted training data for two-level
  • also need to pickle two-level classifiers
  • script to run all experiments should be very similar
• be able to evaluate for MRFs
  • DONE same for One Level: need features and pickled one-level classifiers
  • DONE need to finish code to plug in edge and unary potentials into MRF.
  • DONE script to save all the outputs once we solve the five variables jointly.
• plug the classifiers and the correlations into the markov network
• implement the "two level" classifiers Can Liu
  • the output of the four other classifiers is used as a feature in the "second level" classifiers.
  • DONE open question: for the training set, is the feature set to the actual label from the other languages, or the predicted one? (answer: actual labels)
  • pickle the level two classifiers
• DONE be able to evaluate for One Level classifiers
  • DONE depends on complete feature set
  • DONE depends on script to train and pickle all classifiers (having been run)
  • DONE need to have script save output in some sensible way (look up the file names they want)
• DONE WEDNESDAY AFTERNOON: get the new test data.
• DONE better feature engineering (Read Els again)
  • DONE understand all the features FeaturesFromEls
• convince ourselves that our extracted training data is good enough
  • make sure that it can find all or almost all of the gold standard labels
• DONE try megam maxent classifier
• DONE write to the list and get an idea about the timeline
• get better alignments
  • DONE run aligner for more iterations
  • PROBABLYSKIP symmetrize alignments
  • DONE asked Els for manual alignments (she'll release them after the contest)
    • SKIP we could try to do them ourselves?
  • OPTIONAL use parser to get better alignments
  • this might be done: the problem probably wasn't with alignments, but our interface to TreeTagger
• correlation counts have '' in it, should ignore them when reading the file.
• DONE write the script to extract all training data
• DONE learn the correlations for use with the markov network Can Liu
• DONE need both joint prob, and conditional prob, make them as two versions... Can Liu
• DONE write the script to train all first-level classifiers, and pickle thenCan Liu