public class PennTreebankTokenizer extends TokenizerAdapter
eolString, stNEWLINE_TOKEN, nextToken| Constructor and Description |
|---|
PennTreebankTokenizer(Reader r) |
getNext, isEol, setEolStringhasNext, next, peek, remove, tokenizeclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitforEachRemainingpublic PennTreebankTokenizer(Reader r)