C++ (Cpp) CreateTokenDictionary Examples

Programming Language: C++ (Cpp)

Method/Function: CreateTokenDictionary

Examples at hotexamples.com: 2

C++ (Cpp) CreateTokenDictionary - 2 examples found. These are the top rated real world C++ (Cpp) examples of CreateTokenDictionary extracted from open source projects. You can rate examples to help us improve the quality of examples.

Example #1

Show file

File: EntityPipe.cpp Project: xuerenlv/TurboParser

void EntityPipe::PreprocessData() {
    delete token_dictionary_;
    CreateTokenDictionary();
    static_cast<SequenceDictionary*>(dictionary_)->
    SetTokenDictionary(token_dictionary_);
    // To get the right reader (instead of the default sequence reader).
    token_dictionary_->InitializeFromEntityReader(GetEntityReader());
    static_cast<SequenceDictionary*>(dictionary_)->
    CreateTagDictionary(GetSequenceReader());
}

Example #2

Show file

File: MorphologicalPipe.cpp Project: FranklinChen/TurboParser

void MorphologicalPipe::PreprocessData() {
  delete token_dictionary_;
  CreateTokenDictionary();
  static_cast<SequenceDictionary*>(dictionary_)->
    SetTokenDictionary(token_dictionary_);
  // To get the right reader (instead of the default sequence reader).
  static_cast<MorphologicalTokenDictionary*>(token_dictionary_)->
    Initialize(GetMorphologicalReader());
  static_cast<MorphologicalDictionary*>(dictionary_)->
    CreateTagDictionary(GetMorphologicalReader());
}