void HwcmScorer::extractHeadWordChain(TreePointer tree, vector<string> & history, vector<map<string, int> > & hwc) { if (tree->GetLength() > 0) { string head = getHead(tree); if (head.empty()) { for (std::vector<TreePointer>::const_iterator it = tree->GetChildren().begin(); it != tree->GetChildren().end(); ++it) { extractHeadWordChain(*it, history, hwc); } } else { vector<string> new_history(kHwcmOrder); new_history[0] = head; hwc[0][head]++; for (size_t hist_idx = 0; hist_idx < kHwcmOrder-1; hist_idx++) { if (!history[hist_idx].empty()) { string chain = history[hist_idx] + " " + head; hwc[hist_idx+1][chain]++; if (hist_idx+2 < kHwcmOrder) { new_history[hist_idx+1] = chain; } } } for (std::vector<TreePointer>::const_iterator it = tree->GetChildren().begin(); it != tree->GetChildren().end(); ++it) { extractHeadWordChain(*it, new_history, hwc); } } } }
string HwcmScorer::getHead(TreePointer tree) { // assumption (only true for dependency parse: each constituent has a preterminal label, and corresponding terminal is head) // if constituent has multiple preterminals, first one is picked; if it has no preterminals, empty string is returned for (std::vector<TreePointer>::const_iterator it = tree->GetChildren().begin(); it != tree->GetChildren().end(); ++it) { TreePointer child = *it; if (child->GetLength() == 1 && child->GetChildren()[0]->IsTerminal()) { return child->GetChildren()[0]->GetLabel(); } } return ""; }
FFState* TreeStructureFeature::EvaluateWhenApplied(const ChartHypothesis& cur_hypo , int featureID /* used to index the state in the previous hypotheses */ , ScoreComponentCollection* accumulator) const { if (const PhraseProperty *property = cur_hypo.GetCurrTargetPhrase().GetProperty("Tree")) { const std::string *tree = property->GetValueString(); TreePointer mytree (boost::make_shared<InternalTree>(*tree)); //get subtrees (in target order) std::vector<TreePointer> previous_trees; for (size_t pos = 0; pos < cur_hypo.GetCurrTargetPhrase().GetSize(); ++pos) { const Word &word = cur_hypo.GetCurrTargetPhrase().GetWord(pos); if (word.IsNonTerminal()) { size_t nonTermInd = cur_hypo.GetCurrTargetPhrase().GetAlignNonTerm().GetNonTermIndexMap()[pos]; const ChartHypothesis *prevHypo = cur_hypo.GetPrevHypo(nonTermInd); const TreeState* prev = dynamic_cast<const TreeState*>(prevHypo->GetFFState(featureID)); const TreePointer prev_tree = prev->GetTree(); previous_trees.push_back(prev_tree); } } if (m_constraints) { m_constraints->SyntacticRules(mytree, previous_trees, this, accumulator); } mytree->Combine(previous_trees); bool full_sentence = (mytree->GetChildren().back()->GetLabel() == m_send || (mytree->GetChildren().back()->GetLabel() == m_send_nt && mytree->GetChildren().back()->GetChildren().back()->GetLabel() == m_send)); if (m_binarized && full_sentence) { mytree->Unbinarize(); } return new TreeState(mytree); } else { UTIL_THROW2("Error: TreeStructureFeature active, but no internal tree structure found"); } }
// define NT labels (ints) that are mapped from strings for quicker comparison. void TreeStructureFeature::AddNTLabels(TreePointer root) const { std::string label = root->GetLabel(); if (root->IsTerminal()) { return; } std::map<std::string, NTLabel>::const_iterator it = m_labelset->string_to_label.find(label); if (it != m_labelset->string_to_label.end()) { root->SetNTLabel(it->second); } std::vector<TreePointer> children = root->GetChildren(); for (std::vector<TreePointer>::const_iterator it2 = children.begin(); it2 != children.end(); ++it2) { AddNTLabels(*it2); } }