Skip to content

LinguList/NNTransitionSegmentor

 
 

Repository files navigation

NNTransitionSegmentor

NNTransitionSegmentor is a package for Word Segmentation using neural networks based on package LibN3L. The current version is a re-implementation of segmentor in ZPar.
What is the transition-based framework with beam-search decoding? Please see our ACL2014 tutorial: Syntactic Processing Using Global Discriminative Learning and Beam-Search Decoding

Performance

Take averaged perceptron as an example (CTB6.0, please refer to LibN3L: A lightweight Package for Neural NLP for details):
Both ZPar and this package obtain performance about 95.08%;
The normal sparse model, with max-margin training, reaches a F-measure of 95.24%.

Compile

cmake .
make

Input data format

one line one sentence, with words seperated by spaces

Notice

  • one can remove the length and keyChar embeddings in my implementation to reproduce the results of my ACL paper, because the two kinds of embeddings induce little influences in the final performances, but later experiments I found that this can be more stable.
  • I will make the code more readable in the future. However if someone is interested in this framework, please concat me without hesitatation.

About

Transition-based word segmentation using neural networks based on package https://github.com/SUTDNLP/LibN3L

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 99.8%
  • Other 0.2%