COMPLETED

0.1.1

  • Documentation and the whole project tree updated. No functional improvements.

  • Corrected typos.

0.1.0

  • Notion of binary tokenizer and a library for embedded tokenization.

  • Separation of punctuaction marks.

0.0.1

  • Simple tokenization is desired.

PLANNED

0.2.0

0.3.0

0.4.0

0.5.0

0.6.0

0.7.0

0.8.0

0.9.0

1.0.0