Class: PragmaticSegmenter::Languages::Japanese::Cleaner

Inherits:
Cleaner
  • Object
show all
Defined in:
lib/pragmatic_segmenter/languages/japanese.rb

Constant Summary collapse

NewLineInMiddleOfWordRule =

Rubular: rubular.com/r/N4kPuJgle7

Rule.new(/(?<=の)\n(?=\S)/, '')

Constants inherited from Cleaner

Cleaner::ConsecutiveForwardSlashRule, Cleaner::ConsecutivePeriodsRule, Cleaner::DoubleNewLineRule, Cleaner::DoubleNewLineWithSpaceRule, Cleaner::EscapedCarriageReturnRule, Cleaner::EscapedNewLineRule, Cleaner::InlineFormattingRule, Cleaner::NEWLINE_IN_MIDDLE_OF_SENTENCE_REGEX, Cleaner::NO_SPACE_BETWEEN_SENTENCES_DIGIT_REGEX, Cleaner::NO_SPACE_BETWEEN_SENTENCES_REGEX, Cleaner::NewLineFollowedByBulletRule, Cleaner::NewLineFollowedByPeriodRule, Cleaner::NoSpaceBetweenSentencesDigitRule, Cleaner::NoSpaceBetweenSentencesRule, Cleaner::PDF_NewLineInMiddleOfSentenceNoSpacesRule, Cleaner::PDF_NewLineInMiddleOfSentenceRule, Cleaner::QuotationsFirstRule, Cleaner::QuotationsSecondRule, Cleaner::ReplaceNewlineWithCarriageReturnRule, Cleaner::TableOfContentsRule, Cleaner::TypoEscapedCarriageReturnRule, Cleaner::TypoEscapedNewLineRule, Cleaner::URL_EMAIL_KEYWORDS

Constants included from Rules

Rules::AbbreviationsWithMultiplePeriodsAndEmailRule, Rules::ExtraWhiteSpaceRule, Rules::GeoLocationRule, Rules::QuestionMarkInQuotationRule, Rules::SingleNewLineRule, Rules::SubSingleQuoteRule

Instance Attribute Summary

Attributes inherited from Cleaner

#doc_type, #text

Instance Method Summary collapse

Methods inherited from Cleaner

#initialize

Constructor Details

This class inherits a constructor from PragmaticSegmenter::Cleaner

Instance Method Details

#clean ⇒ Object 



16
17
18
19
 
# File 'lib/pragmatic_segmenter/languages/japanese.rb', line 16

def clean
  super
  @clean_text = remove_newline_in_middle_of_word(@clean_text)
end