public final class WesternTokenizer extends Tokenizer
Constructor and Description |
---|
WesternTokenizer(FTOpt fto)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
int |
compareTo(org.basex.util.ft.LanguageImpl o) |
boolean |
equals(Object o) |
int |
hashCode() |
boolean |
hasNext() |
WesternTokenizer |
init(byte[] txt)
Initializes the iterator.
|
FTSpan |
next() |
byte[] |
nextToken()
Returns the next token.
|
protected byte |
prec()
Returns the precedence of the processor.
|
boolean |
supports(Language ln)
Checks if the specified language is supported.
|
String |
toString() |
supportFor
remove
public WesternTokenizer(FTOpt fto)
fto
- full-text optionspublic WesternTokenizer init(byte[] txt)
FTIterator
init
in class FTIterator
txt
- textpublic boolean hasNext()
public FTSpan next()
public byte[] nextToken()
FTIterator
Iterator.next()
to avoid the creation of new FTSpan
instances.nextToken
in class FTIterator
protected byte prec()
public boolean supports(Language ln)
ln
- languagepublic final int compareTo(org.basex.util.ft.LanguageImpl o)
compareTo
in interface Comparable<org.basex.util.ft.LanguageImpl>
Copyright © 2005–2015 BaseX Team. All rights reserved.