Enum StandardToken 

pub enum StandardToken<Char: Character, CS: CSName<Char>> {
    ControlSequence(CS),
    Character(Char, CommandCode),
    Primitive(PrimitiveIdentifier),
}

The simplest (but not most efficient) way to represent a Token as an enum.

Is Copy iff CS is Copy.

Variants

ControlSequence(CS)

Character(Char, CommandCode)

Primitive(PrimitiveIdentifier)

Trait Implementations

impl<Char: Clone + Character, CS: Clone + CSName<Char>> Clone for StandardToken<Char, CS>

fn clone(&self) -> StandardToken<Char, CS>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<Char: Debug + Character, CS: Debug + CSName<Char>> Debug for StandardToken<Char, CS>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<Char: Character, CS: CSName<Char>> PartialEq for StandardToken<Char, CS>

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<Char: Character, CS: CSName<Char>> Token for StandardToken<Char, CS>

type CS = CS

The CSName type used for control sequence names (e.g. Rc<str> or something interned).

type Char = Char

The Character type for char/catcode-pair tokens.

fn to_enum(&self) -> StandardToken<Char, CS>

Converts to the canonical enum representation of a token, i.e. StandardToken.

fn from_cs(cs: CS) -> Self

Create a new token from a control sequence name.

fn space() -> Self

Create a new space token.

fn eof() -> Self

Create a new end-of-file token.

fn from_char_cat(c: Char, cat: CommandCode) -> Self

Create a new character token with given CommandCode (i.e. conceptually the CategoryCode).

fn primitive(id: PrimitiveIdentifier) -> Self

Create a new token representing a primitive Command.

fn argument_marker(i: u8) -> Self

Create a new argument marker token. i needs to be in the range 0..=8.

fn char_value(&self) -> Option<Self::Char>

The Character value of this token, if it is a character token.

fn command_code(&self) -> CommandCode

The CommandCode (i.e. conceptually the CategoryCode) of this token.

fn is_cs_or_active(&self) -> bool

Check if this token is a control sequence or an active character

fn is_cs(&self, name: &Self::CS) -> bool

Check if this token is a control sequence with the given name.

fn is_primitive(&self) -> Option<PrimitiveIdentifier>

fn is_argument_marker(&self) -> Option<u8>

Check if this token is a argument token, and if so, return its number (in the range 0..=8).

fn display_fmt<W: Write>( &self, int: &<Self::CS as CSName<Self::Char>>::Handler, cc: &CategoryCodeScheme<Self::Char>, escapechar: Option<Self::Char>, f: &mut W, ) -> Result

Display this token to a writer, using the given CSHandler (in case it is a control sequence). In that case, we also need the current \escapechar to optionally insert it in front of the control sequence name, and the current CategoryCodeScheme to determine whether or not to insert a space afterwards - which we do unless the control sequence name is a single character with any CommandCode other than Letter.

fn display<'a>( &'a self, int: &'a <Self::CS as CSName<Self::Char>>::Handler, cc: &'a CategoryCodeScheme<Self::Char>, escapechar: Option<Self::Char>, ) -> DisplayToken<'a, Self>

Returns a helper struct implementing Display for this token.

impl<Char: Copy + Character, CS: Copy + CSName<Char>> Copy for StandardToken<Char, CS>

impl<Char: Eq + Character, CS: Eq + CSName<Char>> Eq for StandardToken<Char, CS>