Enum StandardToken

pub enum StandardToken<Char, CS>
where
    Char: Character,
    CS: CSName<Char>,
{
    ControlSequence(CS),
    Character(Char, CommandCode),
    Primitive(PrimitiveIdentifier),
}

The simplest (but not most efficient) way to represent a Token as an enum.

Is Copy iff CS is Copy.

Variants

ControlSequence(CS)

Character(Char, CommandCode)

Primitive(PrimitiveIdentifier)

Trait Implementations

impl<Char, CS> Clone for StandardToken<Char, CS>

where Char: Clone + Character, CS: Clone + CSName<Char>,

fn clone(&self) -> StandardToken<Char, CS>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<Char, CS> Debug for StandardToken<Char, CS>

where Char: Debug + Character, CS: Debug + CSName<Char>,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<Char, CS> PartialEq for StandardToken<Char, CS>

where Char: Character, CS: CSName<Char>,

fn eq(&self, other: &StandardToken<Char, CS>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<Char, CS> Token for StandardToken<Char, CS>

where Char: Character, CS: CSName<Char>,

type CS = CS

The CSName type used for control sequence names (e.g. Rc<str> or something interned).

type Char = Char

The Character type for char/catcode-pair tokens.

fn to_enum(&self) -> StandardToken<Char, CS>

Converts to the canonical enum representation of a token, i.e. StandardToken.

fn from_cs(cs: CS) -> StandardToken<Char, CS>

Create a new token from a control sequence name.

fn space() -> StandardToken<Char, CS>

Create a new space token.

fn eof() -> StandardToken<Char, CS>

Create a new end-of-file token.

fn from_char_cat(c: Char, cat: CommandCode) -> StandardToken<Char, CS>

Create a new character token with given CommandCode (i.e. conceptually the CategoryCode).

fn primitive(id: PrimitiveIdentifier) -> StandardToken<Char, CS>

Create a new token representing a primitive Command.

fn argument_marker(i: u8) -> StandardToken<Char, CS>

Create a new argument marker token. i needs to be in the range 0..=8.

fn char_value(&self) -> Option<Self::Char>

The Character value of this token, if it is a character token.

fn command_code(&self) -> CommandCode

The CommandCode (i.e. conceptually the CategoryCode) of this token.

fn is_cs_or_active(&self) -> bool

Check if this token is a control sequence or an active character

fn is_cs(&self, name: &Self::CS) -> bool

Check if this token is a control sequence with the given name.

fn is_primitive(&self) -> Option<PrimitiveIdentifier>

fn is_argument_marker(&self) -> Option<u8>

Check if this token is a argument token, and if so, return its number (in the range 0..=8).

fn display_fmt<W>( &self, int: &<Self::CS as CSName<Self::Char>>::Handler, cc: &<Self::Char as Character>::CharMap<CategoryCode>, escapechar: Option<Self::Char>, f: &mut W, ) -> Result<(), Error>

where W: Write,

Display this token to a writer, using the given CSHandler (in case it is a control sequence). In that case, we also need the current \escapechar to optionally insert it in front of the control sequence name, and the current CategoryCodeScheme to determine whether or not to insert a space afterwards - which we do unless the control sequence name is a single character with any CommandCode other than Letter.

fn display<'a>( &'a self, int: &'a <Self::CS as CSName<Self::Char>>::Handler, cc: &'a <Self::Char as Character>::CharMap<CategoryCode>, escapechar: Option<Self::Char>, ) -> DisplayToken<'a, Self>

Returns a helper struct implementing Display for this token.

impl<Char, CS> Copy for StandardToken<Char, CS>

where Char: Copy + Character, CS: Copy + CSName<Char>,

impl<Char, CS> Eq for StandardToken<Char, CS>

where Char: Eq + Character, CS: Eq + CSName<Char>,