syn

Module token

source
Expand description

Tokens representing Rust punctuation, keywords, and delimiters.

The type names in this module can be difficult to keep straight, so we prefer to use the Token! macro instead. This is a type-macro that expands to the token type of the given token.

§Example

The ItemStatic syntax tree node is defined like this.

pub struct ItemStatic {
    pub attrs: Vec<Attribute>,
    pub vis: Visibility,
    pub static_token: Token![static],
    pub mutability: Option<Token![mut]>,
    pub ident: Ident,
    pub colon_token: Token![:],
    pub ty: Box<Type>,
    pub eq_token: Token![=],
    pub expr: Box<Expr>,
    pub semi_token: Token![;],
}

§Parsing

Keywords and punctuation can be parsed through the ParseStream::parse method. Delimiter tokens are parsed using the parenthesized!, bracketed! and braced! macros.

use syn::{Attribute, Result};
use syn::parse::{Parse, ParseStream};

// Parse the ItemStatic struct shown above.
impl Parse for ItemStatic {
    fn parse(input: ParseStream) -> Result<Self> {
        Ok(ItemStatic {
            attrs: input.call(Attribute::parse_outer)?,
            vis: input.parse()?,
            static_token: input.parse()?,
            mutability: input.parse()?,
            ident: input.parse()?,
            colon_token: input.parse()?,
            ty: input.parse()?,
            eq_token: input.parse()?,
            expr: input.parse()?,
            semi_token: input.parse()?,
        })
    }
}

§Other operations

Every keyword and punctuation token supports the following operations.

  • Peeking — input.peek(Token![...])

  • Parsing — input.parse::<Token![...]>()?

  • Printing — quote!( ... #the_token ... )

  • Construction from a Span — let the_token = Token![...](sp)

  • Field access to its span — let sp = the_token.span

Structs§

Traits§

  • Token
    Marker trait for types that represent single tokens.