gm.text.SpecialTokens#
- class gemma.gm.text.SpecialTokens(value)[source]
基类：
enum.IntEnum特殊 token ID。
- PAD: ClassVar[int]
- EOS: ClassVar[int]
- BOS: ClassVar[int]
- UNK: ClassVar[int]
- MASK: ClassVar[int]
- CUSTOM: ClassVar[int]
- START_OF_TURN: ClassVar[int]
- END_OF_TURN: ClassVar[int]
- START_OF_IMAGE: ClassVar[int]
- END_OF_IMAGE: ClassVar[int]