Module technology.sola.script

Package technology.sola.script.tokenizer

Class Tokenizer

java.lang.Object

technology.sola.script.tokenizer.Tokenizer

@NullMarked
public class Tokenizer
extends Object

Tokenizer provides functionality for tokenizing a source string into Tokens to be parsed and interpreted.

Constructor Summary

Constructors

Constructor	Description
Tokenizer(String source)	Creates a Tokenizer instance for desired source string.

Method Summary

Modifier and Type	Method	Description
TokenizeResult	tokenize()	Tokenizes the source string and includes any errors found.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Tokenizer

public Tokenizer(String source)

Creates a Tokenizer instance for desired source string.

Parameters:

source - the source

Method Details

tokenize

public TokenizeResult tokenize()

Tokenizes the source string and includes any errors found.

Returns:

the TokenizeResult