Introduction

• The StringTokenizer class in Java is used to break a string into tokens (smaller parts), based on delimiters like space, comma or any other character.
• Uses:
  • Breaking a string into words or tokens using specified delimiters.
  • Useful in parsing data from text files, CSVs or input strings.
  • Simpler and faster for basic tokenization compared to regular expressions.
• It is a legacy class (introduced in JDK 1.0) and is present in the java.util package.
• Syntax :-

  package java.util;
  
  public class StringTokenizer implements Enumeration<Object>
  {
      // Constructors
      // Methods
  }

• Example :-

  import java.util.StringTokenizer;
  
  public class MainApp
  {
      public static void main(String[] args)
      {
          StringTokenizer st = new StringTokenizer("Java,Python,C++", ",");
          
          while (st.hasMoreTokens())
          {
              System.out.println(st.nextToken());
          }
      }
  }

  Output:

  Java
  Python
  C++

Methods of StringTokenizer Class:

• Below are some common and important methods of StringTokenizer class with examples:

StringTokenizer st = new StringTokenizer("Java Python");
System.out.println(st.hasMoreTokens());

StringTokenizer st = new StringTokenizer("Java Python");
System.out.println(st.nextToken());

StringTokenizer st = new StringTokenizer("Java Python C++");
System.out.println(st.countTokens());

StringTokenizer st = new StringTokenizer("Hello World");
System.out.println(st.hasMoreElements());

StringTokenizer st = new StringTokenizer("Hello World");
System.out.println(st.nextElement());

Properties of StringTokenizer Class:

1. Legacy Class:
   • Introduced in JDK 1.0, it is considered a legacy class and is part of the java.util package.
2. Tokenization:
   • The StringTokenizer class is used to split a string into tokens based on specified delimiters such as space, comma, or custom characters.
3. Simple and Efficient:
   • It provides a lightweight and faster alternative for simple tokenization tasks without using regular expressions.
4. Use of Delimiters:
   • Delimiters can be specified at the time of object creation or during tokenization, giving flexibility in splitting strings.
5. More Useful than split() in Iterative Parsing:
   • Unlike split() which generates an array of tokens at once, StringTokenizer allows you to process tokens one by one using iteration. This is more memory-efficient when working with very large strings.