C++ || How To Split & Parse A String Into Tokens With Multiple Delimiters Using C++

January 23, 2021 admin No comments

The following is a module with functions which demonstrates how to split and parse a string into substring tokens with multiple delimiters using C++.

The function demonstrated on this page parses and splits a string and returns a vector that contains the substring tokens according to the delimiters.

The delimiters can be either a single character or multiple characters. If no delimiting characters are specified, the string is split at whitespace characters.

1. Split – Basic Usage

The example below demonstrates the use of ‘Utils::split‘ to split a string into substring tokens.

In this example, the default delimiter is used to split a string, which is a whitespace.





		
		
			1. Split - Basic Usage
			
C++
			
			// Split - Basic Usage

// Declare data
std::string input = "Kenneth Jennifer Lynn Sole";

// Get string split results
auto results = Utils::split(input);

// Display results
for (const auto& substring : results) {
    std::cout << "Substring: " << substring << std::endl;
}

// expected output:
/*
    Substring: Kenneth
    Substring: Jennifer
    Substring: Lynn
    Substring: Sole
*/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
				
						// Split - Basic Usage
 
// Declare data
std::string input = "Kenneth Jennifer Lynn Sole";
 
// Get string split results
auto results = Utils::split(input);
 
// Display results
for (const auto& substring : results) {
    std::cout << "Substring: " << substring << std::endl;
}
 
// expected output:
/*
    Substring: Kenneth
    Substring: Jennifer
    Substring: Lynn
    Substring: Sole
*/

2. Split – Multiple Delimiters

The example below demonstrates the use of ‘Utils::split‘ to split a string into substring tokens.

In this example, multiple delimiters are used to split a string.





		
		
			2. Split - Multiple Delimiters
			
C++
			
			// Split - Multiple Delimiters

// Declare data
std::string input = "- This, is. a sample string? 3!30$' 19: 68/, LF+, 1, 1";

// Declare delimiters
std::string delimeters = " ,.-':;?()+*/%$#!\"@^&";

// Get string split results
auto results = Utils::split(input, delimeters);

// Display results
for (const auto& substring : results) {
    std::cout << "Substring: " << substring << std::endl;
}

// expected output:
/*
    Substring: This
    Substring: is
    Substring: a
    Substring: sample
    Substring: string
    Substring: 3
    Substring: 30
    Substring: 19
    Substring: 68
    Substring: LF
    Substring: 1
    Substring: 1
*/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
				
						// Split - Multiple Delimiters
 
// Declare data
std::string input = "- This, is. a sample string? 3!30$' 19: 68/, LF+, 1, 1";
 
// Declare delimiters
std::string delimeters = " ,.-':;?()+*/%$#!\"@^&";
 
// Get string split results
auto results = Utils::split(input, delimeters);
 
// Display results
for (const auto& substring : results) {
    std::cout << "Substring: " << substring << std::endl;
}
 
// expected output:
/*
    Substring: This
    Substring: is
    Substring: a
    Substring: sample
    Substring: string
    Substring: 3
    Substring: 30
    Substring: 19
    Substring: 68
    Substring: LF
    Substring: 1
    Substring: 1
*/

3. Utils Namespace

The following is the Utils Namespace. Include this in your project to start using!





		
		
			Utils Namespace
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Jan 23, 2021
//    Taken From: http://programmingnotes.org/
//    File:  Utils.h
//    Description: Handles general utility functions
// ============================================================================
#pragma once
#include <vector>
#include <string>

namespace Utils {
    /**
    * FUNCTION: split 
    * USE: Splits a string into tokens and saves them into a vector
    * @param source: String to be broken up into substrings (tokens)
    * @param delimiters: String containing the delimiter character(s)
    * @return: A vector containing all the found tokens in the string
    */
    std::vector<std::string> split(const std::string& source, const std::string& delimiters = " ") {
        std::size_t prev = 0;
        std::size_t currentPos = 0;
        std::vector<std::string> results;

        while ((currentPos = source.find_first_of(delimiters, prev)) != std::string::npos) {
            if (currentPos > prev) {
                results.push_back(source.substr(prev, currentPos - prev));
            }
            prev = currentPos + 1;
        }
        if (prev < source.length()) {
            results.push_back(source.substr(prev));
        }
        return results;
    }
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Jan 23, 2021
//    Taken From: http://programmingnotes.org/
//    File:  Utils.h
//    Description: Handles general utility functions
// ============================================================================
#pragma once
#include <vector>
#include <string>
 
namespace Utils {
    /**
    * FUNCTION: split 
    * USE: Splits a string into tokens and saves them into a vector
    * @param source: String to be broken up into substrings (tokens)
    * @param delimiters: String containing the delimiter character(s)
    * @return: A vector containing all the found tokens in the string
    */
    std::vector<std::string> split(const std::string& source, const std::string& delimiters = " ") {
        std::size_t prev = 0;
        std::size_t currentPos = 0;
        std::vector<std::string> results;
 
        while ((currentPos = source.find_first_of(delimiters, prev)) != std::string::npos) {
            if (currentPos > prev) {
                results.push_back(source.substr(prev, currentPos - prev));
            }
            prev = currentPos + 1;
        }
        if (prev < source.length()) {
            results.push_back(source.substr(prev));
        }
        return results;
    }
}// http://programmingnotes.org/

4. More Examples

Below are more examples demonstrating the use of the ‘Utils‘ Namespace. Don’t forget to include the module when running the examples!





		
		
			More Examples
			
C++
			
			// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Jan 23, 2021
//    Taken From: http://programmingnotes.org/
//    File:  program.cpp
//    Description: The following demonstrates the use of the Utils Namespace
// ============================================================================
#include <iostream>
#include <string>
#include <exception>
#include <vector>
#include "Utils.h"

void display(const std::string& message);

int main() {
    try {
        // Declare data
        std::string input = "Kenneth Jennifer Lynn Sole";

        // Get string split results
        auto results = Utils::split(input);

        // Display results
        for (const auto& substring : results) {
            display("Substring: " + substring);
        }

        display("");
        
        // Declare data
        std::string input2 = "- This, is. a sample string? 3!30$' 19: 68/, LF+, 1, 1";

        // Declare delimiters
        std::string delimeters = " ,.-':;?()+*/%$#!\"@^&";

        // Get string split results
        auto results2 = Utils::split(input2, delimeters);

        // Display results
        for (const auto& substring : results2) {
            display("Substring: " + substring);
        }
    } catch (std::exception& e) {
        display("\nAn error occurred: " + std::string(e.what()));
    }

    std::cin.get();
    return 0;
}

void display(const std::string& message) {
    std::cout << message << std::endl;
}// http://programmingnotes.org/
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
				
						// ============================================================================
//    Author: Kenneth Perkins
//    Date:   Jan 23, 2021
//    Taken From: http://programmingnotes.org/
//    File:  program.cpp
//    Description: The following demonstrates the use of the Utils Namespace
// ============================================================================
#include <iostream>
#include <string>
#include <exception>
#include <vector>
#include "Utils.h"
 
void display(const std::string& message);
 
int main() {
    try {
        // Declare data
        std::string input = "Kenneth Jennifer Lynn Sole";
 
        // Get string split results
        auto results = Utils::split(input);
 
        // Display results
        for (const auto& substring : results) {
            display("Substring: " + substring);
        }
 
        display("");
        
        // Declare data
        std::string input2 = "- This, is. a sample string? 3!30$' 19: 68/, LF+, 1, 1";
 
        // Declare delimiters
        std::string delimeters = " ,.-':;?()+*/%$#!\"@^&";
 
        // Get string split results
        auto results2 = Utils::split(input2, delimeters);
 
        // Display results
        for (const auto& substring : results2) {
            display("Substring: " + substring);
        }
    } catch (std::exception& e) {
        display("\nAn error occurred: " + std::string(e.what()));
    }
 
    std::cin.get();
    return 0;
}
 
void display(const std::string& message) {
    std::cout << message << std::endl;
}// http://programmingnotes.org/

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

Was this article helpful?

👍 YesNo

Posted in: Advanced, C++ Tags: c++, string split

C++ || How To Split & Parse A String Into Tokens With Multiple Delimiters Using C++

Related

Leave a ReplyCancel reply

Search

Compile Your Code Now

Translate Page

Popular Pages

Recent

Archives

Categories