Member Avatar for Snow_Fox

I need to take in a sentence and break it up based on the individual words.

I've been googling for probably 30+ minutes to 60 ish and I feel it's time I get some help on this one..

Ideally, there should be a way to break up a string possible using white spaces as a delimiter that isn't going to limit the length of the sentence (like a character array would) while allowing me to send the individual words to a pre-existing method I created which can change individual words into pig latin. I just don't know how I'd go about doing this..

(Sorry if I am breaking any rules here, although it is the same project, it is a different issue I figured this would be a better way to go about it)

Edited by Snow_Fox because: n/a
  • 3 Contributors
  • 5 Replies
  • 261 Views
  • 9 Hours Discussion Span
  • Latest Post Latest Post by Snow_Fox

Dani AI

Generated

Short answer: use std::string (not fixed char arrays) and pick the tokeniser that matches how you want to treat punctuation. was right that treating the input as a stream and tokenising on whitespace is the simplest for plain words; it avoids buffer-size limits and you can feed each token straight into your pig‑latin function. If you need to treat punctuation, contractions, or hyphenated words specially, a regex-based extractor or a small post‑processing step is usually better.

A small C++ approach (C++11+): extract only "word" tokens with a regex, then process each token. This keeps punctuation out of the words so your pig‑latin function gets clean input.

#include <string>
#include <regex>
#include <vector>

std::vector<std::string> split_words(const std::string& s) {
    std::regex re(R"([A-Za-z]+['-]?[A-Za-z]*)");
    std::sregex_iterator it(s.begin(), s.end(), re);
    std::sregex_iterator end;
    std::vector<std::string> words;
    for (; it != end; ++it) words.push_back(it->str());
    return words;
}

Python quick options: use sentence.split() for simple whitespace splitting, or re.findall to extract word-like tokens when you want to ignore punctuation.

# simple
words = sentence.split()

# keep only letter-based tokens (keeps don't and hyphen-words)
import re
words = re.findall(r"[A-Za-z]+(?:['-][A-Za-z]+)?", sentence)

Practical checklist for pig‑latin integration:

  • Decide whether punctuation should move with the word or stay outside (e.g., "dog." -> "ogday.").
  • If preserving punctuation: strip leading/trailing non-letters, remember them as prefix/suffix.
  • Remember capitalization: detect if the original started with uppercase and reapply to the result.
  • Apply pig‑latin to the pure word (lowercased if easier), then restore capitalization and punctuation.
  • For non-ASCII input (accents, non-Latin scripts) use a Unicode-aware library (ICU or language-native Unicode routines).

If you explain how you want to treat punctuation and contractions, a short tweak to either the regex or the post-processing will give exactly the behavior you need.

Recommended Answers

All 5 Replies

Member Avatar for Bench

The >> operator is whitespace delimited by default when used with any kind of stream; including cin, fstream and stringstream; so you can use that to your advantage to turn a stringstream into a simple tokeniser

e.g.

#include <string>
#include <sstream>
#include <iostream>

...

std::istringstream hello("the cat sat   on \n\t the mat");
std::string fred;
while( hello >> fred )
{
    std::cout << fred << std::endl;
}

This is also doable for any other type of delimiter, though you need to fiddle a little bit if using something else such as commas etc (std::getline also works with stringstreams).

Edited by Bench because: added headers
Member Avatar for Snow_Fox

The >> operator is whitespace delimited by default when used with any kind of stream; including cin, fstream and stringstream; so you can use that to your advantage to turn a stringstream into a simple tokeniser

e.g.

#include <string>
#include <sstream>
#include <iostream>

...

std::istringstream hello("the cat sat   on \n\t the mat");
std::string fred;
while( hello >> fred )
{
    std::cout << fred << std::endl;
}

This is also doable for any other type of delimiter, though you need to fiddle a little bit if using something else such as commas etc (std::getline also works with stringstreams).

So, I can cin something like?

std::istringstream hello("");
getline(cin,hello);

or is that doing it completely wrong?

Member Avatar for Bench

getline reads into a string from a stream; it's not designed to swap data between two streams. But you can do this:

std::string line;
std::getline(std::cin,line);
std::istringstream hello(line);
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.