strtok help

Question

Joe Shmoe 33 Light Poster

14 Years Ago

I am a VB programmer, so I know a lot about programming. I am just new to C and want to ask a simple question.

How can I read a file and use strtok with space as the delimiter and have access to all tokens in memory? The example provided at cplusplus.com is not exactly what I need. The thread, http://www.daniweb.com/forums/thread55584.html, on this site is also not exactly it.

The problem in the examples provided is that the char array is overwritten each time a new token is read. I need to be able to access all of the tokens from memory.

In VB I would code something like this:

Dim strline As String
Dim strall As String
Dim arrtok() As String

Open "B:\test.txt" For Input As #1
Line Input #1, strall
Do While Not EOF(1)
    Line Input #1, strline
    strall = strall & " " & strline
Loop
Close #1

arrtok() = Split(strall, " ")

MsgBox arrtok(0), vbInformation, "This is token 1"
MsgBox arrtok(1), vbInformation, "This is token 2"
MsgBox arrtok(2), vbInformation, "This is token 3"

How can I do the same in C? I'd like to use char arrays for this, not c strings if possible.

c

2 Contributors
1 Reply
123 Views
11 Hours Discussion Span
Latest Post 14 Years Ago Latest Post by Ancient Dragon

Reply to this topic

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 1 · 2010-07-10T11:26:41+00:00

>> I'd like to use char arrays for this, not c strings if possible.

They are the same thing in C language.

If you want to keep all the tokens (or words) in memory for all lines of the file then just generate a linked list of them.

struct token
{
   struct token* next;
   char* data;
};

Now just allocate new nodes of the above structure and allocate memory for data then copy the return value of strtok() into it.

Warning! The following code has not been compiled or tested. I just posted it to give you an idea of how it can be done in C language.

#include <stdio.h>

int main()
{
   struct token* node;
   struct token* tail = 0;
   struct token* head = 0;
   char iobuf[255];
   char* ptr;
   FILE* fp = fopen("filename.txt","r");
   if( fp != NULL)
   {
        while( fgets(iobuf, sizeof(iobuf), fp) != NULL)
        {
            ptr = strtok(iobuf,' ');
            while(ptr != NULL)
            {
               node = malloc(sizeof(struct token));
               node->next = 0;
               node->data = malloc(strlen(ptr)+1);
               strcpy(node->data,ptr);
               if( head == NULL)
               {
                   head = tail = node;
               }
               else
               {
                   tail->next = node;
                   tail = node;
               }
               ptr = strtok(NULL,' ');
             }
         }
     }
}