determining frequency of words in text file

[Micko]

Hi to all, after two months I decided to give Python one more chance. Here is a little code snippet about counting number of words in text file. (Under word I assume any combination of letters and numbers delimited by standard separators " .,\n\t"

file = open ( "Test.txt", "r" )
text = file.read ( )
file.close ( )

word_freq ={ }

word_list = string.split ( text )

for word in word_list:
    count = word_freq.get ( string.lower ( word ), 0 )
    word_freq[string. lower ( word )] = count + 1

keys = word_freq.keys ( )
keys.sort ( )

for word in keys:
    print word, word_freq[word]

Can this be done a little more elegant?

[vegaseat]

My suggestion would be to use the newer string functions and to add some comments ...

file = open("Test.txt", "r")
text = file.read()
file.close()

word_list = text.lower().split(None)

word_freq = {}
for word in word_list:
    word_freq[word] = word_freq.get(word, 0) + 1

keys = sorted(word_freq.keys())
for word in keys:
    print "%-10s %d" % (word, word_freq[word])

You could also trim off any punctuation marks.
Any chance you could show us how this would look in C code?

[Micko]

Any chance you could show us how this would look in C code?

Well in C++ using STL map container it would be fairly easy, in C it would take more code (possibly more error prone). i suppose i could write that if you're really interested, but I think you ask it just to ahow how Python code is more compact than C code ;)

[vegaseat]

Hi Micko,
glad to see you back here!

C, just to make you think about it!
It's always good to figure out which language would do the job best!

[Micko]

To be honest, I'm afraid of Python. I used to work with C, C++, implementing linked lists, binary search trees, simple math parser and similar, I used to think in terms of arrays, structures, linked lists, stack etc, now when I try to do something in Python I have hard time to thin in terms of lists, dictionaries etc. I suppose I'll need to do to one or two projects in Python to get comfortable. Python's power is really what scares me...
:)
Cheers

[Micko]

I'm having idea to make a simple calculator just like in windows. I suppose I'll need to learn something about GUI in Python... Do you have anything to recommend?

[vegaseat]

For your calculator project I would use wxPython. You could use the form builder that comes with Boa Constructor, or you could easily make your own layout of the buttons and the textbox for the result.

Below is just a little example how to get started with a calculator frame ...

# Template of a wxPython Frame with 2 buttons and textbox
# could be the start of a calculator

import wx

class MyFrame(wx.Frame):
    """make a frame, inherits wx.Frame"""
    def __init__(self):
        # create a frame/window, no parent, default to wxID_ANY or -1
        wx.Frame.__init__(self, None, wx.ID_ANY, 'wxPython', pos=(300, 150), size=(220, 200))
        self.SetBackgroundColour('green')
        
        self.edit1 = wx.TextCtrl(self, -1, value="", pos=(5,5), size=(200, 25))
        
        self.button1 = wx.Button(self, -1, label='1', pos=(5, 130), size=(20, 20))
        self.button1.Bind(wx.EVT_BUTTON, self.button1Click, self.button1)

        self.button2 = wx.Button(self, -1, label='2', pos=(30, 130), size=(20, 20))
        self.button2.Bind(wx.EVT_BUTTON, self.button2Click, self.button2)

        # copy and paste more buttons here ...

        # show the frame
        self.Show(True)

    def button1Click(self,event):
        txt = self.edit1.GetValue()
        txt = txt + '1'
        self.edit1.SetValue(txt)
        
    def button2Click(self,event):
        txt = self.edit1.GetValue()
        txt = txt + '2'
        self.edit1.SetValue(txt)
        
    # add more defines ...


application = wx.PySimpleApp()
# call class MyFrame
window = MyFrame()
# start the event loop
application.MainLoop()

Copy and paste will be easy, just position the buttons at the right place. Eventually you have to parse what is in the textbox and do your calculation, that is the hard part!

Note: php code tags used to work well in the early days of DaniWeb, now they don't. Replaced them with the newer tags.

[mawe]

Hi!

Here's a snippet using Tkinter:

from Tkinter import *

def click(key):
    if key == "=":
        # here comes the calculation part
        pass
        # but that's up to you ;-)
    else:
        display.insert(END, key)

root = Tk()

keys = ["789/", "456*", "123-", "0.=+"]

display = Entry(bg="white")
display.grid(row=0,column=0,columnspan=4)

x,y = 1,0
for row in keys:
    for key in row:
        Button(text=key, command=lambda k=key: click(k)).grid(row=x, column=y)
        y += 1
    y = 0
    x += 1

root.mainloop()

Regards, mawe

[vegaseat]

Nice code mawe, always good to hear from the Tkinter expert himself! The eval() function will do the rest.

Micko and you are almost neighbors. That area of the world seems to have a lot of computer talent!

[Micko]

That area of the world seems to have a lot of computer talent!

Well, I don't know where mawe is located, I'm from Bosnia and Herzegovina, town Tuzla.
I wouldn't consider myself a computer talent, but thanks vegaseat, now I'm motivated even more. Currently I'm waiting a job so I have extra spare time, and I'm planning to use it wisely.
My first forum was on Cprogramming.com where i came to find answer for my homework, and stayed two years. Then I found link on this forum on www.eternallyconfuzzled.com and here I am.
Thank you all, I hope in future I'll get a better internet connection and stay longer...

Cheers

- Micko

[vegaseat]

A note on computer talent:

My favorite image viewer is IrfanView. Irfan Skiljan is from Jajce in Bosnia, he wrote IrfanView when he was a student at the TH Wien in Austria.

[mawe]

Well, I don't know where mawe is located, I'm from Bosnia and Herzegovina, town Tuzla.

I'm from Austria, so we really are neighbours (well, nearly :)).
I wonder how vegaseat knew where we lived? And I wonder why he thinks that I'm a Tkinter expert :)

[vegaseat]

It's the accent that gives it away, even in the written word!

[Bajeed]

Hi,

I have a C++ solution using STL.
This program will find frequency of each word present in a text file.

/* Program to find the frequency of words in a text file */


#include <iostream>
#include <map>
#include <set>
#include <string>
#include <fstream>
#include "vector"


using namespace std;


int WordFreq(char *pcFileName)
{
vector<string> AllWordVec; // vector with all word inc duplicates
set<string> UniqueWords;   // set with only unique words
map<string, int> WordFreq; // Map of words and their frequencies
string word;               // Used to hold input word.


if (0 == pcFileName)
{
cout << "Null input provided" << endl;
return 1;
}


// Open the input text file
ifstream InputFile(pcFileName);
if (!InputFile.is_open())
{
cout << "Can not open the file" << endl;
return 1;
}


// Read all tokens from the input file
while (InputFile >> word)
{
//-- Read unique words/tokens; this is set, so no duplicates
UniqueWords.insert(word);


//-- Read all words/tokens in the sequence
AllWordVec.push_back(word);
}


vector<string>::iterator vWord;     // for iterating thru all words
set<string>::const_iterator UWord;  // for iterating thru unique words


for (UWord = UniqueWords.begin(); UWord != UniqueWords.end(); ++UWord)
{
// check for unique word in the AllWordVec vector
for(vWord = AllWordVec.begin(); vWord != AllWordVec.end(); vWord++ )
{
if (!(*UWord).compare(*vWord))
{
// word matched, count this
WordFreq[*UWord]++;
}
}
}


// Write count, word
map<string, int>::const_iterator iter;
for (iter = WordFreq.begin(); iter != WordFreq.end(); ++iter) {
cout << iter->first << " " << iter->second << endl;
}


return 0;
}


// main function
int main()
{


WordFreq("C:\\Bajeed\\VC++\\list.txt");
system("pause");
return 0;
}

Regards,
Bajeed

[vegaseat]

Thank you Bajeed! Works well on DevCpp. Looks like the C++ version is just a little harder to read.