How to split a large list - Page 2

[Duoas]

I can't reproduce your error. If you can't figure it out by tomorrow then go to your project directory, zip everything up, and send it to me at michael thomas greer (without the spaces)
my provider is comcast.net Use whatever compression program you like (ZIP, TGZ, RAR, etc.) I won't take EXEs though.

[squidd]

will do man... Thank you for all of your help twith this.. Also, I was thinking that when button12 was clicked that a save function would happen to allow the person to name the basefilename and then increment after that.

I doubt I will be able to fix this tonight or ever so i am going to zip it up and send it to you now.

Cant thank you enough for your efforts. Also, I am going out tomorrow and will be buying a book or two on delphi coding for idiots like me for a reference and reading material.

Noone should have to go through this much to get one simple thing to work. Likewise, noone should have to help this much over something like this.

Sending the file to you now as I have been unable to do a thing with it.

[Duoas]

You sent that to me so quickly I took a look at it already. I still can't reproduce your error --the program works fine for me.

Testing
Are you sure you are saving in the same directory you loaded from? Here is what I did:
click "Load List", select my input file "in.txt". The list loads.

My list had twelve items, so I put something like 5 in the edit box. (I also tested with 4. As part of your error checking you need also to make sure that the number in the edit box is strictly greater than 1.)

I click "Go" to save. I select the same directory I loaded from and type an output file name "out.txt". Using explorer (or the command prompt) I look at the directory and I see three files:
- out1.txt
- out2.txt
- out3.txt
Examining the files shows they are properly formed.

The load and save directory need not be the same. Even if you load something the save directory is still at the default directory. You can force them to match by saying saveDialog1.initialDir := extractFileDir( openDialog1.filename ); before calling saveDialog1.execute.

Some errors
My debugger complained about a couple of things, so I looked them over. They are all in Button1Click. Here it is fixed:

procedure TForm1.Button1Click(Sender: TObject);  // load/open file
  var
    txt : TextFile;
    Buffer : String;
  begin
  if openDialog1.Execute() then  //(1)
    begin
    AssignFile(txt, openDialog1.FileName);
    Reset(txt);
    while NOT EOF(txt) do
      begin
      ReadLn(txt, Buffer);
      ListView1.Items.Add.Caption := trim(Buffer);
      end;
    ListView1.Columns[0].Caption := inttostr(ListView1.items.Count); //(2)
    CloseFile(txt);
    end;
  end;

1. If openDialog1.execute returns successfully, the filename will never be an empty string. You will notice I used the same pattern when I used saveDialog1.execute in the code I gave you.
If you just execute, then test the filename, the filename could be anything. Even if the user clicks cancel, if filename is not '' then you'll load the same file again.

2. There is no need to spam the application with updates for every line you load. Set the caption after all the files have loaded, and let the application process messages at its own convenience. It is relatively rare that you need to tell it otherwise.

Your formatting style (indentation and the like) is a bit unique (which is fine) but also a little random. I tend to want all statements "belonging" to another statement (such as a loop or procedure) to be indented under that statement. For example, I type "procedure", then everything under that is indented two spaces. Since the "begin" doesn't belong to "var", both have the same level of indentation. This tells me at a glance which statements belong to what.

Also, you should get into the habit of immediately naming things when you add them. I know that a lot of tutorials and books and code people post use Button1 and Edit2 and the like, but numbers tell you nothing about the function of the object. For example, you could name Button1 as one of the following: BtnLoadList, LoadListButton, etc. It doesn't really matter how you do it as long as you are consistent. When you see procedure Button1Click(Sender: TObject); you will wind-up asking at some point, "what does button1 do again?". However: procedure LoadListButtonClick(Sender: TObject); is immediately apparent.

Epilogue :icon_wink:
I haven't tested all the other buttons in your application, just load list, edit6, and Go button 12. Again, I cannot reproduce your error --It works fine for me.

It is possible that the problem you have is related to the errors produced by the button1click procedure that I made note of above. If that doesn't fix it and you don't have other code in there I don't know about then I don't know what else could be wrong...

Hope this helps.

[squidd]

so you are saying when you click the go button (button12) a save window comes up? That doesnt happen for me. The files may be all set to go, but I cant save them for some reason. isnt that just odd....

I sent you the entire program too...

hmmm. I cant understand that.

[Duoas]

Yes, it gives the dialogue and all.

Ah, I forgot to mention that your program shouldn't have compiled at all because you had listed in your uses clause unit1 instead of unit1_helpme. So I'm not sure exactly what is wrong.

I used Delphi 5 to compile it. You said you were using FPC? Give me a day and I'll see if I can make it work with FPC.

[squidd]

I am using CodeGear RAD Studio 2007, and I renamed the project to help me for sending to you. It is without the helpme on the end after that.

[Duoas]

I've looked over the sources you sent me again and I can't see why it shouldn't work.

CodeGear bought Borland's Delphi and C++ IDE/compilers, so you are actually using Delphi 2007 (the latest version) to compile your code.

The OnClick for Button12 is properly set to Button12Click in the DFM file, and the procedure itself does: lines_per_block := strToInt( edit6.text ); if saveDialog1.execute then so there is no valid reason why the dialogue shouldn't appear... except if edit6 contains an invalid number.

Why don't you change the first line to this and see if it catches the error:

try lines_per_block := strToInt( edit6.text )
except
  showMessage( 'You must specify how many lines to save per file' );
  edit6.setFocus;
  exit
end;

If that doesn't fix it then I'm completely baffled...

[squidd]

I emailed you again about an hour ago. I would like you to see if what I sent you actually works as well. Let me know when you have some free time. Thanks for your help. :)

[Duoas]

Replied in kind. Lots of little fixes, a couple big ones. Most by way of suggestion and organization.

Let me know if your problem disappears with the changes.

[squidd]

ok this is merely a reply just to say that I have a bigger reply in the very near future... it is late and I just want to tell you how much I appreciate your help with tis. the problem IS solved. and I will reflect that after my next post...

I just want to read everything you wrote many more times and grasp it all. And the only problem I am having is simply getting the program to add the .txt extension at the end of the newly split list names.

this problem is SOLVED by DUOAS!

[squidd]

ok, it took a look at everything over 3 times to try and figure out why it didnt work for me. I still dont understand. But maybe it had to do with what you mentioned in the code because I cant see any other reason for it to have given me such grief.

BTW, I used Application.ProcessMessages because when very large lists are loaded, the program locks up until the list has completed the load. when I use Application.ProcessMessages, the program doesnt appear to have 'locked up.'

Anyway, I thank you very much for all of your help in this. It has been solved. :) :)

[Duoas]

Glad to have helped. :)

Now I see why you are calling application.processMessages. I didn't realize you were loading super-gigantic lists.

However, doing so introduces a bug: what if the user clicks, say, Remove Duplicate Entries or (gasp) Clear List while processing messages before the list is finished loading? That's a serious problem! :'(

So, how long do you have to wait loading the largest imaginable list? If it is only several seconds or less, you should instead turn your cursor to crHourGlass and/or put up a little message saying something like "Please wait" --or even, if you want to get fancy, use a a progress bar.

If it is longer, disable all the buttons that can affect the list before beginning to load, and call processMessages as you were during the load, then re-enable all the disabled buttons after loading. Or something like that.

Good luck. :cool:

[squidd]

ok well, the largest list would be somewhere in the area of 100,000 lines... I dont know how to use a dynamic array yet, so I am using listview. It would take almost 1 minute to load a list that large; and that is on a fast computer.

I want to do a progress bar yes! But I havent seen anything out there that really helps me do that so far.

I am trying to add messages as the list does different things (i.e. clear to say clearing list for long lists.. etc...), and of course I am having problems... lol Not to the point that I need to make a post I dont think. Maybe I should make a new post about adding a progress bar, since I have been trying that for about 2 days off and on. What do you think? The progrss bar i would like to do is inside a status bar on the bottom of my form. But learning the progress bar alone would be great as well.

The disabling of buttons that could cause problems sounds like a great idea! Maybe along with the progress bar, i could somehow give a percentage number complete at the top of my mainform. I dont know. What are your thoughts?

EDIT -

Also, Since this has a fair amount of buttons and whatnot.. I thought I may as well start learning some events like mouseover changes button color, or plays a sound, or both... add some eye candy to this program. If i can do it on this and learn this now, I can add the different things I learn into any program I write from now on. Lots of different things to do with cosmetics in a program.. learning how to make it work is another however. I dont wont to have external dependencies for colors and sound for mouseovers unless there is no other way. One exe is all I want for this.

EDIT_2 -

I was thinking since a list FINALLY splits now, maybe to add different types of files to the load button such a *.doc, *.rtf, *.pdf, etc... when I try to l;oad a different type of file other than a text file (*.txt) i get some gibberish and no lines load. If that requires major work then no worries...

I also want to be able to add a "_" in between the numbered split files (i.e. out_1.txt, out_2.txt, etc...) I tried to l;ook at the code and make some changes but had no luck with that. Again, if it is a major hassle, then no worries there either.

I did find I could save a file in different formats... that was kinda neat.. maybe I could add an edit box to have it save automatically to whatever format. Again, some reading material on how to do that would be a great thing to have. lol

[Duoas]

The windows progress bar displays the percent-complete in the center, so that is a good choice. Delphi status bars are a pain in the nose to use, but I think you can add a panel containing a progress bar... I don't think I would bother with the form caption.

The reason it takes so long to load is because you are using a TListView... sorry to say.

Take a look at the OnMouseMove event for mouse over stuff.
Also, the PlaySound Win32 function. You'll probably want to compile your sounds into your program, so you'll need to read up on using resources.

Your program is designed to read and write text files. Binary files are a no-no. You'll damage them irrevocably. To auto-append a specific extension, just test the filename once you've got it to see if it has an extension or not, and add it if not.

The filename itself is just a string, so add an underscore the usual way basename + '_' + intToStr( 12 ) + fileext Just because you can save a file named "foo.pdf" does not make it a PDF file. It is still just a plain-text file. Remember, PDF, etc. are binary files. RTF is a text file, but it is not your run-of-the-mill text file. Its contents have a specific format. Load one into Notepad to see.

Good luck.

[squidd]

Thank you for the reply kind sir. I will look into that right now. :) and edit will follow shortly...

[squidd]

The reason it takes so long to load is because you are using a TListView... sorry to say.

I know.. an array would go MUCH faster... exponentially faster even. One thing at a time though....

Take a look at the OnMouseMove event for mouse over stuff.
Also, the PlaySound Win32 function. You'll probably want to compile your sounds into your program, so you'll need to read up on using resources.

I will be doing that tomorrow for sure... thanks for the heads up on where to look. :)

Lastly, (I have everything saving to a txt format now) binary files are totally different than huh? Any ideas about reading up on how to do the same thing with those types? If that is above me, then once again, no worries. :)

I think I need to start a new topic. lol

[Raged]

hey!, i know this threads dead, but I've got a related issue which is really bugging me.

I have been making a program to get items from a txt file and move them into a combo box, of course i got that working great, but for all my testing i just had to remove the txt file because it took so long to load, and this topic started moving onto using application.message or something, to stop the lag....

so I'm wanting help with how i can get my program to either load my list faster (i wouldn't mind storing the information in the program if its nesessary) even though i have 1 big problem with the application already and that is its storing the information in the program, but if i remove the txt file, it can't use the information. so my applications a large file size already.

anyway, i was thinking about having a progress bar for it loading the lists up.
and i can't find any sort of progress bar code to use it while the pc's lagging on loading a file.

but if its possible to have the application loaded, and be able to use the combo box while its feeding in the information, that would be great.

also there was a post about putting progress bars in status bars, that deals with parenting, and is to over complicated for simple programs unless your really visual needed =D

Thanks in advance =)