Saving Program Data

[CoilFyzx]

Good day folks.
Problem: My program needs to import data from a database and display them in a table then manipulate and display the data in various ways. That bit is solved the next part is where I need help.

The program then needs to save locally...

1. the newly processed data (what I mean is the data from the database plus all related data created by way of data manipulation).
2. the last view of the program? (What I mean is, when the program is restarted I want it to display the last thing the person saw; GUI and data.
3. and finally I want to save the user settings like database credentials and user preferences.

So I want to save all this data in such a way that I have a cohesive program. Please give me some leads. I'll do the research myself.

Thank you.

[cool_zephyr]

store all the details in a class implementing Serializable interface..then save it in the file system..when the program starts you can initialize the class directly from the file.

[peter_budo]

@cool_zephyr I will say it is everyone personal opinion/preference how to proceed. Because as to your recommendation I can say "why to bother with file read/write logic when there is already database connectivity?"

@CoilFyzx it is difficult to give deceisive answer without knowing more about functionality of your application.

1st point, as ong there is no heavy data processing (thousands of entries to go through before displaying result, which means delay before user sees anything) you more then safe to do it on each read

2nd point, you may create special table to hold data of last seen UI, simple if working with single UI view (school assignmen). However that may be impractical if you have multiple UIs you will require number of such tables.

For 3rd point implementation is really up to you if you want to keep config file or database config table, same goes for credentials

[Ewald Horn]

To save data locally, why not look at something like the embeddable H2 database from http://www.h2database.com/html/main.html - it is a brilliant, small SQL database, written in pure Java.

As for the last view, you can easily serialize and deserialize the state of your application, if your design supports it, of course. If you followed the MVC pattern, it should be easy to add a layer that persists the program state to the database, and then read it back on startup. You shouldn't try to save the GUI components, just the data that led to that UI being built. That way, once you load the data, you can "replay" the sequence that led to that UI being built.

You can save user preferences and database credentials in the H2 database as well. I'd use the H2 database for all three your requirements, it is very easy to use and makes sense given that you already have database experience.

[JamesCherrill]

In defence of cool_zephy's suggestion...
Storing a Serializable object - no matter how complex its internal structure and content - is a single call to writeObject. Restoring it ditto.
Storing/restoring data with multiple occurences of complex data by mapping it into relational tables can be very complex and code intensive.
I'm not saying that using a database mnager is a bad idea, just that writing a currentState object to an ObjectOutputStream could be vastly simpler.
Personally I would
1. Start with making my state object a valid javabean, then use java.beans.XMLEncoder to write it as an XML file
2. Encapsulate the whole save/restore behind a simple interface so I could change ithe implemenattaion later.

[CoilFyzx]

@Ewald Horn. Thanks but I'm interested in creating this without just what Java gives me. I appreciate your help nonetheless.

Thank you very much for all the help guys.

So let me see if I'm clear here. For my requirements(in the same order that I listed them)

1. Use serialization. (This generates it's own file)
2. Don't save the GUI elements just the data that lead to them. But save the data how? Should I serialize this as well?(And would I serialize thi to tshe same file?)
3. As for the user preferences?

@JamesCherrill

I want to say that I understand what you said in your numbered list but I don't. Added to this you mentioned "write it as an XML file". How would I set up file type association then? (In the end I want to be able to open my file just the way we open all our other ones; that is, by simply double clicking them(OPEN action))

[JamesCherrill]

Serialization allowes you to write/read arbitrary objects to a binary file, in a Java-specific binary format that may be subject to change (not just an empty threat - I had a live system fail when the serialisation of Image objects changed somewhere in the later releases of 1.6).
A very similar alternative is to use the XMLEncoder/Decoder classes to write the objects to a .xml file in standard XML format. It's much the same as serialising, excepy that you have a readable standard XML format file, not an obscure proprietary binary file.

Re GUI objects - you can't serialise open WIndows etc because they are tied in to native OS components - you would have to create a little class to hold the types of the open windows with their x,y coordinates and sizes, from which you could easily reconstruct the user's visual layout.

[CoilFyzx]

@JamesCherrill Addressing your first comments: Are you saying to me that my save file may become useless over time if I use Serialization?

Maybe I'm not being clear. I do understand that I can't just freeze a window and save it to a file. That is why I agreed with the advice to save the data then reconstruct it. My question is now: Do I save that data to the xml file as well?

[JamesCherrill]

There is a problem that, from tine to time, Oracle change the serialisation format for one or more classes - in the Image case I mentioned before it was to close a security loophole. That's why its only really safe for transient storage or data transfer. A file written before the change may not be readable after the change. In practice this is a rare event, and you may never encounter it if you are just using serialisation to hold status info from one run to the next.

If the data came from a database in the first place, then maybe it's not necessary to save another copy - all you need to save is anything changed but not yet comitted to the database, or additional status info that's not part of the actual data (eg window sizes).

Without more understanding of your application it's hard to be specific. My instinct (and that's all it is) leans towards:

• data should be updated back into the source database
• credentials and preferences should use the java.util.prefs.Preferences class
• session info like window configuration should be saved in an XML file

... but without knowing the full requirements this is just a provisional opinion.

[CoilFyzx]

Thank you for the heads up re: the xml for window configuration and preferences for credentials.

The main requirement is that I must store the data locally(on a physical storage device). The information that the program is manipulating comes from a database. Then, it is processed to create data that does not belong in the database. So I have to save that data locally, so that each time the user goes back to the program he can continue working on it.

I'll give you a thought aid: What I'm planning to do is much like what we do with our office software such as Microsoft Word. Data is put in the application, saved as a .doc file to be further edited in the future.

And so I'm looking for guidance in accomplishing that bit.

Any thought as to what I could use to accomplish this please sir?

[JamesCherrill]

it is processed to create data that does not belong in the database. So I have to save that data locally,

In that case I'd go back to just serialising or XMLing the highest-level Java object (ArrayList or whatever) that holds all that data, but I'd be sure to implement that via an interface so I could change it later.

Can't comment on the last question without a LOT more info. Maybe jsut start with something simple an go from there???

[CoilFyzx]

Okay thank you. May I ask what sort of information you need? To be fair I have not started coding yet. I'm just planning my development approach. SO the pieces that are missing in your mind, aaaare probably missing in mine.

It wasn't until last night that I decided to use ArrayList to store the table data(But I'm still not even sure if that's the best thing to use...I'd need some assistance here as well).

I am completely new to these arenas of program development.

[JamesCherrill]

What sort of info? I woul;d start by trying to name all the main classes that will represent the data and all the things you want to do with it (ignore the GUI for now). Write the names, return types and parameters for the main public methods you will need to call. Now, on paper or even just in your head, try to run through a few typical use cases using those classes and methods. Keep refining them until the use cases work smoothly.
Then, and only then, code all that up in Java along with some simple test methods to conform that your code works as expected.

You will inevitably need to add to those classes (eg to support the GUI) and maybe re-write bits when you run into problems. That's OK. The main thing is to develop one piece at a time and keep testing as you go.

[CoilFyzx]

Hold up....sigh we seriously need to resurrect the phone call. :D

You asked me:

What sort of info?

I think we're hitting and missing here. MAy I ask why you've asked me that question? Is it in response to my question:

May I ask what sort of information you need?

If that is the case, then let me clarify. The reason I asked that question was because of your statement:

Can't comment on the last question without a LOT more info

So if I got all of that correct, let me please ask: What sort of information do you need in order to answer the question that triggered all of this please :D?

I'll give you a thought aid: What I'm planning to do is much like what we do with our office software such as Microsoft Word. Data is put in the application, saved as a .doc file to be further edited in the future.

And so I'm looking for guidance in accomplishing that bit.

Any thought as to what I could use to accomplish this please sir?

[JamesCherrill]

"What sort of info" was a summary of your question, followed immediatly by an attempt at an answer (classes & methods etc) - I wasn't re-asking it back to you :)

The problem here is that there are many quirky details about how to select and implement the various examples of persistence in your question. Apparently tiny details in the requirements can swing the choice one way or another. Within the time/resource constraints of a volunteer service like this there's only so much time we can spend. To give an answer noticibly better than the one I already gave I would need to understand the whole picture (complete requirements specification, use cases, technical constraints, success citeria, financial/timscale constraints etc etc.), and complete a first-pass design.

That's obviously unrealistic, so I was suggesting that you should take what's useful from the suggestions from this thread and attempt the first-pass class diagram, which would be a good basic for further review and discussion.

[CoilFyzx]

Thank you. NOW I'm very much up to speed with where you were carrying this conversation.

Well I followed your suggestions and this is the data flow that I plan to use:
(This, I hope, is the last time I ask you to give suggestions as to how save data)
As I was saying, here is how the data will move:

1. Data is imported from the SQL database.
2. This data is stored in a an ArrayList of custom class called Student Objects.(This class stores data such as first and last name, number of subjects chosen, as an array with the names of their subjects)

3. The Data from the ArrayList<Student> is displayed in a JTable. There are however some cells in the JTable left to be filled/edited. When any cell is edited the row of data in which it falls is used to update another ArrayList with another created class called Subject. In the Subject class there is also an ArrayList of Students, and other elementary data types.

4. The Data from the ArrayList<Subject> is also used to up data the GUI.

So that is how the data will flow through the program. A bit of a mouthful.

But the data that needs to be stored is found in the ArrayList of Students, the Object data inside the Table(even though I essentially store it the ArrayList of Subjects, I still wish to save it) and the ArrayList of Subjects.

• I want to use a proprietary file extension.
• The data must be there for as long as the user need it.

Now the question: What is a suitable way to save this information please?

[JamesCherrill]

In that case I would ensure that Student and Subjst follow the rules for JavaBeans, and use XMLEncoder to write the ArrayLists of Students and Subjects to a file in XML format. You can use any extension for the file you like. (I have done this myself, it really is simple.)

To be fair: Others may still prefer to use SQL to store the Students, Subjects, and the Student/Subject pairs. Its certainly do-able, but a lot more code than XMLEncoder. If any SQL enthusiasts out there want to make the case for doing that, now would be a good time...

Maybe you should use a custom TableModel that' simply a wrapper for the data that's in the two ArrayLists - then there's nothing more to store for that.

ps Why do the Student objects store subject names, why not Subject instances?

[CoilFyzx]

Wow this has been so helpful. Thank you very very very much.

As for your question: I only need a record of the names of the subjects that each student does. Having instances of the Subject class, seems like using a chainsaw to cut a thread. Or is there sometthing that I may be overlooking? Did you notice something out of the ordinary?

[JamesCherrill]

That's the object-oriented way of thinking! Sudents study Subjects, not "subject names".

Suppose you want to know how many credits a Student can earm from his current set of Subjects - you loop thru his List of Subjects adding their credit points. If all you have is subject names you have to use those to look up the corresponding Subjects in some kind of Map in order to get the credit points.
(Same for checking whether he has met minumum requirements for spread of Subjects, of has forboidden combinations of Subjects, etc etc.)

It's not "heavy". Either way the Student class just has a list of refernces to other objects, be they Strings or Subjects

[CoilFyzx]

I see. Hmm. That actually makes more sense. So instead of using a map, each instance is complete in that it possesses it's all the data related to it.

Thank you sire.

[CoilFyzx]

By the way is using Preferences in anyway better than using, say a Properties File?

[JamesCherrill]

Only in that it "just works" regardless of platform. Why re-invent the wheel?

[CoilFyzx]

Thanks again.

~Fin~