I have been struggling with the principles of creating a client-server design where the server maintains a data structure that multiple clients can view and manipulate with various restrictions on which client can access which parts of the structure. I don't know what to properly call a system like that, but on the surface it seems like a simple problem. Unfortunately, I've found ways to make it complicated for myself and I've been slowly teaching myself how to simplify it to manageable levels. For example, one of my top design goals is that each client should have a local copy of the accessible part of the structure. This is necessary for creating a fast user interface, but it complicates the design by creating partial versions of the data structure and requiring consistency between client data structures and server data structures.
With or without a consistency requirement, I would want the server to send out messages to all clients who have permission to view a change when that change happens because keeping clients promptly notified about such changes is another one of my design goals. So I need the client to serialize and send each change that it wants to make to server, then the server makes that change to its copy, then it resends the change to all relevant clients so they can update their local copies. At first I wanted to have the original client update its local copy immediately, but I later realized that consistency is easier to maintain if all clients see the changes in the same order, so now I prefer to have the original client wait for the change to bounce back from the server before making the change locally.
Before I realized that this might be difficult, I jumped straight into the problem without thinking about it carefully. I had an application in mind. I knew what objects I wanted the server to manage and I designed classes to represent those objects in the straight-forward way, then I created serialization methods for each of those classes and serialization methods for every modification that might legally happen to objects of those classes. I used special values in fields to represent when the true value of the field was absent due to lack of permission to view it. I knew what sorts of permissions clients might have for each object and added fields in the classes to store each possible permission. I managed to get all of this mostly working before things started to fall apart. The whole system was complicated and unmanageable. It had problems and those problems were very difficult to find and fix, and making any sort of change was a huge effort.
In the hope of avoiding the problems that plagued me from my first effort I restarted from scratch and designed the system more carefully, but it was still designed on the same principles. This time I had less patience for putting in a large amount of work because I was painfully aware of how badly it might end up, and when I saw the same sorts of problems starting to form halfway through the project, I abandoned the entire idea for a long time.
Now I am attempting it again, but this time I have studied more object oriented design and have radically different ideas. I have learned from my past failures and this time I won't create the system for just one application. A system like this is too difficult and too much work to be practical unless it can be reused many times in many different applications, so I will try to reject anything application-specific from the design. I also realize that I was working at too low a level; instead of creating a class to represent each kind of object in the data structure, I should create a class to represent the concept of a kind of object. My kinds are now objects, not classes, and each object contain a description of what sort of content might appear in an object.
The objects of the data structure are now all uniformly instances of a single class. For example, call that class "Thing." The various roles of the objects are represented by a field of each object that points to an object of class "Kind" that represents that role. So, to represent cars, trees, and houses in the server data structure, the server would have three objects of class Kind: car, tree, and house. And each Thing object x would have a field "kind" and x.kind == car, or x.kind == tree, or x.kind == house. The Thing object stores the details of a car or a tree or a house in an uninterpreted way, like a hash table from field identifiers to objects of class Data, where Data is a totally abstract class with very few members. Thing can't do much with Data objects, but each Kind object knows what Data objects its Thing should have, so it can downcast to extract the necessary values for serialization. The real power of the Thing class is that it can keep track of permissions, modifications, and fields that point to other Things, and so all the difficult parts of the system no longer need to be designed and implemented for every kind of object in the structure; it can all be done just once in the relatively simple Thing class.
This seems like a powerful and promising technique and I have rushed ahead with an implementation, but questions and doubts nag at me and I want to look into how this sort of problem is handled by people on the Internet. Unfortunately I don't know what to google to get the answers I want. One question I have is what sort of structure Thing should be. My first thought was to make it two hash tables, one mapping identifiers to Thing objects and one mapping identifiers to Data objects. Then I realized that a list of Things would be needed by almost any application. Do I have the applications create a linked list where each cell is a Thing with a reference to one of the Things we want to store and the next cell in the list? If I do that then perhaps I should take a lesson from Lisp and simplify the Thing structure to nothing more than a cons cell, where the car can be either a Data or a Thing and the cdr can be either a Thing or a null. Perhaps I should even have more than one Thing subclass. One subclass could use hash tables while another stores a list of Things, but this seems to complicate the system and would introduce more downcasting which repels me because I have a natural fear of downcasting.
Using cons-cell-Things seems to simplify the server, but it might complicate the clients. To make a meaningful change, a client would need to send many small changes to update all of the cells involved. The client would need to acquire a lock to prevent other clients from reading or writing until all of the changes had been completed to prevent other clients from seeing the structure in an illegal state. Clients also need to interpret the change notifications that they get from the server. When a client gets a sequence of cons cell modifications such as new car and cdr values, it would need to figure out whether an item is being added or removed from a list so that it can update the user interface. Even if the client knew that an item was being added to a list, determining which list was being modified would be troublesome when all it has are cons cells.
It's possible that Kind objects can contain more than just serialization capabilities. A Kind object could also have a reference to objects that perform complex operations upon a client's request and sends out appropriate messages. This could help the system be customized for specific applications and also allow lists to be made out of cons-cell-Things without great burdens being place on clients. On the other hand, it seems like a step backward, away from a general solution and toward a solution that needs a large amount of application-specific work.
Another issue about lists is that I can imagine a wide variety of permissions one might have with them. In comparison, a hash table seems quite simple: you either know what value is associated with an identifier or you don't. In a list, you might know all of the list or none of it, or only the first element, or only the last, or only the first three elements. You might know the content of the list without knowing the order they are in and you might know the number of things in the list without knowing what they are. How can I have a generalized solution to permissions for lists in the data structure?
Most of all I'd like to be directed to examples of systems like this that are well designed so I can learn from them, and I would appreciate learning the correct terminology for this sort of project so that I can do useful web searches on this topic. Any comments or ideas would also be appreciated.
