Design of large projects and tracking errors

[Kanoisa]

Hi everyone,

I'm not so sure if this belongs in here or in computer science but I think as I want to use c++ this may be the place.

I was wondering if anyone could point me towards some tutorials/references or books on methods for approaching the design of larger projects.

I keep finding that part way through a development of a project I have to change some amount of the functions to support new features I want and editing classes and this sometimes causes me to have to "hack" code to get some things working how I expected again. I have tried various approaches to debugging by utilizing visual studio's built in debugger to watch variables and using both a debug log file generator in code to record to a file the results of various operations and also using #ifdef tags to mark debugging console printouts per module i.e. as below (code here is typed not copied pasted so its to demonstrate the method and may not be totally correct code but I hope its right)

//using my errorLogg function
/*pass typedef'd struct of openGL settings to my openGL class which will then call the init functions i need set up view mode, pixel format, required features etc*/
if(!initOpenGL(oglSettings))
{
   errLog(OPENGL_INIT_FAIL);//OPENGL_INIT_FAIL is an enum value i have for my errLogger
   cout<<"Error: "<<enumToInt(OPENGL_INIT_FAIL)<<endl; 
   exit(-1);
}

//using #ifdef's
#ifdef _OGL_DEBUG_
if(!initOpenGL(oglSettings))
{
   cout<<"OpenGl failed to initilise\n";
}
#else
if(!initOpenGL(oglSettings))
{
   exit(-1);
}
#endif

I think the error log approach is more effective as I can provide more information and of course it does not require a recompile to simply debug, but as I have no real software experience being self taught I don't know how its done. I've seen both approaches used in places in code I looked at.

My code goes from usually quite neat and tidy to a complete mess and I find as the project develops I spend more and more time finding the right variables I want to use or functions and get lost. I also find that I spend more and more time adding error enums to my error logger which becomes a massive switch statement.

Recently I have also looked into exceptions but have not used them as i'm not comfortable with them, would it be a good step to use them in place of or in conjunction with my error logger? or in the #ifdef where I have no debugging set throw an exception when the else version is used so I still get indication of an error?

Would anyone consider these design issues or a lack of experience as I have only done smaller code projects before and need to learn by experience what to use and where because if so I guess I need a heck of a lot more experience?

The design approach i'm taking is a bottom up design (i think thats the correct name) in that i start with basic functionality and then expand features as i go, i tried a top down design where i knew the features i wanted but then got lost as where to start to impliment everything seemingly at once.

Any help or advice on any of the above items would be greatly appreciated and if this is the wrong place for my topic could you advise where to move it and how to delete this thread as I couldn't get a previous thread to delete only edit :(

Again thanks for your time and effort in helping me

[Taywin]

In my opinion, I don't think it is different for anyone who works on large project. The only different is how you prepare for a project. In pure computer science, there is no class that teach you how to create or develop a large project, but you learn about how to solve problems, implement, and develop/improve techniques of how to solve problem. In software engineering, they teach you more abstraction in how to create, design, and standardize stuff but not very deep in how to deal with implementation.

One thing I learn from software engineering class (and I still do not really like it :P) is that a project should be divided into 3 parts - design, implement, and test - before you release it. In design, you need brain storming in order to cover as much need as you want. A good design will save a lot of time later when implement. In most cases for you (and for me as well), the design is not good enough. One reason would be from unforeseen issues or features that you may want to add later on. This is why brain storming is very important. Then, the implementation would be another issue later on, but it is not that bad (at least for me because I like to implement). I think the implement part is most likely for computer scientist. The test part is boring for many people (but not for me because I also like testing).

So you could have messed up the whole code before you even release it if your design is not up to par. Don't feel bad about it. A large project is supposed to be done with more than only you. Also, the design part could be from others, not from you as well. :)

[thelamb]

I've had the same 'problem' as you when I started (I'm also selftaught)... but I don't really understand what error handling has to do with it... but let me comment on that first:

You created your own macro that in debug-mode prints some information about the error.. this is a fine idea, you could also use assertations (lookup ASSERT). But in many cases, you also want to have some error-reporting in a release build.

I mostly use an exception class for this (CExcept), which inherits from std::exception. I overload operator<< in this class.
I can then do error checking:

void myFunction()
{
    if( hasError() )
        throw CExcept( SOME_EXCEPTION_ENUM ) << "myFunction failed horribly";
}

int main()
{
   try {

   } catch( CExcept& except )  // or std::exception& 
   {
        if( HORRIBLE_EXCEPTION_CODE == except.getCode() )
            err_out << except << "\n"; // in Debug build, this won't do anything
   }
}

err_out either functions as cout, or some file stream object or it could even be a class which emits the message as messagebox etc...
And it implements a << for ( err_out, except ) of course.

This way, if I want to change something as to how errors are handled in general I can do it in one place (Want errors to output to file? Just change err_out).
Also... a lot of functions set some error code (GetLastError in Windows or errno in Linux) with a simple #ifdef _WIN32 or #ifdef __linux__ in the overloaded operator<< you can include this information in any exception that is thrown, but only writing it once (of course only printing if GetLastError() > 0 ).

I hope that gives you an idea of my way.. I don't know if it is the best way, but I really like it, and I only have to write this CExcept class once for all of my projects, and make some minor adjustments for each.

[Kanoisa]

Thanks to you both, some great info there and i feel better knowing its not just me who finds my design sucks later :P. Im going to have a better look at these exceptions as they actually seem quite handy and hopefuly with more expierence we will each write better designs Taywin. :)

Like you i also like the impliment and build phase as from my uni course we were working on embedded systems (mainly siLabs C8051 chips) so building those up and seeing lights flash when you made it work can be very magical (at least to me)

Again thanks for both of your help and suggestingi will take them aboard and keep trying :)

[Bench]

This thread is missing the most important step IMO.

Software development always starts out with the statement of a problem; that is to say that Programming IS problem solving. With this in mind, you cannot expect to design a solution without first defining, in detail, what exactly you're trying to solve.

Most people can usually visualise a small/medium problem, but anything complicated needs more thought; the thought process might go as follows:

• Define the problem
• Define the requirements of a solution (i.e. What tasks/activities do you want to be able to perform?)
• Evaluate each of the requirements
• begin to define the specifics (i.e think about what the solution should actually do)
• Create one or more prototypes as necessary for various bits of functionality - this in itself will often unearth sub-problems which you hadn't previously considered

Not all of those steps will necessarily occur; and if they do, they do not have to occur in any specific order (except for the step where you initially define the problem). The key point to note is that unless you understand at least part of the problem, you have little hope of being able to create a solution.

Just as an example, think about a web browser

The problem: Unable to browse the web

The requirements (Why i want to browse the web and how i want to browse it):
- Specify an URL containing an HTML document
- Connect to the URL via the WWW, then view the HTML document on the screen
- Interact with forms, hyperlinks, etc.
- Automatically load a homepage on startup
- (etc.)

Evaluation (trying to understand all the various unknowns - involves alot of reading/googling/asking questions/etc) :
- Investigate/Define: What is an HTML document?
- Investigate/Define: What is an URL?
- Investigate/Define: How do you connect to the web?
-- Investigate/Define: What is HTTP? How does HTTP work?
-- : what do i need to connect to? how does a web service work?
- What development Tools/Libraries/API's/Frameworks are available which have already partially solved some of these problems?
- Define what it means to interact with web forms and hyperlinks
- (etc.)

Specification (the various bits of the puzzle as separate features which can be treated in isolation)
- Network connectivity
- HTML parser
- Address bar
- HTML Document window
- Menus, buttons, other GUI stuff
- Simple diagram showing the high-level architecture (i.e. client/server, web requests, internal interfaces..)
- (etc.)

Prototypes (The bit where a lot of your understanding will develop from) :
- Simple console app which parses HTML
- Simple console app which connects to a web server and handles a few HTTP request/replies
- a 'mock' GUI front end with the right visual components
- Console apps to test the functionality of various API's/frameworks/libraries

If you've managed to get that far (Bet there's a few weeks work involved there alone..) Then the architecture of the program is probably already in your head, and the design should be able to flow.

[Kanoisa]

@Bench

Thanks allot thats pretty much what i was looking for, one small final question, I generally can do the requirements bit, but moving from there is where i had problems geting to the specification bit. Would you say in your expierence that this all comes down to expierence of participating in larger projects and programming expierence?

Many thanks for a great post

[Bench]

I don't think experience in large projects really has anything to do with it - even very small 1-man-band projects have to go through the same motions; the difference is that mini-projects can probably be sketched on a scrap of paper, and the amount to think about and learn is usually far more managable :-)

Writing a draft specification requires some understanding of the technology which you're working with (Note the word 'draft'. Specifications don't need to be anywhere near complete or concise before you start working - they usually end up being continually changed right up until the end of a project);
You'll probably be defining some of your inputs and outputs in your specification, this can depend on what the API's and libraries you're working with are capable of.

For example, You start out thinking that you want to display something on the screen using Library X, but you don't know how Library X displays something on the screen, then you have little chance of being able to determine what inputs you need until you know how Library X works. Later you decide that the output you thought you wanted is no good anyway, but having found your inputs, and learned something new about how Library X works, you decide that there's a better way anyway, and change the specification to match.

Various ways to reach that point involve spending time trying sample programs (from your own prototypes and/or from trying to understand someone else's), reading documentation/google results, and challenging your assumptions about how things work (that bit can be difficult - usually this is easier if you have someone with you to point out the obvious).

The added bonus of trying sample programs is that you'll probably end up with a whole bunch of mini-projects which can be tweaked, then later on reused; for example, a console-box HTTP client prototype might have to handle sessions, fire out HTTP requests, wait for replies, maybe even handle threads. If your program only wants to retrieve files, then you might just be able to wrap the whole prototype with a very simple "retrieve file" interface - knowing that you've done the hard work in setting up the client already; (This is a reasonably common approach if you like "OO" methodology).

Its perfectly OK to write yourself a long list of things which you don't know how to do - the more the better; keep the lists going until you have as many prototypes as you need to confidently understand enough of the libraries which you're going to use; before you know it, you'll probably have solved 70% of the problem just by repeating this process.