Hi guys, I'm doing a dissertation on the skills necessary to do a CS degree, within which I'm going to design a website that assesses prospective CS students and their ability to cope with the harder more technical and analytical skills.

I'm basically just looking for some experienced perspectives, brainstorming really helps me. One of the reasons i chose this is to gain some experience of high level website design. Some ideas would be great, because my knowledge of websites is still quite limited. Is there any other programming languages that might be useful? someone said i may find silverlight useful.

So far i've thought that i would quite like to have some form based html questions that i can validate client side with javascript and then server side php. Which i've stored in a textfile but never in a database like mySQL. Which i think i will do here. I also thought a forum might be useful, and i really am looking to increase my knowledge so i would like to learn how to do that. Any further ideas, advice, criticism of my current path or suggestions are more than welcome! thank you.

Dani AI

Generated

A short, practical addendum that pulls the thread into a testable plan.

Separate aptitude from learned skill (as emphasized), then map what you want to measure to recognized CS outcomes. Use a curriculum taxonomy (for example, the ACM/IEEE CS2013 knowledge areas) to decide which cognitive and non-cognitive abilities matter for success in a CS degree. (acm.org)

Target three classes of measures: (1) cognitive aptitudes needed for analytic coursework (abstract/logical reasoning, pattern extraction, basic numeracy), (2) behavioral/personality traits that affect persistence and study habits (conscientiousness/time-management), and (3) observable learning behaviours (time-on-task, iterative debugging tolerance). Meta-analytic evidence shows conscientiousness predicts academic achievement and adds incremental predictive power beyond prior grades or test scores, so include a short, validated measure rather than inventing one. (research-repository.griffith.edu.au)

Build the test like a psychometric instrument, not a quiz. Follow established testing standards, pilot items, and run item-level analyses (difficulty, discrimination, point‑biserial) to remove or revise poor items. After a pilot and classical item analysis, consider Item Response Theory (IRT) and an item bank for adaptive delivery — IRT lets you estimate ability on a common scale and make shorter, more informative tests. (ncme.org)

Treat this as human-subjects research: get institutional review/consent, anonymize data, and store only what is necessary. Document versioning and metadata for each item (when added, times administered, discrimination index, flagged bias) so the site becomes an improving measurement tool rather than a static quiz. (hhs.gov)

Practical next steps that build on advice from and : pick a small set of validated short measures to pilot, recruit a convenience sample, run classical item analysis, iterate, then expand to an IRT-based adaptive model if piloting shows stable item statistics. Log everything and align items to the CS taxonomy so results can be interpreted against standard undergraduate learning outcomes.

Recommended Answers

All 7 Replies

Skill is something you get from your education. Talent, ability, "raw horsepower" is something you have before you get the education. You aren't looking for skill, but ability, if I understand your intent; so I think that skill assessment is probably not to the point. A good assessment tool would indeed be very valuable, though I reserve the right to believe you will not be able to make one that is much above, say, 60% accuracy.

Which brings up the next issue: Are you going to assess your assessment? How?

Next issue: There is a big difference of opinion among educators about what are the educational goals of a CS degree. At one pole, you have folk who think it means providing course work that leads toward a doctorate in CS. At the other pole, you have people who think it means teaching how to program (in a particular language, even, sometimes), how to administer various machines, how to do networking, etc. I have participated in many heated discussions among the faculty and grad students where I was learning CS. You will probably need to clearly distinguish your goals here (you can assess people's abilities in both directions, of course).

I recommend using a free database rather than a text file: Much faster, more robust, deals nicely with multi-threaded access, makes it easier to write queries, etc. etc. It probably even makes sense to keep questions in a question table, along with some columns that allow you to keep track of things like when added to the table, when last asked, how many times asked, relevance (along at least two axes) if you can get such a measure, and probably a few things I haven't thought of. This allows you to use the same basic table access code for all your data, which simplifies your task. I like both MySql and Postgresql, and think either will be adequate to this task.

Wow thank you so much, you are totally right i do mean abilities, and the whole answer you've provided is fantastic. You seem to know a about skills/abilities can i ask you how you would approach assessing them? i've considered looking into things like psychology, hard and soft skills etc but i'm worried it's not enough and that i'm not heading in the right direction.

Measuring skill is reasonably like "easy".

Assessing ability and desire/drive (which I forgot to mention in my first response) are much more difficult. The only tools that I believe in are based on a long time baseline: You administer general tests that try to measure things like innate logic, creativity, verbal aptitude, spacial cognition, the ability to move between concrete and abstract, etc. These tests are given to people in your target age. Then, several times, years apart, you follow up with the participants to see how patterns of test results correlate with employment and satisfaction. This is not something that you can reasonably do on your schedule with tests of your own making. However you may be able to find existing studies that you can either follow up on to measure current status, or even better, maybe longitudinal studies that come close enough to your thesis to make a translation possible.

The other option is to pretend that training/education won't change the score on aptitude tests. (I sincerely doubt it: Training even increases IQ scores at least somewhat). If you make that assumption, then you can administer tests of various kinds of ability to CS students who are on track to graduate, maybe correlate with GPA. You will definitely have to try to avoid measuring anything where taking CS classes gives an advantage, so no programming at all, for instance.

Don't forget to measure some non-mental aptitudes: You cannot be a good CS Student if you are unable/unhappy to spend hours in one place while writing and debugging programs, to pick one obvious thing.

There are a myriad of existing personality, aptitude tests, many with many years data. I would certainly start with a study of that literature. There is a large corpus of data regarding how well various personality types fit into various job categories. Some are more or less "direct", but some are about how well your preferences correlate with the preferences of people who are already successful in a particular job category. One great resource here would be your student councilor's office. At the very least, they should know the names of such tests, and might even be able to point you to related studies.


I recognize that you set out to measure success at getting a CS degree, and I'm mostly talking about success in a career that may hinge on getting that degree. Maybe because I really don't see a degree as a particularly useful goal, but merely as a stepping stone along a career path.

Here's a test I'd like to see the results: Give a general description of a problem, maybe like "I need a way to get from my car to my office without getting too wet/cold/blown/seen-by-paparazzi" or "I need to serve dessert to my friends". The problem is: "What else do you need to know in order to begin working on this?" And the answer is measured only by the number of distinct things mentioned. I would be willing to bet that successful programmers list more things (though I have no proof)

While Stumbling around this evening:
- Alan J. Perlis of Yale University

Seriously Griswolf i cant thank you enough that's a massive help.

Hi guys, I'm doing a dissertation on the skills necessary to do a CS degree, within which I'm going to design a website that assesses prospective CS students and their ability to cope with the harder more technical and analytical skills.

I'm basically just looking for some experienced perspectives, brainstorming really helps me. One of the reasons i chose this is to gain some experience of high level website design. Some ideas would be great, because my knowledge of websites is still quite limited. Is there any other programming languages that might be useful? someone said i may find silverlight useful.

So far i've thought that i would quite like to have some form based html questions that i can validate client side with javascript and then server side php. Which i've stored in a textfile but never in a database like mySQL. Which i think i will do here. I also thought a forum might be useful, and i really am looking to increase my knowledge so i would like to learn how to do that. Any further ideas, advice, criticism of my current path or suggestions are more than welcome! thank you.

A small suggestion, once I made a site involving PHP, and using the PHP XML DOM I was able to store my information in an organized fashion without any MySQL. Just a suggestion for if you want to move out of using simple text files.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.