|
|
The immediate task in the self-assessment process is to ensure that there are questions to be answered. Although this may seem self -evident, it is nevertheless important that there is a ever changing portfolio of questions. Without this questions will get stale and students will have the opportunity to memorize or even worse collude to pass answers from one to another. |
There are a wide variety of question forms that can be created:
- Multi-choice questions. A question is posed and a number of possible answers provided. The objective is to chose the answer that most resembles the correct answer. Usually in multi-choice questions there is more that one answer that could be the correct choice. Therefore the student has to make a decision to select the answer that they think is the most correct one. Sometimes multi-choice questions will have different scores for different answers. The following question is an example of a multi-choice question:
- An organisations mission is?
- Important.
- Only useful at times of disaster.
- Fulfilled by processes that are purposefully designed around the mission.
- Written on paper and displayed in the reception area.
- Short-answer questions. Is a question where a sentence or two, usually not more that 100 words is required to answer the question. Attention should be made to make sure that you (the student) understand the key words in the question. These words will give you a clue as to how to phrase the answer. For example the key word - criticize - is different to - justify - or to - list. For more details about these subtle distinctions you will need to attend the Academic & Business Report Writing workshop. The following questions are examples of short-answer questions:
- Justify the application of the hard HR approach for managing a hospital ward?
- Describe some of the effects of organisational change on employees?
- List and summarize the content of a project charter document?
- True-False questions. These questions require that a single answer is provided, this could be a yes/no or true/false. The following questions are true-false questions:
- Every organisation has a mission? TRUE/FALSE
- The mission is fulfilled by processes? TRUE/FALSE
- Process are a series of work activities? TRUE/FALSE
- Work activities create products and/or services? TRUE/FALSE
- Matching questions. Are questions where you need to match the correct answer from a number of possible answers. The following questions are matching questions:
- When closing a project it is important to record?
- The charter must include?
- EVA means?
Earned Value Analysis / Lessons learned / The name of the sponsor
- Numerical questions. These questions expect that the answer will be a number, date, quantity or something of numerical value. The following questions are examples of numerical questions:
- What percentage of allowable fault represents six sigma?
- What decade was Taylor's scientific model popular in?
- If a sales person is 'on quota' what percentage of target has been achieved?
- Missing word questions. The missing word question requires you to complete a sentence by inserting a missing word. The following questions are examples of missing word questions:
- When a sales manager allocates ______________ to sales people then they need to focus on building relationships with customers in those ________________.
- The ___________ is a financial document that shows the assets, liabilities and equity at a point in time.
- A ________________________ is the ultimate authority on a project.
|
To record the questions a file format known as a GIFT file format is used. GIFT is the most comprehensive import format available for importing quiz questions from a text file into the learning management system (LMS). GIFT supports Multiple-Choice, True-False, Short Answer, Matching and Numerical questions, as well as insertion of a _____ for the Missing Word format. Various question-types can be mixed in a single text file, and the format also supports line comments, question names, feedback and percentage-weight grades. GIFT is a format that is accepted by the Moodle LMS. |
|
An important attribute of the self-assessment questioning mechanism is to ensure that there are different balances or richness in the questions within a question set. A question set could be a specific subject grouping of questions round a certain topic. It could be a particularly important set of questions that carry a higher weighting for Vocational post-graduate students, or a lower weighting for Certificate IV students. In other situations the same set of questions could be applied to different subjects. For example business management questions could are applied to both project and business management subjects, in each case the set of questions can be weighted with differing mark weights. Key questions sets that must be answered can be designated as red flag questions, thus ensuring that each student undergoing the assessment is forced to answer some common questions.
The weighting on question sets is transparent to students, and become important when the GAP analysis is performed. The result of the weighting can vary depending on the effect that the assessor or the organisation (if the student is working within an organisation) wishes. For example, a state government department may wish to weight questions differently for successive tiers of administrative officer position. So that an administrative officer 5 (AO5) role is weighted differently to an AO8 role. The weighting has an important impact on the GAP analysis mechanism and is intended to enable a fair and equitable representation of the students performance.
The second step in the process is to apply the weighting to the GAP analysis tool. |
Once the questions have been written, along with their answers, then the GIFT file is uploaded into the LMS. The upload can be performed in a batch basis or performed interactively. The questions are stored in the LMS host database according to the designated question set and available for students to undertake their self-assessment. |
The learning management system or LMS host server is the primary repository where all questions, courses, and student details are stored. Residing on an internet or intranet server it is maintained in a secure environment. Students, teachers and administrators can access the LMS by login in and use an interactive interface that provides a virtual classroom for courses and the learning experience.
|
|
The third step in the process occurs when the student undertakes the self-assessment. Each student has a unique identity in the LMS and can be allocated, by the assessor or teacher, assessments for them individually or for their class.
Once the student logons to the LMS they then select the self-assessment instrument and answer the questions as they are prompted. Questions can be repeated, or randomly selected depending on the teacher or assessors requirements. Immediately and as part of the question and answer process the student can be given feedback about their answers and results. Hints, repeat attempts, prompts and the correct answers are all provided by the LMS to the student. The duration of the assessment is recorded and time limits can set by the teacher for answering questions. When this happens the student is warned of the limited duration or time limit for the assessment and a count down process activated. |
|
The questions, answers, date and time, that the student takes are recorded in the LMS. Both the student and the teacher can view the results of the assessment interactively or after the assessment process. For teachers or assessors logs and tables are available to easily look at groups of students and their recent activities and statistical tables and simple analysis can be easily performed. Automatic grading is another nice feature whereby result ranges, from single or multiple question sets can be allocated specific grades. Students then immediately upon answering a set of questions can be graded and even awarded passes to classes. |
|
The database used in the LMS is a highly secure SQL database accessible only to the LMS administrator. Encryption of student details, passwords, and identities are managed by the LMS. The data structures contained in the LMS are normalized into a form that can be accessed by a structured query language (SQL) program with the appropriate permissions. Thus allowing complex external results analysis (the GAP analysis process) to be performed. |
|
ODBC or online database connectivity is a programming interface that enables applications (programs) to access data in a database management system that uses SQL as a data access standard. The ODBC driver connects to the database and according to the SQL statements supplied will select, update, insert or delete data in the database. Each type of database ODBC driver has been written for a specific database. Many drivers exist and obviously the correct driver must be used for the database that storing the self-assessment questions in the host LMS. |
|
The weighting and analysis server holds summaries and histories of the assessments that have not only been downloaded to it, but also of sample question sets. As previously discussed in step 2b, generating the weightings is an important feature of the assessment process. Once the latest self-assessment for a student has been downloaded to the weighting and analysis server then a comparison of those results with the history of similar students and the weighted question sets is made and an assessment of the student performed. |
|
The GAP or (Groups, Algorithms and Programming) is a computer algebraic system for performing computational discrete algebra. There is an emphasis in GAP for the type of computational algebra that is made through computational group theory (or the study of groups). More specifically the GAP analysis process in the self-assessment is performed around abelian groups (sometimes called commutative groups) to generate a permutation group for the set of questions, weightings, histories and student answers. Group theory is the branch of mathematics concerned with the discovery of groups and the description of their properties. For a good quick and brief explanation of group theory please download a PDF file explanation from the Rutgers Professor of Mathematics Charles C. Sims 1998, Computational Group Theory, Rutgers, the State University of NJ, USA.
Although this may sound awfully complicated, there is an important reason for going to this effort. Naturally, it is as desirable as possible that in performing a self-assessment and producing a comparison we ensure that there is a fair and equitable representation made for the students self-assessment results. The GAP process, the weighting, the historical sets are all important components in trying to make the process as fair as possible (also see the comments associated with Step 2 Generate Weighting). The underlying mathematics is ultimately significant as we want to be able to confidently state that "the current performance of student [S1] in relation to other students [S2, S3, S4, ..., Sn]" is accurate. In this sense the groups being represented by the data must have the appropriate relationship properties and those relationship properties must be valid according to the principles contained within group theory. |
The radar or spider web chart is useful for rating an item or items along 3 or more axes and is used to represent the current state of the students results. Although it is very tempting to make immediate interpretation from the radar chart, we should be careful considering all the factors in the light of the weighting and the GAP analysis performed. Nevertheless, the radar chart is a great way of quickly generating a pictorial map of this student in relation to other students. The following diagram illustrates the performance of a student in relation to other sets of students at the AO5 and AO8 level (this example was discussed previously in step 2b). The target represents a weighted target set for this student. The weighted set could be unique to this student or common to other students groups. |
|
In conjunction with the radar chart an information report is generated. This report with the radar chart is then used to evaluate the students performance and the next actions that should be performed. The format of the report can vary but is presented in a similar format. The example below shows the major headings:
|
Self-Assessment
|
| Student Name: |
Student ID: |
Date of Self-Assessment: |
| QSet |
Question subject |
Y/N or Score |
Red Flag? |
Major/ Minor/ Wt. |
Comments/Notes |
|
As a consequence of the self-assessment process students will be able to evaluate their performance and study results with their assessor and develop a self-improvement plan. As has been extensively mentioned, it is important for students not to place too much reliance on the output from the self-assessment process and start to become discouraged or de-motivated. The whole process is subjective and can be artificially skewed by the questions, weighting, and the GAP analysis process. What is important is that students and their assessors use the process over time and improvements will become apparent and the student will start to learn. Learning without making mistakes is near impossible. Learning is a discovery process, and to quote Samuel Smiles (1812-1904) the Scottish author and social reformer who said "we learn from failure much more than from success; we often discover what we will do by finding out what we will not do; and probably he who never made a mistake never made a discovery."
The following table is an example of the headings from a self-improvement plan:
|
Self-Improvement Plan
|
| Student Name: |
Student ID: |
Date of Self-Assessment: |
| Learning Aspect |
Findings |
Impact |
Recommendation |
Action Plan (to be done by student): |
Other parties |
| |
|
H/M/L |
|
Action item |
Person Responsible |
When implemented |
|
To undertake a self-assessment a student should follow these steps:
- Access the LMS and complete the questions in the self-assessment test for their current competency.
- Review the test results online and repeat the test if necessary.
- Once the test is complete and if they wish request a GAP analysis.
- Review the GAP analysis assessment results.
- If the results meet their needs for competency then start the self-assessment process for the next competency.
- Otherwise, meet with their assessor and create a self-improvement plan then update their evidence plan.
- After undertaking activities advised by the assessor in the self-improvement plan (such as undertaking a workshop session) then go back to step 1 and undertake another assessment.
|
| |
