APP
Assessing
Pupil
Progress
   
Discuss the issues.  Hold a meeting in your school.
 
  Home
  News
  Mailings
  Associations
  DATA
  Campaigns
  DAT
  Learning Reps
  Reports
  DATAbase
  Links
  Contact Us

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 
 
APP: What does it mean to you?
 
We are looking to make policy on APP and to do this we need help.  We need to know how APP is implemented in your school.
 
Here are three case studies of the implementation in three schools.  Click to read
 
Case Study 1
Case Study 2
Case Study 3
 
But what are the issues involved?
 
APP – Thoughts to consider (in no particular order)
 
Possible functions of APP
 
  1.  Raising teachers’ awareness of needs of individual pupils: by focussing on the levels reached, the teacher becomes aware of the pupil’s needs, and the focussing would get overlooked without APP in place.
  2. Computing progress made by individual children.
  3. Computing progress made by  a class.
  4. Computing progress made by a school.
  5. Computing the effectiveness of a teacher.
  6. Identifying strengths and weaknesses of a teacher by subject.
  7. Identifying strengths and weaknesses of a school by subject.
  8. Moderating standards and clarifying the meaning of NC levels through the collection and comparison of sample work.
 
Issues Concerning Function
 
The benefits of APP should be compared against the true cost, which includes the time taken to operate it.  It also includes the lost opportunities, and the effect of focussing on one approach at the expense of then ignoring a different approach.
 
The effectiveness of APP refers to whether it achieves its aims and the benefits of its aim; not to be confused with whether they APP system operates as expected.
 
Going Through Each of the Functions
 
Function 1
This function is only relevant if APP covers all children, on the assumption that it is not acceptable to have a policy which sets out to benefit a section of pupils.
It is self-evidently true that by focussing you will observe things, and that teachers will notice that the “seeing of things” followed the act of focussing.  But it is not self-evident that the argument has then been won.
 
  • The teacher may have noticed it anyway.
  • Teachers may have other ways of “noticing things”, eg through their marking scheme.
  • Does APP cover all the skills or aspects of a subject?  If APP focuses on particular skills or a limited number of skills, do the other skills get ignored?
 
In analysing the effectiveness of APP in particular, the devil is definitely in the detail.  It is very dependent on how the scheme is operated: how many children?  How many subjects?  How many features of a subject?
 
Functions 2 – 5
Remember Goodhart’s law: as soon as you measure a variable for the purpose of controlling it, then you will alter its behaviour.  IE teachers and schools will cheat.  Whether this is an important consideration depends on how much pressure teachers are under, and how many will cheat, and to what degree.  If these are important functions, and cheating is a problem, then there is a knock-on effect on cost.  Because a rigorous system of moderation and evidence collection will be needed.
There are other ways of achieving these functions already in place in most schools, aren’t there?
 
Functions 6 – 7
Are these functions rated as important?  Do they get a mention?
There are other ways of achieving these functions already in place in most schools, aren’t there?
 
Function 8
 
This function has little purpose outside of APP, it makes APP operate smoothly, but it is an expensive way of making sure that a level 3 in one school is the same as a level 3 in another school.  [This argument appears logical, but is it missing the point?].
 
Concluding Thoughts
 
  • The workload varies enormously depending on how it is operated.
  •  I guess that once the system is up and running, and if the planning process and schemes of work have been altered to get the APP outcomes, then it is more manageable.
  •  What have been the benefits for the quality of teaching and learning?  Are these benefits clearly discernable?  If they are not clear and obvious, it is not likely to be worth the cost.
  • Even if it is beneficial, it may not be workable if teachers do not have the time.
 
 
 
David Furness
29/9/09
 
 
Case Study 1
Secondary School 1
 
1. How it is used
My main reference will be to the English dept – here is used to:
1.track students’ progress in line with preset targets – these are set by us based on prior data.
2. provide information to put on profiles – this is in an ideal world but fine tuning of synchronisation of our marking an assessed piece and the data being required for profiles has yet to happen-: we are only assessing certain pieces according to our own curriculum and this need to be done sooner to fit in with the first profile where data is inputted mid October.
3. to clarify teacher understanding of national curriculum as we will have internal moderation and moderation overseen by an advisor.
4. See above – to check teacher judgements through moderation meetings and bring them in line with countywide levels as the adviser will help us.
 
2, How APP is being implemented
At present in ICT, maths and science and English. Maths and science still have SATs – in English we do not. As far as I know, every pupil is being assessed through app in these subjects – most of the others have fewer assessment foci than eng – we have 4 subjects – reading writing speaking listening – each of which has on average 7 foci – S & L is yet to be formalised – not until 2012 or 11 but the rough assessment foci have been issued.
In science they are piloting the APP through new and existing schemes of work and if it hasn’t worked well they are re-writing the APP aspects and taking their time to do it as it suits them
ICT have said that APP fits in easily with their existing schemes of work
Maths ?
 
Problem issues
Eng – we have found that a tremendous amount of time is needed to update our schemes of work and bring in elements of AFL and APP as we have so many foci from which we are trying to select for each assessed piece so we do not duplicate what is being assessed nor leave out any elements. Without gained time at the end of last term it would have been a Herculean task and now we are coming to the end of the 2 units we prepared then and time will be an issue.
Also – the language in the APP sheets is not child friendly even to a level 7 or 8 year 9 so we are spending a lot of time rewriting it to help pupils self-assess/be aware of what they need to do to progress etc – we do it ourselves for hours to produce one self assessment grid or cannibalise others we find on the net etc – all of which takes a lot of time.
 
How we will run APP
Some pieces will be done in test conditions – this will prepare them for the new GCSE where coursework is written at school, all pieces are common assessment tasks so everyone in year 9, for example, will have dome the same reading response to poetry (for example)  so moderation will be more straightforward as we are comparing like with like. We are very experienced at moderating GCSE work and will apply the same skills – in fact, whole dept moderation of year 7 and 8 work has been going on for years as we have done no ‘optional ‘ SATs in those years and instead have a portfolio for each pupil as well as common speaking and listening tasks. Now we have a portfolio for year 9 pupils too.
 
Final thoughts – if we are left alone to get on with this and given enough time to recreate our schemes of work, are not asked to assess any more pieces than the 4 written (including reading responses) and 2 or 3 spoken (which we plan to include reading responses in too), are not rushed to assess too soon just for early profiles etc etc then we may be alright – but once it starts APP can take on a life of its own and we should try to be sensible and resist pressure to expand our workload.
 
Case Study 2
Primary School No 1
 
1) To provide a detailed picture of every child’s progress in Reading, Writing and Mathematics.
 
2) To set targets for individual children (using AF’s).
 
3) To clarify teacher understanding of national curriculum levels.
 
4) To check teacher judgements through moderation meetings.
 
5) To measure pupil progress against national curriculum levels.
 
6) To back up/moderate/check(this has not been clear) SAT and optional SAT results. (Yes we are doing both!).
 
7) To provide a fantastically detailed picture (looking through APP glasses) of every child and their progress through APP.
 
Problems with APP at our school (As things currently stand).
 
1) The moderation system, while helpful, requires the collection of evidence for a spread of children across the class. The collection of evidence invariable involves photocopying 4 pieces of work a term from four children across a range of subjects. This is time consuming. If APP is to replace SAT’s one can only wonder what level of evidence collection the government would require to ensure accountability.
 
2) Each child’s progress in the subjects detailed above is recorded on the corresponding  APP sheet.  Progress is recorded by highlighting the bullet points from the assessment foci as they are achieved. 
 
In reading each child has 4 sheets detailing progress from L1-:L2, L2-L3 etc). In writing the methodology is replicated.
 
In numeracy all the levels are covered on one sheet but each sheet only covers one of the 4 areas of mathematics.
 
This means for my class of 31 I have potentially 372 sheets to fill. In reality the real figure is lower as a child is not likely to move from level 1 to level 5 in a year. Hover a child could easily move from the L1-L2 Sheet to the L2-L3 sheet in reading and writing.  This would mean I have a total of 248 sheets to check and highlight a term. The current recommendation is that sheets should be filled in once a term (6 times a year).
 
3) The assessment foci lack the necessary clarity to make a concrete judgement. Teachers are advised to view the appropriate standards file if they are unsure if a particular child is meeting an assessment foci.
 
4) For some children the process of filling in APP grids is entirely unnecessary as the teacher is clear on their level and knows what they need to do to progress. However, the teacher still has to fill in the APP sheets.
 
5) The language in the APP sheets is not at all child friendly and we have had to rewrite each of the sheets so we can set consistent and understandable targets for the children.
 
6) APP is being used alongside SATs and optional SATs. This means teachers in years 1-5 have to mark and deliver SATs as well as using APP.
 
7) We have marched headlong into a complex and complicated system with very little staff consultation and contrasting guidance from the local authority (I am sure other schools have done the same). If the government decides to introduce APP as a replacement for SAT’s then one would expect they would want a standardised system because that seems to be the way they work.  I would be very worried about what happens to these systems if a standardised system is introduced.
 
Concluding Thoughts on APP
 
I have looked hard and I struggle to find positive aspects to APP. I’m still unsure as to its principle aims and messages are mixed. Many of the things we are told it will deliver (such as the moderation of work) are already in place and working perfectly well.  Quite simply APP is a huge workload burden delivering very little other than the recording on standardised proformas what teachers already know about children.
 
Case Study 3
Primary School No 2
 
As a staff we hope that APP will be adopted to track a range of children in the class.  We have currently not decided on how many children this would be, as 3 out of 4 Key Stage 1 classes are mixed Year 1 and 2.  We feel this has an impact on how many children need tracking with APP.
 
Currently we are considering 6.
 
Each Progress Week (we have 3 a year), we would collect evidence for writing for these 6 children and moderate it in teams to give a National Curriculum Level.
 
We hope that this will support new staff with levelling work and ensure that our assessment in the area of writing is fair.
 
Currently we are only looking to adopt APP in the area of writing.
 
Positive Aspects
  •  Support new staff with levelling
  • Ensure teach levelling within school is same
  • Portfolio of work to support judgements at end of KS1
  • Hopefully this assessment will go towards abolishing SATs
 
Negative aspects
  • Increased workload in a mixed class (may have to increase to 8 children)
  • The misuse of APP to track every child in the class (very paper based)
  • Too many tracking sheets for 1 child
  • The vocabulary used on tracking sheets
 
If this is used only to track a small percentage of children in the class it could be a valuable assessment tool in school.  However the second this model becomes the assessment tool for all pupils it will develop into an uncontrollable workload.  It should only be used to ensure teacher assessment levels are correct and fair from school to school.
 
You can e-mail your thoughts on this issue by clicking here.
 
  If you have any further mailings you can contact me by clicking on this link.

All suggestions will be welcome