This is some initial training on our Quality Monitoring and Evaluation Platform (QMEP) but the second phase of the project which is focused on annual reports that our External Examiners submit, I'm going to talk through some introductions, give you a bit of information around why this project took place and then I'll pass to Jen, my colleague, for the second half so that Jen can talk through some of the step by step guidance that you'll be needing to follow when you come to submitting your annual report.
At the end of the session we're going to have time for question and answers so we'll obviously open it up to the virtual room. What we have been doing in previous sessions is of course please go ahead and use the meeting chat if you want to pop questions in there as they come to your mind and just so you don't lose sight of things but if okay we will kind of go through all of the chat and answer those questions in that part at the end of the presentation just so Jen and I are able to fully engage with that rather than flitting between the chat and presenting so I hope that's okay with everyone, thanks.
So hopefully you know these names already but we just wanted to introduce our External Examiner team. We sit within the Centre for Quality Assurance here at the University of Derby and these are just the key contacts that are involved in the External Examiner work stream. So we have our Head of Academic Quality, Richard Monk, and our Quality Manager who's here with us today, Paula Bushby, so they're in the Senior Leadership team and support this work stream. There's myself so I'm the Quality Monitoring Manager and I lead on the key projects for External Examiners and manage the team operationally. For those of you that are new Externals and have attended our central induction you've probably come across some of these names already, Jen Kerr is with us and she'll be talking after myself and Jen is our Senior Quality Monitoring Officer and then we have Helen Crooke who's our Quality Officer and then five Quality Administrators, so they're the names that hopefully you're used to interacting with when you email our External Examiner mailbox and take you through lots of steps throughout your tenure here at Derby. That's our generic mailbox at the bottom of the slide, hopefully you're familiar with that and we'll talk about that more as we go through the presentation anyway.
So I'm just going to give a bit of a recap, hopefully this is information you're all fully aware of but as hopefully you know we had to make an imminent change to how our External Examiners were set up at Derby and this was launched last September, so the official date it went live was the 17th of September. This was on the back of actually the Quality Monitoring and Evaluation Platform project and when the platform was launched it became clear that our Externals were having issues with licensing permissions and access issues to that platform so it triggered us to make a quicker change in quite a short time frame in terms of how External Examiners are set up in terms of their accounts here at Derby, having a mailbox and also how they access our key systems. So hopefully this was communicated clearly, as I say it's one of those projects and things that you wish you had more time for but hopefully that's been communicated out positively.
So as mentioned the External Examiners now have a ment address so you would have always had this ment number but now you have a mailbox that's attached to that account so as stated in communications this is the main point of communication in terms of us with our External Examiners and will also be where the automated notifications go to for the key systems that you need to use. Your ment number is provided by us and it is detailed on your initial appointment letter following your approval of tenure but if at any time you're unclear on what that is then of course you can contact our team and and we can provide that. As mentioned, that ment number, so those credentials and login, you use those to sign into Blackboard Ultra which is our VLE and also QMEP which is the platform we'll be talking about today.
As mentioned previously, all of the automated notifications to do with these platforms will go to your ment email address as well so as you know we've been encouraging External Examiners to check that regularly and make sure it's an email that you use alongside others. We've been picking up lots of tips and tricks along the way so there are guidance and things we've provided to Externals and can also do so around merging that mailbox to your other institutional one in Outlook and receiving notifications a bit more smoothly so hopefully we can support you further with that if you need it but the support we've been able to offer so far has hopefully gone down quite positively.
So where are we now, as mentioned our team has been supporting our Externals as much as possible with account setup and access and we've also established a really good working relationship with our Digital Services and Solutions team so that's enabling us to escalate issues quite quickly. Obviously blocks and access issues can be linked to different things so we've even been doing Teams calls with Externals so we can share screens and support you and there's guides and information that we've been sending out as needed as well. The more that that mailbox is accessed hopefully you won't miss important communication and the notifications that will trigger that knowledge to you that work needs to be actioned. Also just to state that auto-forwarding hasn't been permitted and that's a University decision based on their security policy so that's not something we're able to enable but as I said we have been sharing guidance to kind of support with other ways of accessing that mail.
I'm going to move on to QMEP which is our Quality Monitoring and Evaluation Platform, so this is an online application that has simplified and streamlined our essential Continual Monitoring processes, it's digitised our previous process and as External Examiners hopefully already you've had touch points with this system and it's something that you're familiar with and have used already.
In terms of the QMEP system you would need to access to complete assessment verification and also the moderation of assessed student work. So just some key points we've been sharing on this and hopefully to kind of expand your awareness, so in terms of assessment verification and moderation the process steps through the platform and that would be created by the Module Leader, would go to an internal verification stage and then onto you as an External Examiner for assessment verification. The documentation that you would need to review tends to be included within QMEP so once a Module Leader creates a form anything that you need to view in terms of assessment verification so exam papers or assignment briefs they should be attached within that form in QMEP and you should just be able to view that before you answer the questions that are required. For the moderation of student work we still use the External Examiner folder on Blackboard Ultra for that so that's why you will still need access to our VLE. So the form will be triggered within QMEP and the questions that you need to complete will be within that platform, there's also an option within the form for a link to be shared with you to Blackboard but as I say that sample of student work is still in the place that you would have accessed it to date and and that hasn't changed.
So just to make you aware that QMEP is also used internally for self-evaluation so that's at a module level, programme, School and at College level and it supports a lot of our reporting into Committees and other aspects as well.
So the purpose of today was to set the scene for the next phase of our project, so QMEP was produced working with an external company, we had a little bit left in the pot to expand the project a bit further and we're terming that Phase two, which has taken over Jen and my world for quite a while now. So the focus of phase two has been to digitise our External Examiner reports and responses process and we've currently come to sort of the end of the main project phase. We've been doing some testing and we're going into the pilot phase but the aim is for it to go live and be ready for our External Examiners to submit their annual reports for the 24/25 academic year.
So, the purpose of today is to give you that initial introduction and just to highlight really, that further communication will be coming around this. And there will be other opportunities for training, and resource and guidance, for you as Externals, to be supported through that process.
So, the benefits of this, the main focus of digitising the reports and responses was to have all of the things as an External, that you do as key tasks throughout your tenure, within one platform. So, it would mean that all these processes now are on QMEP and hopefully that's beneficial for Externals, but also for our Derby staff and our academic partners as well.
So, you can access most of the things that you need to do in that one place, in that one system. The process in terms of submitting basically, like the manual process that you do now, it's not changed too much. There's a slight tweak, that I'll go into on the next slide, but it's mainly been digitising that process and hopefully making it more seamless and embedding that within the QMEP platform. So, the process flow is automatic, and that will include the sending of email notifications and reminders to relevant people, so that hopefully that process is more efficient.
As with QMEP already there's a system that we use internally called PeopleSoft, and that enables a pre-population of key details within QMEP. So, for your awareness PeopleSoft is updated by myself, or the team that we work with, and they will make sure that when your tenure is approved here at Derby, all relevant details are inputted within PeopleSoft and this hopefully means things flow into QMEP accurately, and is all up to date. So, if there were ever any problems with that data then our team would be able to amend that, so that means the annual reports that you will create within QMEP will be prepopulated where possible. With that data we're hoping that it means the process is more streamlined, and more than anything, involves less emails, so not only for yourself but for our team, and and other colleagues as well.
So, basically the External Examiner will create their annual report within QMEP, and Jen will give some more information about that shortly, and then you would receive your formal response within QMEP as well from the Programme Leader, or Head of School if they needed to be involved. So, hopefully less emails and a slicker process for all.
One of the positives about digitising this process is that we've been able to tailor the form types within QMEP, so the questions within the current word document that you will see at the moment covers all types of External Examiners, as well as where you are at, what point in your tenure so sometimes there are questions that aren't applicable on that hardcopy form, but by digitising it within QMEP, we've been able to kind of set this up. I think there's about six form types. So whether you're a new external, existing external, you have EPA duties, hopefully those questions are more tailored now and more specific and you have the ones that you need.
The process changes that I've mentioned is something just for awareness really, but we submitted a change to our Academic Development and Quality Committee (ADQC) before Christmas now, and it was approved. The purpose behind that change was to do with the response to External Examiners for their annual report. So, our standard process has been that once you send that in via email, our team will do a quality check and then we send that onto a Programme Leader for their response. Our standard process was that the Head of School would also need to approve that response, before it comes back to you as an External. What we found was that in most cases that response wasn't changed, there was nothing that needed to be updated and that there was sometimes delays with getting that response back. So, because of the reporting loop, and trying to make that more efficient, the approval at ADQC has meant that once that a Programme Leader has done that response, it comes straight back to you as an External Examiner. However, if there was a major concern raised or something of concern to you as an External, that is when the Head of School involvement would be included, and they would need to obviously get involved in formulating that response and approving it, before you receive it. But if it's a standard report, that's hopefully a quicker process that's been implemented.
All External Examiner duties will be in one place. Hopefully, that's better from a user perspective and there's also the option to access your annual report once you've created it, and the response from the UoD member of staff. So, you're able to access that and view that as a PDF, as well as emailing that to yourself as well within the platform. So, you can keep a copy and that would be emailed to your MENT email address.
So as mentioned we have been doing lots of testing and we did some testing with a group of our External Examiners here at Derby, and here are just some quotes that we received on the back of that testing. So, it was great to have that engagement and obviously we wanted to test that functionality from an EE perspective and and make sure that that was a positive experience and seamless where possible. So, the feedback we've had is that it was easy to complete and the workflow seems quite logical, and really that it's great just to have it all in one place and makes everything a little bit easier in terms of that workflow.
So, our next steps. We have obviously been running some External Examiner training sessions. So, we did two last week and it's our final one today. There is a plan to launch another two of these sessions, so I think we're going to be doing one around June time, before that sort of main point of submission for some annual reports, and then around September time, for those that have like postgraduate duties.
Today's session is being recorded and as mentioned this will be shared with other Externals and will also be available on our EE-website. As well as over the course of the next couple of weeks to a month, there will be further communication from our team to you all and following this session, there'll be a user guide that we'll be sharing, which will cover some of the points that Jen goes into, but it's basically a guide for you to use when you're submitting your annual report, and this will hopefully support you in that process. Then as mentioned, we're hoping that for the next cycle of annual report submissions, so for the 24/25 academic year, these will then be completed and submitted within QMEP, and we can go live with it as planned.
So, I'm now going to pass over to Jen, who's going to go into a bit more detail about the system and step through some of the guidance linked to that. So, thank you Jen.
Thanks, Laura. Yeah, we're just going to step through a bit more of the process in terms of how you would submit, and we've got some screenshots from the platform as well, to give you an idea what that's going to look like. We're aware today, that some of you might be new External Examiners. This might be the first report that you're going to submit, or you might be partway through your tenure, or at the end of the tenure, and this will be your last report, but we just wanted to recap some key points.
So, all External Examiners are required to submit a report within 4 weeks of the main assessment board, and like Laura said, this will now be via QMEP. Just a few key points to note, and this was the same for our previous process when it was on the word template, and it'll be the same for now within QMEP, so just to be mindful when you're completing the report to be specific in the free text boxes as you make comments about the modules, programmes, locations and delivery methods, that your comments relate to just so that it's nice and clear for the relevant programme teams that'll be reading.
We ask that you please ensure your report contains clear feedback on academic standards, advice on good practice, innovation and areas for enhancement, because this will be really helpful for us, and the programme teams who are developing those programmes further. We're aware that as part of your duties some of you might have a range of programmes, and delivery methods as part of that. You might have academic partners as part of your duties, and apprenticeships or online learning. So, again we just mentioned that any comments that you make within the report, it's really clear which element or delivery method you're referring to. Just to note though that all the academic partners would be able to to see those comments, if they're part of your duties so, just consider that while you're writing comments. On the same theme, we ask that you please don't name any individuals. So, student names, Programme Leader names, you don't need to name anyone specifically in your report. And just to note that the University may use anonymised extracts from your report in promotional material, or for the purposes of internal or external audits.
So, I'm just going to have a look, kind of step by step, now about the process of submitting a report via QMEP, and what that'll look like. Hopefully, you've all accessed QMEP and used that as part of your External Examiner role already for verification and moderation purposes. So, you would access the platform in the exact same way as you do now. Currently you would see ‘My Tasks’ when you log into the Home screen, when we go live and launch the report form within Home you would then see a new button and that would be this, ‘Create an Annual Report form’, and it will just be the External Examiners that see this, because it's the role of the External Examiner to start that process and create the form. So, you would select that option when you go to the Home screen. When it's time for you to submit your report, there'll then be a few simple dropdown questions for you to complete, so that the report can tailor appropriately for the type of External you are, and your duties.
So, you would be selecting the relevant academic year, the submission point, and the External Examiner type, which Laura referenced previously. So that would relate to if you're a new External; if this was your final report; if you're a current External; or you, if you have Endpoint Assessment programmes as part of your duties. We can, of course, confirm any details as needed though, so don't worry about that. There will then be dropdowns to select the relevant programme or module, that you are the External Examiner for and with all of these you can submit multiple options as well. If you are a programme External you don't need to select all your modules as well, just the programme would be sufficient, but we were aware that some externals just have modules as part of their duties, so we wanted this to be an option to be able to select those.
There will be a dropdown then for delivery type. So, this is how the programmes delivered, so that would include where there's academic partners, apprenticeships, or online delivery included, and again multiple options can be selected, based on the option that you've selected from the dropdown for program or modules. The system will prepopulate the relevant academic staff, so Programme Leaders, Module Leaders and Heads of Schools. If those don't look correct though, you have the option to change those from the dropdown. And, also, our team, when your report comes through, we can make any changes as needed if it doesn't look correct throughout the form. Any questions with an asterisk they're mandatory, so you would need to complete those before you're able to move on through the form on that first screen. Once you've populated all of those, you'll see that the create new form button becomes live. I’ll just show you a screenshot on this next one that just gives you an option of the dropdowns that you've selected, and then the create new form would have been greyed out. But once you've selected all the appropriate responses, that will become live. And that's created that form for you now, with all the relevant questions and you can work through and submit your report.
For External Examiners who have been in post for a year or a few years already, and have submitted reports previously, you'll notice as you go through this, that we haven't actually changed the report itself that much. We did review it and tweaked the odd question as needed, but essentially it's very much still the same format as the word based form, but we've just digitised the questions and the report.
So you would work through the form, once you've gone into that, enter in all the responses. Some would be yes/no questions, some you have text boxes to do your free text. Like I say, some questions are indicated where they're mandatory, and you would need to do a response to those. If you try to submit and you haven't completed one of those, the form will kind of really handily take you back to the question that you've missed, so that you can populate it as you work through the form. You can save it if you wanted to to save and come back to it at any time. That's absolutely fine, but once you've worked through all the required fields you can click submit. That will automatically send that form through to us as the quality team. You willlget an automated email notification noting that we've received that.
As an External Examiner team, we would carry out an initial quality check on that report, just working through making sure all the questions have been populated correctly. Like I say this is where we could change the names of Programme Leader or Head of School details, if they didn't look correct to us.
If we found that we needed to come back to you because perhaps something had been missed, or we just needed a little bit more information, there is an option for us to refer the form back to you. There's a comments function within the form where we could just type the question that we had for you or just ask you to populate a little bit more information for a certain question. Again you would get an automated email just to note that we've referred that back to you. You'd be able to access our comments and just see what else is needed.
If when we do that initial quality check, we feel that you've raised a a major concern within your report, or something serious that needs to be escalated. We would work and escalate that with the relevant program team and Head of School accordingly.
This was just a screenshot to give you an example of the form as you work through, so you can see there's some yes, no options, we've also put not applicable as an option where we felt that was appropriate as well. Hopefully you'll see from the format it looks very similar to when you have used QMEP previously to do moderation or assessment verification. You've got the subway function at the left-hand side where it just shows the different stages that the form is at. You can see this form will start with the external examiner if that little circle is filled in as orange. That means it's at that stage already and is in progress. Once you as an External have submitted that form that would go green. It would move on to the Centre of Quality team which is us the External Examiner team. We would do our checks and then it would move on to the responder. You'll see at the top the different functions that you've got, so the save button where you can save your form as needed. You then have the option to submit once you've completed everything and then the comment's function that we referred to. They would be the three buttons that were appropriate for you to use, so comments would be where we've put anything, if we needed to refer that back to you and again you could use that functionality as well.
Once we've done the quality check then the report would go off to the relevant Programme Leader or Leaders, because there might be multiple if you've got more than one program as part of your duties. They would have six weeks to do their formal response to your report. Once that's been completed, you will get an automated email notification. There will be a link to QMEP on the email as well so that you can go in and have a look at the response. If you've got multiple programs as part of your duties you'd get the responses individually once they've been completed. We feel that's a really positive function so that you don't have to wait for all responses to be completed, you can just get them as and when the Programme Leaders have completed theirs. You would be able to go into QMEP view that response as well and like Laura referenced, you have also got the ability to send a PDF version of the form and send it as an email, so you've got that for future reference as well.
This is just an example of the kind of automated emails, so we've tailored them to each stage of the process, so the appropriate information gets sent out to the relevant colleagues. It would always have a title, that would be your name as the External Examiner, the academic year and the submission point. Then the stage would indicate where that reports at so this is an example of one where it's been through the whole process and it's completed, all the responses are ready and they can be viewed. Hopefully you find those notifications useful just to see where your report is up to and how you can access and go back into it.
I'm going to pass back to Laura then who's just got a few more key points to note in general about the annual report submission process. Thanks Laura.
Thank you, so also just to highlight in terms of the annual report submission, our team will send out our general reminders to all Externals around those dates. We're always there as that support to you and to give you that information. Keep an eye on your mailbox and then that will hopefully support you in knowing when you need to submit your report. In terms of other key points to note, the Program Leader and your College contacts are the people really that will be able to advise you of those specific timelines of when moderation's due but also assessment verification. It's not something that we hold locally because obviously that is very specific to the work you're involved with in terms of the College, and that subject level. However, it is something that we promote in all our guidance to internal staff. We really ask that our externals are provided with that timeline upon the local induction. It really should be at the start of every academic year. We also know that this will be supportive to externals as well in terms of when to access our systems, and when they're likely to get more important emails. Please establish that link with your Programme Leader and ask for that information if it's needed.
If you've got any queries around Assessment Boards specifically, that would be the College Registry Operation teams that would provide that information, for dates of assessment boards and anything that's needed. We've listed the email addresses for each College and our Online Programme Services team. Just to highlight, because we had this query in a previous session, here's not an obligation for every External Examiner to attend Assessment Boards, our process changed during COVID and hasn't reverted. We have College Level Externals that attend our Assessment Boards, and they basically are there to assess the formality of the board, and check that everything's being done with due diligence. Some Externals request to attend Assessment Boards or might be invited if there's PSRB elements, for example they have specific requirements. You might be invited to Assessment Boards, but you might not and the questions within our annual report will give you the option to answer either way as well. If you do need information, it's the College Operational teams that would be able to give you that information.
In terms of your fee, hopefully you know already but you're able to claim your annual fee once you've submitted your annual report. It's just to outline that the process for that hasn't changed so you won't be submitting your fee within QMEP, everything is done as it is at the moment in terms of that finance process. Once all your moderation work is completed and you know that you've covered that for all of your duties and levels of study, you'd submit your annual report within QMEP. One of the acknowledgement notifications will let you know that you can then claim your fee and go through that process to get paid. That will be via the external personnel claim form that's on the finance system. There's a link there but our other communications detail this information as well. You will need your MENT number as a reference. This is just to outline that if we don't receive an annual report submission, we're unable to process your fee. It is really key that the report is submitted and then your fee can be processed on the back of that.
For any queries relating to the annual report process or really anything to do with your External Examiner duties and tenure here at Derby there's our team, that I mentioned available on the external examiner generic mailbox. Any kind of queries or if you need any support please come to that and we'll support you accordingly. In terms of QMEP itself and the system, the mailbox to contact if you're having difficulties with QMEP would be the Quality Monitoring mailbox. It's the same colleagues behind them but it just means that your queries are going to the right place, and we can answer them hopefully a little bit more efficiently.
There will also be guidance and there is guidance on our external examiner website. This is still in the process of being updated with some of the Phase 2 information but will be up to date and where you can access a wealth of useful resources and information linked to your tenure here at Derby.
Thank you for coming and listening, I'm going to have a quick look at the chat but we're going to open up now as well for questions and answers.
QMEP Training Session (May 2025) video
Back to Completing the Annual Report