We have had 2 support issues along the lines of:
"My assessment is set up to use questions from question pools, but when I preview it, the questions that appear aren't in the question pool that it uses."
The way the T&Q UI is structured, users (and support staff) were under the impression that an assessment that draws questions from a question pool does so whenever it's previewed or published. However, how it actually works is that when a new part is created that uses questions from a question pool, the questions are copied to the assessment. The selection of questions is not updated unless the part is edited and saved again.
In the worst-case scenario, we had an assessment that had been copied across successive sites and contained parts and questions that dated from 2007, although the related pool had been completely changed since then. The "old" questions appeared when previewed or published rather than the new questions.
We believe that this mismatch of user expectations with actual behaviour needs to be resolved in one of two ways to remove the potential for confusion:
1. Change the UI to clearly reflect that the questions have been copied into the assessment from the pool at a specific date/time, rather than linked live to the pool. For example phrasing such as:
The questions for this part are generated randomly from a question pool.
could rather read something like:
The questions for this part are generated randomly from a set of questions imported from the question pool "pool name" on XX-MMM-YY.
Similarly in the Add/Edit Part page, it appears as if the part is connected "live" to a question pool, whereas it could reflect that the questions were imported from a named pool on a particular date, and saving the part will update the questions.
2. Change the behaviour so that the assessment is linked live to the question pool, and the questions are generated from the pool or re-copied whenever the assessment is previewed or published.