Journal of the Society for Psychical Research 54 186-198, 1987
The report which we publish below is based on the 1979 report deposited in typescript in the Society’s library where it was available to interested persons on request. The author has now decided that it would be fairer to all concerned if this report were now brought into the public domain. Accordingly we are now publishing it in a slightly emended form and have invited those who consider themselves to be explicitly criticized therein to write a rejoinder—Editor.
In 1979 I visited the laboratory of Dr. Carl Sargent at the University of Cambridge to observe highly successful ganzfeld psi experiments then in progress. I observed 13 sessions, of which six were direct hits. I considered whether the results might be accounted for by sensory leakage, experimental error, cheating or psi. I made observations of the sessions to test these hypotheses. The experimental design effectively ruled out sensory leakage. However, I observed several errors in the way the protocol was observed. Most of these occurred in the cumbersome randomisation procedure. It was not clear how these errors came about. Their origin might have been clarified by either (a) a statement from Sargent or his colleagues, or (b) by reanalyses of the raw data. However neither has been made available. Sargent’s nine ganzfeld studies form a considerable proportion of the total ganzfeld database. In view of Sargent’s unwillingness to explain the errors found, or to make his data available to other researchers, I suggest that these results should be viewed with caution.
In November 1979 I went to visit Carl Sargent’s laboratory at the University of Cambridge. He had carried out numerous ganzfeld experiments with highly successful results (Sargent 1980). Meanwhile I had been unsuccessful in superficially similar ganzfeld experiments at the University of Surrey (Blackmore 1980).
The objective of the visit was to observe the methods and conditions used at Cambridge and compare them with those used at Surrey, to see whether any reason for the discrepancy in the results could be determined. Because of the possibility of a psi-mediated experimenter effect, Sargent and I hoped to carry out experiments in which we would both act as experimenter while using the same subjects and procedure. Sargent kindly invited me to visit his laboratory for a month. The Society for Psychical Research (SPR) provided a grant to cover my expenses while there. In the event I was only able to stay eight days from November 22-30 1979.
During the visit I observed several errors in the way that the protocol was observed and the randomisation procedure carried out. The source of these errors was unknown. After the visit I wrote a report for the SPR (a condition of the grant) which was placed in the Society’s office and was available to any member who wished to see it.
The account which follows is based on four sources of information. 1. My original report for the SPR (which is still available from them). 2. My notes which I made during the visit to Cambridge. 3. My private diaries written each day. 4. Letters between myself, Sargent and other interested parties.
THE EXPERIMENTAL PROCEDURE
At the time of my visit three ganzfeld experiments were in progress. I observed a total of 13 sessions. I either watched the experimenter and subject, or the agent, or acted as one of these myself. The experimenters were Sargent, Trevor Harley and student experimenters (G. M., J. L. and K. R.). The subjects and agents were all of these, plus other students and friends of the experimenters.
All experiments used the same procedure, outlined below, with the following variations. In one experiment subjects could remain in ganzfeld for as long as they wished. In another, sessions lasted either 15 or 30 minutes, and the third involved a study of subject-agent pairs. There was also one session conducted at a private house for the benefit of the BBC and for this the procedure was, necessarily, slightly different. With these variations, the procedure was as follows.
The subject arrived at the experimental room where Sargent or the student experimenter gave them coffee, chatted with them and, if the subject was a novice, explained the purpose of the experiment and the procedure. There was often music playing and the atmosphere was very informal and relaxed. In some cases the subject brought a friend to be agent, but in most cases an experimenter acted as agent.
When the subject was ready the experimenter gave him or her a pre-session questionnaire to complete. The subject then lay on a comfortable mattress on the floor and was prepared for the ganzfeld. Half ping-pong balls were fixed over the eyes with sellotape and cotton wool and white noise was played through headphones, adjusted to be comfortably loud. A red light was shone on the ping-pong balls. The subject was then left alone and the door shut. The experimenter’s and agent’s watches were synchronised from the start of the ganzfeld session.
The experimenter then retired to the control room from which he could watch the subject throughout the session through a one-way mirror. A microphone near the subject’s head picked up everything that was said. This was relayed to the control room and was both recorded on tape and written down by the experimenter.
Meanwhile the agent alone (if one of the experimenters was to be agent) or the agent with an agent’s experimenter, went along the corridor into Sargent’s office to select the target for that session. There were 27 sets of pictures, each containing four black and white or coloured pictures, chosen by Sargent and Harley to be as different as possible from each other. One of these was selected by using random number tables. There were two copies of each set. One contained the four pictures in individual sealed large envelopes, for the agent. The other, duplicate set, had all four pictures in one envelope and this was left in the office.
In each set the pictures were lettered A to D.
Next a small sealed envelope containing a letter A-D was selected and used to determine which of the four pictures in that set was to be target. The randomisation procedure is outlined in more detail below.
The agent took the four large envelopes and the small envelope (all still sealed) to a different building and into a soundproof booth. At a pre-arranged time (depending on the experiment) the small envelope was opened. This contained a letter A-D. The corresponding large envelope was then opened and the agent took out the picture and looked at it for the prescribed length of time making notes on a sheet provided. He or she retained the small envelope with its letter.
The other three large envelopes remained sealed. Afterwards he or she waited near a telephone in another room on that floor of the building.
At the end of the ganzfeld session the experimenter went into the subject’s room, turned off the white noise, removed the headphones and ping-pong balls and gave the subject a post-session questionnaire to complete. He then went into the office and collected the duplicate set of pictures, left there by the agent He laid them out in order in front of the subject and then went through the transcript of everything the subject had said. Each picture was marked by the experimenter and subject together, on a scale of 0 to 2 for correspondence with each item of the transcript.
The various experimenters differed somewhat in their approach to the judging and in the extent to which they encouraged or guided the subject, but in all cases the total score for each picture was added up and the subject then asked to rank and rate (on a scale of 1-100) all four pictures.
Once the ranks and ratings were recorded the experimenter telephoned the agent and asked him to come over. He always used the same words when ringing. When the agent (and agent’s experimenter when applicable) arrived they disclosed which picture was target and showed this, together with the other unopened envelopes, and the letter A-D, to the experimenter and subject. The rank allocated to the target was then known and a z-score based on the ratings was calculated.
The randomisation procedure is briefly described in Ashton, Dear Harley & Sargent 1981 and Sargent 1980. It was rather complex and I shall therefore describe it in more detail.
There were 27 sets of four pictures, numbered 1-12 and 14-28. First the agent selected one of these by taking an arbitrary starting point into the RAND random number tables, and taking the first number between 01 and 28 (excluding 13). This determined which set was to be used.
The pictures in each set were lettered A-D. Which was to be target was determined as follows. There was a pile of 20 small brown sealed envelopes constantly on the desk in the office. Each contained two pieces of white card enclosing a slip of paper bearing one of the letters A, B, C or D. There were five of each letter in the pile, the envelopes being all of the same type and unmarked. The agent (or agent’s experimenter) opened the book of random digits arbitrarily selected an entry point and took the first number between 01 and 20. He counted this number of envelopes down the pile and cut it. He then took the next number in the list, and counted down the pile again, taking the envelope indicated. Once in the soundproof room he opened his envelope and used the letter it contained to determine which of the large envelopes would be opened.
Afterwards the pile obviously contained only 19 envelopes. To restore it to 20 the one used had to be replaced; and by one of the same letter. In four drawers adjacent to the desk, spare envelopes were kept: A’s in the top drawer, down to D’s in the bottom drawer. Like those in the original pile they were, of course, of the same type and unmarked. Their contents were only known by which drawer they came from. After each session the experimenter looked up which letter had been used for that session, took an envelope from the corresponding drawer and placed it in the main pile. In this way the main pile could retain its contents unchanged.
During my visit I observed 13 sessions. Of the 12 conducted at the laboratory, six were direct hits. This is a hit rate of 50 per cent when 25 per cent is expected by chance. Obviously the number of sessions was small but the results seemed to confirm Sargent’s previous high rate of scoring.
OBSERVATIONS PHASE 1. DAYS 1-3
During the first three days of my visit I observed five sessions (not counting the BBC session). During these sessions I did not take part but just watched either the experimenter or the agent. I took detailed notes, intending to compare the procedure with my own. Of the five sessions, three produced a rank 1 or direct hit, one a rank 2 and one a rank 4. That is a hit rate of 60 per cent when 25 per cent is expected by chance (p = -12). These results seemed quite unlike my own chance results.
The whole purpose of the visit was to try to determine the reason for the difference in results between my experiments and those of Sargent. I considered the following five hypotheses and made observations accordingly.
Sargent’s setting and procedure appeared to be potentially far more psi-conducive than mine. The room was much larger and more pleasant. There was music and coffee and the whole environment was much less like that of a laboratory.
It was quite clear that the main experimenter was extremely confident about the expected results and conveyed this confidence to the subjects The experimenter’s role during the judging was also much more active with the experimenters, especially Sargent, often encouraging the subject making suggestions and pointing out correspondences. This would allow for more influence by the experimenter, which might be good or bad. With a skilled experimenter, it might maximise the use of the available information.
The design seemed to exclude very efficiently the possibility of sensory leakage. Duplicate target sets were used so that no handling cues were available the subject and subject’s experimenter were entirely isolated from the agent from the time the watches were set until the phone call was made By this time the subject had made his choice. In the sessions I observed I could see no means of sensory leakage unless protocol were violated. I observed no such violations of protocol at this stage.
Three questions arise here. First does the procedure allow for errors to take place? Second are those errors likely to be important to the results, and third did any errors actually occur?
The sort of accidental errors which might occur include incorrect replacement of pictures in envelopes, errors in the timing, in giving the right questionnaires in the addition of marks or the calculation of z-scores.
First, the complex randomization procedure seemed to allow for errors to take place reasonably easily. For example, if an envelope were incorrectly replaced in the pile this would lead to a bias in the pile which might never be detected. Second, however, such a bias would produce only a small effect on the overall scores.
Third, only one error was observed during this stage. On one occasion, when the duplicate set of pictures was brought in for judging, it was found to contain only three pictures instead of four. The problem was efficiently resolved. J L (a student) rang the agent (G. M.) (in my presence) and asked him to see whether he had an extra picture by mistake and if so, to place it on the ground floor of the other building and then return to his place by the phone. Sargent then went to fetch it and the judging proceeded as usual. This sort of error can easily arise in experiments of this complexity but, if handled correctly like this, could not produce spurious results. No other errors were observed at this stage.
I had no reason to suppose that anyone might be cheating. However, parapsychology is still a controversial subject and it is conventional to consider whether a protocol is proof against obvious methods of cheating, even though completely cheat-proof designs are not to be expected and are probably unattainable. My intention was to look for any obvious methods and to ensure that they were not taking place. In this way I could be reasonably certain that the only remaining hypothesis was that of ESP.
Before the visit I had thought of several possible methods. These involved the experimenter finding out which picture was target, and pushing the subject towards it, or the agent opening a different picture from the one specified by the randomization. I could now see that the experimental design made any of these methods extremely difficult. However, the complex randomization procedure seemed to allow for several methods of cheating. The observations necessary to check up on these were simple and unobtrusive and I believed them to be necessary if I was to convince myself and others of the validity of the results. This led to the second phase of the observations during which I checked various new hypotheses.
OBSERVATIONS. PHASE 2. DAYS 4—5
During these two days I observed a further five sessions. I was subject in one of them. There were three direct hits, one rank 2 and one rank 3. This is a hit rate of 60 per cent, where 25 per cent is expected by chance. For these few sessions alone, the results are almost significant (p = -055). For the ten sessions observed so far the sum of ranks was 17 (p = -016).
Clearly chance was very unlikely to account for these results. Sensory leakage or simple experimental error had been excluded and so the remaining possibilities seemed to be either ESP or cheating. I should point out that it was probably clear to everyone in the lab that I was sceptical about the possibility of ESP. I believe that having a sceptical observer there was not particularly pleasant, but on the other hand the sessions proceeded in a relaxed and pleasant atmosphere and the results were not adversely affected by my presence. I did not tell anyone about the specific hypotheses I had in mind. I hoped only to make some simple observations which would exclude them to my own satisfaction.
With this in mind I considered whether any simple methods of cheating were possible within this experimental design. I considered the following hypothetical methods and ways of detecting them.
These methods would all be detectable. If (b) occurred a small envelope would disappear from the drawer during the trial instead of afterwards (during replacement). The pile might also remain at 20, instead of 19, during the trial. Or if it were reduced to 19, two envelopes would be used during one trial instead of one. Also the pile would become biased because the one removed would not match the one later replaced.
All these methods involve violations of protocol. Some would be easily detectable and I therefore decided to make certain simple observations which in no way interfered with the running of the experiment or with anyone’s privacy. If I found no indications that any of them were happening, then I could be reasonably confident that the results were due to ESP.
The effects predicted were as follows:
To check on these last two possibilities I decided to count the numbers of envelopes in each drawer both during each session and afterwards, and to watch the replacement procedure whenever possible.
To recap—my hypothesis was that if cheating were taking place I would expect envelopes to disappear from the drawers during, rather than after, a session, or for more than one envelope to be used for each session.
The main pile did not seem to be marked and was not switched during these two days. I counted the envelopes in the drawers from session 8 onwards. The results are shown in Table 2.
From this table it can be seen that between sessions 8 and 9 two envelopes, not one, disappeared from the drawers. One was a ‘D’ (which is correct to replace the ‘D’ which was target for session 8). The other was a ‘B’. This was target for session 9, but of course the ‘B’ for that session should have come from the main pile; only being replaced later by one from the drawer.
I later observed the replacement procedure and this was carried out correctly. i.e. another ‘B’ was taken from the drawer and placed in the main pile.
If the ‘B’ for trial 9 had come from the drawers instead of from the main pile (as hypothesised in 2b above) this would probably result in there being an extra ‘B’, instead of some other letter, in the main pile. I was unable to check on this at this time.
I noted certain other problems all concerning the same trial. During this trial (No. 9) I stayed with the experimenter (K. R.) and watched the judging. Sargent was not officially taking part, but he came in during the judging. He said he wanted to help, because it was a particularly difficult session, the subject having said only a few words. He seemed to push the subject towards picture B. I wrote this observation in my notes at the time and K. R. independently mentioned it tome as well. Note that I wrote down this observation before I counted the envelopes in the drawer. Of course this ought not to matter because Sargent should not have had any way of knowing the identity of the target (but see later).
On the same trial there was also an arithmetical error: it was later discovered that the experimenter had added up the marks wrongly. Picture B had not been given the most marks and so this session should not have been a direct hit. When he discovered this Sargent checked the addition for all previous trials and found nothing else wrong. Rejudging would be one way to clarify whether there really was a good correspondence between the subject’s mentation and the picture B. Sargent said that he intended to do this rejudging.
OBSERVATIONS PHASE 3 DAYS 6-7
I intended to continue observing. I also considered asking Sargent whether we could open the envelopes in the main pile to see whether it had become biased as hypothesised. However, Sargent became ill with ‘flu’ and was away on Day 6. I was therefore unable to observe any more sessions or to ask him about the main pile.
In Sargent’s absence I discussed the experimental design and its potential problems with Trevor Harley. I told him that I was worried that the main pile of 20 envelopes might become biased, and no-one would know it had happened.
He assured me that Sargent always did the replacement himself and that he would not make such errors. Nevertheless, he thought it was a good idea to open them to find out. He checked that there were new envelopes of the same kind available. I then opened all the envelopes. There were 19, the replacement for the previous trial not yet having been done. There should have been 4 ‘A’s, 5 ‘B’s, 5 ‘C’s and 5 ‘D’s. There were in fact 5 ‘A’s, 6 ‘B’s, 4 ‘C’s and 4 ‘D’s. As I had predicted there was an excess of ‘B’s.
Harley and I discussed the possible ways this error could have come about. These include:
We then opened the envelopes in the drawers. Drawers B-D were correct but the ‘A’ drawer contained 2 ‘D’s in addition to several ‘A’s. Harley and I replaced all the letters in new envelopes and reconstituted the main pile correctly.
Because of finding these errors I discussed with Harley the reasons I had for worrying about them. I explained about the missing ‘B’ on session 9, and the other observations made concerning that session. Harley immediately recalled that on that session there had been a change from the official procedure.
Harley was to be agent. It was an experiment in which there was little time for the agent to do the randomization. Harley therefore asked Sargent to prepare things for him; apparently meaning him to get all the envelopes, tables and so on ready. In fact Sargent actually carried out the randomization and handed Harley the set of pictures and the small envelope. Harley took them and used them for that session. This should not have mattered since officially Sargent was to have no further role in that session. However, of course, we now knew that Sargent had come into the judging session on that occasion and had apparently ‘pushed’ the subject towards the correct picture.
The following day Sargent was still away ill. Harley and I wished to check up on some details of previous sessions and therefore looked for the book in which they were recorded. We could not find it, but in the process Harley found a sealed envelope, like those used in the randomization, under some papers. We decided to look for any further ones. We found a single one in a drawer and a pile of three under some papers. We opened them all. The single ones were a ‘C’ and a ‘D’. The pile of three were all ‘A’s. We found no ‘B’s.
We discussed possible reasons for them being there. One possibility appeared to be the method 2c, outlined above. If there were no ‘B’s concealed, then only method 2b could be used and would result in a ‘B’ going missing from the drawers, as observed on trial 9. We discussed alternative explanations.
Harley said that the envelopes for this series of experiments had been specially prepared all at once and placed either in the main pile or the drawers. Envelopes of that size and colour had not been used in any previous experiment. He could think of no reasons for there being any extra ones around the room.
Two further sessions were conducted, by student experimenters, in Sargent’s absence. These obtained ranks 3 and 4; both misses.
When Sargent returned after his illness Harley presented him with the findings so far. These were:
Sargent denied that any of these errors had come about deliberately and supplied alternative explanations for them. I hoped that Sargent would write his own account and provide these explanations himself. Since he has never done so I shall try to be fair to what he told me. We now have two alternative hypotheses to account for the findings.
Sargent said that the errors in the pile must have come about by accidental errors in replacement.
He calculated the maximum size of any spurious effect that could be created by this bias and found it to be only 3 per cent; a negligible effect when the average hit rate was about 45 per cent. Clearly if the bias were accidental it could not account for the successful results. On the other hand if it came about as a by-product of those methods of cheating, a very large effect size could be obtained.
At this time the error in addition (mentioned above) was also found. Neither Sargent nor I had any explanation for the ‘D’s in the ‘A’ drawer.
Sargent explained that the extra envelopes had been left over from a previous experiment, although Harley had previously said that this was very unlikely.
Sargent said he had done the randomization because Harley asked him to. Sargent said he had removed a ‘B’ because it was bent and therefore distinguishable from others. He said he had already told Harley about this. Harley now said he remembered being told although he had not remembered this previously when he and I discussed the problem.
Sargent said there was no harm in him coming into the judging since he did not know the identity of the target, even though he had done the randomization. He denied ‘pushing’ the subject.
There are therefore two hypotheses to consider. The hypothesis of cheating led to the discovery of the errors. It explains them fairly neatly and could, if extrapolated to the whole experiment, account for the large effects observed’.
The alternative is ad hoc, and cannot account for the large effects (these would have to be attributed to psi). It would imply a good deal of carelessness in the running of the experiment.
I considered that the evidence was not conclusive in favour of either hypothesis and that more evidence was needed. I did not wish to make any accusation, or even implication, of cheating, without conclusive evidence that it had occurred. It therefore seemed essential to gain further information which might support one or other hypothesis, and in the meantime not to publicise the findings.
There were several kinds of information which would be relevant:
In January 1980, I wrote a report for the SPR archives. This was to be available to SPR members on request, but I hoped it would soon be made redundant by a published version. Sargent and I agreed that we would each write our own version of the events. I wrote mine and sent it to him. He wrote an early (confidential) version, but never produced a final one. He continued to promise he would and therefore I waited and did not publish my own account.
When it became clear that Sargent was unlikely to produce a report, I discussed with Harley the possibility of publishing a joint account. We differed in some respects but agreed that we could write a report together if the points of disagreement were made clear. Harley did not write a report. I finally concluded that no written explanation was likely to be forthcoming from either Sargent or Harley.
There were several ways in which the raw data might help to test the hypotheses. For example, according to some methods of cheating one would expect the most popular picture in any set to have been target more often than predicted by chance. I asked whether I could check this. However Harley said that the pictures in each set were changed from time to time, without any record being kept, and that it would be impossible to check this from the existing records.
Another hypothesis was that, if one person were cheating and pushing the subject towards the target, rejudging should give poorer results than the original ones. This would be easy enough to do and Sargent said that he intended to do it. However he never published the results of any rejudging.
Thirdly, if one person were cheating, the most significant results should occur when they were acting as agent or experimenter, though of course this could also occur because of a psi-mediated experimenter effect. In fact there is evidence that scores were higher when Sargent took part in the few sessions observed during this visit and in published data (Ashton, Dear, Harley and Sargent 1981).
I hoped to be able to check the entire data base for this effect. This would mean having the Blue data book in which the names of all participants are recorded.
Finally, another suggestion was made by Parker and Wiklund (1982). Cheating could take place by manipulation of the randomisation combined with knowledge of the subject’s likely responses (as in 2a-c above). The easiest way to find this out is by looking at the subjects’ responses on previous trials. Wiklund and Parker suggested that in those trials where Sargent was responsible for the randomisation, and the subjects did not make direct hits, there would be above chance scoring if the target were matched with the subject’s mentation on a previous trial (Parker and Wiklund). This could be checked from the raw data and they therefore asked Sargent for those data.
These suggestions provide definite ways in which the implications of cheating could be lifted. If Sargent supplied the raw data other researchers could check them for these effects. If these effects were found, that hypothesis would be strengthened. If they were not found then the cheating hypothesis would lose much of its force.
I kept hoping that this would happen and the truth become clearer. However Sargent refused to make his data available. Several informal requests for the data were made. Then when these failed to elicit any data, official requests were made through the Parapsychological Association. Sargent still did not supply the data, nor any reason for withholding them.
In 1984 the PA Council asked Martin Johnson to head a committee to investigate the case. The final report of this committee is now available. Council reprimanded Sargent for failing to respond to their request for information within a reasonable time.
In view of this lack of cooperation it is not possible to test any of these hypotheses against the data. Also there now seems little hope of obtaining any new evidence and therefore we must assess the case on the basis of what evidence we already have.
I have been criticised for not publishing a full account earlier. I hope I have now made clear my reasons. I did not wish to publish something which discussed the hypothesis of cheating, (a) while there were still promises that others would supply alternative explanations for my findings and (b) while there was still some hope that further evidence would come to light.
I think there is still doubt as to the correct hypothesis. However, any hope that this will be speedily resolved now seems to be unrealistic. I am therefore presenting the evidence I have, as accurately as possible. I hope that others will add their versions to mine.
There has recently been considerable controversy concerning the value of the ganzfeld database in providing evidence for psi. The many experiments involving Sargent as experimenter form a very substantial and important proportion of that database. According to Hyman (1985) Sargent’s 9 studies and Honorton’s 5 account for one third of the total. According to Honorton (1985) Sargent’s experiments have the second highest effect size, after Honorton’s own.
If Sargent’s findings were removed from this database it would be considerably weakened as evidence for psi.
Brain and Perception Laboratory
University of Bristol, Bristol.
Ashton, H.T., Dear, P.R., Harley, T.A. and Sargent, C.L. (1981) A four-subject study of psi in the ganzfeld. Journal of the Society for Psychical Research, 51, 12-21
Blackmore, S. J. (1980). Extrasensory Perception as a Cognitive Process. Unpublished PhD Thesis, University of Surrey.
Honorton, C. (1985). Meta-analysis of psi ganzfeld research: A response to Hyman. Journal of Parapsychology, 49, 51-91
Hyman, R. (1985). The Ganzfeld Psi Experiment: A Critical Appraisal Journal of Parapsychology, 49 3-49
Parker, A. and Wiklund, N. (1982). The ganzfeld: A methodological evaluation of the claims for a repeatable experiment. Unpublished.
Sargent, C. L. (1980). Exploring Psi in the Ganzfeld. Parapsychological Monographs No 17
Sargent, C. L., Harley, T. A., Lane, J. and Radcliffe, K. (1981). Ganzfeld psi optimization in relation to session duration. Research in Parapsychology 1980, 82-84.
Sargent, C. L. and Matthews, G. (1982). Ganzfeld GESP performance in variable duration testing. Journal of Parapsychology 1981, 159-160
I wish to thank Dr. Carl Sargent for inviting me to visit his laboratory, and the Society for Psychical Research for financial support.
See also the following responses published in the same journal issue.
Harley,T. and Matthews,G. (1987) Cheating, psi, and the appliance of science: A reply to Blackmore. Journal of the Society for Psychical Research, 54, 199-207
Sargent,C. (1987). Sceptical fairytales from Bristol. Journal of the Society for Psychical Research, 54, 208-218.
This article has been scanned. Please let me know if you find any mistakes.