00:16this week surveys and I'm going to be
00:19talking about two things one is the the
00:22the process of doing a survey its
00:25position as a major research design and
00:28the the process you go through and talk
00:31about some of the different kinds of
00:32surveys yes thanks very much
00:34some of the different kinds of surveys
00:36that that are done I think I was played
00:40a bit of time talking about online
00:42surveys and things like that and then
00:44I'm going to be talking about sampling
00:45issues and most surveys are done based
00:49on some kind of sampling approach and
00:52something actually getting a good sample
00:54I particularly think back to what I
00:56talked about last week the issue of
00:58generalizability sampling is there to
01:01try to ensure that the samples you take
01:03in a survey are as generalizable as
01:06possible so generalizability is the big
01:09issue for that something tries to
01:11address and I'll talk about some of
01:12those things and I've got an exercise
01:13which I'll give you half way through
01:16which is looking at some of the some
01:19questions really about sampling and some
01:20some ideas about how you might draw
01:22samples of the best possible kind in
01:25different kinds of research
01:26circumstances so okay first of all then
01:30talking about surveys in in a general
01:32sense as a research design and here are
01:36the major characteristics I'll talk
01:38about the same thing when I come to case
01:40studies and to experiments in in
01:42subsequent sessions but this week just
01:46this to start with on surveys surveys
01:49collect a small amount of data
01:53relatively small that is I mean so it
01:55can be quite large I mean I once filled
01:57in a questionnaire that was over 100
01:58pages long I think I've got a 20-pound
02:01token for doing it not exactly much for
02:03the amount of work involved but so it
02:06can be you know physically a large
02:08amount but comparatively small amounts
02:10and in fact a lot of survey is based on
02:13questionnaires that is one or two pages
02:14long so that that's a not unusual
02:16standard so the key point is a
02:19relatively small set of data first of
02:22all but it comes in a standardized form
02:25and of course as I've already said the
02:28that is the question there isn't always
02:30you can do surveys without
02:32questionnaires you can do surveys based
02:34on interviews for example or surveys
02:36based on observation as a way of
02:38collecting the data so there are
02:39different ways of collecting data but I
02:41have to say that by and far the most
02:43common kind of survey is a at one based
02:47on a questionnaire so that's the
02:48instrument issues but the important
02:51point is whether it's a questionnaire or
02:53observation or it's interview whatever
02:56it's a standardized form and what that
02:59means is that every person that you
03:02approach in your sample gets the same
03:05thing the same set of questions there
03:08may be some variation it may not all be
03:10able answer the same question you might
03:11have some questions that talk about you
03:12know the children in their family and
03:14some responders may not have a family or
03:16might not have children to talk about so
03:18that won't be relevant to them but
03:20nevertheless they have to go through the
03:22same standard set of questions that have
03:23you know asked if they have a family and
03:25if not they go on to the X when it's on
03:26so that the standardized form in which
03:29you start is the same for everybody so
03:32that's the other another key point of
03:33the the survey is the standardized form
03:38and then a relatively large number of
03:41people again it's relative its relative
03:44to things like a qualitative case study
03:50for him where he won't mind simply
03:51taking a handful of people or a dozen or
03:54so 20 people that kind of number in a
03:58survey you're normally doing with the
04:00numbers in the hundreds if not better a
04:02thousands of people so these are much
04:04much bigger numbers of individuals who
04:06come into the the sample then you would
04:09get in say a qualitative work or for
04:12that matter an experiment experiments
04:13often done on just a you know 20 people
04:1525 people that sort of thing so much
04:18bigger numbers so that's the key point
04:21small amount of data relatively a
04:22standardized instrument being used a
04:25questionnaire most commonly and on a
04:27large sample a large number of people
04:29and this other key point here which is
04:33why we only talk about sampling in the
04:34second half is that it's key to the the
04:38survey that the sample is representative
04:41of course because of that you can
04:42generalize to the population and
04:45representative from a known population
04:47so you have both those issues how to
04:49make it representative and how to to
04:51know the population how to get a record
04:54or listing if you like or some
04:55definition of what the population is
04:58from which you're drawing your sample
05:00I'll say this later on but of course not
05:03every survey takes a sample you don't
05:05have to sample sampling is convenience
05:08really because the population is so
05:10large usually so if you're looking at
05:12doing a survey of the Kirklees this this
05:16region with about a quarter million
05:18people they almost certainly you would
05:20sample but you don't have to the most
05:24common most well-known example that
05:26isn't something of course is the census
05:27which takes every single person in the
05:30country or tries to anyway doesn't
05:33always succeed but if you're talking
05:35much much smaller population so the non
05:38population might be wrote to be small
05:39for example if you want to do a survey
05:41of this class then you might seem to
05:44take every single person you wouldn't
05:46have to sample it might not be necessary
05:47in fact the numbers are that small you
05:50wouldn't actually bother about sampling
05:51you just simply survey everybody and so
05:55your sample can be a hundred percent in
05:57that kind of case okay
06:01surveys I'm sure she recognized very
06:04common I mean almost all of us have done
06:06surveys that is we filled in
06:08questionnaires we've answer things and
06:09so on you know commercial companies are
06:12forever putting out questionnaires about
06:14their products and things like these
06:16days I mean it's almost impossible to
06:18buy something online for it sample
06:20without getting an email coming a few
06:21days later saying please really fill in
06:23this little short questionnaire about
06:24your purchasing experience and so on so
06:28it's been taken up in all sorts of areas
06:29but in the the more serious social
06:31scientific research area surveys are
06:34also very common as well as I said here
06:36most calls for research most invitations
06:39to bid for money often call for for
06:42surveys at least as part of the work
06:44they're doing so it's a it's a you know
06:47very very common technique if not the
06:49most common technique and be
06:51used a an interesting development here
06:56by the ways is the mixed methods
06:57approach I think I mentioned the
06:58previous week mixed methods mixes
07:00together typically qualitative and
07:03quantitative and one thing you often
07:05find these days is a mixture of a survey
07:08done quantitatively based on a
07:10questionnaire where you're counting and
07:11and and so on and the calculation
07:14percentages combined with some kind of
07:18qualitative work often based on some
07:20kind of interviews of a subset so a
07:22mixed methods approach often is a survey
07:24plus some kind of a smaller number of
07:28case studies or interviews or whatever
07:30them alongside it here's some references
07:35and I've given you a much longer list of
07:36references but I thought I'd mentioned
07:37some of the key texts given surveys are
07:40so frequent given it's quite likely
07:41you'll come across surveys that the
07:44things that you have to read have to
07:45appraise or even at some stage yourself
07:48design and do it's extremely likely
07:50involved at some point in your career in
07:52doing a survey it's worth knowing that
07:55there are some key books probably the
07:57most popular is the device book I'm not
08:01sure that's the most recent edition you
08:03can see it's in his fifth edition even
08:04in 2002 I think it's probably another
08:06one come out since then in the last ten
08:08years it's what all these books are
08:11thick volumes the device I think is the
08:13thickest about two inches thick it's a
08:15really held heavy weight and it they all
08:17include everything they talk about how
08:19how to design the surveys how to carry
08:21it out how to design the questionnaire
08:23which I'll come back to you by the way
08:25in another session I have a session just
08:27devoted to questionnaire design later on
08:28and they also cover the sampling issues
08:31that I'll be talking about later on
08:32today the orders weather is a similar
08:36volume from the University Press Floyd
08:40Fowler is is Sage's entry into this
08:42market and another heavyweight book too
08:46at the bottom are putting their relief
08:47historic reasons I don't think these are
08:49being published any mores but you'll
08:50find in the library that their particle
08:53is the ones I started with when I
08:54started in this area most important
08:57mostly was the key text you can see from
08:59the date 71 even the publisher and along
09:02with this scour has been taken over
09:05it was republished in several times but
09:07it's often cited as a classic text in
09:10the area so and again it's healed it's
09:12been the library and still being used
09:14along with home didn't gel that came out
09:16slightly later so that the ones at the
09:18top of the more frequent ones and of
09:19course there are other books on the list
09:21I given you almost every publisher has a
09:23book on survey work of some kind and
09:26there are other books on aspects of
09:29survey work on particularly focusing on
09:31questionnaire design for example or on
09:33sampling issues and things of that kind
09:35so that's why I've given you a longer
09:36list as well to work with okay right how
09:42would you mount a survey how do you
09:43actually carry out a survey why it's
09:46called mounting sounds like a horse
09:48doesn't it mounting a survey and but you
09:51mount a survey you carry out a survey
09:53you do the survey whatever whatever in
09:56particular they'll be using here well
09:59first of all there are the stages and a
10:01bit like I talked about last week this
10:03is the normative kind of approach is how
10:04it should be done and actually it makes
10:07sense most of the time if you're
10:08carrying out a survey to go through this
10:10order in particular working out the
10:13questionnaire design before you start
10:15sending it out you must all that first I
10:18mean yeah think of it logic there's no
10:20point in sending out a very badly down
10:24questionnaire design and you're gonna
10:26spend quite a lot of money and time
10:27sending it out so make sure it's good so
10:30doing that upfront is important so you
10:33start of course with your initial design
10:35what and I think an important point is
10:38bear in mind if you aren't doing it
10:40a mistake that people often make when
10:42doing surveys and designing
10:44questionnaires is they start thinking of
10:47questions to write on the questionnaire
10:49and that's not a good idea you need to
10:53do that at some stage but right at the
10:56beginning think about the research
10:58questions you want answered and then
11:00later on think about how you're going to
11:03find out the answers by asking the
11:05questions on the questionnaire so start
11:07with what you need to not find out start
11:09with your research questions your more
11:10general ideas your ideas based in the
11:13the social theory you're looking at and
11:15so on and then later on think
11:18the actual way of wording questions and
11:20asking questions of people so try to
11:22avoid getting in the I'm going to ask
11:24this question coming right at the very
11:25beginning and think more generally about
11:27what the questioner is trying to achieve
11:29what the the survey is trying to achieve
11:33part of that is thinking about the the
11:36sample itself and obviously your
11:38research questions will be related to
11:40some kind of Psalm for it might be the
11:42whole population of a country but it's
11:43more likely to be some kind of subset
11:45it's going to look at just I don't know
11:47women who turn up to the hospital with a
11:50certain disease between the ages of 30
11:52and 50 something of that kind so that's
11:55the way you define your your sample so
11:57whatever your interest is whatever your
11:58research questions are that you think
12:00that your sample and of course with that
12:02you have to think about how you're going
12:03to draw your sample you how you're going
12:05to take that sample from the population
12:07how you're going to find them and so on
12:09I'll come back to those issues it's a
12:11bit later on today okay so I imagine
12:16you've got past that now you've got
12:17you've got an idea of who you're going
12:19to do it on that you know what kind of
12:23research questions are going to ask the
12:25next stage is to draw up the
12:26questionnaire itself drop the instrument
12:29traditionally of course that's a piece
12:31of paper only many cases it still is a
12:33piece of paper but it can be other
12:35things it can be done by telephone we
12:39still start with the piece of paper and
12:40you read it out or it can of course
12:42these days beyond on the internet most
12:44typically as a website although they can
12:47be a file you send by email and so on
12:49but most commonly is is by by by using
12:53webpages so you draw it up on on on the
12:56web actually it's not a bad idea even if
12:59you're doing it on the web to start by
13:00drawing hell on paper to get some idea
13:02that and then kind of enter it into the
13:04particular site you're using for your
13:15because people I just couldn't usually
13:18know about rather than having to know
13:19it's not much yeah I mean I talked about
13:24this little detail later on but but
13:26there are several issues here and the
13:27answer is yes and no you're right it's
13:29it's quite easy to do and it's certainly
13:33much easier to send it back than it is
13:34to put a postage stamp on it and post it
13:36or to have to take it physically to some
13:38kind of return box and so on which is
13:40not about doing it and it is a big issue
13:42getting stuff back and you know making
13:44sure you have a good response rate
13:46basically that's the issue so it helps a
13:49bit there on the other hand it's equally
13:51easy to ignore it and there is something
13:55about the psyche somehow I think the
13:56human psyche that says well I'm got a
13:58piece of paper that someone spent some
14:00time making sure it looks good and
14:02printing it out and so on and sending it
14:03to me personally with my name on it and
14:05so on but I'm more likely to fill it in
14:07than an email I get along with thousands
14:09of other emails and if anyone like me
14:11and when I get hundreds of emails a day
14:13almost and you know I can't respond to
14:16them all and and and I don't and it's
14:17very easy of course to ignore things
14:19like that on email so and even if it's
14:23on the web again you normally get an
14:24email message asked me to do it on the
14:25web so it's the same kind of issue there
14:27so it's both yes and no I think in that
14:30case there are other problems as well
14:32I'll come back to later on with with
14:33online versions of things to do with
14:35something okay and so questionnaire
14:39construction as I said I've got a whole
14:40session about how to do a questionnaire
14:42as do it on paper or online whole set of
14:47issues to think about there I'm not
14:49going to dwell on those now I'll come
14:50back to those later the one thing though
14:53to think about all the ways in doing
14:55questionnaire design is testing it out
14:56called piloting you have to pilot the
14:58questionnaire design so you have to test
15:00it out ideally with the kind of people
15:03you're going to be using in your actual
15:05survey not the actual people so don't do
15:08it you know don't send the people you're
15:10going to contact an early version in
15:12another version and another version
15:13because given in mind what I said last
15:15week that's that could be biasing them
15:17it prepares them for that for the
15:18answers and maybe change their answers
15:20so pick people like the people you're
15:22going to survey so if it's a particular
15:25for the population choose other members
15:26of that subgroup who you're not going to
15:28use in your final sample and let them
15:30have a look at it and comment on it and
15:32of course also you can give it to your
15:34matron your friends your colleagues and
15:35so on to have a look at it's always good
15:36to get other people's eyes on things to
15:38get other people's views on what you've
15:40written I think one of the one of the
15:44reasons why that's so important is
15:46because we all think we know what other
15:49people are thinking like we think we can
15:51write a questionnaire because we think
15:53we can get into their minds and see the
15:55world from their perspective and
15:56therefore write questions that address
15:57the way they see the world the way the
16:00way they're thinking and so on and I'm
16:02it's I'm afraid sadly true that that's
16:04just not true you just cannot do that
16:07even when you're well experienced mean I
16:09I do it I've done questionnaires the
16:11last few years and even there I have to
16:13pilot them to test them out and I find
16:15things that I haven't thought about that
16:17I think differently from other people
16:18and so on and therefore the questions
16:20need to be reworded or redesigned to
16:22accommodate that change to a view of
16:25other people so piloting or testing out
16:28the design with other people and
16:30particularly with people like your
16:31actual sample is very very important to
16:34do when you're doing that one of the
16:36things you can do is to ask them to give
16:39feedback you can actually tell them if
16:41you're doing the pilots like it just
16:43don't answer the questions write
16:44comments as well now people do that
16:47anyway people can be quite rude actually
16:49on the huntsmen question if if they
16:50don't think they apply or and somehow
16:52the other that they don't fit into your
16:54your rigid categories of things on your
16:56questionnaire so and you know you can
17:00invite them to do that as well and
17:01that's actually very helpful sometimes
17:03to get that kind of feedback as well in
17:04the polishing stage okay so now you've
17:09done that you've got an instrument
17:10you're using or you're using something
17:11else if you're doing interviews for
17:13example you have a schedule of questions
17:14you'll ask in the face-to-face interview
17:16or if you're doing it by observation you
17:18might have a set of things you're going
17:20to observe and ways of doing it so
17:22standardized form but most commonly it's
17:24a questions you're not as people and
17:26then you you do the survey you you you
17:28distribute your questionnaire or
17:30whatever you do the fieldwork fieldwork
17:32by the way is a general term in social
17:34science used here to represent
17:38the research lab if your research office
17:40out into the world to do your research
17:42to collect the data to observe and to
17:45talk or whatever that's fieldwork
17:47doesn't literally mean going into a
17:48field it means in the metaphoric sense
17:51going out into the town the city the
17:54workplace whatever you wherever you're
17:56doing study fieldwork I've set the most
18:01it depends what kind of work you're
18:03doing actually in in surveys if you're
18:05doing it through the web it certainly
18:07it's by mode the fifth most efficient
18:09stage once you've done all the Premio
18:11the pre-planning and preparation work
18:13you press a button and out it goes and
18:15that's it you sit back and wait for a
18:17few weeks while the whole stuff comes
18:18flooding in so that's very very easy if
18:21you're doing it by postal questionnaire
18:22is not that different either you've sent
18:24the things out you've got the stamps on
18:25the envelopes whatever they're all sent
18:26out or you put things in the internal
18:28mail if that's what you're using and out
18:30they go and you can sit back and wait
18:32for things to come back here so that's
18:34quite efficient if you're doing it by
18:36face-to-face interviews and some surveys
18:38are done that way or by telephone
18:40interviews then obviously it's not that
18:42efficient it still takes a lot of time
18:44and someone has to do the interviews and
18:46so on so there's a slight difference but
18:48most surveys are done by some kind of of
18:51self response on paper or on the web so
18:54it is quite efficient as already
18:57mentioned there are different ways of
18:59delivering it and there's a question
19:00about how to do that really about how
19:02you should decide whether to do it post
19:04early or electronically the important
19:08thing about that is it's very efficient
19:09it's easy to do it's usually easy to get
19:12to people because you can send it to
19:13them or just just you mail it to them
19:16the downside is that there you're not
19:19there when they fill in the the
19:22questionnaire so any problems they have
19:24with it you can't answer anything that
19:26you've forgot that's why the polishing
19:28is so important because anything that
19:29goes wrong at that stage you cannot fix
19:31it you are not to there and all they can
19:34do is work around the the mistakes
19:36you've made in your design or the
19:37ambiguities you have in your design and
19:39deal with that as a whole
19:44I mean I've mentioned most of these
19:45things already the web email giving
19:47handouts I mean if you're seeing a group
19:49like if I were doing a survey of you
19:51I would actually bring along the
19:52questionnaire to the class and just hand
19:54it to you like the handouts are giving
19:55you earlier on and now I kept it in
19:57perhaps next week or actually probably
19:59ideally catching at the end of the
20:01session before you disappear because the
20:03chances of you all remembering to bring
20:05it back next week are not 100 percent so
20:08whereas if you're still here so you have
20:10to think about all those kind of things
20:11in designing the survey how they're
20:13going to get it back to you how people
20:15are going to be reminded to get things
20:17about you and I talked a bit more about
20:18that that response rate issue in the
20:23the alternative is in person by
20:25interview or by telephone interview it's
20:28it's very expensive and so on but it
20:30doesn aybe you to be a bit more flexible
20:32in the way you're giving the questions
20:33and then while you're getting the
20:35answers and if if you're unsure about
20:37how to design the questions sometimes it
20:40might be a good idea to do it in person
20:42but it's done quite rarely these days
20:45because it's just so expensive to do and
20:47if you're going to do that why not do
20:49proper depth interviews where you can be
20:51a lot more kind of flexible about how
20:53you're doing you're doing the study and
20:57then there's dealing with non responses
20:58and I say I'll come back to that a bit
21:00later on to talk about solving issues
21:02but there are some tactics you can use
21:04to try to improve the response rate the
21:07response rate is essentially is the
21:09number of proportion of questionnaires
21:12you actually get back that you can work
21:14with so it's the number you get back
21:17divided by the number you sent out and
21:19response rates in surveys come very
21:22enormous ly a very good survey done in a
21:26close situation amongst people you know
21:29and so on so if I did it on you for
21:30example and I did it in the class and I
21:33catch it at the end I might expect to
21:35get a hundred percent response rate I
21:36might not get it maybe want to serve you
21:38who object to it or or write the city
21:40things on the questionnaire that that
21:41means I can't use it so there's not
21:42great slightly smaller but it's quite a
21:44high response rate more typically though
21:46if things are done post early or by the
21:48web response rates drop dramatically
21:52so we're talking here about 30 percent
21:54twenty percent ten percent response
21:56rates a good response rate in a postal
21:59survey is more like about fifty sixty
22:01percent so it gives you some idea if
22:04you've done that done a post
22:05survey and you get 16 then you can
22:07celebrate it's done really well so so
22:10that's that's why it's an issue you can
22:12but it can be as low a below 10% if
22:14particularly on the worldwide web there
22:17are other problems actually with
22:19measuring response rates on the web
22:20which I'll come back to later okay so
22:24imagine you've got past this you've now
22:26got your final dates you have to give a
22:28final date on that you know for things
22:30to be sent back to you or collected and
22:33now you've got all the things you're
22:34going to work with what you do with all
22:35these questionnaires well you have to
22:37edit and code them anything doesn't mean
22:40changing them it means making sure you
22:41can make sense of them if they're
22:43ambiguities in the way people have done
22:45things you need to resolve there's any
22:47big ambiguities or ignore the question
22:50if that's appropriate and then you have
22:52to code them and coding in this area in
22:56the area of surveys means translating
23:00the answers on a questionnaire the ticks
23:03on the boxes or the things people
23:04written in in in open-ended questions
23:07translating those into numbers and it's
23:11the numbers you need at the end that's
23:12what you're then put into your stats
23:14package or in the old days onto your big
23:16spreadsheet your physical spreadsheets
23:19but sheets original of course what bits
23:20of paper with with squares and rows and
23:22columns on and that's how you did it but
23:25these days almost everybody uses a stats
23:27package to to actually analyze the data
23:29one we use here is called SPSS and it's
23:33available on all the machines here to
23:35use so you book you're translating the
23:37answers on the questionnaires the actual
23:39sheets you get back into a set of
23:41numbers you can then input into that
23:43status package so then you have computer
23:46entry editing you may need to do some
23:49I've just been involved in a project
23:50called coping it's a European funded
23:53project looking at what we did actually
23:57was we surveyed families where one of
24:03the parents was in prison usually the
24:07father but sometimes the mother in
24:08prison and we contacted the prisoner the
24:14the the other parents or the other
24:17carrier if it wasn't the the partner and
24:19the children and in fact we surveyed
24:23them and interviewed them as well we had
24:24two stages of mixed method design in
24:26that sense and on the survey stage we
24:31had to spend an awful lot of work
24:33sorting out the ambiguities in the
24:35survey partly it was because it was done
24:38in four countries in four different
24:40languages and so we had to have had
24:42problems of different ways in which we
24:44are putting Terp really the instructions
24:45we have some scales we were using and
24:47some other questions and so on and they
24:49were interpret differently and we had to
24:51try and bring those four things together
24:52to get a unified data set so a lot of
24:54what's called data cleansing or data
24:57cleaning had to go on to just sort out
25:00what these things actually meant we
25:02weren't changing the results but rather
25:03we were interpreting them to make sense
25:05against the other numbers adjusting them
25:07the scale zone always the same scale
25:09some how to adjust the scales to to make
25:11them to normalize them to bring them
25:13into line with the other groups and
25:14so in large-scale surveys you can get
25:17involved in quite a lot of work it was
25:19very frustrating going through this
25:21particular research has involved trying
25:22to sort out these kinds of issues once
25:24the survey was finished we got all the
25:26data back and then we still had to
25:27cleanse it in that kind of way too to
25:29actually use it fortunately for most
25:32things I suspect you'll be involved with
25:34certain if you're doing a survey on your
25:35own that's much less of an issue
25:36normally it's a fairly simple procedure
25:38of converting the question is into
25:41numbers and inputting them and then of
25:44course the analysis analysis is actually
25:46very very quick usually if you know your
25:48stats package you know what questions
25:50you're asked you go back to the
25:51beginning to the initial design you got
25:52the right research questions it's a
25:54matter of you know clicking a few
25:56buttons on the stats package and
25:57outcomes the results of course
25:59interpreting those results takes a bit
26:00more effort you have to understand what
26:01the stats mean and what he's telling you
26:03about your research questions and so on
26:05and but he actually can be quite a quick
26:07procedure and then of course you finally
26:10have to write up your results and and we
26:13have published them or make them
26:14available to others okay so that's the
26:17that's the procedures you go through and
26:19that you can see that the the important
26:21ones I think here are the the sampling
26:24issues which I'll come back to later
26:25today the questionnaire design issues
26:28another session on and the some of the
26:32decisions to be made about about
26:34fieldwork about your design on that let
26:39me dwell a bit more on some of those
26:40questions should you do it
26:43remote remotely this is where you send a
26:46questionnaire out that somebody else is
26:47sorry your respondent is filling it in
26:50when you're not there this is some kind
26:53of interview and the interview can be on
26:55the telephone it can be face to face we
26:58actually go and visit the person in
27:00their home sometimes or it can be these
27:02days of course on the on the internet
27:04using some kind of you know Skype or
27:05something some technology like that
27:07where you can again interview them face
27:08to face but electronically face to face
27:11perhaps ok what are the pros and cons
27:14well first of all the remote sending
27:15stuff out you know so that either by the
27:19web or by email or or by paper giving it
27:22out to people sending it through the
27:23post the pros well very often its
27:27easiest way or the only way to get hold
27:28of them and you know it's very hard to
27:31kind of contact people sometimes two to
27:34get to face-to-face so giving out
27:37questionnaires in that kind of way may
27:38be the best way of doing it certainly if
27:41you're a company trying to to interview
27:43your your customers and they've just
27:46bought something from you online it is
27:47by far the easiest way of getting a hold
27:49of them to send out something that way
27:51as I already mentioned it's efficient
27:54and it's low cost once you've done the
27:56design and got a piece of paper it's
27:58relatively cheap to print it resolute
28:00you to post it people but I certainly
28:02compared we're going in person to talk
28:04to them it's cheap and of course on the
28:06web it's even cheaper there's almost no
28:08cost at all once you've done the design
28:09work itself is a matter of you know
28:11sending stuff out so electronically so
28:14it's low cost and it's also anonymous
28:17usually I mean you can of course ask
28:19people's names and details and sometimes
28:22that's done as part of the reward system
28:25on a questionnaire so usually at the end
28:26somewhere they say you know we'd like to
28:27enter you into a competition for a small
28:29prize etc as a way of rewarding you for
28:31the effort you've put into this in order
28:34to do so you must put in your name and
28:36address so we can contact you if you win
28:37and they normally tell you at that stage
28:41name won't be user puff the
28:43questionnaire and so on but of course
28:44you needn't do that you can leave offer
28:46together in which case the certainly the
28:49paper form is is is very anonymous I
28:51mean it's almost impossible to tell
28:53where it comes from and who who filled
28:54in so anonymity guaranteed on the web
28:58it's slightly different if it's a web
28:59page then usually that will be anonymous
29:03too it's quite hard to tell where it
29:05comes from and certainly it's very easy
29:07not to give away who you are if you're
29:09doing it by filling in it in a web form
29:11if you're sending out the questionnaire
29:13by email as an attachment that it has to
29:16be sent back to you then of course in a
29:18sense you know where it comes from
29:19because the email address will be there
29:21when it's sent back to you but it
29:22doesn't have to be actually attached to
29:24the questionnaire itself so again you
29:25can at least keep that separate and you
29:27can of course reassure people that
29:29anonymity will protect it and so on in
29:31that situation okay so that's the pros
29:35of having a remote one the the cons the
29:38the the against and whatwhat's against
29:41well first of all the response rates an
29:43issue if you're doing things
29:45face-to-face then you get a hundred
29:48percent response rate just about I mean
29:49there might be one or two K's of it but
29:51withdraw and things like this in halfway
29:53through the interview but in general
29:55face to face interviews give you very
29:57high response rates once you've got the
29:58person to answer the questions doing it
30:01in a remote sense by sending it out and
30:03someone will give you a low response
30:05rate and I've already said how low those
30:06can get ambiguity is not detected again
30:10that's another big issue you've got to
30:11make absolutely sure that your
30:12questionnaire is well designed because
30:14you're not going to be be there when the
30:16people fill it in so any problems any
30:19mistakes any ambiguities and you can't
30:22clear them up because you're not there
30:24and respondents may not treat it
30:26seriously I mean again you're not there
30:28to see if you're doing it face to face
30:30you can tell if they're joking and
30:32they're unlikely anyway to do that
30:34they're going to take it more seriously
30:35because they're face to face with you
30:36but if they're remote who knows now
30:39sometimes you can tell people are
30:40kidding and you know you can actually
30:42see from the way they're answering the
30:43questions they're not serious and you
30:45have to throw that data away but it's
30:47not always obvious what's going on what
30:50making what Woody's making people answer
30:51in that kind of way I mean they can be
30:54kind of just taking it
30:55random and how will you know that
30:56ticking boxes at random isn't it's just
30:59them not taking it seriously as opposed
31:01to being genuine about their answers so
31:03that's that problem slight uncertainty
31:05about how genuine people are in
31:07answering their questions I think most
31:11the time people actually are pretty good
31:13most people are generous people most
31:15people want to help and if they're going
31:17to fill it in there they're disposed to
31:19help you in your survey so I think most
31:22cases people are doing it genuinely but
31:24you never know there'll be some who get
31:26bored halfway through and I mean you do
31:29find sometimes when you do these kind of
31:31surveys that you get some questionnaires
31:33back that a half-filled in the first
31:35half filled in they get hopper to get
31:36bored can't be bothered and that's it
31:38and then just send you back in half half
31:39done it's up to you whether you keep
31:42that data or not in some case you
31:44perhaps shits right all the way in other
31:46case you might keep the half that's been
31:48filled in and all the others so that's
31:51an issue about serious treatment what
31:55about the other way doing it face to
31:56face in some way I use the term
31:59face-to-face to mean either in person
32:01you know their bodily with the person or
32:04by telephone or by internet interview
32:07and something of that current remote
32:09interview now obviously it's the other
32:12way around the pros are that you can
32:13clarify the questions I've said already
32:15and I think by being there face to face
32:18you're encouraging you know involvement
32:21you're encouraging them to take things
32:23to think seriously about the questions
32:25they're asking and not just be flippant
32:26about how they're answering and of
32:30course you're encouraging actual
32:31response if you can get hold of them you
32:33can contact the people and get there
32:36you're going to get a response once
32:37you've started the interview almost
32:40certainly compared with the distance
32:43approach the remote approach the cons
32:46that the the arguments against this
32:48interviewer bias I've already talked
32:50about that last week those kind of
32:52possibilities but interviewers can buy
32:53us the way that the answers are given
32:55and so on I think I mentioned it to you
33:01so far there's a classic example again
33:04this is back in the nineteen fifties I
33:05think this research was done I thought
33:07that dates that yester fit
33:08to be the fifties because somebody did
33:12some research looking I think the topic
33:14of the the survey was something to do
33:19with with marriage and are probably
33:21sexual relations something a bit kind of
33:24sensitive to ask people about and the
33:28interviewers were all women and the the
33:33way it was set up was on half the
33:35interviews they all dressed exactly the
33:37same each interview they they were the
33:38same kind of clothing dress the same
33:40kind of way you know can smart casual
33:42kind of clothing but on half the
33:45occasions they wore a reading ring on
33:47their finger and half the occasions they
33:50didn't wear a reading wedding ring so
33:51half of them had this what you might
33:54call read reasonably subtle symbol that
33:56they were married and half that they
33:59weren't married and what they found they
34:01got on this kind of area of sexual
34:03relations on they're asking about they
34:04got a significantly different set of
34:06results despite the fact that everything
34:08else is the same the same questions
34:09asked the same survey use the same
34:11questionnaire used women dressed the
34:13same way operating the same way and so
34:15on the only difference was the wedding
34:17ring but they found significant
34:18difference in the answers between the
34:19two groups and so even quite subtle
34:22differences in the interviewer can buy
34:25us the results in that way so it is an
34:27issue and I guess less so if you're
34:29doing it remotely and of course on the
34:31telephone it's only the voice that
34:32matters but even that you know the way
34:34questions are asked and the way people
34:36respond and so and can bias things so it
34:38is an issue I suppose it's saying the
34:42interview a respondent affects is I
34:43guess that's that's actually probably
34:45what the the ring is its interior
34:46respondent effect and rather than bias
34:49issue but bias can be a problem as well
34:52and of course it may not be seen as
34:54enormous the fact you're face to face
34:55you know here it is you're talking to
34:57means that it's you've had to have more
34:58reassurance about anonymity if if if
35:00that's what you're guaranteeing and of
35:03course because of that response may be
35:05less honest and they may not tell the
35:07entire truth because they don't want to
35:08tell it to your face and there may be
35:10some advantage in a a remote you know a
35:15non face-to-face questionnaire use
35:21okay what kind of designs can you have
35:24in a survey I mean I think the ones that
35:27most of us have experienced tend to be
35:29this first case that what I call the
35:31simple survey the way I like to think of
35:35it is the term I've used here in
35:37brackets the snapshot it's a bit like
35:39taking a snapshot of the mill year the
35:42society you're trying to investigate but
35:47like getting your camera out and taking
35:48a quick shot the point about that is
35:50it's a picture at a certain period in
35:51time at the point you give the
35:54questionnaire that's the picture you get
35:56of the the state of affairs in in that
36:00group in that in that population I say
36:04it sound hawk it can often is that hawk
36:07it's just we suddenly think we need to
36:08know this let's do a survey and that's
36:10what's carried out of course you can
36:12repeat things I mean that another
36:15typical server you'll come across is the
36:17opinion poll and they tend to be
36:20repeated so they'll maybe you can use
36:24year after year but certainly at the
36:25time of Elections you'll get several of
36:28these these kinds of polls going on
36:30these kinds of surveys but they're all
36:33kind of simple server they're all kind
36:34of generally get a population and then
36:37just ask the questions that population
36:39that's it that's a snapshot based of
36:42course on on really good sampling issues
36:44so we can generalize the whole
36:45population a second for most slightly
36:50different it's it's what's called a
36:51panel survey and in the panel survey you
36:54go with the intention of going back more
36:58than once and to do that you don't have
37:01to ad hoc or simple surveys you have a
37:04panel of people you go back to again
37:06this this actually happens during
37:08opinion sorry during election times with
37:11opinion poll surveys they'll have a
37:12panel of voters who they ask questions
37:15over and the strength of the panel
37:17survey is judging change over time
37:20because that's the interest of these
37:22opinion pollsters is in how people's
37:25views protocol views are changing over
37:27the panel is a group of people a sample
37:31usually done you know pick fit for good
37:34statistical reasons as to being
37:36representative of the population but
37:38then there are lost back on several
37:39occasions so you you go back on two or
37:42more occasions to ask them the same set
37:44of questions or maybe different
37:45questions but often the same kinds of
37:48questions again and as I said the real
37:52power of this kind of approach is to
37:53judge change in the case of opinion
37:56polls change over a matter a few weeks
37:57of course now that's quite difficult to
38:02do and it's quite difficult to get that
38:04panel to get them to come back with them
38:06and they often they're paid to do that
38:08and but even then people don't always
38:11come back again so this third version
38:14the rotating sample survey is sometimes
38:16used you just repeat the survey with
38:20some respondents the same but others
38:22changing so you get some comparison and
38:25with previous occasions so it's going
38:26mixture of one and two and then number
38:30four which actually is to some extent a
38:34bit like the panel survey but it's
38:37normally much long much much larger
38:39scale and much longer in time panel
38:42surveys tend to operate over a few
38:44months so the issues about keeping
38:46people are relatively easy to deal with
38:48you just ask them to come back you know
38:50in a few weeks time to have another
38:51interview or to tape on another survey
38:54whereas longer to dnal surveys are often
38:57done over years if not decades but
39:01longitudinal surveys in a sense are the
39:04the you know the the best possible form
39:06of survey in getting ideas of things
39:08over time and it has to be said also of
39:12making judgments about causality as
39:14things change over time you get a much
39:17better idea of what is causing what a
39:19snapshot it's quite difficult to make
39:22any kind of judgment about course
39:24Thomasson you find this is what happens
39:25this they do this they do that and so on
39:27but answering the question why do they
39:30do that why is that group different from
39:31that is much harder to do with the
39:33snapshot with a simple survey with a
39:35longitudinal survey over a period of
39:38you can get some handle on those those
39:41questions those white questions why is
39:42that group different from that and so on
39:44usually through some kind of correlation
39:46approach so it's typically a former
39:52panel over a long period of time the
39:53panel is normally kind of chosen at
39:55certain point I mean one of the most
39:57famous examples of this is is the study
40:00of children born in a certain year of
40:02course I can't remember off the top of
40:03my head what the off year is now I think
40:05was back in the 1960s when they started
40:07and every X years after that they go
40:11back and find the same children and they
40:14become adults and so on and you can then
40:15find things changing over time health
40:18issues issues of you know marriage and
40:22divorce and things of that kind
40:24job issues exceptional educational
40:27things changing over periods of time so
40:28you can you can make connections and you
40:30can see that the same people are doing
40:33these things 10 20 30 years later now of
40:37course that's very hard to do you've got
40:39to go back and contact the same people
40:40who are moving around the country and
40:42they're dying and getting yiling they're
40:43getting married and changing their names
40:45and so on so they're quite difficult to
40:47get hold of so a lot of effort to go
40:49into keeping in touch re finding people
40:52who've moved away and so on but it's
40:54worth it because you get such good
40:56quality data from it now of course
40:58because of difficulties often it's
41:00government's that do this so that the
41:03money you need to set up this kind of
41:05language was done these quite large and
41:06so he's government funding that often
41:08does it and of course the time pro2 is
41:11an issue if you know if you're thinking
41:12about doing a PhD in this a 3-year PhD
41:15you really haven't got time to do it a
41:17longitudinal study and in most cases it
41:21is possible I mean I did have one PhD
41:23student a few years ago who did a study
41:27where she went back to family shoots
41:29picks on families and looking at
41:30families with long-term illness or one
41:32of the members of family had long-term
41:33illness and she went back on three
41:35different occasions over a period of a
41:37couple of years to see how things had
41:39changed as the the prognosis developed
41:41and as the the actual experience of the
41:43illness changed and so on so she gets
41:46some idea throughout longitudinal study
41:47but it's really limited
41:50I encourage men in a short and study
41:53period okay so that's some ideas of
41:56different kinds of studies that you can
![Welp, Here's what the Fed *Just* Said [Fed FOMC Minutes]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fl8u23oq1E9E%2Fhqdefault.jpg&w=3840&q=75)
