00:00hi everyone welcome to the a 6nz podcast
00:02today's episode hosted by Frank Chen is
00:05a conversation with Stanford senior
00:07scientist and astronomer Ivan Linscott
00:09and it is all about the juicy nitty
00:11gritty mind-blowing details of how
00:12exactly you send a signal to Pluto
00:15Ivan Linscott is part of the team that
00:16helped the radio science experiment or
00:18Rex experiment that went up with the new
00:20Horizons probe and will use radio
00:22transmissions to gather info about Pluto
00:24he focuses on digital signal processing
00:27radio occultation experiments I don't
00:29even know if every noun set right
00:30observational radio astronomy and
00:32particle physics and has worked at
00:33dudley nasa and unsteady all of the work
00:36has in common listening for signals and
00:38doing radio science experiments in the
00:39outer planets a 16z research and
00:42investing team head Frank begins a
00:44conversation by sharing how he came
00:46across this work so I'm sitting at home
00:49and it was July of 2015 when all of the
00:53amazing pictures were coming back from
00:55New Horizons and I'm having a bunch of
00:59Wi-Fi problems at home and I'm thinking
01:01to myself here we are there's a probe
01:03billions and billions of miles away
01:05that's sending photos and I have this
01:08huge Wi-Fi dead spot at home like wood
01:10is going on and as it turned out a
01:13couple of weeks later I was talking with
01:14the fabulous Theresa Johnson
01:16shoutout to Theresa who is now a data
01:19scientist at Pinterest but she studied
01:21with Professor Lynn Scott and she's like
01:23oh I know the guy who helped design that
01:26system let me introduce you and so I got
01:28introduced to Ivan and he started
01:30telling me stories about the design
01:33process and the mission and it was just
01:36so fascinating that I was convinced that
01:38you would be excited to hear Ivan's
01:41stories and so here we are and maybe you
01:43can tell us a little bit about yourself
01:44and how you got involved in
01:46communication systems design thanks
01:48Frank I've got to Stanford a long time
01:51ago because of my interest in the SETI
01:53project we had been developing
01:56high-performance spectroscopy in radio
01:58systems for looking at pulsars in small
02:02Observatory W Observatory in the East in
02:04Schenectady New York as a postdoc there
02:07one of the applications of this
02:09spectroscopy was looking for alien
02:11signals on the sky and promoting that
02:14got me an invitation to be a NASA fellow
02:18an RC fellow at the Ames Center which
02:21was then pioneering the work on the
02:23development of the study instrument so
02:24that got me to Stanford working with
02:27Professor Alan Peterson who was then
02:29leading the signal processing group in
02:32the Department of Electrical Engineering
02:33and Allen and I became responsible for
02:37the 10-million point before you
02:39transform analyzer that we built for the
02:42SETI project at his prototype that has
02:44then since been used and thinking other
02:46versions for the search that they've
02:48conducted it ever since but I but I
02:50haven't been involved since the late 80s
02:53when I started working more seriously
02:55with the space sciences group and
02:57electrical engineering under professor
02:59Tyler there and it was his interest to
03:01do radio science experiments in the
03:04outer planets which attracted me very
03:07much one of the things you can do with
03:09radio is like remote sensing and their
03:12specialty lens specialty with
03:14professional and at the time was to do
03:16what they call radio occultation it's a
03:18process by which radio signal is
03:21transmitted from one direction and like
03:23refracted or diffracted okay you know an
03:26atmosphere or soften rings or some other
03:28process on a planet and it's received on
03:31the spacecraft or vice versa
03:33and using that you can extract the
03:36properties of the refractor and in this
03:38case we get from that we can recover the
03:40temperature and pressure profiles of the
03:42atmospheres of the objects and so that
03:44was our plan for Pluto when Pluto
03:46mission was announced initially there
03:48were several incarnations of it and one
03:49of the early ones was in the night mid
03:511990s and lynn tyler at the time said
03:54you know i have done every occultation
03:55at every planet in the solar system
03:57except pluto it was still a planet right
04:01and he said this is gonna require a
04:04novel Architecture from the acquisition
04:07of the signal it's gonna have to be
04:09transmitted from the earth not to
04:10spacecraft just gonna have to receive on
04:12the spacecraft means the receiver is
04:14gonna have to be modified they don't
04:15like to do that they'd like to fly
04:17things that have flown before
04:18yep they don't like novelties about
04:20want to do with this we're gonna have to
04:22do the modification and so we proposed
04:24in the early 90s to to develop the
04:27technology nASA has a program for
04:30technology development that is
04:32applicable to future missions and so
04:33they gave us a support to build
04:35prototype receivers that would do this
04:37would be able to capture a narrowband
04:39signal and record it and then send it
04:42back with the opportunity to extract
04:44those properties that allow us to do the
04:47inversions that resulted in a sequence
04:49of proposals and so we teamed with us
04:52with a group of Southwest research led
04:55by Allen Stern to do this new Pluto
04:57mission they could call new horizons we
04:59became the radio science experiment
05:01called recs that had the objective of
05:03doing the radio alkylation of Pluto's
05:05atmosphere and Charon which doesn't have
05:07an atmosphere and to do what we call
05:10radiometry to measure the temperature of
05:12Pluto we using the radio antennae and to
05:15do an additional experiment where we
05:16bounce signals off the surface of Pluto
05:18called a by static radar experiment that
05:20would characterize the surface to a to
05:23even greater degree and in the ensuing
05:25years we had students develop the
05:27technology we had them build it we had
05:30them tested it was integrated into the
05:33spacecraft and you know you pretty much
05:35know the rest of the story we got our
05:37we got a exquisite profile of cudos
05:41temperature and pressure we did
05:44radiometry measurements and discovers
05:46some really quite surprising things like
05:49the date the Nightside is a whole lot
05:51warmer than the de site and we got to by
05:53static signal which is itself kind of a
05:56small miracle yeah that's amazing so
05:58funnest PhD projects ever right to
06:01design the system a novel system that
06:03would fly for the very first time right
06:04further than we've ever flown before is
06:07well the pioneers are out there further
06:09yeah this is the farthest radius of
06:12occultation experiments has ever been
06:13done and the furthest by static radar
06:16for this radar experiment that's been
06:17done so in a way they're a lot of firsts
06:20associated with this tell me how New
06:23Horizons got its power
06:24well close Pluto is 48 roughly 37 right
06:29now you know 40-ish a you out from the
06:30Sun 40 times the distance you can't use
06:34the square of the distance and so 40
06:37squared is a 1600s so you can 1/16
06:39hundredth of the power of the Sun that
06:41you would get at the Earth's orbit and
06:42that's not enough so common these other
06:46of these deep-space missions is to use a
06:48electric source called an RTG a radio
06:50thermal electric generator which uses
06:53radioactive nuclides to generate the
06:56heat which is presented to a thermo-pile
06:59that generates the electricity we use a
07:01radioactive element called plutonium to
07:03generate the heat and that generates
07:05about 250 watts for us power yeah so if
07:09you think about 250 Watts that is a very
07:12modest power budget so entry level PC
07:15without a graphics card properly 200
07:17watt power supply probably you know cup
07:19it's a couple of there's a couple of
07:20lightbulbs yeah yeah it's not a lot yeah
07:23but you ran a whole mission on that I
07:24mean everything it's amazing
07:26propulsion and telemetry education and
07:28all the scientific scientific
07:30instruments 250 watts you Menten it will
07:32turn out you can't run them all at the
07:34same time oh and my steering or am i
07:37doing science so you got you can do four
07:38out of five instruments and so one of
07:41them has to has to be turned off but
07:43fortunately we're maybe not for the for
07:46the instruments they're body mounted
07:48unlike a lot of other spacecraft this
07:50means that all of instruments are not
07:52all facing the same direction at the
07:53same time so you can only use one or two
07:56at the same time right it turns out
07:57right and be pointed in the right
08:01now plutonium has this property that it
08:04degrades over time it's radioactivity
08:05and so did you need to figure that into
08:07the design of the system oh sure there's
08:10a bunch of plutonium isotopes the
08:13longest-lived ones are mmm about a
08:15quarter of a million years but they're
08:17also the ones that don't generate as
08:20much heat because they're not popping
08:21off as much okay the ones that are the
08:23hottest are the ones that are the
08:25shortest-lived and they're about 84
08:26years so what you want to do is you want
08:29to get a fresh supply of plutonium
08:30socket on a spacecraft send it off so
08:33that in the first year so you don't lose
08:35you know more than a few percent to ten
08:37percent of the power in our case that
08:39process got started there is a plutonium
08:42processing facility that the US
08:44Department of Energy maintains
08:46and that there was an about halfway
08:49through the process there was a leak
08:51that was discovered in the radioactive
08:53leak when they shut down the process of
08:55production indefinitely well that
08:58doesn't sound good it wasn't cuz you
08:59have a window yeah you can get to Pluto
09:01we had a launch window that was narrow
09:04for a lot of reasons one was we were
09:06looking for a gravity assist at Jupiter
09:08which would have boosted day which did
09:10boost the arrival time by about four or
09:12five years it also was true that we were
09:16suta has served an elliptic orbit at the
09:18high point in your 'but yeah the apogee
09:21it's colder significantly than in the
09:24perigee which is the closest part and it
09:26had just passed the warmest part and so
09:29what happens if with the same model for
09:32Pluto at the time was kind of like a
09:33comet so when it got warm in the
09:35atmosphere would bloom when it got cold
09:37the atmosphere would collapse and if
09:39there was going to be an atmosphere
09:40there that we were going to measure its
09:41temperature and pressure we doggone well
09:43better get up there before too much time
09:45it didn't pass because it was very
09:47likely to have to collapse sometime soon
09:49so there was a lot of pressure on the
09:51mission to design build and launch yeah
09:54so you have to build this thing that's
09:55never been built before you're racing
09:57against time because otherwise the
09:58atmosphere you want to absorb isn't
10:00there and then the radiation processing
10:02plant springs a leak that's right and
10:04then they tell you this might not be
10:05possible so we had at the time the
10:08project had assigned a liaison to the
10:10Department of Energy within the program
10:13who is a was the face craft was being
10:15developed by APL Johns Hopkins Applied
10:18Physics lab and Glenn fountain was the
10:20person who was in charge of this process
10:23of acquisition of the RTG and mitigating
10:27problems that would come along with
10:29getting the licensee license for this
10:31from the Department of Energy and it has
10:34to go through but you know a whole lot
10:36of layers including a congressional and
10:38approval and environmental impact
10:40studies and oh and people who had
10:43protested the launching of plutonium
10:45before because other spacecraft had had
10:48it right but there they were a lot of
10:50protests about you know putting that on
10:51a rocket and sending it up in the yeah
10:52what could possibly go wrong
10:53yeah what could go wrong with that so we
10:55had to prove that if something went
10:58that and the payload exploded which they
11:01do every now and then rockets explode
11:03and well and they actually put that
11:04capability in there in case the thing is
11:06going in the wrong direction after a
11:09launch failure so you want to be able to
11:11you know mitigate that threat you know
11:13boom so you have to prove that if you do
11:15that that the plutonium won't disperse
11:17it'll fall down and intact and in such a
11:20way that it won't be potentially harmful
11:22all that in addition now the thing has
11:25been shut down and so Glen Glen - his
11:29extraordinary credit got into motion and
11:31he started discussing with the US State
11:33Department alternative sources of this
11:36rare and precious quantity called
11:39plutonium which you probably understand
11:42is actually one of the reasons why it's
11:45scarce is that it's this it's nuclear
11:47fuel in addition it's nuclear bomb fuel
11:50so to contrive many people on the planet
11:52can make it and my god and and there are
11:54quite a few that do that we don't want
11:56to and some that have done so that we
11:58haven't had in excess of it the Russians
12:01under this former Soviet Union made a
12:04significant amount of plutonium and they
12:06had it in storage and it was my
12:09understanding that what Glenn discovered
12:11was that the Soviets or the Russians now
12:13were anxious to maybe use it as a
12:16marketable option that they were looking
12:18for cash they desperately needed the
12:20cash they would have possibly
12:22entertained by offer from a variety of
12:26vendors and the US did not want that to
12:29happen because they didn't want it to
12:31fall in the wrong hands so Glenn
12:32brokering through the US department got
12:35us to make an offer to the Russians to
12:37buy their plutonium for this mission
12:39unbelievable for the missing piece of it
12:41wasn't the entire piece but it was about
12:42enough so that was brought over to the
12:46processing facility that could package
12:48it which was not shut down and they did
12:51that they packaged it put it up tested
12:53it blink got the licenses it was all
12:55under extraordinarily short fuses the
12:57last time that that happened for the
12:58Cassini mission the license itself took
13:01like three years and maybe pushing for
13:03and we had to have it
13:04in under six months that's amazing and
13:06it was amazing I just didn't think our
13:08odds were we're very good for that to
13:11happen nobody really did but then tell
13:12us some more about Clinton so what was
13:14his role on this mission well Glenn
13:15became our our program manager that the
13:18program manager of the mission at the
13:19time became ill and they were looking
13:21for somebody to step into John's ample
13:24shoes and and Glenn was asked and he
13:26agreed Glenn brought to the mission that
13:28same quality of statesmanship and
13:31problem-solving that he demonstrated in
13:34the brokering of the plutonium and it
13:36really eased so many problems that
13:39occurred a long way that had to do with
13:41funding irregularities recalcitrants on
13:44the part of the US Congress and the
13:48White House and all of those other
13:50problems that pop up day to day that you
13:52would otherwise any one of which could
13:54have stall this for too long
13:56Glenn made a work of and I think we all
14:00credit him with making this possible
14:02that's fantastic and so you get part of
14:05the plutonium that you need and who was
14:07the supplier of the last part that was
14:09Department energy okay good so now we've
14:11got enough plutonium actually we were
14:13still a bit short and the do II say well
14:16you know we have put aside a few extra
14:18pellets for the next space mission we
14:21might be willing to trade those for some
14:24new ones that will come along and the
14:27problem is that the old that the ones we
14:28have put aside are now bit old right so
14:31they don't have so much juice and you
14:32won't have the power that you expected
14:34we had maybe thought we'd start out with
14:36like 300 watts and we didn't we started
14:38maybe with 280 to 75 and by the time we
14:41get there it's you know pushed down the
14:43room to 45 did was a succession of
14:46compromises and brokered deals but yeah
14:48eventually it all kind of worked yeah
14:50well let's work our way back to the
14:53impact on the communication systems
14:54right because you had assumed a power
14:55budget assuming fresh plutonium from the
14:58Department of Energy and you got kind of
15:00tired plutonium from a variety of
15:02sources and now you have a much lower
15:04power budget right we were initially
15:06offered a chance to implement our new
15:09technology for acquisition of the signal
15:12from that was transmitted from the earth
15:13to the spacecraft like I said we had a
15:15sponsored research development program
15:18knew how to do it we could we thought we
15:19could fit it into a signal processing
15:22system but it would require in the
15:25existing technology of the late of the
15:27early 2000s I mean these have to be
15:29radiation hardness because after all
15:32you're on a spacecraft and not too many
15:34not too meters away is this variety
15:37source of radioactivity pumping out
15:38gamma rays and neutrons like crazy so
15:41not good for chips yeah you have to have
15:43a radiation tolerant implementation and
15:45that was possible in a signal processing
15:49element called of FPGA which are field
15:51programmable gate arrays that act L was
15:53making and they were based on a
15:54technique of hardened by design but also
15:57what they call triple redundancy you
15:58have each element in the each gate in
16:01the end the array is triple as Triple E
16:03implemented and you vote them in pairs
16:06so if if all three agree fine if two
16:09agree you chose that you chose that one
16:11right and if none of them agree well
16:13you've got a problem the Minority Report
16:15yes sir that's right but it will turn
16:18out that it will be the case that you'll
16:20get three or two well essentially all
16:23the time so those are limited in
16:25capacity and the time I think you could
16:28get a million gates and one of the
16:29highest density aktel you could get for
16:31this case which is you know it has to be
16:33face to face flight qualifying so we had
16:35been offered the opportunity to
16:37implement in two of these and as you say
16:40Frank has the power budget was reduced
16:42and the scope was was reevaluated they
16:45came back to us and said well guess what
16:47we really have to have one it's just one
16:50all the way to Pluto you've got one got
16:52one PGA now we knew that we were at the
16:56time about 50/50 now that we could put
16:58half of the design in one and half on
17:00the other and they could talk to each
17:01other and it would be all just fine and
17:03and that was good from a lot of reasons
17:06because the one of the design guidelines
17:08from the mission is that you have to
17:11have a margin of gates in these FPGA
17:13that is high enough to give you
17:16assurance that the implementation in the
17:19gate is going to be successful one of
17:20the biggest problems in implementing an
17:22FPGA design is routing and being able to
17:26successfully route to all the gates in
17:28this million point of gets exponentially
17:31located as you get closer and closer to
17:33the head to the top of the line and so
17:35they say that got NASA guidelines as you
17:38need at least we would like you to have
17:4025% margin minimum number is 15% which
17:43means 50% of million gates 150,000 gates
17:47just sitting there unnoticed and we were
17:49fine we had 50% okay now we have to
17:52stuff it into one and we said well we're
17:54not sure we can do this
17:55and they said well I don't think you
17:56have a choice so I had a student at the
17:59time who had designed the signal
18:01processing for this application
18:05this was Kamakshi she were my Krishnan
18:07Kamakshi wanted to take on a challenge
18:10we said well this is the challenge we've
18:13got a signal that's coming in in this
18:14big wide bandwidth it's 4 and 1/2
18:16megahertz and this actual signal is
18:17about kilohertz which is basically ten
18:21thousand to one reduction in bandwidth
18:23and we got to make that reduction we
18:25have to throw away all that other stuff
18:26and pick that one band that we need and
18:30we have to do that in such a way as to
18:31guarantee linearity in the process that
18:34filter has to be perfect it has to
18:36suppress the out by a noise and inside
18:38the band has to be flat it has to be
18:40linear Lisa called fi RS finite impulse
18:41response filters but a 10,000 element F
18:45IR is unimplemented all there is no
18:48algorithmic stability that could you
18:51could possibly find in an any computing
18:54technology even today that would do that
18:56so come on cheese task was to figure it
18:58out and do it in a way that was not
19:01computationally expensive and she did
19:02she discovered a old radar idea by RCA
19:08employee Glenn Hagemeyer who was working
19:11in the 1950s and 60s when they didn't
19:15have a lot of transistors in a chip and
19:16they had vacuum tubes and they were
19:18doing radar on planes and they wanted to
19:20be able to reject the clutters and the
19:23jamming and so they want to be able to
19:25filtered and so Hagemeyer had developed
19:27these computationally efficient
19:28techniques for filtering linearly and we
19:33do vacuum tubes and Kamakshi the design
19:34was I think this could work if we did it
19:36this way so she did then I had another
19:39couple of students that tried to take
19:40that and drop it down into a
19:42programmable device FPGA and well so I
19:45tell gives you all of the tools and
19:46they'll even give you a course in how to
19:48do it and they give you the development
19:51environment and they give you program
19:53languages and they give you simulators
19:56they give you routers they give you is
19:58simply gave you the emulator to test it
20:00a couple of students one after the other
20:01I went through that whole process got it
20:03down into the chip I actually bought
20:05some chips and the programming the
20:07stations and we burned them in and we
20:08turned them on and we ran a signal in
20:10and it didn't work no we spent the
20:12better part of a year not getting it to
20:15work wow that long yeah yeah and after
20:17and that it wasn't a whole design it was
20:19just a small test piece that uses maybe
20:22a tenth of the hundred thousand gates it
20:24didn't work there was something
20:26fundamentally broken in the process of
20:28how that language assembled into code
20:33that was then dropped onto the device is
20:35something fundament we didn't understand
20:37after a year the program manager that
20:39oversaw the entire spacecraft's Glenn
20:42was the payload guy and there was a
20:44manager at Southwest San Antonio which
20:47which had responsibility for the whole
20:49program he's getting worried he's saying
20:51Stanford doesn't look like it's gonna
20:53succeed we could have to do something so
20:56he sends a SWAT team in to review your
20:59come to fix and it's until already my
21:02hackles are up right now I'm real real
21:04prickly about this right the guy who is
21:07head of that evaluation was a former
21:10Stanford graduate student of not too
21:12long ago who was now the payload chief
21:14engineer and Mark worked happily and we
21:17became close friends quickly it was
21:20amazing how that compatibility seemed to
21:22get to kick in so mark went back and he
21:25recommended to Bill Gibson the program
21:26manager you know that there what staff
21:29needed help but they didn't need it
21:31wasn't probably a good idea to take the
21:33whole project away and give it to
21:34somebody else so how can we help him
21:36well Southwest had a fpga programmer for
21:41qualified solutions Mark Johnson marks a
21:45motorcycle-riding guitar-playing
21:50Thun I have this great mental image he's
21:53got out of Texas right yeah so mark
21:56shows up and we you know wants to know
21:58what's going on what are we doing so
22:00come out she's there you know we give
22:02them a story we give them the story we
22:04tell them what's going on
22:05two things really important happened one
22:07is Mark became really fond of Komachi
22:10always helps the other was that mark
22:13says you know I have a very strict
22:15discipline about how I code there are
22:18certain choices you can make in the
22:20language of the code that are absolutely
22:23guaranteed to fail he said you only know
22:27that would zeal or buy you know a
22:28lifetime of trial and error and mostly
22:31air but these are just this is a
22:33discipline you have to have he said
22:35students PhD students in Stanford in
22:38electrical engineering are too clever
22:40they see the opportunity to recast an
22:43algorithm into a more efficient form and
22:45they'll do that they can't resist doing
22:47that Steve said that's the problem mark
22:50took over the code and with Kamakshi x'
22:53interaction they succeeded in getting
22:56the first attest module to work and then
22:58the succeeding modules one after the
23:00other and when he was done he had think
23:04it was twenty five gates left over in
23:06them out of the million that was in the
23:07FB 25 Kate's not 25 percent no 25 said
23:11less than 1% so we went to the project
23:13and said okay here's the deal
23:15it works it works it barely fits it it
23:18works it fits we can we've demonstrated
23:21it across the board there is there are
23:23no test procedures that show any
23:26vulnerability we've stressed it
23:28temperature radiation I mean we've done
23:30all that stuff so you can you got a
23:33choice you know you can take it as it is
23:34and can waive the margin or give us the
23:37other chip so they waived the margin
23:39then they said well actually there are a
23:41couple of other functions that this
23:43ought to have do you think you can fit
23:45them in its voices I'll take a look I
23:49said you kidding he said no I think so I
23:52think so when you so he did yeah and
23:54there were a couple of just simple
23:55protocol acknowledgments on the
23:58communication link that they wanted to
24:00seed him to make sure that the thing was
24:02handshaking correctly
24:03so he was that he had five five days
24:06left and that's the way we saw the FPGA
24:08on the mission to Pluto has five days
24:10under Yukon you know non-use gates yeah
24:13lord knows how that actually the routing
24:15worked but again that was Mark's magic
24:18be able to kind of groom the routing you
24:20do these look you do these and that's
24:22first and then you do the next in the
24:24set and you just sort of layered and you
24:25don't give it all at the same time and
24:27you just try to slowly ease its way into
24:29completion its herding cats but yeah get
24:32them all into the corral let's back up a
24:34little and talk a little bit about how
24:35the probe actually communicates its
24:37signal so where did the signals go where
24:39they received so we transmit from the
24:42earth with the deep-space schnitz
24:45stations which are there's three of them
24:47that ring the earth and we used two of
24:50them because at the time of the
24:51encounter Buddha was up above the
24:53horizon for Goldstone California and
24:55Canberra in Australia they have antennas
24:59that are two sizes ones a 70 meter
25:01diameter and the others are 34 s they
25:04all have these 20,000 watt transmitters
25:07so during the occultation portion of the
25:11fly-through of the encounter we had four
25:14of those transmissions going
25:16simultaneously actually five and i'll
25:19tell you though about the fifth it's the
25:20by static one so the seventy and a
25:22thirty-four from each of the two
25:23stations transmitting at frequencies
25:26that are within this one kilohertz of
25:28each other spaced about a hundred Hertz
25:29apart to the spacecraft what's going to
25:32happen is as the spacecraft flies to
25:34pluto and then past and in flight oh
25:36it's at an angle and it flies into the
25:38shadow of pluto and seen from the earth
25:40and so that's the occultation now you
25:42come on the way in to the shadow and
25:44then coming out on the other side and
25:45then and in those moments before ingress
25:47as we call it an egress the refraction
25:50of those four signals through Pluto's
25:52atmosphere has the effect of shifting
25:54the frequency of the received signal and
25:57that's because it the direction of the
26:00ray bends slightly and you're getting
26:02the Doppler from an from a slightly
26:04different direction than you were before
26:06so what we do is we capture that central
26:10thousand Hertz and we sample it into the
26:14present in the signal and the waveform
26:16has got a lot of amplitude in other
26:18words we've sent 20,000 watts on an
26:21aperture that's about 70 meters in
26:23diameter so the instantaneous power in
26:26the beam is huge it's over a million
26:28watts per square meter at Pluto which is
26:313.7 billion miles out and a system that
26:33captures this those waveforms are
26:35sampled with noise that's one thousandth
26:37of the amplitude of that signal so it's
26:39like you see a sine wave thoughtyou
26:41sampling many times recycle each dot is
26:43a as a sample value hasn't has noise
26:46associated with it associated with this
26:48with the receivers noise and the sky
26:49noise and this noise from coming down
26:52the radio path from the earth you're
26:53going through you know three and a half
26:56billion miles of solar wind and plasma
26:58irregularities it's extraordinary that
27:00all that happens and you still get a
27:03signal where the amplitude is noisy only
27:07a thousandth of the waveform
27:08now the power which is this which is a
27:10square of that you square the amplitude
27:13of the voltage squared is power so a
27:16thousand squared is a million so we got
27:19signals that were a million to one above
27:22it's Bluto yeah Pluto so what you can do
27:25with that is a lot the way we sample it
27:28is we have a clock that's running the
27:30sampler and on the broadband channel
27:32it's running at ten million ten million
27:34samples a second that clock is derived
27:36from an oscillator on board that
27:38oscillator itself is what we call a uso
27:41an ultra stable oscillator that
27:42oscillators quartz it's a slice of
27:44quartz encapsulated in a glass
27:47sarcophagus you know carefully suspended
27:50and thermally insulated and kept at
27:52temperature barely changing by a
27:54thousandth of a degree and as a
27:57consequence it doesn't change its
27:59frequency by one part in ten to the
28:01that's one tenth of a trillionth of the
28:04frequency added some accurate
28:06oscillation so what I have is I have a
28:08million-to-one Selah noise ratio sample
28:10to a precision of one part in 10 to the
28:1313th basically you can measure a gnats
28:16eyebrow at a billion miles with that
28:18thing yeah and and that's what we did
28:19that's amazing the oscillator itself is
28:21its own story because us owes were
28:24commonly implemented by
28:26the Department of Defense during the
28:28Cold War they were used for surveillance
28:32if he ever read blind man's bluff about
28:35submarine espionage it was conducted
28:38during world the Cold War and using
28:40atomic submarines the US would not tell
28:43any secrets this is in the book they
28:45found and the undersea cable that the
28:47Russians had laid from Vladivostok
28:49across the bay and they they they and
28:52they snuck in under the Soviets
28:54surveillance and they put a box onto the
28:58cable that had been designed by MIT
29:00signal processing lab which would sample
29:03the fluctuations in the voltages that
29:06were induced by the communication
29:08traffic in the cable that's awesome and
29:11then they would compact months later and
29:13retrieve the box and put another one
29:15down bring the box back to the DS
29:17scramblers headed these a bundle of
29:19cables with you know dozens of lines of
29:21communication sometimes running you know
29:23simultaneously they'd had a descrambler
29:25they had a decoded it translated and
29:27they basically knew what the Russians
29:30were talking about you know four years
29:31unbelievable by putting a box over the
29:34cable over the Kate didn't they didn't
29:36puncture it right they didn't there
29:38wasn't an invasive to monitoring because
29:41it would have caused the cable to leak
29:42and fail so you couldn't do that right
29:44yeah I count that you know among the top
29:48ten actually nice stories of all time
29:51yeah anyway to do that you needed the
29:53ultra-stable oscillator and I may be
29:55rambling on here but no because the Cold
29:58War when it went over was when that
30:00happened the former Soviet Union was
30:02disassembled the market in u.s. owes
30:05disappeared mmm we didn't need them
30:07anymore we didn't need him anymore there
30:09was really very few people that even
30:10cared about having a part in tennis 13
30:1410 to the 14th thing right anyway so
30:16here you are in the year 2000 and we won
30:18one of these UFOs where do you get one
30:20right and it was the same story like
30:21with the plutonium we've started looking
30:23for suppliers they didn't exist there
30:26was a French company that was making
30:28them and they'd had gone bankrupt there
30:30was a US company that was making him
30:31they'd gone bankrupt interestingly at
30:35frequency laboratory high frequency
30:37stability laboratory that they had been
30:39kind of encouraging this technology
30:41internally and they had a program then
30:44of exporting technology to startups and
30:47they thought well this might be a good
30:48opportunity to do that so they did they
30:51transitioned the capability that they
30:54had to a small start-up and they started
30:56making these things Wow
30:57gave him like three years and so here we
31:00are six months before launch we have to
31:03actually have one of these things to
31:04integrate into the payload and that
31:07company delivered five Wow we tested the
31:10five - okay Wow you made two met specs
31:14and they were put on mission
31:16unbelievable so we've got one of these
31:18like incredibly rare oscillator yeah
31:21we've got the sort of mix of old and new
31:23plutonium we've got the FPGA they've got
31:25five gates left over on it and you
31:28launch the sucker all right like so now
31:29we're going and then so it's gonna take
31:32years it's almost a decade right before
31:33it gets to its destination and so you're
31:35testing the communication all along yes
31:37every just called the annual checkout
31:39and space crafts in hibernation once
31:42here we come I quote wake it up we put
31:44we exercise a set of tests we transmit
31:46to it we do performance evaluation like
31:50we were going to do the occultation z'
31:53tests and we make sure that everything's
31:55running okay and episode once a year for
31:57the for the nine years we went through
31:59the and we'll check out plus we did two
32:01rehearsals we ran the whole process of
32:03occultation and radio metric measurement
32:06in rehearsal yeah and then somewhere
32:08along the way am i right that you lost
32:11communication with the satellite at the
32:13very moment almost of the encounter Wow
32:16so you're almost there we're almost
32:17there and it checked out year after year
32:19during your yearly check out everything
32:21is you're almost home and there's a
32:23message from the Mission Operations
32:25Manager who call her mom / mission
32:28operations manager Alice Bowman yeah and
32:30so mom calls that's not a good sign I'm
32:33so Saturday morning we're about to go to
32:36Baltimore my wife Margot and I for the
32:39encounter a lot of the other members of
32:41the team are already there and it's two
32:43Saturday morning we're having breakfast
32:44and I don't normally even open
32:46laptop to see messages and and I do and
32:49there's one from Alice and she says we
32:52have a conference call you know in an
32:54hour to discuss the spacecraft went safe
32:57when the spacecraft goes safe those are
32:59not good words because it meant that
33:01something something went wrong and the
33:03spacecraft tucked its head under its
33:05wings uh-huh and and shut everything off
33:08and tried to find out where Earth was
33:10and and turned on a tone that said one
33:13of several things happened and you get
33:15something to figure out what it is
33:17Wow and so you joined this conference
33:20call and the satellite is nowhere to be
33:22and he trying to find it well first of
33:23all they were fortunate to get there was
33:26it was relatively easy to acquire the
33:29beacon and the beacon said that there
33:31had been a Fault in the onboard computer
33:34and without knowing exactly what that
33:36was the team needed then to establish a
33:40set of protocols as to how to go and
33:43wake up enough in the spacecraft give us
33:45the commands to wake up enough
33:46functionality to look at the error log
33:50and transmit the error log back so they
33:52could figure out what happened and then
33:53so they said they had done that they
33:55would they would do that we did get back
33:56to us and they then in a few hours
33:59because it's a four and a half hour
34:01flight travel time so essentially
34:03round-trip is nine hours so yeah
34:05basically and then we'll give us a
34:06couple hours to figure out what's going
34:07on we'll have another conference call in
34:10in ten hours and and in ten hours it
34:13turned out that the computer had faulted
34:16because it encountered an overload a
34:18timeout actually that occurred from the
34:21fact that it had us process had starved
34:23out watchdogs timer in the clock so
34:26something had taken over that are taken
34:28too much time and we weren't sure what
34:30it was then it wouldn't take too long to
34:32figure out what it was I was in X
34:34iteration of the communication so in the
34:36next ten hours later what we knew was
34:39that the computer had been taking an
34:43image that was taken earlier of Pluto
34:46against a small imaged against the dark
34:48sky it was not very big and they were
34:51compressing it to be using it to JPEG
34:53algorithm and it was going to be then
34:57scent than telemetry there was the
34:59compression algorithm that wasn't done
35:01when they had sent a command to update
35:06the onboard flash memory within with the
35:09up-to-date instructions as to when to do
35:12so it turns out that the computer wasn't
35:15finished the command to update flash was
35:18received it was given a very high
35:19priority the highest priority it said I
35:22don't know what to do
35:23and it went safe it just did they raised
35:26the fault flags shut itself down went
35:27safe so we're all thinking why did that
35:30happen who you know who could possibly
35:33have screwed up and made that happen
35:36that isn't that isn't that's not
35:38something that's supposed to have
35:39happened right now that shouldn't have
35:40happened right now we know what this
35:41stuff does why is this a problem it took
35:44and why didn't we find it when we ran
35:46the rehearsal no no it's six months ago
35:49we did exactly the same thing we've ran
35:52exactly the same programs we went we
35:54sent the exact same update command to
35:56flash well I wasn't in a problem there
35:57it turned out that six months ago the
36:00picture of the sky was dark there was
36:02nothing but black and a few stars so
36:04that JPEG compression took a whole lot
36:06less time right so it was done when
36:09there when the flash commands came in
36:12and stars confuse the algorithm just
36:15enough or delayed the algorithm just
36:16enough to get into a race condition yeah
36:18nobody had seen that which of course in
36:20the course of spacecraft missions you
36:23come to understand that things always go
36:25wrong it's not for the faint of heart
36:27that is amazing and so you managed to
36:30reboot the computer computer was yeah
36:32restarted the real concern was that we
36:36were on a time line knows everything on
36:38a spacecrafts run autonomously and in
36:40order for the encounter sequence to
36:43proceed with the science that we
36:44intended to obtain that timeline needed
36:47to start only it was like three hours
36:50from when the fault occurred so to their
36:53extraordinary credit the operations team
36:56was able to do that they restarted the
36:58timeline just moments minutes before it
37:02was needed to get it on track to do the
37:04encountering the way that captured all
37:06of the scientific opportunity
37:07so we were minutes away from not getting
37:10any other pictures from screwing up
37:12completely yeah unbelievable well not
37:14completely because what it would have
37:15done is we do sacrifice some of the
37:17early scientific measurements for the
37:20later ones all right for those of you in
37:22the listening audience go now Google New
37:25Horizons pictures and much deeper
37:28appreciation hopefully when you see
37:29those pictures realizing how close we
37:31were to not getting any of those so one
37:33other thing I'd love to talk to you
37:34about those are an amazing set of
37:35stories so thank you for sharing them
37:37and I talked to you a little bit about
37:38the dish at Stanford the big dish the
37:40big dish right and so most people think
37:42of it as sort of a landmark on your
37:43hiking meet you at the dish I mean it is
37:45and it is I have torn it down if it
37:47wasn't oh him work by now exactly but
37:50it's actually being used in scientific
37:52experiments so tell us a little bit
37:53about that right from time to time the
37:55dish is actually in the stewardship of
37:58SR I miss our international they were
38:01partnered with the team men at Stanford
38:04electrical engineering in the 1960s when
38:06it was proposed as a facility to support
38:09initially investigative technique that
38:13US Navy and Department of Defense was
38:16interested in for communication over the
38:18horizon they were interested in talking
38:21to ships at sea that were over the
38:24visible horizon hoping to get out
38:26significantly maybe two as far as Asia
38:29and in both directions I mean you want
38:31to be able to send a signal over and you
38:33want to be able to maybe listen over the
38:34horizon as well that was a that would
38:36have you know listening at that time of
38:38course of course I'm very popular yes so
38:40there were ideas from again Alan
38:43Peterson and his colleagues in the
38:45electrical engineering department that
38:47suggested that there were techniques for
38:49taking advantage of propagation of
38:51phenomena that would allow you to but
38:53for radio signals to refract over the
38:56diffract over the horizon and be able to
38:57get something so they figured they
38:59needed a big antenna and they needed
39:02transmitters and they needed to have it
39:04high enough to be able to point and be
39:06close enough to Stanford to actually use
39:08it with from from a campus perspective
39:11and so they proposed to the US Navy to
39:15they were partnered with SR I to do so
39:18and they got the funds to do that that
39:21would then emerge that there was a a
39:23subsequent interest from NASA to
39:26investigate the properties of the solar
39:28wind that measure the total electron
39:30content in the solar wind out through
39:32the solar system and that meant that you
39:35because Pioneer spacecraft at the time
39:38were being developed and sent into the
39:40inner and outer solar system that if
39:42they carried properly the you could do
39:44this if you have two frequencies and so
39:45you build a spacecraft with receivers
39:47that can operate at two frequencies
39:49ideally an octave up more apart and so
39:52they did that and they built facilities
39:54into the dish to transmit and receive on
39:57those frequencies and so four years
39:58after the experiments to do over the
40:01horizon work faded away did the there
40:04was a long epic NASA work that was done
40:07with really characterizing the solar
40:11wind throughout the solar system it was
40:13an epic body of work and they did a
40:15wonderful job with it that's fantastic
40:17and then after that period did they shut
40:19it down yeah by the late 1970's and
40:21early 1980s that work was largely
40:23complete that task actually had been
40:25taken up to some degree by the other
40:27this next generation of spacecraft to
40:29have their onboard plasma instruments
40:31and they were able to do sample
40:33Institute the conditions of the solar
40:35wind and so it became you know an
40:38unnecessary task and they stopped that
40:40there was very little else that stepped
40:43in largely because there was a
40:45transition in personnel and capability
40:48an interesting aspect I think of
40:51operations at the dish was in order to
40:53facilitate the tracking in some cases
40:56they wanted to track satellites that
40:58were earth orbit and so you want to be
41:00able to follow for low-earth orbit you
41:02want to yes the thing rises above the
41:04horizon you want to be able to it's an
41:05altitude azimuth drive which means that
41:08you have the ability to go up and down
41:09and rotate and azimuth and then up and
41:12down elevation so you have to so you
41:13want you pointed in azimuth at the
41:15horizon and you and it's think comes up
41:17and you start to raise in elevation
41:19slightly and azmuth and then as it gets
41:21close to the zenith you're suddenly
41:22rotating very rapidly in azimuth and
41:25then coming down so the azimuth rates
41:27have to get high enough and you have to
41:29take this giant structure and you have
41:31to twist it fast in the 1960s there were
41:35no electric motors that could do that
41:36electric motors are great if they're
41:38running fast and the transmissions that
41:40you would need to gear them down so that
41:43the dish which is from you know rambling
41:45we're impractical to even to use those
41:47would mean that the point you'd
41:48sacrifice pointing accuracy it's a big
41:50dish the pointing the beam is tiny you
41:52have to be able to control it to a
41:54fraction of a degree you couldn't have
41:55done it and you can't do with electric
41:57motors so but they found out that you
41:59could with with something else otherwise
42:01they wouldn't built it there's something
42:02else is hydraulic motors uh hydraulic
42:05motors are very good at having a very
42:07high torque at low angular velocities
42:10and the hydraulic motors that were the
42:12bet that were best designed for that
42:13were the were the anti-aircraft gun in
42:16control on u.s. battleships right and
42:19after World War two those were being
42:23uh and so what they did was they got two
42:27of them well maybe they got a few more
42:28to per spares and they stuck them into
42:31the altitude azimuth drive system for
42:33the dish the hydraulic motors are a very
42:36peculiar beast they use very high
42:37pressure on a set of pins that push
42:40against a off-center plate and what you
42:43do by differential pressure is you push
42:45that plate and it rotates
42:46it's a funny concept and it's hard to
42:49explain without images but the key
42:53ingredient is high pressure and one of
42:56the problems with this battleship is
42:59that you know you're under fire and one
43:01of those high pressure hydraulic lines
43:03is cut the line becomes a weapon it will
43:07spray tens of thousands of pounds per
43:10square inch oil around the control room
43:12and it doesn't sound good
43:14yeah it was a very dangerous job to have
43:16which was which was the anti-aircraft
43:19control guy who was sat in that seat
43:20moving that thing around because if
43:22something went wrong you probably
43:24weren't going to survive it anyway how
43:26they got approval to stay
43:28that on the banana dish in this era
43:32teaches reserva division I don't think
43:34that would have ever happened but the
43:36desperation being what it is and
43:38innovation being the better part of
43:39valor they did yeah but by the 1970s the
43:43technicians that knew how to prepare
43:45that were retiring mmm the need for
43:48repair was increasing and so the general
43:52consensus was we really shouldn't be
43:54running this anymore and so the
43:56combination of lack of objective with
43:59the age and the growing sense of risk
44:03meant that there really wasn't very much
44:05you should be doing with that dish yeah
44:07so they shut it down I remember when I
44:09was on campus in the 80s and 90s it
44:11wasn't because it was just a landmark
44:13right but then you had the idea that
44:15maybe we should be using it yeah long
44:17about the same time I stopped work with
44:19saying I still carried that nagging
44:21sense that gee I really should be
44:23looking in for something anyway and so
44:26this is sitting up there yep you know
44:28fallow and I started asking around and I
44:31was led to s RI and a colleague over
44:34there Mike last name is cousins my
44:36cousins who was there like the chief
44:39engineer that knew most about it and had
44:41been using the dish up until then still
44:44did from time to time but so Mike and I
44:46developed a really lifelong friendship
44:48and the desire to keep the dish alive
44:51and so what we found was that there were
44:54a couple of things that were in need
44:55clearly one was the control and the
44:58other was the receivers the era when the
45:01dish was used those receivers could be a
45:04relatively low performance they were
45:06good at the time but they were
45:07unsuitable for listening for a rating
45:10like alien signals on the sky or looking
45:12for subtleties within the Galactic
45:14background and so we needed to
45:16demonstrate that you if you had the
45:18right receiver you had in this
45:20environment enough sensitivity to with
45:23that dish to make a difference one of
45:25the big concerns was that dish is
45:28sitting fully visible from the Bay Area
45:30one of the highest sources of radio
45:32interference that you can imagine and so
45:34we were necessarily concerned that even
45:38if you had the most sensitive receivers
45:40that they would be swamped
45:42and by the interference in the
45:43environment indeed that was true and so
45:46what that spawned was a succession of
45:48research into the way the means of
45:52isolation and protection really excision
45:54how do you eliminate without destroying
45:57what you're looking for the unwanted
45:59interference and that became a research
46:02project for me a couple of students that
46:04I had pioneered some very nice solutions
46:07as well as Mike's ability to develop to
46:10implement those and it was Mike though
46:13that heard about electric stepper motors
46:16so this replaced the hydraulic motor so
46:18we placed the hydraulics so what
46:21kind of wrinkle in the technology story
46:24was that over the era from the 1970s to
46:27the 1990s and 2000's the progression of
46:31the disk drive industry to
46:33ever-increasing capacity had led to the
46:36sophistication of these electric stepper
46:38motors for controlling the drive on the
46:40yoke of the magnetic readers going in
46:42and out and they had a step with
46:44increasing precision and they had to do
46:47it through precise rotation of the motor
46:49and it had to do it quickly had the
46:51quicker the better to because they were
46:53increasing speeds and densities and so
46:54what happened was that this drive
46:56industry spawned a technology for the
46:59electric stepper that was for purposes
47:01of industrial control and so now instead
47:04of something that was you know a cubic
47:06centimeter in size in your little
47:08disrobe you could buy something that was
47:09the size of a Volkswagen and would turn
47:12the dish and they did we replaced the
47:14hydraulics with electric steppers and
47:16the dish woke up we were able to do not
47:19only a high accuracy high speed tracking
47:23on the sky which we were involved in a
47:27couple of failed spacecraft there's a
47:28couple of spacecraft were launched and
47:30then they stopped talking to the ground
47:32couple by us a couple of by you know
47:36NASA code by the our colleagues in
47:37Britain the realization was if the thing
47:40is alive but the radio system is broken
47:42so it hit transmitters say died but the
47:45receiver still alive maybe we can still
47:47figure out what happened maybe it's just
47:50stuck we could send a command to unstick
47:52it but we need to know what's going on
47:54so if this thing is still alive up there
47:56then the telecom system is partly awake
47:59and the reference frequencies from their
48:01oscillators are leaking out from the
48:05spacecraft the thing is that the leakage
48:08is tiny some it's a millionth of a lot
48:10coming out of the spacecraft in orbit a
48:13thousand kilometers away yeah so to find
48:16that needle right you realized that the
48:20dish has a sufficient aperture and
48:22sensitivity to see a millionth of a watt
48:25had a thousand kilometers that's a it's
48:27a strongly I mean this thing's stuck up
48:29stuck up like a sore thumb NORAD was
48:32tracking the space junk it said here's
48:33here's the ephemeris of the thing and we
48:36programmed that into the tracking
48:37computer attract it and turned on the
48:39Doppler compensation and bam there was
48:41and there was he found least we found it
48:43with satellites it was still alive yeah
48:45unfortunately I think except from one he
48:48couldn't fix it but was there yeah
48:52that's fantastic well professor Linscott
48:54thanks so much for coming to share these
48:56stories their amazing stories and I'm
48:57glad we got to share them with a broader
49:00I appreciate the invitation so very much
49:03it's not often you know we get to ramble
49:06about this stuff man it's a pleasure to