Update on the 'zombie' ISEE-3 spacecraft

Watching my local news last night, I noted an interview with Dennis Wingo that was labeled Zombie Spacecraft Rescue Planned by Private Group. It seemed an apropos title so I’m going with it – Anthony

ISEE-3 Reboot Update, Ranging The Spacecraft

Guest essay by Dennis Wingo

Folks

It has been a bit since my last update.  Tomorrow I hope to have a more thorough one for you all but this one here should be of interest, especially for the more technically inclined.  We had a pass from the NASA Deep Space Network (DSN) that was to be used to improve our knowledge of the position of ISEE-3 in space.  This is not an easy task, though NASA’s DSN team makes it look easy.  

Every measurement, in on way or another, depends on accurately measuring the time it takes for a signal to either come from the spacecraft (passive mode), or a two way round trip signal, (ranging mode) or the most accurate means, called coherent ranging where the spacecraft locks to a signal from the ground, and matches its phase coherently.  The ISEE-3 spacecraft, even thought it is of a 1975 design vintage has this ability.  However, things did not go as planned last week.  Here is what I posted on our http://www.spacecollege.org site and this was also sent to our NASA sponsors.

_________________________________________

The DSN pass on June 18th that went from 1:45 to 2:45 Pacific Daylight time was not a success. Here is a recap of the pass activity.  The DSN pass started at 1:45 pm PDT.  Here is a graphic of the pass through a very nice DSN Now web app:

Figure 1: NASA DSN Now Web Interface Showing ISEE-3 (ICE) Pass Via DSS 24 Goldstone

The pass began with a +/- 3 KHz sweep across frequencies representing the input frequency of transponder A (2090.66) MHz + the Doppler offset + an additional 11.25 KHz that came from our most recent command session. The additional offset is due to thermal and or aging issues with the spacecraft transponder. The sweep is done with a carrier only, no modulation, to get the receiver on the spacecraft to lock to the DSN transmitted signal. The output of transponder A will start to vary in a 240/221 relationship when the carrier is locked. Then ranging can occur. The sweep was unsuccessful in establishing a coherent lock. The sweep rate was 60 Hz/sec. This conforms to the procedure used in 1985 by the DSN for the spacecraft during the ICE comet encounter.[1]

A second sweep with a bandwidth of +/-6 KHz was initiated at the same sweep rate. This was also unsuccessful. It should have been successful as a total sweep of 12 KHz encompasses more than the offsets that we have successfully used to command the spacecraft. After this did not work, the DSN ops team then did a sweep at +/- 20 KHz with the center frequency set at our Doppler + 11.25 KHz offset. This surely should have worked in obtaining a lock, but it did not. Due to the time involved to do the sweep this exhausted our available time on the Goldstone dish, and thus we completed the pass without any indication of lock from the spacecraft.

Troubleshooting the Pass

Link Margin

The configuration of the Goldstone system was using the DSS-24 34 meter dish, running 10 kilowatts of power. Table 1 gives the estimated link margin:

Table 1: Estimated Link Margin for Ranging with Carrier at Goldstone for 6-19-14:

Looking at the link margin, it is evident that the link was not the problem.

Failed Subsystem or Procedural Issue?

There are two possibilities for why the ranging failed, when we know for certain that transponder A is functional for commanding the spacecraft. The first is that the ranging function on the transponder has failed. The second is that a procedural error regarding commanding the communications system into coherent mode is at fault. Thus by eliminating the DSN link as an issue and looking at what could be the problem otherwise we have the beginning of a fault tree established for an investigation.

Failed Subsystem

The possibility that the ranging function has failed, or was not functional was taken into account by our team. The spacecraft has two transponders, A, and B, and both are capable of ranging. There are some issues with lower gain on transponder B’s antenna that make it less desirable but with the robust margin from Goldstone seen in figure 2 this is not a problem. On our 6-15-14 Arecibo pass we placed transponder B into coherent ranging mode as well as transponder A. However, due to time constraints the DSN did not have time to attempt ranging to the B transponder. Also, due to lack of licensing to transmit to the B transponder from Arecibo, we have been unable to verify the functionality of that receiver. Thus one possibility would be to range to transponder B for our next DSN pass on 06-25-14. However, this does not address the issue of what the problem might be for transponder A.

Our success with transponder A has not been 100% in sending commands to it. The receiver input frequency seems to be drifting toward a higher frequency over time. We do not know at this time whether that is a random drift or one that is predictable. We overcome that when we do our commanding by sweeping the expected frequency plus Doppler plus an offset when we send commands. Since our first commanding session the offset frequency has appeared to drift to higher frequencies each time. This is not completely unexpected as we found test data that indicated a positive curve for transponder A. If this were the issue the +/- 20 KHz sweep would have locked the receiver. While this does not verify that the ranging mode is not functional, it does narrow the possibilities.

Operational Procedure

With the incomplete documentation at our disposal it is not unlikely that we have issues in operational procedure that preclude the coherent function of transponder A from working. On top of this problem is the legal issue that we can’t just use transponder B because we have been unable to obtain a license to transmit on the frequencies for that system from Arecibo. We have no schematics or vendor documentation on the transponder. We do have test data and we know the specifications and requirements that it had to meet. Worst of all is that as far as we can determine from the digital subcom that provides verification of communications operation, it is not functional, through either the Data Handling Unit (DHU) A or the redundant DHU B. This is not that surprising in that the DHU’s have absorbed more than five times the radiation that they were designed to handle. The only way that we have to verify a function for the communications system is to command that function and then see if it works in the intended way.

We actually were able to verify that the coherent mode is working and to validate the operational issue. Figure 2 shows our Eureka moment for coherent ranging:

Figure 2: Post Doppler Correction Spectrum Trace During Commanding for Coherency

The above graphic, produced by Phil Perillat from Arecibo, shows that our command bits for coherency did go up and did push the transponder into coherent mode, shown by the 18 kHz frequency jump by the downlink.  There was a 1.6 second lag afterwards and we send the ranging command again, which stabilized the carrier at the offset frequency.  After the end of that command, which at 256 bits/sec took 2.4 seconds, and after an additional 1.6 seconds, the downlink shifts back to its base frequency.  This was our first solid indication, on June 15th that the ranging transponder coherent mode was working.

We had tried ranging through the transponder on the previous pass we had on June 9th.  We did not think that this had been successful but after looking at some of the analyses by Phil Perillat we noticed that the ranging mode had indeed worked on that day as well.  Figure 3 shows this:

Figure 3: Ranging Tone Spectrum With Frequency Offset, Transponder A

In looking at this spectrum we were able to completely verify that the coherent ranging function with tone had worked.  The spectrum above shows this.  We verified this by looking at a document with test data on it from the spacecraft acceptance test.  This is shown in figure 4:

[2]

Figure 4: 20 KHz Ranging Tone In the Time Domain on Oscilloscope from 1978

The difference between the signal in figure 3 and figure 4 is that the first is shown in the frequency domain, and the second in the time domain.  With the correlation between the acceptance test data in figure 4and the waterfall plot in figure 3 we pondered why the DSN pass did not work.

Even though we have incomplete information, we still have a lot.  In our discussion related to the issue we recalled that somewhere in a document it was stated that the spacecraft had to be commanded into coherent mode and that if the carrier dropped for more than three seconds it would automatically drop out of that mode.  After searching this document was found and here is what it said:

Figure 5: Documentary Evidence of Procedure for Coherent Mode Operation of Transponder A

This provides verification of why we were able to get into coherent mode and the DSN was not able to also do so.

Final Verification of Procedural Issue

Friday June 20th we were going to do the propulsion system test and spin up maneuver.  However, one of our pass/fail criterion was real time telemetry and reliable commanding.  Neither of these criterion were met and thus we cancelled that activity early in the pass.  This gave the team time to focus on operationally testing transponder A’s receive system and to retest the coherent mode to determine whether or not we could command that action and record the results.  This we were able to do and figure 6 is our final evidence of Coherent mode operation:

Figure 6: Coherent Mode Operation Confirmed 6-22-14

What we were doing, as shown in the waterfall plot above, is that we were sending dummy commands to the spacecraft in order to get the command counter to increment.  This was a test of the command counter but we also send a coherent mode command.  The difference between figure 2 and figure 6 is that we changed our procedure slightly and sent commands one after another without allowing the carrier to drop, which maintained receiver lock on the spacecraft.  This can be seen in the shifting of the waterfall plot and at the end the three seconds of drift before the carrier snaps back to the baseline frequency.

Findings

  1. Coherent Mode Operation

Coherent mode is operating in a manner consistent with the original acceptance test report.

  1. DSN Failure to Go into Coherent Mode

The failure of the DSN frequency sweep to lock the spacecraft into coherent mode is due to the lack of a command to go into coherent mode before the sweep.  The sweep method must be coupled with a command for coherency for coherent mode to be entered.

Solution

The ISEE-3 team can provide to the DSN a packaged coherency command that can be broadcast to the spacecraft as the frequency sweep happens.  At this time our best estimate for frequency offset is Doppler +10-15 KHz.  This is due to aging of the transponder on the spacecraft.  After lock is achieved ranging tones should be sent without dropping the carrier.  Discussion with the DSN should be centered around how our command can be integrated with DSN operations.

[1] DSN_Operations_Plan_1985.pdf (page 5-8)

[2] 1978-02_compatibility_test_report_isee-c_flight_model (page 162)

0 0 votes
Article Rating
26 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
william
June 24, 2014 2:42 pm

this is cool!

June 24, 2014 3:20 pm

If only a command to become coherent would work if directed at the WH!

Robert of Ottawa
June 24, 2014 3:34 pm

Thanks for the update. As an engineer, I love this stuff.

Doug Proctor
June 24, 2014 5:10 pm

Wow. You guys really are rocket scientists..
The abilities of “amateurs” to work at the level of former mainstream professionals is a 20th century marvel. It is also the reason why the poltical scientists and politicians who like to use science to their own ends are in an angry pickle. Too many people these days can justifiablly comment on what they are doing.
I’m not sure what post-normal science or citizen scientists are. But I see both as reflections of the modern, technically savvy non-academic, non-professional science observer who is equal to and, at times, superior to, the elected and non-elected “expert”. I am in the oil and gas business and I (and my colleagues) regullarly recognize gross misrepresentations of our business by various self-described experrts. The previous, WWII generation considered the leaders – government or industry or university or think-tank – to be smarter, wiser and in the possession of more pertinent information than the common man. My/our experience is that often the working stiffs, who have to be pragmatists in the for-profit world, by the way, actually know more about what is going on that those white-coated talking heads.
You guys are running a spacecraft (well, will be). You are not NASA, you are not the European or Russian or Chinese space agencies. Really, this is a remarkable accomplishment, not just for you as individuals, but for the progress of a society that creates such a generally knowledgeable and competent class …. without the prior intent to create a Class as such.
Hats off.

KevinK
June 24, 2014 5:10 pm

Dennis, very impressive efforts, I also hate “drop outs”. Maybe you can sell the movie rights to a cell phone company; “Can you hear me now ???”.
Best of luck with your efforts, thanks for the update.
Cheers, Kevin.

Rud Istvan
June 24, 2014 5:47 pm

So the thing is old and creaky. No surprise there. It always was a lucky shot you were attempting. Your teams technical competence is awesome.
The inability to get permission for transponder B communication is revealing of what is wrong with today’s government regulation. good enough for NASA then, but not for you now???

Mac the Knife
June 24, 2014 6:20 pm

Dennis and Crew,
Looks like ‘root cause’ has been determined and ‘corrective action’ is in the works… Good Job!
I’ll say a prayer for Y’all and the good ship ‘ISEE3’ success!
Hey – Maybe its really ‘ICY3’…. and it just needs to ‘warm up’ to its new masters! And you guys are still learning how to ‘sweet talk’ (“…packaged coherency command..” Oooooohh Baby! Oooooohh Baby!!!) this cool critter into joining your orbit?
Best Regards,
Mac

RoHa
June 24, 2014 6:28 pm

Zombie spacecraft? We’re doomed!

Paul Penrose
June 24, 2014 7:31 pm

This is the most interesting series of articles that’s been on here for a long time. Real rocket science – I love it!

Mac the Knife
June 24, 2014 7:58 pm

David says:
June 24, 2014 at 6:42 pm
David,
Thanks for that link. Geez – I love this!
Wingo and Crew are real close to clearing this major hurdle!
Filipinos have a saying: “Softly… Softly…. Catch the monkey.”
I think it applies here. Softly…. softly…
Mac

stan stendera
June 24, 2014 8:54 pm

I am so happy I contributed some money to this venture. I suggest everyone join me.

Greg
June 24, 2014 11:11 pm

Rud Istvan says:
June 24, 2014 at 5:47 pm
So the thing is old and creaky. No surprise there. It always was a lucky shot you were attempting. Your teams technical competence is awesome.
The inability to get permission for transponder B communication is revealing of what is wrong with today’s government regulation. good enough for NASA then, but not for you now???
===
Just at a guess, there may be a very good reason for that , not just bureaucratic perversity.
Most bands in this part of the spectrum are fairly fully exploited. That’s why licencing exists, to stop everyone trying to use the same band and turning it into a radio frequency shouting match.
Now imagine that there is some military or commercial service at or near to the drifting frequency of transponder B. Maybe they don’t want to shut down operations and give up thier communications link so that this excellent but basically “fun” project can use those frequencies.
This slot may have been available to NASA 30 years ago, but the air waves are busy these days. If NASA have moved to using different bands or have allocated transponder B’s freq to something else, well you sadly you can’t just come in and take over the airwaves.
It is indeed unfortunate for the project, but I imagine they have fully explored the possibilities.

Steve C
June 25, 2014 12:40 am

Fascinating, top-class stuff. Thanks again, Dennis.

Barry Sheridan
June 25, 2014 1:00 am

Magic. I really hope the repackaged signal via the DSN will work. Keep the updates coming.
Thank you Antony for allowing this article to be printed on your site.

Stephen Richards
June 25, 2014 1:39 am

Denis, this is just so brilliant and exciting. I bet there are a good number of engineers here that would love to be working with you. Me for one.

Bloke down the pub
June 25, 2014 4:06 am

Steely eyed rocket men.

beng
June 25, 2014 5:55 am

After reading this, rocket-science really is rocket-science.

June 25, 2014 8:41 am

Thanks everyone! We are having fun and it could not be done without a lot of smart people helping.
NASA’s Deep Space Network does awe inspiring work, for which we are grateful. I do look forward to the day when this is commercial!

DesertYote
June 25, 2014 9:32 am

Figure 2 looks like its from a HP 141T. Sure brings back memories.

June 25, 2014 3:37 pm

As for licensing a ground station to test transponder B from South America, be aware that regulations vary by fiefdom, and administrative common sense varies as well.
Two examples:
– Boeing gave up trying to use wireless connectivity on the 787 airplane for passenger seat control of things like reading lights, because of the zoo of rules and bureaucracies around the world.
– Last I heard, HF radio cannot be used in the lower 48 states of the US, but is allowed in Canada and AK. So to test HF Datalink in 1994 we had to go more than 200 miles off of the east coast of the US.
(What aviation people call “HF” is about 2-30MHz, for reference I note the commercial AM radio station band in North America goes from just below that down to about 0.5MHz.)

David Rodgers
June 25, 2014 4:09 pm

Is this a wind up?

David Rodgers
June 25, 2014 4:17 pm

Guess I’m dumber than I thought lol

June 25, 2014 7:50 pm

This should be an example to those who produce “climate models”.
We try something.
If it works, we’ve got something.
If it doesn’t, we got it wrong.
Try to figure out what’s wrong and try again.
Repeat the process as often as needed.
No room or need to pretend what didn’t work actually did, not if you’re looking for something that reflects reality.

June 26, 2014 6:16 am

“Controlling the ISEE-3 space shuttle via SDR Austin Epps John Malsbury Balint Seeber @spenchdotnet #dorkbotsf”
From Here –
http://instagram.com/p/psWnN6vXYD/

Keith Sketchley
June 27, 2014 4:05 pm

As for licensing a ground station to test transponder B from South America, be aware that regulations vary by fiefdom, and administrative common sense varies as well. Two examples:
– Boeing gave up trying to use wireless connectivity on the 787 airplane for passenger seat control of things like reading lights, because of the zoo of rules and bureaucracies around the world.
– Last I heard, HF radio cannot be used in the lower 48 states of the US, but is allowed in Canada and AK. So to test HF Datalink in 1994 we had to go more than 200 miles off of the east coast of the US. (What aviation people call “HF” is about 2-30MHz, for reference I note the commercial AM radio station band in North America is just below that down to about 0.5MHz.