Is this the real launch, or is this just fantasy?
A tremendous explosion bursts out from the Sun. Just a few minutes later and the spacecraft is bathed in radiation, blinding the star trackers it relies on to know what direction it's pointing in. The spacecraft goes into Safe Mode, like a turtle retreating into its shell. Most units are powered off, the onboard computer is rebooted and spacecraft points its solar arrays to the Sun. The control teams are left wondering what on Earth – or in space – has gone wrong.
Fortunately, it was all a simulation, one of many that teams at ESA’s mission control go through to prepare them for the countless hazards of space.
“My most stressful moments doing this job, thankfully, have all taken place within simulations on the ground”, explains Tomas Ormston, Spacecraft Operations Engineer.
“By the time you’ve completed sometimes 15 simulations in preparation for the real launch, you feel pretty confident you can handle anything that could happen to that thing in space”.
The simulation campaign
Many years go into planning a mission, not to mention large sums of money. The time spent flying the mission is when everything needs to go as smoothly as possible – there’s no time to practice or take unnecessary risks.
So, before every launch and critical operation, teams at ESA’s ESOC mission control centre go through an intensive simulation campaign made up of dozens of real-time ‘nominal’ and ‘contingency’ scenarios.
Nominal simulations are practice-runs where everything goes to plan. Contingency simulations are designed to prepare the teams for the potential hiccups or even disasters that could take place, from sick team members to potential collisions with space debris.
“Through simulations we are training and learning to be prepared for any crisis. 95% of what we exercise will never be applied, but in rare crisis scenarios we will 100% appreciate an educated and trained team,” explains Head of Mission Operations Simon Plum.
“The value of simulations is not obvious in daily business, but becomes undeniable when problems occur”.
Disasters abound
Last night’s chicken curry
It’s launch day! But someone’s missing. One of the team leaders hasn’t come in. It turns out a dodgy curry from last night has left her feeling unwell, and the team needs to figure out how they will launch this new mission without her expertise. Either someone needs to step in to perform her tasks or the team leader for the other shift needs to be woken up and called in.
Don’t forget to save
One person hasn’t saved the critical commands they’ve been working on in the last 15 minutes and these have to be sent to the spacecraft soon. All of a sudden, their computer crashes and they have to start again, now under extreme time pressure, to be ready to command the spacecraft on time. Of course, the crash was timed perfectly by the Simulation Officers with control of all the computers in the control room and all other involved servers and infrastructure, but as we all know, they happen all too often in reality as well.
Where’s it broken?
Suddenly, the flight control team notices that data arriving at the Mission Control System used to observe the behaviour of the satellite does not seem to make any sense – the values would indicate a major problem with the spacecraft.
Investigations begin. Different elements of ‘operations infrastructure’ are tested such as the Mission Control system, ground stations and their internal signal processing. It is confirmed that all ground systems are behaving as they should, so the issue must be ‘upstream’ at the spacecraft.
Now that the issue seems to be in the spacecraft, the flight control team looks into the many previously validated flight operations procedures and finds one could help with the present case – one of the units onboard the spacecraft that generates the signal is misbehaving. By executing the fitting procedure the team switches this specific unit to its backup and the data coming down is correct again.
The people who dream up disaster
In an underground room beneath the Main Control Room, the simulation officers sit intensely watching what the flight control team is doing throughout every scenario - they observe their individual monitors, listen in on their calls, and access cameras while controlling the ‘mission simulator’.
We spoke to two of four simulation officers at ESOC whose job it is to design and conduct simulations, from “boring” nominal sims to those in which our beloved space missions could go terribly wrong (but in interesting, informative ways).
“Our job is to unite mission control teams in front of a common enemy – and that has to be us,” explains Petr Shlyaev.
“I have never played Dungeons and Dragons but I am familiar with the concept. Sometimes, it feels like we are the Dungeon Master organising a several-day long, massive game. Except for us, the outcomes have very real consequences,” adds Filipe Metelo.
Stay tuned for the full interview with Petr and Filipe, the people who dream up disaster.
Watch The burn
Access the video
