Creating a Solar Stormwatch Catalogue from YOUR clicks.

Hello, I’m Luke and I work with Chris Scott (formerly Davis!) at the University of Reading as a postdoctoral research assistant. Recently I’ve been doing some work with the large amount of Trace-it data that has been generated over the last few years. We thought it was a good time to update everyone on the work we have been doing.

The short story is that we have turned the roughly 40000 time-elongation (t-e) profiles generated by Trace-it into a catalogue of CMEs seen by Heliospheric Imager (HI) instruments aboard STEREO-A (STA) and STEREO-B (STB). The Solar Stormwatch catalogue provides profiles of the CME fronts in the remarkable field-of-view (FOV) of the HI instruments. The HI FOV covers regions of the inner heliosphere not accessible to the coronagraph instruments that are more commonly used to build CME catalogues. Therefore the Solar Stormwatch catalogue should allow us to study the structure and dynamics of CME fronts in a way not previously possible using other presently available CME catalogues. This has been written up into a paper which is currently under review for publication in the journal Space Weather. So, first things first, thank you to everyone that has contributed to Solar Stormwatch. I think that we have produced a useful catalogue of CMEs, which will hopefully be of use to the wider space weather community – this wouldn’t have been possible without all of the contributors to Solar Stormwatch.

Let’s begin with a quick review of the raw data produced by Trace-it. Trace-it analysed J-maps made from HI1 and HI2 images, for both STA and STB, over 18 distinct position angles separated by 5 degrees, except for one position angle, which was centered on the ecliptic plane.

The J-maps covered a time span of January-2007 to February-2010. As of a few months ago, the Trace-it results consisted of database of 38171 t-e profiles, 22007 from STA and 16164 from STB, generated by 4599 Solar Stormwatch users.

If elongation angles and position angles are unfamiliar to anyone reading this, Figure 1 shows an image from HI1A, over which contours of constant position angle (in blue) and constant elongation angle (in red) have been overlaid, to make these coordinates clear.

Figure 1

Figure 1. An example of a differenced image from the HI1A camera, overlaid with contours of

constant PA (in blue) and constant elongation angle (in red). The elongation and PA contours

are in 5◦ increments. A CME is visible to the right of the image, between 5◦ and 10◦ elongation

and with maximum extent in PA between 65◦ and 135

To separate the t-e profiles into groups which represent individual CMEs we looked for periods when many t-e profiles were clustered in a short space of time. To do this, we counted how many t-e profiles began in a 7-hour window, for every hour covering the data set, and whenever the count of profiles was higher than a threshold of 22 counts we defined that as an event. This happens whenever lots of us have seen features over multiple position angles but at similar times. Figure 2 shows an example of this. Panel A) shows a STA J-map, at PA=110 degrees, overlaid with the t-e profiles generated by the Solar Stormwatchers as red dots, whilst the blue dots mark the earliest occurring point in each profile. In this instance, this position angle was tracked 11 times by 8 different Solar Stormwatchers. Panel B) shows the count of these profiles as a function of time, using the 7 hour sliding window. Note that this count is done over every position angle, whereas the J-map shows the t-e profiles at one position angle only. There is a well defined maximum in the count, which we use to define the onset of this event and identify the t-e profiles that describe it. The thresholds we picked are arbitrary but sensible, we could have used different ones and had similar results – for anyone interested in how we picked these numbers, we go into a bit more detail in the paper.

Figure 2

This gives us groups of t-e profiles for each CME – 113 from STA and 80 from STB. However, we had to do a bit of quality control, as it is not good enough just to have to t-e profiles that start at similar times – they could come from coincident but unrelated solar transients that are widely separated in position angle. So we used another set of rules to exclude any t-e profiles which look like they may belong to a different solar transient. This process is detailed more in the paper, but the result of it is that we have to discard 6 events that we are too unsure about, 3 each from STA and STB. This leaves us with 110 events from STA and 77 from STB.

At this point, we have defined sets of t-e profiles which we think robustly identify CMEs seen by the Solar Stormwatchers. The next step is to average these profiles along each position angle the event was observed. An example of this averaging is shown in Figure 3. The black dots show the t-e profiles generated by the Solar Stormwatchers for one event and along one PA, which includes 13 t-e profiles, generated by 9 different Solar Stormwatchers. The red-dots and red-lines show the average profile and the uncertainty in the average profile.

Figure 3

Figure 3. An example of an average t-e profile, for CME number 59 from STA, tracked along a PA of 110 degrees. The black dots show the individual t-e profiles and the red dots mark the consensus profile while the two red lines indicate the uncertainty in the mean time coordinates.

Now we can turn this around and overlay the average t-e profiles for each position angle back onto the original differenced images that made the J-maps they were tracked in. Figure 4 shows a movie of the evolution of an event through the HI1A field-of-view. The yellow lines mark the maximum extent of PAs that the J-maps used by Stormwatch cover, whilst the regions bounded in red mark the locations where the consensus profiles (like figure 3) suggest the CME front should be. The width of the bounded region arises from the uncertainty in the consensus profile at that position angle, so that wider regions mean we are less sure where the CME front is.

Figure 4

Figure 4. This movie shows a sequence of HI1A differenced images in which a CME can be seen to enter and propagate across the HI1A field-of-view. The yellow lines mark the outer limits of the position angles of the J-maps analysed by Trace-it. The red lines mark the location of the CME front, and are calculated from the averaged t-e profiles (see Figure 3) along each position angle the event was tracked.

We are in the process of making this CME catalogue available in an easily usable form so that the rest of the space weather community can get involved and hopefully start using it for some research. In November we will be taking this work to the European Space Weather Week conference in Belgium, to present this work to other researchers. In the meantime, we have some plans for some things we would like to do with the Solar Stormwatch catalogue, which we will update you with when there is more to say.


Solar wind causes sparks to fly

You may have noticed a bit of a splash in the press last Thursday, when I and my co-authors at the University of Reading had a study published showing that the solar wind appears to affect lightning rates over Europe. If you are interested, you can download the paper here;

Environmental Research Letters, Solar wind modulation of lightning;

or, if you don’t fancy wading through a scientific paper, you can see me trying to explain it without waving my arms around too much in a short video, here;

And, if neither of those approaches appeals, you can read on for a short summary of the work (I’m assuming you’re interested otherwise you wouldn’t have chosen to read this blog, right?).

It’s long been thought that cosmic rays (very energetic particles generated throughout the galaxy, accelerated on shock-fronts created by supernova explosions) could be responsible for causing electrically charged clouds to discharge to ground in the form of lightning. As the cosmic rays fall through the atmosphere, the argument goes, they ionise the air, free electrons get accelerated further by the electric field present in the cloud and a runaway breakdown of air results, ending in a lightning flash.

What does the Sun have to do with this? Well, the Sun is an active star with an eleven year solar cycle. The solar wind drags the solar magnetic field into space where it shields the Earth from some of the cosmic rays. When the Sun is active, the solar magnetic field around Earth is stronger and we see fewer cosmic rays reaching the ground. There is also evidence that there is less lighting at these times. So that’s the long-term view, but what happens over shorter timescales?

While you can use solar storms, or Coronal Mass Ejections (CMEs) as they are known, with their enhanced magnetic fields, to look for short-term enhancements of the interplanetary magnetic field, relatively few events travel Earthwards to make a statistical survey conclusive. Instead, we looked at fast solar wind streams. While these produce a smaller depletion in cosmic ray flux (around 1%) compared with CMEs (around 10%) they co-rotate with the Sun and so wash past Earth at regular intervals. We were expecting therefore to see a reduction in lighting but instead we saw that the lightning rates went up (there is a moral here; never try to anticipate the result of an experiment!). The answer, we think, lies with ‘solar energetic particles’ that are accelerated ahead of the solar wind stream, like surfers on a huge wave. While these do not reach the energies of cosmic rays, it is likely that they nevertheless are able to penetrate the Earth’s atmosphere to the height of thunder clouds where they presumably do a similar job to that thought to be done by cosmic rays in initiating lightning.

There’s loads more work to do in order to fully understand where these particles end up and how they influence lightning but if we can understand this effect, there is the tantalising possibility that we could use our observations of solar wind streams from space to forecast the severity of lightning events several weeks in advance. With around 24,000 lightning associated deaths occurring worldwide every year, anything we can do to predict the severity of lightning in advance has to be useful, doesn’t it?

While all this has been going on, we have been analysing the Stormwatch data too, and it’s been very informative. More on these results soon.

Thanks again for your enthusiasm and time, keep clicking! (don’t forget Trace It!)


Calling All Stormwatchers : BBC 2 documentary needs your help!!

‘Here Comes The Sun’ is special one-off documentary for BBC 2 investigating the nature of the Sun during this period of heightened solar activity – the solar maximum. The presenters, Kate Humble and Helen Czerski, along with a team of experts will explore how the Sun works, how its secrets could power our future and what the current behaviour of the Sun means for us. One strand of the programme will focus on Space Weather Prediction: examining the fundamental mechanisms that cause solar storms, the impact they may have on the Earth’s infrastructure, and how scientists are working to predict this type of solar behaviour, which is why they are looking for your help…!

The team are coming to film at the Rutherford Appleton Laboratory (RAL), in Oxford, on the 28th of February and 1st March and are looking to recruit a group of around 20 UK-based Stormwatchers to come to RAL and be part of this programme! It is a wonderful opportunity to bring the Solar Stormwatch project into the public eye and illustrate the important job of every single Stormwatcher. It will be great fun and a good excuse to get together with other fellow Stormwatchers!

This is a great opportunity and we are hoping this exposure on a national level will encourage more people to get involved!!

If you are interested please do not hesitate to contact Fay Finlay for more information. or 0141 422 6991.

See your data analysis like never before!

Attention Stormwatchers!

Remember all that data analysis you’ve been doing for us? All those storms you’ve tracked in both archive and real time data have now been used to create an animation of what the Sun has been up to over the first three years of the STEREO mission. Over the summer, a student of mine, Amy Skelt, wrote a program to enable us to view your data analysis in a unique way. By taking all your CME tracking information and combining it with my analysis of smaller solar wind features, we can now create animations showing the activity of the solar wind throughout the first three years of the STEREO mission. Just in time for Christmas I’ve used Amy’s software to create a movie of the entire Stormwatch analysis so far. You can view the movie here;

It’s incredible! You can now see the constant stream of solar wind material as it erupts into space and even the spirals created as the various sources of solar wind rotate with the Sun. And when a solar storm erupts, you can see which planets are in the firing line!

We’ve had to make some assumptions about the rate at which the solar storms expand and so any differences between this movie and the real world will help us understand how realistic our assumptions are. Amy made the software very flexible so that you can view the solar system from a fixed point (as in the attached movie), from above or even from a moving object. You can even go for a ride on comet Encke and see how it fares as it rides the solar wind!

As usual, many, many thanks for your time and efforts so far. In the New Year my group and I at the University of Reading will be using your data analysis to investigate what we have learned so far about using STEREO HI data to make real-time forecasts. Working with the UK Met Office, we will ultimately be applying what we learn to improving the operational space weather forecasting model that they will be running. In the current climate, there is much talk of ‘impact’. I can confidently say that you are helping us with our impact. Both metaphorically and literally!

I hope that those of you that are about to celebrate Christmas have a wonderful holiday.

See you in the New Year for more Solar Storming!


New York to Rome in 2 seconds

This post is part of Citizen Science September at the Zooniverse.

Some of the fastest solar storms (coronal mass ejections or CMEs) that Solar Stormwatch volunteers measure have speeds of around 450 km/s. That’s a billion tonnes of plasma being blasted off the Sun at a million miles per hour. If it’s pointing at Earth the CME will arrive 3 days later. Not all CMEs are Earth bound though and any of the other planets in the solar system might be in the firing line.

On July 23 2012 the fastest CME recorded by STEREO blasted away from the sun at a staggering 3,400 km/s (7.6 million mph.) That’s New York to Rome in 2 seconds. This rates as Extremely Rare on NASA’s CME speed scale and we can expect one this fast to occur only once every 10 years. While CMEs might be big you won’t be able to see one if you look up. Solar spacecraft are fitted with very sensitive cameras to capture the tenuous outflow of particles. Here’s how STEREO Ahead’s Cor2 (coronograph) recorded the event.

click for video – published on YouTube 23 Jul 2012 by ve3en1

The source of the CME was sunspot Active Region 1520. An active region (AR) is an area with an especially strong magnetic field and is often associated with sunspots and solar activity. AR 1520 was very photogenic when it was on the Earth-side of the Sun earlier in July.

The Sun on 15 July 2012. AR 1520 is the feature at 5 o’clock.
(photo ©Julia Wilkinson)

Luckily AR 1520 unleashed the super-fast CME after it had rotated to the far-side and despite being an 80 degree wide CME all of the inner planets, including Earth, escaped its blast. A YouTube summary can be found here.

So what’s the big fuss about Earth-bound CMEs fast or otherwise? Well most of the time you wouldn’t even know one had hit us but just occasionally they can cause problems. If the direction of the CME’s magnetic field is southward, (opposite to the Earth’s magnetic field), the two magnetic fields interact producing a geomagnetic storm. Such storms enhance the activity in the Earth’s radiation belts which presents a radiation hazard to low-Earth orbit satellites and the ISS astronauts as they pass through the South Atlantic Anomaly. Geomagnetic storms also generate electric currents which can cause electrical surges through power lines and damage power grids. In 1989 a geomagnetic storm blacked-out most of Quebec as circuit breakers tripped on Quebec’s Hydro-electric power grid. The super-fast storm of 23 July might have missed Earth but the STEREO Ahead spacecraft took a battering from a very large proton storm associated with the CME. Proton storms can damage a spacecraft’s electronics.

This is how Solar Stormwatchers saw the CME in the STEREO Beacon mode (near real-time) feed. The effect on STEREO A is clear but it did survive to carry on its mission.

It’s pretty useful then to be able to predict the speed and direction of CMEs but there is still much to learn about what triggers them. This is where Solar Stormwatch comes in. Data from the project are being used to track and measure solar storms to learn more about how they begin and how they evolve. If you want to contribute to solar science, it’s easy, just click and join in!

Sunspots ahoy!

Over on our sister site, the Old Weather team are digitising ship’s logs from first world war Royal Navy vessels in order to capture the valuable meteorological information that the fleet collected. These records also contain information that can be surprisingly useful for a whole host of other reasons.

One such example was identified by HebesDad and brought to our attention over at Solar Stormwatch by Caro.


In the log of H.M.S. Hilary on Saturday February 10th 1917, the observer wrote;
“at noon aTS observed spots near centre of sun like this [diagram] it appeared to be two with a narrow passage between them, I make this note, for although I have seen sun spots before, I have never seen such large ones”

This is an unusual sighting and, if observed with the naked eye, (something you should never try to do as you risk damaging your eyes) the spots must have been very large indeed.

I consulted my colleagues at the UK Solar System Data Centre at the Rutherford Appleton Laboratory and they were able to confirm that this observation was indeed correct. Photographs taken from the Dehra Dun observatory show quite clearly that on that day, and on the days leading up to the 10th, a large sunspot group was indeed observed almost exactly as the observer on H.M.S. Hilary described.


My colleague Dr David Willis, an expert on historical sunspot observations, informs me that the accepted limit above which sunspots can be seen with the naked eye is taken to be when a sunspot exceeds an area of 500 millionths of the Sun’s visible hemisphere. On my screen, the Sun has a diameter of 220mm while the spots are around 10 mm. Using the formula for the area of a circle (3.14 x [radius squared]), I estimate that the area of each of these spots is around 6195 millionths of the total area of the Sun. These sunspots are clearly above the threshold to be viewed with the naked eye!

David also provided information from the Greenwich observatory publication “Sunspots and Geomagnetic-Storm Data Derived From Greenwich Observations 1874-1954” (HM Stationary Office, 1955). In a list of the 55 greatest sunspots in the interval 1874-1954 this sunspot group (number 7977) recorded the 8th largest maximum area.

The reason we are so interested in observation of sunspots is that these regions are closely linked with solar mass ejections – vast eruptions of material from the Sun’s atmosphere. If one of these eruptions comes towards Earth, it can generate spectacular auroral displays, disturb the Earth’s magnetic field – leading to anomalous compass bearings, deplete the ionosphere (the electrically charged layers in the Earth’s upper atmosphere) – leading to disruption of radio communications, and cause surges of electricity along any long cables such as power and communications networks.

Matthew Wild at the UKSSDC generated a plot of the aa index for February 1917. The aa index is a measure of the variability in Earth’s magnetic field. This index compares the measurements made by two magnetic observatories on opposite sides of the World in the UK and Australia. Measurements started in 1868. Originally the UK measurements were made at Kew in London but they are now made at Hartland in the UK while the Australian measurements are made in Canberra. Large values of aa represent a disturbed magnetic field.


Looking at the aa index for February 1917 we can see that the values around the 10th were not particularly high but that on the 15th February, aa levels increased from around 20 to over 150! It is likely that this is a result of the Earth being hit by a solar storm launched from this enormous sunspot group. It is difficult to say how big or fast this storm was since we have no record of when it was launched but these storms travel at speeds between 400 and 2000 km per second, reaching Earth in 1-3 days. It would be interesting to see if any ships reported erratic compass bearings during this time as the cloud of solar material buffeted the Earth’s magnetic field in space. Variable magnetic fields set up electrical currents that heat the Earth’s upper atmosphere, depleting the ionisation there. As a consequence, wireless operators, using the ionosphere to reflect their wireless signals around the world, would also have noticed a weakened signal strengths in the days that followed the arrival of the solar storm at Earth. The Greenwich publication noted a ‘small’ geomagnetic storm on the 15th and rather curiously relates it to the much smaller sunspot group number 7990 though there seems to be no conclusive proof of this.

So, there you have it. A rare observation of a large sunspot group visible with the naked eye recorded with precision by a diligent observer who presumably knew the potential impact that such an observation could have to both the ship’s navigation and radio communication (if it was equipped with such modern technology).


If you are interested in finding out more about the Sun and solar storms feel free to join us over at solar stormwatch where you can even help predict the arrival of solar storms at Earth. Meanwhile please keep us posted on any sightings of aurora, sunspots, erratic compass needles or poor wireless reception. They are all clues about the activity of the Sun in a time long before spaceflight.

My thanks to Matthew, Sarah and David at the Rutherford Appleton Laboratory for helping to piece this story together. If you would like more information about the UK Solar System Data Centre and the records it contains, visit Solar images are under Crown Copyright.


Storms, data-analysis and fruitcake

Thanks to everyone who helped track the most recent bunch of solar storms. Several of them came by Earth but their magnetic fields were mostly the same polarity as Earth’s and when this happens, just like in the school experiment, the two magnetic fields just bounce off each other. I was amazed that we could track an Earth-directed storm at all from where the spacecraft are now, particularly with the gaps now appearing in the data coverage in incoming-traceit. You are really showing us what the limits are on our mission! Keep tracking the real-time events though. Even if we can’t track them to Earth very easily we can still test out our tracking skills by watching them go by other planets. We have spacecraft around all the inner planets at the moment:- Messenger at Mercury, Venus Express at Venus, the sister-craft to Mars Express which (amongst others) is orbiting Mars.

Thanks to your efforts we now have two publications that used your clicks to help track storms and dust. The next task we’re aiming to do is to analyse the huge amount of data you have processed in the trace-it archive. With this dataset we hope to track each storm in detail as it expands out into the solar wind. Experts like Solar Stormwatch’s own Neel Savani can turn your clicks into a comprehensive survey of all the storms we have seen. Not only will this analysis tell us how fast and how big each storm was, it should also reveal how much it was distorted by the solar wind into which it was expanding – vital information in tracking their progress towards our planet.

So, please keep clicking on trace-it archive data too. The real time stuff may be the most exciting but the careful consideration of the archived data will be an important part of the Solar Stormwatch legacy too.

And the fruitcake? Ah… I appear to have eaten the last photograph of one that Jules posted to the forum… sorry.

Thanks once again for all your enthusiasm, time and efforts. It really is a privilege working with you all.