Manual pre-processing

By Dominique Dhoosche

Introduction

This tutorial explains how to pre-process manually an astro image shot with a digital camera (DSLR), but the processing of images from astronomical cameras is almost identical. Of course, Siril features scripts , which carry out this process automatically. But in some cases , you will need to perform manual processing:

  • You want to understand what scripts do: the best way is indeed to do a manual pre-processing!
  • You want to fine-tune the preprocessing: the scripts indeed take default values, which can be optimized
  • Your images contain gradients, which are difficult to remove during processing. It is very interesting to remove them on the raw images: the gradient is simpler and easier to remove.
  • You have no flats: of course, it is formally not recommended, but the shooting sessions do not always go as we would like!
  • And many other reasons … which belong to you :-)

Needed files

For this tutorial, we will assume that you have images of the object, biases, flats and darks.

File storage

It is best to store your files in different subfolders. We will follow the pattern used in the scripts: lights for images, biases, darks and flats for other images.

Principle of pre-processing

Image preprocessing is made by the following formula: $$ \frac{\text{image}-{\text{dark}}-\text{bias}}{\text{flat}} $$ Basically, the biases are contained in all images we shot: the lights ones of course, but also in darks and flats. It will therefore be necessary to remove the biases of the darks and the flats. But let’s analyze this formula. $$ \frac{\text{image}-{\text{dark}}-\text{bias}}{\text{flat}} = \frac{\text{image}-({\text{dark}}+\text{bias})}{\text{flat}} $$ We note that one only need to subtract the UNPROCESSED darks (which therefore also contain the biases) to remove the dark AND the biases in one pass: saving work and time!

To preprocess the images, the proposed operating mode is as follows:

  1. Set the working directory.
  2. Convert the files.
  3. Create the master bias.
  4. Create the master flat, by subtracting the master bias on each frame.
  5. Create the master dark, without subtracting the master bias.
  6. Calibrate the images, with flat and dark masters.
  7. Process sky background.
  8. Register the images.
  9. Stack the images.

Set the working directory

Set the working directory.

Set the working directory.

  1. Click on the Home icon
  2. Find the folder where the images are located, then create a working directory (the scripts will create a process folder)
  3. Double click on the process folder to enter it
  4. Validate

All images will now be saved in this folder

Converting files

Convertir ses fichiers

Convertir ses fichiers

Siril works with files in the FITS (Flexible Image Transport System) format. You must therefore convert your raw files. For people with FITS images, obtained from dedicated cameras, conversion is not necessary but renaming is. It is also possible to create a sequence without copying the files by creating symbolic links. An explanation for doing this on Windows is given here .

  1. Click on the Conversion tab
  2. Click on +Add
  3. Find the folder where the images to convert are located (by default, we are in the working directory)
  4. Select the images to be converted
  5. Add
  6. Give a generic name to the images to convert (lights, flats, …). Siril adds a number as a suffix to this name to differentiate the images, and this allows the creation of the sequences.
  7. Click on Convert

Siril converts images to the .fit format into the working directory. You must then apply the same procedure with the lights, the flats, the biases and the darks, each time giving them an evocative generic name.

Sequences

A few words about sequences that are very important in Siril. A sequence is a sequence of consistent images: lights, darks, biases, etc. These sequences are listed in small files with the extension .seq. So when you convert biases images giving them as sequence name: bias, the converted images have the names bias_00001,bias_00002, bias_00003, etc. The name of the corresponding sequence is bias and is contained in the bias.seq file which we recommend never to edit by hand.

It is quite possible to have several sequences in the same folder. However, keep in mind that you should ALWAYS choose the sequence of work before proceeding with any processing. Most often, by default, Siril loads the last computed sequence, but please keep that in mind!

In the Sequences tab, the Search sequence button analyzes your working directory and searches for sequences:

Search sequences

Search sequences

You can force Siril to recompute all sequences in the working folder, by checking the Force .seq recomputation box. Be careful, however, that this erases all the data (normalization, alignment, …) which have been calculated before.

Biases

Processing biases

Processing biases

  1. Click on the Sequences tab, then choose the sequence of biases.
  2. Click on the Stacking tab.
  3. Set the parameters as above.
  4. Click on Start stacking.

The computation starts, Siril displays the console on which we can follow the operations carried out.

Once the calculation is complete, check that the rejection rate is within a range of 0.1 to 0.5%. These values are completely arbitrary and allow the beginner to get an idea. It is not useful to want at all costs to obtain a value lower than 0.5%: a value of 1% can be just as good. Just keep in mind that you need to check these rejection rates. If this rate has an outlier, resume stacking by changing the values of the Sigmas low / high.

Checking rejection rates

Checking rejection rates

ATTENTION: when you increase a sigma value, you lower the rejection rate!

At the end of the calculation, Siril displays the stacked image. It is saved here under the name bias_stacked.fit

Flats

Preprocessing flats

Preprocessing flats

Preprocessing flats

  1. Click on the Sequences tab, choose the sequence of the flats.
  2. Click on the Pre-processing tab.
  3. Set the parameters as above.
  4. Click on Preprocess.

Siril subtracts the biases from the flats and creates new processed flats files by appending the prefix pp_ to the filenames. You can of course change the prefix (pp_ = pre-process in English).

NOTE: we recommend to check the button “Equalize CFA” for color images in order to preserve the original balance of the channels.

Building master-flat

Stacking flats

Stacking flats

  1. Click on the Sequences tab, then choose the sequence of the preprocessed flats (normally, it is already loaded at this stage).
  2. Click on the Stacking tab.
  3. Set the parameters as above.
  4. Click on Start stacking.

Siril stacks the flats, and creates the file whose name was defined. As with the biases, check that the rejection rate is within a range of 0.1 to 0.5%. If the values are outliers, resume stacking by changing the values of the Sigmas low / high. When you increase a sigma factor, you lower the rejection rate!

Darks

Stacking darks

Stacking darks

As we saw above, we will not remove the biases from the darks. Let’s proceed directly to stacking.

  1. Click on the Sequences tab, then choose the sequence of darks.
  2. Click on the Stacking tab.
  3. Set the parameters as above.
  4. Click on Start stacking.

Siril stacks the darks, and creates the file whose name was defined. (Here dark_stacked.fit, but you can give the name you want).

As with the biases, check that the rejection rate is within a range of 0.1 to 0.5%. If the values are outliers, resume stacking by changing the values of the Sigmas low / high. When you increase a sigma factor, you lower the rejection rate!

Lights

Preprocessing lights

Preprocessing lights

Preprocessing lights

  1. Click on the Sequences tab, then choose the sequence of light images.
  2. Click on the Pre-processing tab.
  3. Set the parameters as above.
  4. Click on Estimate to know the number of pixels corrected by the cosmetic correction. If any of these numbers turn red, change the sigma value. If the value remains red despite the sigma value changes, then you can uncheck the corresponding cosmetic correction (hot or cold pixels).
  5. Click on Debayer before saving to save the images in RGB.
  6. Click on Preprocess.

Siril processes images with darks (and the biases they contain) and flats and creates as many processed image files, with the prefix pp_ (which you can change).

Background extraction

The method described below became obsolete when Siril 1.2.0 introduced a new algorithm much more efficient to remove the gradient. This method is well described in the gradient tutorial.

If your images show a gradient, generally due to light pollution or the Moon, it is very likely that this gradient will change during your nigh session and therefore its appearance will change on your images, especially if you shot for several hours. It is therefore much preferable to remove this gradient directly from the raw images. In fact, stacking would make this gradient much more complex to eliminate later.

Removing gradient

Removing gradient

  1. Menu Image processing, Background extraction....
  2. A window opens. We are going to put samples on the image. Siril computes the background level of the sky under these points, and deduces a polynomial interpolation of the gradient: a synthetic sky. The first thing to adjust is the order of the interpolation. Here, on a processing of a sequence of raw images, an order of 1 is generally used because, as we have said, the shape of the gradient is simpler on the raw images. We then give the number of points to be located on a horizontal line. Here, I reduced the suggested value to 10. I also limited the tolerance, so that Siril does not put too much points near the nebulae. We then click on Generate. Siril then automatically add the samples (the green squares): we can delete one (right click on a sample) or add one (left click on the image location we want to add it). Regarding light pollution gradient, the type of correction to choose is Subtractive. Then, we do not forget to check Apply to sequence and we also can choose an output prefix for the new sequence of images that is created.
  3. Click on Apply. Siril then recalculates a new image sequence free of the gradient, starting with prefix bkg_, if you have left the default value. It may be necessary to redo a background extraction during processing, but the latter will then be much easier to perform.

Images registration

Preprocessing images

Preprocessing images

  1. Click on the Sequences tab, then choose the sequence of preprocessed light images.
  2. Click on the Registration tab.
  3. Set the parameters as above.
  4. Click on Go register.

Siril aligns the images and creates as many registered image files, with the prefix r_ (which you can change).

Note: Since Siril 1.2.0, the registration process can be split in two steps: first you choose the registration method (as descibed above), then you Apply the registration. It allows to compute the registration, and then to apply and modifiy the images.

2-pass registration, then Apply

2-pass registration, then Apply

Analysing images

It’s time to check the quality of your images, and eliminate the bad ones.

Checking images

Checking images

  1. As usual, we check on witch sequence we are working (if you have just aligned, no need to check, Siril will have chosen the registered sequence)
  2. In the Plot tab, Siril shows the image FWHMs of the loaded sequence. To remove the worst images from the sequence, we right click on an image with high FWHM, and we validate Exclude image XX.
  3. By clicking on this icon, the list of images in the sequence is displayed, and we see that the deleted images are unchecked. You can work on the graph or in this window.
    Checking images

    Checking images

    Once done, you can close this window.

Since Siril 1.1.0, the sorting tool is offering much more features. It is now possible to change the X and Y axes thanks to the scrolling menu shown.

Changing the X/Y axes

Changing the X/Y axes

The sorting parameter (right colomn) in the frame list is changes accordingly, and you still can sort the images as described above.

Stacking lights

Stacking lights

Stacking lights

  1. Click on the Sequences tab. And choose the sequence of registered images.
  2. Click on the Stacking tab.
  3. Set the parameters as above. We see that Siril has removed the images eliminated during the previous stage: we are only processing part of the sequence.
  4. Click on Start stacking.
  5. It is also possible, before stacking, to add new image selection criteria like fwhm or roundness.

Siril stacks the selected images, and creates the file whose name we have defined. As before, check that the rejection rate is within a range of 0.1 to 0.5%. If the values are outliers, resume stacking by changing the values of the Sigmas low / high. When you increase a sigma factor, you lower the rejection rate!

Once again, since Siril 1.2.0, this last tab also changed a bit as shown:

New Stacking lights tab

New Stacking lights tab

The option RGB Equalization allows you to get a well balanced image at the end of the stacking process. It avoids the well known green cast due to the Bayer filter in OSC.

So what now?

This is the end of pre-processing with a clean image, but still linear. Don’t be surprised if it’s still all black, it’s normal! It remains to carry out the process, but here we are overflowing with this tutorial. :-)