Next: Chandra Level 3 Processing: Creating an Automated Source Catalog for Chandra Data
Up: High Performance Computing
Previous: Developments in the Starlink AST Library - an Intelligent WCS Management System
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Pety, J., Gueth, F., Guilloteau, S., Lucas, R., Teuben, P. J., & Wright, M. C. H. 2003, in ASP Conf. Ser., Vol. 314 Astronomical Data Analysis Software and Systems XIII, eds. F. Ochsenbein, M. Allen, & D. Egret (San Francisco: ASP), 416

Interoperating GILDAS and MIRIAD

J. Pety1
Observatoire de Paris, France

F. Gueth, S. Guilloteau2, R. Lucas
IRAM Grenoble, France

P. J. Teuben
Astronomy Department, U. Maryland

M. C. H. Wright
Radio Astronomy Laboratory, U.C. Berkeley

Abstract:

We describe here the first step we took to interoperate GILDAS and MIRIAD, two current state-of-the-art data reduction packages of millimeter radio astronomy.

1. Introduction to GILDAS and MIRIAD packages

Figure 1: Results of the evaluation of the GILDAS and MIRIAD packages for compliance with the ALMA off-line data processing requirements. Those requirements have been prioritized: All takes into account absolutely all the requirements while Critical only evaluate the 49% more important requirements. In the legend, A means Adequate, A/E Enhancements are desired, I/N Inadequate of Not available and U Unable to evaluate. Low/Medium/High is the severity of the failure to meet the requirement. Many more details may be found in Pety et al. (2003a).
\begin{figure}
\epsscale{0.85}
\plotone{P4-14_f1.eps}
\end{figure}

GILDAS and MIRIAD (Sault et al. 1995) are two state-of-the-art data reduction packages for the current generation of millimeter instruments. GILDAS is used daily at IRAM instruments ( i.e. the Plateau de Bure Interferometer and the 30 m) as well as several other single dish telescopes ( e.g. CSO, HHT, Effelsberg) while MIRIAD is used daily at BIMA, ATCA, OVRO and WSRT. Moreover, as revealed by publicly available evaluations for compliance with the ALMA off-line data processing requirements (Gueth et al. 2003 and Wright et al. 2003), almost 2/3 of the core functionalities needed by the next generation of mm interferometers are adequately covered by GILDAS and MIRIAD. Going further, Fig. 1 shows that, between them, GILDAS and MIRIAD cover more that 75% of the ALMA requirements for off-line. Indeed, although the core functionalities needed to reduce millimeter interferometry data are well covered in both packages, the strength of both packages are complementary. GILDAS has good data analysis and visualization tools, while MIRIAD has a complete set of calibration and imaging algorithms including polarization (for more details, see Pety et al. 2003a). It is thus interesting to study the possibility of interoperating both packages in a user-friendly way (Pety et al. 2003b).

Both packages have rather different implementation philosophy. GILDAS is made of large stand-alone programs like CLASS and CLIC for single-dish and interferometric data calibration and MAPPING for data imaging, deconvolution and analysis. In addition, GILDAS has the ability to launch fully separated tasks. MIRIAD is build around a large collection of individual, well focused tasks, that run under the native operating system. Both packages also have different user interfaces. GILDAS is build around SIC, an homemade command line interface enabling direct insight into the data, and GREG, a very efficient, homemade plotting environment. Since MIRIAD programs are simple standalone tasks, every UNIX shell may serve as MIRIAD user interface. MIRIAD developers have in the past experimented with graphical user interfaces, but in practice users write shell scripts to perform their analysis pipelines. Finally, although the data models used in GILDAS and MIRIAD are very similar, their implementation is very different.

In a first step toward interoperability, we thus decided to run both packages under python and to exchange data using FITS. The porting of both packages under python is described in the next section. Section 3 describes a concrete example of GILDAS/MIRIAD interoperability. This gives us the opportunity to compare the imaging and deconvolution algorithms of GILDAS and MIRIAD.


2. Python Ports

The main difficulty to port GILDAS under python is the different characteristics of SIC vs. python, i.e. strongly vs. weakly typed languages and procedural vs. object oriented languages. As a first approach, we used f2py (a FORTRAN to python interface generator) to produce a generic interface to the SIC command line interpretor and a direct interface to some of the most commonly used commands. All GILDAS capabilities are thus directly available from the python prompt. No one-to-one correspondence between FORTRAN and python variables is yet provided impairing good interaction with data. Progress is currently being made in this direction. Another foreseen evolution is the possibility to swap on-the-fly from python to SIC and vice-versa, as this is an important requirement for the community of current users.

In the MIRIAD case, a generic python interface was written to pass information to MIRIAD tasks and recover their output. Neither direct access to MIRIAD internal data structure nor automatic translation of template procedures from csh to python are yet available. The next improvement will be to automate the building of task access in python, decreasing the size of current python script by about 50%. Finally, the ATA group have recently implemented a Jython interface to MIRIAD dubbed J-MIRIAD (Harp & Wright 2003).


3. Interoperability Demonstration

Figure 2: Top images: The 1 mm continuum imaged in GILDAS (right) and MIRIAD (left), both using the same map resolution and restoring beam. Bottom left: $\vert$GILDAS$-$MIRIAD$\vert$. Bottom right: GILDAS/$\vert$GILDAS$-$MIRIAD$\vert$. In all panels except the fidelity image, the same color scale and 3-$\sigma $ contours levels have been used. The difference map sigma is $1.27\times 10^{-3}$ compared to $1.76\times 10^{-3}$ for GILDAS and MIRIAD images. Please, note the absence of contours on the difference image and the high fidelities similar as the image dynamic, which imply an excellent agreement between GILDAS and MIRIAD.
\begin{figure}
\epsscale{0.85}
\plotone{P4-14_f2.eps}
\end{figure}

An example of the current interoperability capabilities of GILDAS and MIRIAD is available with all needed details to rerun it on your own PC3. In this example, we start from the raw visibilities (in native PdBI format) of the Young Stellar Object GG Tau (Guilloteau, Dutrey & Simon 1999). The data is calibrated inside GILDAS, imaged and deconvolved inside MIRIAD and finally visualized back inside GILDAS, both packages being called from the same python process. This is about 1.5 GB of data that has been processed in about 10 minutes end-to-end on a 1.5 GHz AMD Athlon with 768 MB of RAM memory. We had to adapt our FITS readers/writers as uvfit is an ill-defined standard that does not fully support the flexible GILDAS and MIRIAD data structures. No others difficulties were encountered. The main results are velocity channel maps of the HCO$^{+}$ (J=1-0) line (not shown here) and a 1mm continuum map.

As a by-product, we also imaged and deconvolved the 1mm continuum map inside GILDAS. Fig. 2 shows a detailed comparison of the GILDAS and MIRIAD images, which allows us to conclude that GILDAS and MIRIAD imaging and deconvolution performances are similar.

References

Gueth, F., Guilloteau, S., Lucas, R., Pety, J., & Wright, M. C. H., 2003, BIMA Memo 96 & IRAM Memo 2003-2.

Guilloteau, S., Dutrey, A. & Simon, M., 1999, A&A, 348, 570.

Harp, G. R. & Wright, M. C. H., 2003, BIMA Memo 98 & ATA Memo 58.

Pety, J., Gueth, F., Guilloteau, S., Teuben, P. J. & Wright, M. C. H., 2003, ALMA Memo 464.

Pety, J., Gueth, F., Guilloteau, S., Teuben, P. J. & Wright, M. C. H., 2003, ALMA Memo 465.

Sault, R. J., Teuben, P. J. & Wright, M. C. H., 1995, in ASP Conf. Ser., Vol. 77, Astronomical Data Analysis Software and Systems IV, ed. R. A. Shaw, H. E. Payne, & J. J. E. Hayes (San Francisco: ASP), 433

Wright, M. C. H., Teuben, P. J., & Pety, J., 2003, BIMA Memo 95 & IRAM Memo 2003-1.



Footnotes

...J. Pety1
IRAM Grenoble, France
... S. Guilloteau2
Observatoire de Bordeaux, France
... PC3
The URL is: http://www.iram.fr/~gildas/gildas-miriad/gildas-miriad.html.

© Copyright 2004 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: Chandra Level 3 Processing: Creating an Automated Source Catalog for Chandra Data
Up: High Performance Computing
Previous: Developments in the Starlink AST Library - an Intelligent WCS Management System
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint