Next: The Automated Data Processing Pipeline for SIRTF IRS
Up: Data Management and Pipelines
Previous: The Next Step for the FUSE Calibration Pipeline
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Patz, A., Harbo, P., Moran, J., Van Stone, D., & Zografou, P. 2003, in ASP Conf. Ser., Vol. 295 Astronomical Data Analysis Software and Systems XII, eds. H. E. Payne, R. I. Jedrzejewski, & R. N. Hook (San Francisco: ASP), 249

Middle Tier Services Accessing the Chandra X-Ray Center Data Archive

Alexandra Patz, Peter Harbo, John Moran, David Van Stone, Panagoula Zografou
Harvard-Smithsonian Center for Astrophysics, 60 Garden Street, Cambridge MA 02138

Abstract:

The Chandra Data Archive team at the Chandra X-ray Center has developed middle tier services that are used by both our search and retrieval applications to uniformly access our data repository. Accessible through an HTTP URL interface, these services can be called by our J2EE web application (WebChaser) and our Java Swing application (Chaser), as well as any other HTTP client. Programs can call the services to retrieve observation data such as a single FITS file, a proposal abstract or a detailed report of observation parameters. Having a central interface to the archive, shared by client applications, facilitates code reusability and easier maintenance.

These middle tier services have been written in Java and packaged into a single J2EE application called the Search and Retrieval (SR) Services. The package consists of a web application front-end and an Enterprise Java Beans back-end. This paper describes the design and use of the SR Services.

1. Introduction

The Chandra Data Archive team has two search and retrieval applications: Chaser, a Java Swing application and WebChaser, a J2EE web application. Both perform similar functions: they search the Chandra Data Archive for observations matching a set of search criteria, display information about the found observations, and allow the user to download data for these observations from the archive. Until recently, they have used two different bodies of code to perform these functions causing maintenance issues. In an effort to make our code more reusable, these shared functions have been broken out into a separate middle tier that both applications can use to access and retrieve data from the archive. This middle layer is comprised of a collection of services, accessible through an HTTP URL interface, called the Search and Retrieval (SR) Services.

In addition to being used by these two internally developed applications, the SR Services can also be used by external programs needing to access the Chandra Data Archive. There is a growing need from the astronomical community for programmatic interfaces to the astronomical data archives. Interest in Web Services is growing with virtual observatory projects needing to query various data archives for observation data. While we have not yet implemented actual Web Services, these HTTP URL services do provide programmatic access to the Chandra Data Archive.

2. The Interface to the SR Services

2.1 Description of Individual SR Services

The following SR Services are available:

Observation Summary Service

Given a set of search criteria (position coordinates, target name, observation start date etc.) this service returns a summary report of the observations found. The data is returned in one of two formats: RDB table format or HTML format. In an effort to comply with emerging virtual observatory standards, there will also be a future option for the data to be returned in VO Table format.

Observation Details Service

Given the same set of search criteria used by the Observation Summary Service, this service returns a detailed observation report including instrument settings and multiple ACIS window settings if applicable. The data is returned in the same formats as the Observation Summary Service.

V&V Report Service

Given a single observation ID, this service returns the Verification and Validation Report. The data is returned as a formatted text file.

Proposal Abstract Service

Given a single observation ID, this service returns proposal information, such as the principal investigator, proposal title and abstract. The data is returned as an HTML page.

Archive File Retrieve Service

Given an observation ID, filetype and data-processing level, this service returns the content of a single file from the Data Archive. The file is returned as a stream of binary data.

Image Service

Given an observation ID and an image filetype, this service returns a JPEG image.

ADS Wrapper Service

Given a single observation ID, this service looks up the NASA Astrophysics Data System (ADS) bibcodes in our Data Archive. It then uses these bibcodes to forward to the relevant page on the ADS website.

2.2 Calling the SR Services

To connect to the SR Services the HTTP URL query mechanism is used. Both Chaser and WebChaser connect to the SR Services using the HttpUrlConnection Java class, as could other Java programs. Clients running scripts could access the SR Services using an application such as wget to retrieve the content via HTTP.

HTTP error codes are returned to the client application when the user passes in a bad parameter, when no data is found, when authentication fails or when proprietary data not belonging to the user is requested.

2.3 Handling proprietary data

If an observation has not yet been made public, the Image Service, the V&V Report Service and the Archive File Service require authentication to retrieve proprietary data. Client applications must send a username and password to these services using www-authentication to retrieve proprietary data, otherwise only publicly available data may be requested.

3. Applications Using the SR Services

Both internally developed applications and externally developed applications could use the SR Services.

3.1 Search and Retrieval Applications: Chaser and WebChaser

Chaser

The Java Swing application Chaser makes calls to the individual SR Services and displays the results in a manner dependent on the data type the service returns. For example, when a service, such as the Observation Details Service, returns an HTML page the results are displayed in the Swing JEditorPane widget. When a service, such as the Observation Summary Service, returns data in RDB table format the results are parsed and displayed in a JTable control.

The WebChaser Observation Viewer

After an initial observation search has been performed in WebChaser, the new Observation Viewer page can be used to examine information about these observations before products are selected for retrieval. The Observation Viewer page includes a set of menu options: Summary, Detail, V&V Report, Proposal Abstract, Images and Publications. Each menu option makes a call to the corresponding service in the SR Services middle tier to display the requested data.

3.2 Other Applications

There has been interest from other astronomical data archives in providing links to Chandra data from their client search applications. These links could be made to the SR Services through the HTTP URL interface.

There has also been interest in using these services from educational astronomy software, astronomical analysis programs and virtual observatory applications.

4. Design of the SR Services

The SR Services have been written in Java and packaged into a single J2EE application. This consists of a web application front-end and an Enterprise Java Beans back-end.

The web application follows a Model-View-Controller design pattern implemented on top of the Apache Struts framework. The Struts framework provides a controller for the application which invokes the relevant SR Service object. The model layer is made up of form beans that encapsulate the set of parameters each service accepts. The view layer consists of JSP pages that display the results.

The Enterprise Java Beans (EJBs) are called from the web application to do the actual archive lookup. The EJBs connect to the back-end archive servers using the Sybase JConnect JDBC driver.

Putting all the SR Services together in one application allows the services to share code. The front-end components share authentication and logging code, for example, while the back-end components share code that checks the proprietary status of the data.

Acknowledgments

This project is supported by the Chandra Xray Center under NASA contract NAS8-39073.